1. Conference takeaways:
   Helsinki Data Week was nearly sold out. Juha said Big Data London felt less crowded than two years ago, with context, AI trust and governance, and data products prominent in the discussions.

2. The missing enterprise context layer:
   Catalogs focused on tables and metrics are too narrow. He envisaged a shared reference layer that helps agents find relevant data, documents, processes and APIs—but said he had not seen it working in production yet.

3. Making business meaning explicit:
   Definitions for the book and companion app, the distinction between a business glossary and a data dictionary, and why relationships such as “customer places order” and “customer cancels order” need their meaning captured.

Thank you to everyone who tuned into our latest AURA Ask Us Anything live session..

Join us for the next AURA session in the app

Got a Question you want to ask and have answered in the next AURA session, leave a comment and we got you!

Leave a comment

Shane runs a 6 hour workshop on the Information Product Canvas, a repeatable way to gather data requirements in 30 minutes. The approach he actually uses with stakeholders, not a textbook version that falls apart on contact with messy data and vague requirements.

He runs it regularly.

Latest dates : Gather data requirements in 30 minutes using a shared language

Workshop details : https://maven.com/agiledataguides/info-prod

Another AURA session is scheduled!

and I will chew the data engineering fat on Friday, 9th October 2026, 1pm EDT / 6pm UK.

You can join the Substack Live and ask your questions live here:

https://open.substack.com/live-stream/393056

If you cant make it, the recording will be published on the Agile Data N’ Info substack after the live session.

Feel free to ask any questions you have in the comments here and we will answer them in the live session.

While you wait for the session you can read Chris' awesome content here:

Chris Gambill

No hype. No buzzwords. Just real data strategy, engineering, and leadership insights from 25 years in the trenches. If you’re tired of noise and want to actually understand how to build, scale, and lead with data—this is where you start.

The AURA Session Format

AURA is aimed at working Data Practitioners who are tired of polished webinars and want the real, in-the-weeds conversation, where the live audience sets the agenda and the unedited recording gets published afterwards

Data Practitioners get to ask us questions and we will try to answer them.

There is no agenda, each of the AURA foursome have “a very particular set of skills, skills they have acquired over a very long career …”, the live audience will drive the focus of the conversations.

If the room goes quiet, we will just chat amongst ourselves about what we see is actually happening in the data world right now and what we each worked on and wrestled with last month.

Come join us

It will be quite the ride!

Shane runs a 6 hour workshop on the Information Product Canvas, a repeatable way to gather data requirements in 30 minutes. The approach he actually uses with stakeholders, not a textbook version that falls apart on contact with messy data and vague requirements.

He runs it regularly.

Latest dates : Gather data requirements in 30 minutes using a shared language

Workshop details : https://maven.com/agiledataguides/info-prod

1. Personal OS / “Watson OS”
   a privacy-conscious household management system that filters emails, checks shared calendars, routes responsibilities, and sends daily/weekly digests rather than becoming another task manager.

2. Context layers and shared organisational memory
   AI summaries may be more useful for copilots to retrieve on demand than for people to read; the broader challenge is consent-based creation, sharing, retrieval, and curation of context.

3. Scaling innovation from individuals to the organisation
   power users need a path to turn useful experiments into trusted, governed products, while data teams may increasingly own mini-apps, write-back workflows, and source-system-like capabilities.

Thank you to everyone who tuned into our latest AURA Ask Us Anything live session..

Join us for the next AURA session in the app

Got a Question you want to ask and have answered in the next AURA session, leave a comment and we got you!

Leave a comment

Shane runs a 6 hour workshop on the Information Product Canvas, a repeatable way to gather data requirements in 30 minutes. The approach he actually uses with stakeholders, not a textbook version that falls apart on contact with messy data and vague requirements.

He runs it regularly.

Latest dates : Gather data requirements in 30 minutes using a shared language

Workshop details : https://maven.com/agiledataguides/info-prod

Another AURA session is scheduled!

and I will chew the fat on Wednesday, 7th October 2026, 12pm EDT / 5pm UK.

You can join the Substack Live and ask your questions live here:

https://open.substack.com/live-stream/397370

If you cant make it, the recording will be published on the Agile Data N’ Info substack after the live session.

Feel free to ask any questions you have in the comments here and we will answer them in the live session.

While you wait for the session you can read Tim’s awesome content here:

Self Taught Engineer

Self-taught, engineer to leadership and patterns for learning.

By Tim Frazer

The AURA Session Format

AURA is aimed at working Data Practitioners who are tired of polished webinars and want the real, in-the-weeds conversation, where the live audience sets the agenda and the unedited recording gets published afterwards

Data Practitioners get to ask us questions and we will try to answer them.

There is no agenda, each of the AURA foursome have “a very particular set of skills, skills they have acquired over a very long career …”, the live audience will drive the focus of the conversations.

If the room goes quiet, we will just chat amongst ourselves about what we see is actually happening in the data world right now and what we each worked on and wrestled with last month.

Come join us

It will be quite the ride!

Shane runs a 6 hour workshop on the Information Product Canvas, a repeatable way to gather data requirements in 30 minutes. The approach he actually uses with stakeholders, not a textbook version that falls apart on contact with messy data and vague requirements.

He runs it regularly.

Latest dates : Gather data requirements in 30 minutes using a shared language

Workshop details : https://maven.com/agiledataguides/info-prod

1. How to migrate legacy data platforms safely
   audit first, choose greenfield vs. lift-and-shift early, define the target architecture, and validate with a thin-slice POC rather than a big-bang migration.

2. Using AI in migration work, with human oversight
   LLMs can help translate code, create plans, surface outliers, and review alternatives, but edge cases, over-engineering, and validation mean people remain essential.

3. Preserving business context and governance
   retain raw migration conversations, document why decisions were made, separate logic/context from the execution platform, and establish governance, cost monitoring, and FinOps from day one.

Thank you to everyone who tuned into our latest AURA Ask Us Anything live session.

Join us for the next AURA session in the app

Got a Question you want to ask and have answered in the next AURA session, leave a comment and we got you!

Leave a comment

Shane runs a 6 hour workshop on the Information Product Canvas, a repeatable way to gather data requirements in 30 minutes. The approach he actually uses with stakeholders, not a textbook version that falls apart on contact with messy data and vague requirements.

He runs it regularly.

Latest dates : Gather data requirements in 30 minutes using a shared language

Workshop details : https://maven.com/agiledataguides/info-prod

I have ended up with two separate “harnesses” for now, one called ‘AgileData Practi’ that assists me with the Data Work and one called ‘Information Product Builder’ that assists me in building and deploying the Information Products (yes I know ‘Builder’ is not the most creative name).

The Information Value Stream for “Builder’ looked something like this.

1. Upload completed Information Product Canvas and ask Builder to tell me what data in the Consume layer is missing.

2. Hand that off as a requirements brief to Practi to help do the Data Work.

3. Back to Builder to confirm its now has access to the data it needs.

4. Builder then builds a first cut of the Information Product.

5. I become the human in the loop and review it, ask for iterations until I don’t hate it.

6. Builder deploys it, and I ask our Customer Stakeholders to review it and tell me what they want iterated.

7. Builder and I work together to iterate it based on their feedback.

8. Rinse and repeat #7 until the Stakeholder is happy.

9. Deploy the Information Product as a ‘Live’ version.

There are a few hand-offs in that Value Stream at the moment, they are there on purpose, so there are natural breaks I can check things before we spend more time and tokens on the next step.

But thats an article for another day.

One of the things I have learnt in the current Value Stream is to recognise when I need to get the hell out of the way.

Originally step #7 would go something like this.

1. Stakeholder would email me the things they wanted iterated.

2. I would read it and summarise what I think they mean’t.

3. I would hand the summary to Builder to start the iteration.

I summarise the problem with this process as this.

• The Stakeholder is speaking German

• I translate that German to English

• I give English to Builder and the underlying LLM translates it to French

There is no shared language

As a result a lot of context gets lots at each step of those language translations.

So I tested removing my language translation from that process. It made a big difference.

Now I just give Builder the content from the Stakeholders email and it goes from German to French with aplomb, hell thats what LLM’s are good at right, language.

And it got me thinking how much could I push that Builder work to the Stakeholder directly and let them iterate without me in the loop at all.

Given the manual hand-offs that I have in place at the moment, I didnt want to go full YOLO on this one in the first iteration.

While the dream is a magical box where the Stakeholder writes one line of text and the GenAI tooling builds them a trusted Information Product from the messy raw data in the source systems, we are far from that magic being reality yet.

So as is the Agile Data way I decided to do a McSpikey (research spike) to experiment with a small part of the process.

So I created an experimental Information Product called ‘Pioneer’.

This allows a person (can’t decide what exact Persona its targeting at the moment, think its a novice Data Practitioner rather than a Information Consumer) to do three things:

1. Dig down into the Consume data to see what that data looks like.

2. Prototype a page of an Information Product for that data

3. Share that prototype as a Brief I can put directly into Builder to build and deploy a supported Live Information Product.

A few key points on the Way of Working experiment:

• They can only access trusted Consume data

• They cant share the prototype with anybody, they can only share the generated brief

As I started working on it I ended up going down a rabbit whole on the style of graphs Pioneer was producing.

I have spent quite a bit of time working on Builders Design system for graphs to make them look attractive.

As they say sex sells and I personally didn’t want to produce graphs that looked like they were produced by a Data Analyst using R code.

The first versions were ok.

But not quite the same styles as we have in the Design System and our Information Products, and so down the rabbit hole I went, building out things to force Pioneer to comply with the Information Product Builder Design System.

And then I remembered a Pattern I have used for years.

When I start off the design process for an Information Product in the pre GenAI days I would do a rough wireframe sketch, typically by hand on a piece of paper or a whiteboard.

This let the Stakeholder know that it was a rough sketch, it was the beginning of an idea, it was ok to change it, the cost of change was very very low.

As soon as I created that sketch in a tool and it had an increased level of fidelity, a hogh level of resolution, then people were less likely to requests changes to it.

And so I stopped on the high fidelity resolution quest in Pioneer and embraced the idea of a sketch.

Of course when I asked Claude to make this change it asked me what I wanted to do about the Prospect page. Did I want the same sketch design applied to that as well?

The Prospect page allows the person to explore the Consume data to understand it more.

Data in the Consume layer is deterministic, it is trusted.

And so the person using that page had to feel like they were exploring data they could trust.

So high fidelity graphs it was.

I think if I do open Pioneer up to allow people to explore the data in the Designed or History layers in the AgileData Information Platform, I will have Pioneer change the style of these visualisations back to the sketch format, ie lower fidelity.

But that is a decision for the future.

For now its back to the original question the McSpikey was designed to answer.

Does Pioneer help to make our Way of Working better or worse.

One concern I have is now the Information Product Canvas is decoupled from this process. I have no line of sight how the Pioneer prototype links to an Action, an Outcome and Value,

I am worried it will just move us back into a Dashboard feature factory that has been a problem in the data domain for decades.

But again that is another problem for another day. And no doubt another McSpikey.

• Data platform architecture and orchestration
  whether agent meshes can replace traditional DAG-based orchestration, and when platform-native tools are sufficient.

• Choosing platforms vs. best-of-breed tools
  integration gaps, feature parity, and using explicit principles to decide whether to adopt, lease, or build.

• AI tooling, privacy, and context ownership
  custom AI harnesses, model cost/visibility problems, multiplayer workflows, and the risks of organisational and personal data being exposed or sold.

Thank you to everyone who tuned into our latest AURA Ask Us Anything live session.

Join us for the next AURA session in the app.

Got a Question you want to ask and have answered in the next AURA session, leave a comment and we got you!

Leave a comment

Shane runs a 6 hour workshop on the Information Product Canvas, a repeatable way to gather data requirements in 30 minutes. The approach he actually uses with stakeholders, not a textbook version that falls apart on contact with messy data and vague requirements.

He runs it regularly.

Latest dates : Gather data requirements in 30 minutes using a shared language

Workshop details : https://maven.com/agiledataguides/info-prod

Join us for the next AURA session in the app

Agile Data N’ Info

AURA - Tim Frazer - Wednesday, 02 September 2026 - 12pm EDT / 5pm UK

Another AURA session is scheduled…

Read more

a month ago · Shagility

Got a Question you want to ask and have answered in the next AURA session, leave a comment and we got you!

Leave a comment

Shane runs a 6 hour workshop on the Information Product Canvas, a repeatable way to gather data requirements in 30 minutes. The approach he actually uses with stakeholders, not a textbook version that falls apart on contact with messy data and vague requirements.

He runs it regularly.

Latest dates : Gather data requirements in 30 minutes using a shared language

Workshop details : https://maven.com/agiledataguides/info-prod

One of the bits of work was going through SOC2 certification, we are finally ready for the Type 1 audit, I have to say that doing SOC2 as a two person company is interesting, think I will write more on that later.

Another bit of work was the result of winning a competitive public tender to extend out the VIEW system we have been building and supporting with New Zealand on Air for a few years.

The RFP wanted new data collections for a bunch of social platforms and extended capability for the current Linear, Video on Demand and Web Journalism data sources we already had been running for a while. As well as providing these capabilities to two other NZ media funding agencies.

It also allowed us to package up the new VIEW360 capabilities as a commercial product to take to the global market. What we are branding as MediaView360.com. (still working on the messaging, branding and website for that one).

Until we embarked on iterating these capabilities, we had always used Google Data Studio as our primary last mile tool for our customers to consume their information from.

Data Studio was well integrated with Google BigQuery, it was free (well no license costs) and it did a lot of what the other more expensive BI tools and platforms did.

It was good enough. But it wasn’t great. And we wanted to make Information Products that were great!

When we started AgileData almost 8 years ago, the one thing I said we would never build is a BI front end, the table stakes were to high and there were too many other good options. the problem to be solved was the complexity of the data in the messy middle not yet another BI tool.

So guess what, we decided now was the right time to move that stake.

The ability to use GenAI tools to build what you need and more importantly what you want in this space is no less than fricking amazing.

Don’t get me wrong, its not all honey and marshmallows, there are a lot of things that you still need to wrangle, a lot of still muttering under your breath, a lot of four letter words. I think my go to skill in Claude Code at the mo is /wtf.

But we are managing to build and run things with just Nigel and I, just two co-founders, which we would have normally needed a full team to do.

And we were lucky (or maybe clever) that a lot of our DataOps patterns were easily adopted for our Information Products, our new “Information Product Builder” and “AgileData Practioner” capabilities.

For example I can push a change to the Information Product github repo for a customers AgileData Tenancy and it will deploy that new version automatically into the private Google App Engine container for that customers tenancy and give us all the serverless cost and performance benefits we have had for our AgileData App for years, by resuing that same pattern.

But again ill leave that Pattern as a post for another day.

So for this article I thought I would share a couple of things that we have built lately that I found exciting.

I have previously written how we are building out a Design System for the Information Product Builder to use.

Agile Data N’ Info

Exposing our Design System for AgileData Information Products

As I have been working on using Claude Code and our Information Product templating system to build and deploy Information Products for our customers I have found that I actually need to build out a Design System at the same time…

Read more

5 months ago · 2 likes · Shagility

This gives us scale through the DORO principle (Define Once, Reuse Often)

But we have learnt as we have worked in this new way that building and deploying cookie cutter “Dashboards” that all look and behave the same wasn’t going to cut it. If we were going to do that we might have just reused an open source BI tool or licensed one of the many BI Dashboarding platforms.

What we have found is we should reuse the core of our Design System, but then focus on the unique areas the Stakeholder or Subject Matter Experts really want.

As part of the VIEW360 Information Products one of the data sources we collect is Linear TV data.

One of our key customer Stakeholders asked there any way we could present this data in the form of a TV Guide.

I was sceptical, but as is the AgileData way, I thought lets do a McSpikey, a research spike, and see what happens.

This is what happened:

These are screenshots of the component from our Design System with dummy data, not from our Customers tenancy or their data.

I was surprised how easy it was to “vibe” code this component.

And then of course the flood gates of ideas opened. Would we be able to click on the Programme and see the underlying measurements for it, was the next thing we were asked.

Back to another iteration with Claude Code to which the answer was of course yes.

This Way of Working of small iteration, small builds with constant feedback before we move on to the next small iteration really is the key, that is the agility.

Initially I used to take what I was asked by the Stakeholder via email or via a meeting transcript and summarise it and give it to to Claude Code. I found that I was doing the equivalent of listening to the Stakeholder speaking German, translating it to English so I could understand it and then translating it to French so Claude Code could understand it. That translation process caused no end of problems.

Now I just paste the selected part of the email or the selected part of the transcript and get the hell out of the way.

Again, this is not YOLO or a fully AutomatedAI pattern. I am always the Human in the initial loop, reviewing and asking Claude Code to refine things to both make sure it uses or improves our Design System (this process is a lot harder than it looks) and to provide a strong opinionated view on what is good enough to share for feedback.

I am however doing a McSpikey at the moment to see if I can take myself completely out of the loop and have the Stakeholder or Subject Matter Expert interact directly with an Information Product, called PIONEER, to do the initial ideation.

We will see how that one goes.

Another example from another customer.

For this customer we are building a bundle of Information Products that help with debt recovery optimisation.

We are building them to solve that customers problem and they will take that solution to market and resell it. Build it Once, Sell it Many (BIOSIM doesn’t really have the same ring as DORO, or maybe it does …)

For one of the problems we needed to solve in this space the Stakeholder said they thought a “Vintage Analysis” might be useful.

What the hell is a Vintage Analysis I thought.

So off goes another McSpikey.

This is a Vintage Analysis.

Its what I know of as the Cohort Analysis Pattern. Something we have used a lot in SaaS companies.

Again screenshots of the component from our Design System with dummy data, not from our Customers tenancy or their data.

I didn’t know what a Vintage Analysis was, the LLM did.

We now have a bunch of these and each one is a reusable component in our Design System that is parameterised. They can be used by each customer tenancy, but tailored for that customer.

And when we update one of the components in the Builders Design System we can cascade those changes efficiently to a shit ton of customer tenancies, without losing their personalisation. Again that pattern was harder than it looked.

That is how we can scale as a two co-founder company.

I often say writing helps me think.

We have been so busy lately I have not made the time to write.

So this article really started off as oh I love those components lets finally make some time to write about them.

And at the end of the writing process I have three takeaways in my head:

1. GenAI tooling allows us to quickly iterate in small chunks and then get feedback, it helps us in increase our agility.

2. Personalised components that do exactly what the Staleholder wants, really does provide Stakeholders with a magical experience.

3. To scale, spend as much time on automation, resuse, iterating your Way of Working as you do on the build itself.

Now back to Vibe coding the next component a customer Stakeholder has imagined.

Shane runs a 6 hour workshop on the Information Product Canvas, a repeatable way to gather data requirements in 30 minutes. The approach he actually uses with stakeholders, not a textbook version that falls apart on contact with messy data and vague requirements.

He runs it regularly.

Latest dates : Gather data requirements in 30 minutes using a shared language

Workshop details : https://maven.com/agiledataguides/info-prod

Join us for the next AURA session in the app

Got a Question you want to ask and have answered in the next AURA session, leave a comment and we got you!

Leave a comment

Shane runs a 6 hour workshop on the Information Product Canvas, a repeatable way to gather data requirements in 30 minutes. The approach he actually uses with stakeholders, not a textbook version that falls apart on contact with messy data and vague requirements.

He runs it regularly.

Latest dates : Gather data requirements in 30 minutes using a shared language

Workshop details : https://maven.com/agiledataguides/info-prod

Join us for the next AURA session in the app.

Got a Question you want to ask and have answered in the next AURA session, leave a comment and we got you!

Leave a comment

It will be quite the ride!

Shane runs a 6 hour workshop on the Information Product Canvas, a repeatable way to gather data requirements in 30 minutes. The approach he actually uses with stakeholders, not a textbook version that falls apart on contact with messy data and vague requirements.

He runs it regularly.

Latest dates : Gather data requirements in 30 minutes using a shared language

Workshop details : https://maven.com/agiledataguides/info-prod

The Problem

The problem to be solved is that we collect data from multiple platforms and multiple publishers, and the Content Titles don’t align, so being able to show a single measurement number for a piece of content is a challenge.

For example the Content Titles used for Shortland Street on Linear are often slightly different to the Content Titles used for Shortland Street on Video on Demand.

When we get to YouTube then the problem get a lot bigger, content publishers want to be able to tweak the titles they use to try and increase engagement and reach, but we always need to know which funded content those YouTube videos relate to.

The Blueprint

As we build this out we always start with a light architectural Blueprint so that Nigel and I are on the same page and talking about the same things. (we call this our agile-tecture sessions)

For this one we ended up with 5 pattern components we cared about:

1. Matching Engine
   (AutomatedAI)
   The Pattern(s) that create the suggested matches for Content Titles

2. Matching Store
   (Operational Context)
   The place we store those suggested matches

3. Human in the Loop
   (AssistedAI)
   The interface that allows a Human to be in the loop and approve/decline/pass the suggested match

4. Mastering Policy
   (Business Context)
   The place where we store those human matching decisions as policies

5. Mastering Execution
   (Business Context and Code)
   The business logic that applies those matching decisions in a repeatable way every time we collect or consume the data

Experimenting with Agility

We tend to work on these architectural components in parallel as a decision for one will often impact the decision for another.

And we will often discover something we haven’t thought about that means we end up needed an additional component or two.

For example what we decide for #2 and #4 impacts what we build for #3.

We can use a relational physical data model or a graph model and we can use Spanner, Spanner Graph, BigQuery, BigQuery Graph. These decisions impact what we build for #3.

But also the design of #3 will also impact teh choices we make for #2 and #4.

In the past we had awesome humans help us prototype our app, Chris did the awesome Design and Richard did the awesome Build. Nigel had of course automated the Deploy.

The challenge was the cost of change. It reduced our agility.

So I have been experimenting with the use of Claude Design and then Claude Code to see if we can reduce the time and cost of those experimentation loops.

My Claude Design Process

Here is the process I go though using the Human in the Loop #3 component above as the example.

1 - Pattern Storming ideas, the Town Plan

The first thing I do is give it some light instructions on the intent of the app and then ask for 10 options.

As you can see I don’t worry about spelling etc.

Claude Design came back with this:

What I am doing in this step is looking for UI design ideas.

I always have an idea of what I think these screens will look like before I start the design process and how they will flow. But often Claude Design will come up with a UI or UX pattern I hadn’t thought of, and I then get it to help me rift with those ideas to see if they have legs or not.

2 - Explore the possible Houses on the Street

Next I step back and look at the UI Patterns across all the options. Either one will stand out as the my clear preference and I will start iterating that or I will want to experiment at the higher level, lets call it the town plan level.

In this case I went with the town plan level and so asked for another round.

If you have seen any of the designs in our AgileData UI you will see how Claude Design is re-using the “tile card” component from our Design System for these iterations.

While I love reusing that design component, i’m not sure it fits well for this use case. So we need to iterate that.

3 & 4 - Design the Rooms in the House

One of the things I love about Claude Design os the way it numbers each option. So it allows me to say, take this bit from 2b and this bit from 2d and then give me new versions

It also allows me to add comments to specific parts of each option and then get it to generate new versions based on those comments.

In this flow I did both.

I wanted to focus on one Component and validate it is what we want, so first I picked a specific option and asked for a new version of it.

Focusing on the design of one room in the house so to speak.

It still wasn’t quite want I wanted so back to designing all the rooms again.

This time I provided more guidance using comments. This allows me to tag comments about different parts of the options, highlight the bits I liked and want to keep and the bits I don’t like and don’t want reused.

5 - Design the flow between the Rooms in the House

Once I think I have an idea of the moving parts, I start to refine each one.

In this round I want to focus on the different flows through the screens depending on the Matching Pattern that was used to selected the recommended match.

I want to be clear to the Human in the loop which matches are pretty confident and which require more curation.

6 - And back to the Furniture in a Room

Once I ham happy we have the high level flow of the screens sorted, then I start to focus on the detail.

This time I want to return to focussing on the components within the match that provide the context for the recommended match.

I can’t quite decide how I want those to look, so rather than specific instructions for that part of the design, its back to the “give me 10 ideas” pattern storming

7 - Iterate a specific item of Furniture

8 - Time for the Human

I could keep going iterating this way for ever, but I would quickly hit the diminishing returns anti-pattern.

Plus the closer I get to supposed Done Done, the less likely the people who will validate these screen designs will feel like they can can provide feedback or suggest changes.

So now its time to create a prototype app they can try and see what they like, dislike and hate.

Claude Design creates an prototype you can click though. Haven’t tried that in anger as a way of getting feedback, so keen to give that a try and will report back later.

Mature Design System

One of the gifts we have is Chris our Designer was fastidious on creating a Design System for all the UI components he designed.

This was in Figma and we were able to import that from Figma into Claude Design, as the basis of the design system for prototyping.

I think that saved us a lot of time and effort in the initial setup.

Claude Design will still ignore some of the design system components, specifically fonts for some reason, so I always have to reinforce for Claude Code that its to use the Prototype designs as the ideas not the spec and it must reuse the AgileData App Design System when its creating new screens for our AgileData App UI.

Define Once, Reuse Often (DORO)

One of the things I keep in mind throughout this design process is reuse.

This series of screens will be used for more than just matching Content Titles, we will use it to help match Customers, Suppliers, Products etc for other AgileData Information Products for other customers.

But I don’t want to over engineering it right now, so I focus on the use case in front of me.

That is a bit of a balancing act.

Next Steps

Next I will get feedback from the key stakeholders on how they find this design based on the Claude Design interactive prototype.

And I will share it with Nigel as context for our conversations on what we are going to do for the other moving parts,

The key being this was a minimal investment in time and effort to get this far, and so the cost of change is low.

Its the value we get from this AssistedAI pattern.

And that give us agility to iterate this as much as we need to.

Shane runs a 6 hour workshop on the Information Product Canvas, a repeatable way to gather data requirements in 30 minutes. The approach he actually uses with stakeholders, not a textbook version that falls apart on contact with messy data and vague requirements.

He runs it regularly.

Latest dates : Gather data requirements in 30 minutes using a shared language

Workshop details : https://maven.com/agiledataguides/info-prod

1. Architecture and tests matter more than code generation.
   AI can build quickly, but sustainable work depends on clear outcomes, strong test coverage, and system-level judgment.

2. AI makes feature creep and context-switching easier.
   Both of you saw overbuilding, parallel sessions, and longer test cycles creating new delivery problems—so thin slices and simplicity still matter.

3. “Company brain” will be valuable only with usable context and privacy.
   Shared definitions, selective/high-quality source material, automated capture, and careful privacy boundaries are more important than simply dumping documents into an LLM.

Thank you to everyone who tuned into our latest AURA Ask Us Anything live session.

Join us for the next AURA session in the app.

Got a Question you want to ask and have answered in the next AURA session, leave a comment and we got you!

Leave a comment

Shane runs a 6 hour workshop on the Information Product Canvas, a repeatable way to gather data requirements in 30 minutes. The approach he actually uses with stakeholders, not a textbook version that falls apart on contact with messy data and vague requirements.

He runs it regularly.

Latest dates : Gather data requirements in 30 minutes using a shared language

Workshop details : https://maven.com/agiledataguides/info-prod

Join us for the next AURA session in the app

Got a Question you want to ask and have answered in the next AURA session, leave a comment and we got you!

Leave a comment

Shane runs a 6 hour workshop on the Information Product Canvas, a repeatable way to gather data requirements in 30 minutes. The approach he actually uses with stakeholders, not a textbook version that falls apart on contact with messy data and vague requirements.

He runs it regularly.

Latest dates : Gather data requirements in 30 minutes using a shared language

Workshop details : https://maven.com/agiledataguides/info-prod

Join us for the next AURA session in the app

Got a Question you want to ask and have answered in the next AURA session, leave a comment and we got you!

Leave a comment

Shane runs a 6 hour workshop on the Information Product Canvas, a repeatable way to gather data requirements in 30 minutes. The approach he actually uses with stakeholders, not a textbook version that falls apart on contact with messy data and vague requirements.

He runs it regularly.

Latest dates : Gather data requirements in 30 minutes using a shared language

Workshop details : https://maven.com/agiledataguides/info-prod

Join us for the next AURA session in the app

Got a Question you want to ask and have answered in the next AURA session, leave a comment and we got you!

Leave a comment

Shane runs a 6 hour workshop on the Information Product Canvas, a repeatable way to gather data requirements in 30 minutes. The approach he actually uses with stakeholders, not a textbook version that falls apart on contact with messy data and vague requirements.

He runs it regularly.

Latest dates : Gather data requirements in 30 minutes using a shared language

Workshop details : https://maven.com/agiledataguides/info-prod

Join us for the next AURA session in the app.

Agile Data N’ Info

AURA - Chris Gambill - Friday, 10 July 2026 - 1pm EST / 6pm GMT

Second AURA session is booked…

Read more

3 months ago · Shagility

Got a Question you want to ask and have answered in the next AURA session, leave a comment and we got you!

Leave a comment

Shane runs a 6 hour workshop on the Information Product Canvas, a repeatable way to gather data requirements in 30 minutes. The approach he actually uses with stakeholders, not a textbook version that falls apart on contact with messy data and vague requirements.

He runs it regularly.

Latest dates : Gather data requirements in 30 minutes using a shared language

Workshop details : https://maven.com/agiledataguides/info-prod

Its been a while since I was back riding on the AgileData N’ Info podcast horse so to speak.

So decided time to mount up and this season I am trying something a little different.

I have been lucky enough to meet some awesome data experts over the years, and extra lucky to be able to chew the fat with them on a semi regular basis to discuss what we see happening in the data, product and AI domains.

Its also a great opportunity for me to ask them my latest burning questions.

So I thought what would happen if we had those chats in public and let the audience ask questions.

Me + one data expert, ~1 hour, fully live, audience drives it, published raw with no editing.

Then I thought you know what is better than 1 data expert, 4!

Four regular guests on a monthly rotation, one guest a week.

And so I asked four awesome data experts if they were up for it and they all said yes!

This is going to be an experiment.

• We will use Substack Live for it (first time I have used it)

• Who knows if anybody will turn up and ask questions (but if they dont that is ok, we will just chew the fat anyway)

First awesome data expert scheduled in for next Wednesday, 1st July.

If you want to see the schedule as we build it out, there is a new section for it here here:

agiledata.info/s/aura-a…

Come join us, it might be quite the ride!

Shane runs a 6 hour workshop on the Information Product Canvas, a repeatable way to gather data requirements in 30 minutes. The approach he actually uses with stakeholders, not a textbook version that falls apart on contact with messy data and vague requirements.

He runs it regularly.

Latest dates : Gather data requirements in 30 minutes using a shared language

Workshop details : https://maven.com/agiledataguides/info-prod

“One Shot BI Apps” aren’t good enough for us

As I build and deploy Information Products in production, I have the benefit of the deployment and running of the Information Products being completely automated using the AgileData Information Platform templating system built by Nigel a wee while ago to enable this.

This means I can focus on what the Information Product front end looks like and what data and information it presents to our customers users, rather than how I deploy and manage them.

But as I build more of these Information Products in production I find the need to define and reuse ‘widgets’ in a repeatable way is becoming compelling.

For example I typically need a Date Picker in an Information Product.

But I don’t want Claude Code to randomly generate a new one with different code, different features and different styles every time, I want a default and repeatable version that I can choose change or replace if I need to:

Same with Page Intro’s, I want a certain style:

And with KPI Cards, I want to be able to define a standard style and have all new Information Products use that style:

And in the near future when I add to the KPI Card component the ability to see period on period change, which I know I will be doing soon, I want all my Information Products I have already deployed to inherit that change, where it makes sense.

Define Once, Reuse Often (DORO)

This behaviour is just following our DORO (Define Once, Reuse Often) principle.

Design System

Luckily there is a well proven Pattern that solves this reusability problem called a Design System.

I am out of Claude Code tokens as I write this so over to my other friend Perplexity to find the quick answer for me:

Claude Code of course has access to a lot of content about Design Systems and so with some simple prompts it can start to build one out as I am building Information Products.

There is a whole problem space on how you build these components in a way that can be inherited, but that is a problem and Pattern conversation I will write up in another article.

Making our Design System visible

One of the other challenges is we are not the only people reusing the AgileData Information Product templating capability.

Our AgileData Network partners also use it to build, deploy and manage Information Products for their customers.

So how do I let them know what Design System components are available so they can choose to reuse them, customise them, or ignore them and build their own?

Luckily again there is a proven Pattern that solves this problem.

Companies like Google solved it when they published their Design Systems like Material.

https://m3.material.io/

They created a website that let you explore and see the components that were available.

So that is the Pattern I reused.

First an overview of the Design System Context.

Then each ‘thing’ is visible.

And each Component is interactive so you can quickly see how it works:

And then provide visibility as things constantly change:

Dynamic Context not Static Documentation

In the past these Design System websites would be static, i.e they would be generated and then hosted so the AgileData Network partners could view them at will.

But that always took effort to keep it up to date.

Yes I could automate that refresh each time I get Claude Code to iterate the design system, but given how often I am iterating it at the moment just to get to “BI tablestakes” (that is a whole nother article) it will burn tokens constantly updating it, and there would always be those times where Claude Code will “forgot” to update it.

So applying our “Context not Code by Default” principle, I have embedded the capability to generate the Design System into the Design Systems / Information Product templating system itself.

What this means is when an AgileData Network partner wants to see the latest version of the Design System they just ask Claude Code to generate it. Claude will then read the Design System, generate a standalone localhost app and the partner can explore it to their hearts content.

That way it is always up to date, as it is generated when it is needed using the Context that is current.

(and it uses the AgileData Network partners Claude Code tokens, not mine …. )

Iteration One Done

This is just the first iteration of the Design System.

Now we need to test it for a while with our AgileData Network partners and prove it actually makes their lives easier and what we need to change.

Shane runs a 6 hour workshop on the Information Product Canvas, a repeatable way to gather data requirements in 30 minutes. The approach he actually uses with stakeholders, not a textbook version that falls apart on contact with messy data and vague requirements.

He runs it regularly.

Latest dates : Gather data requirements in 30 minutes using a shared language

Workshop details : https://maven.com/agiledataguides/info-prod

As I have been working with Claude and Claude Code to do some “vibe coding” development over the last wee while I have also been working on a “harness” that is designed to make me more efficient.

Its following the DORO (Define Once, Reuse Often) principle I try and follow.

One of the surprising thing (to me anyway) is when I mention the concept of a “harness” to other people I sometimes get blank stares.

I actually did a LinkedIn poll and this feeling was reinforced by the reponses.

https://www.linkedin.com/posts/shagility_based-on-a-few-chats-and-a-bunch-of-reading-activity-7458771767482839041-cO_e/

No seriously what is an “AI Harness”

One of the problems I have is I can’t share the harness files we have developed within our AgileData platform and I don’t want to share the harness files from my PersonalOS version.

I thought about mocking some up, but then they would’t be real examples and so wouldn’t be that helpful in showing what they do, how they do it and their real value.

I actually started my harness journey based on playing with OpenClaw, listening to a few podcasts on the subject and chatting to . Nick did an in-person meetup on how he built his PersonalOS harness and how he uses it, I learnt a lot at that session.

You can watch a version of this talk online here:

There is also a great article by on LinkedIn that explains what an AI Harness is:

https://www.linkedin.com/pulse/i-learned-ai-harnesses-you-should-too-andreas-kretz-v2okf

Claude Write up on how my harness works

One way I thought might be useful to share this part of my journey is to get Claude to go through all the chats and code we have worked on together over the last we while to create my harness and explain it.

This has been based on iterating the harness for three specific use cases:

1. To support my PersonalOS

2. To support the vibe coding of the open source Pattern Template Standalone Apps I have been creating and sharing. (https://github.com/AgileDataGuides)

3. To support the creation of an Information Product templating system in the AgileData platform and using it to build and deploy Information Products for our customers.

The key is each time I work on one of these use cases, the learnings from doing that work is used to enhance the harness. And hopefully makes that work quicker and easier for me next time.

So here is what Claude wrote.

Universal harness     — applies to every Agent
   Family templates   — shared patterns for one Agent family
      Real Agents     — specific deployments

Weekly retro.md (per Agent)
   ↓ proven across sessions
"Retro learnings" in wow.md (per Agent)
   ↓ would benefit other Agents
Harness inbox (cross-Agent review)
   ↓ accepted by curator
Universal conventions / shared skills (every Agent)
   ↓ sync
Sister Agents pick it up

proposed_destination: agiledata     # universal Agile Data rule
# or: custom-app-template           # all Agile Data custom-app Agents
# or: demo-template                 # one Agent family
# or: shagility                     # personal preferences only

Markup version of the text at the bottom of the article if you want to copy and paste it into your own LLM.

Some More Context

I asked Claude to explain:

1. What the harness is

2. How it is inherited across “agents”

3. How it differs from a bunch of skills.

When it talks about “Agents” there is nothing fancy happening, its is just a different directory on my MacBook, that has separate code in it that I am working on for a specific Use Case.

Nothing I am doing is unique or magic, a lot of people are building these harnesses as they use these tools.

Why not just use somebody elses “harness”

There are a bunch of harnesses out there you can download and use.

I tried a few and found they didn’t help me as much as I had hoped.

It might have been the type of work I am trying to use them for.

It might have been my lack of coding skills.

Try one and see if they help you.

Part of this whole journey is learning as I am doing so I also naturally leaned towards craft my own.

My Agile Data Way of Working Language

You can see the Agile Data language I use coming through strongly in this harness:

• Persona

• Policies

• WoW

• Retro

• Defintion of Ready

• Definition of Done

• TODO / DOING / DONE

• Press Release

I have naturally being applying the Patterns and Pattern Templates I coach a human Data and Analytics team to use to my machine buddy.

Others use different terms for the same things (Identity, Soul etc). They may also include the things I hold in multiple files in a single file for their harness.

And I do worry that applying Human centric patterns to the Machine may not be the optimal approach.

A journey not a proven Pattern or Pattern Template

All the above is just a brain dump on part of my journey so far.

It is not a well formed Pattern or a tested Pattern Template.

But hopefully Sharing it in its half arsed state is still Caring.

Markdown Version of the Claude Content

# The Agile Data Way-of-Working Harness

A reusable convention package that gives every AI-assisted coding session in our repositories the same Way of Working — the disciplines, guardrails, and patterns we've evolved over months of working with AI coding assistants.

## The problem

AI coding assistants are stateless. Each session starts fresh, so the team re-explains conventions, hits the same gotchas, and watches the same mistakes recur across Agents. When a learning emerges in one Agent, there's no clean path to apply it everywhere it matters.

## The structure

Each repository carries a small, predictable bundle the AI reads at session start. Together this bundle defines a specific **Agent** — the AI is no longer a generic assistant; it has an identity, hard rules, a way of working, and a memory of what it has learned.

**Agent identity** — four short markdown files defining the Agent's character:

- `persona.md` — what this Agent is, its voice, who it serves
- `policies.md` — hard rules and guardrails (will not / will always / boundaries)
- `wow.md` — Agent-specific way of working + accumulated retro learnings
- `retro.md` — running weekly log of new learnings

**Universal conventions** in a single `CLAUDE.md` that the AI loads automatically:

- Task tracking (TODO / DOING / DONE files)
- Commit discipline (logical chunks, no work pileup)
- Definition of Ready / Definition of Done checklists
- Press release format for user-facing changes (working-backwards style)
- Estimation as t-shirt sizes (XS to XL), not minutes
- Pre-push verification (explicit checklist before push to auto-deploy branches)
- Design system compliance (every UI value traces to a design token)
- Scoped tool / data access per Agent (prevents wrong-environment bugs)

**A skills system** — discrete procedures the Agent invokes by name when triggered (e.g. *"draft a step-page"*, *"verify after change"*). Skills load on demand, so they don't bloat every session's context.

## The inheritance hierarchy

```
Universal harness     — applies to every Agent
   Family templates   — shared patterns for one Agent family
      Real Agents     — specific deployments
```

A new Agent cloned from a family template inherits all three layers and starts with a clean identity it customises. Every layer follows the same convention set.

## What it gives you

- New AI sessions reach productive work in minutes, without re-explaining basics
- Conventions stay consistent across Agents without anyone policing them
- Bugs caught once stay caught — they land as policies or guardrails
- AI estimates and pre-push verifications are predictable and comparable
- Cross-Agent learnings compound rather than re-occurring

## What it doesn't do

- Replace human judgement on architecture or product direction
- Generate code without context — good prompts still matter
- Work across tools that don't honour structured conventions

The harness is small, opinionated, and modular. Drop in what fits, leave what doesn't.

---

## The retro process — how learnings compound

The retro flow is the engine that keeps the harness alive. Without it, the conventions calcify and the harness becomes a relic. With it, every Agent that's harnessed benefits from every learning every other Agent surfaces.

### Three tiers, one direction

```
Weekly retro.md (per Agent)
   ↓ proven across sessions
"Retro learnings" in wow.md (per Agent)
   ↓ would benefit other Agents
Harness inbox (cross-Agent review)
   ↓ accepted by curator
Universal conventions / shared skills (every Agent)
   ↓ sync
Sister Agents pick it up
```

### Tier 1 — Weekly retro per Agent

Mid-session, when a learning emerges, it gets added to `retro.md` under the current week's heading. Examples of what lands here:

- *"Tooltips inside scrollable tables need to portal to `<body>` or they get clipped by overflow."*
- *"Don't fabricate descriptions when the catalog is silent — use an explicit `(no description in catalog)` fallback."*
- *"Status of an Information Product is about trust level, not dev stage. Code-complete + data-unvalidated is `experimental`, not `live`."*

The retro log is cheap, time-boxed, and read by humans during the weekly review.

### Tier 2 — Promote into wow.md when proven

If a retro entry has stuck across multiple sessions and continues to serve the Agent well, it's promoted into `wow.md` under "Retro learnings" — becoming a permanent Agent rule. The model reads `wow.md` at session start, so promoted entries shape future behaviour automatically.

### Tier 3 — Promote to the harness for universal value

A learning that would help every Agent, not just this one, gets sent to the harness inbox via the `retro-promote` skill. The promotion is a single markdown file with a frontmatter block declaring its proposed destination:

```yaml
proposed_destination: agiledata     # universal Agile Data rule
# or: custom-app-template           # all Agile Data custom-app Agents
# or: demo-template              # one Agent family
# or: shagility                     # personal preferences only
```

The curator reviews the inbox in a separate session, applies the universality test ("would another Agent benefit from this?"), and either accepts into the appropriate layer of the harness, rejects, or defers.

### The reverse path — sync down

When the curator accepts a learning into the universal harness, it doesn't magically appear in every Agent. Each Agent pulls it on the next `harness-sync` call. This explicit pull preserves Agent autonomy — a fork can choose not to sync a change it doesn't want.

### Why this matters

Without the retro loop, the harness becomes static reference material that quickly drifts out of date. With it, the harness is a living convention base that gets sharper every week. The cost of a hard-won learning is paid once; the value compounds across every Agent that follows.

---

## How this differs from "just a collection of skills"

It's tempting to imagine the harness is just a pile of skill files. It isn't. Confusing the two leads to a system that feels organised but doesn't actually change behaviour where it matters.

### Skills are tactical. The harness is strategic.

A **skill** is a discrete procedure for a specific task: *"draft a LinkedIn post from a published article"*, *"verify the changed code by running the test suite"*. Loaded on demand, by keyword match, when the user asks for the thing.

The **harness** is the identity, conventions, and rules of engagement that apply BEFORE any specific task starts. *"This is what this Agent is."* *"Here's what we never do."* *"Here's how we track work."* *"Here's the press release format every user-facing change must follow."*

A skill answers *how* to do a particular task. The harness answers *who is this Agent*, *what are the non-negotiables*, *how do we operate*.

### Skills load on demand. Harness conventions are always-on.

The model only loads a skill's full body when the user's task triggers its description ("write a press release" → load the press-release skill). Without that trigger, the skill sits silent.

Agent identity (persona, policies, wow) is loaded into the system prompt every session start. Every conversation, every response, the model is already operating inside that frame. That's why a stray "wrong tenancy" mistake gets caught — the policy is always in scope, not waiting for a keyword.

### Skills don't enforce discipline. Conventions do.

You can write a skill called *"do the Definition of Done"*, but it only fires if someone asks for it. The DoD checklist as a convention in `CLAUDE.md` shapes every response by default.

The pre-push verification rule that catches wrong-tenancy data, the press release format that protects brand voice, the t-shirt sizing rule that stops minute-level commitments — none of these would be reliable if they were skills the user has to invoke. They have to be the air the Agent breathes.

### Skills don't propagate. The harness has a sync path.

If you write a great skill in Agent A, it stays in Agent A. Agent B's session has no idea it exists. The retro flow + harness sync is what makes a learning in one Agent show up everywhere it matters.

A pile of skills with no propagation path gets reinvented in every repo. The harness pays the cost of a learning once.

### Skills don't carry Agent context. Identity files do.

A skill called *"build a marketplace page"* is the same procedure no matter which Agent it's in. But what a marketplace page *means* for an Information Product Agent with five live products is different to what it means for a single-product demo Agent. The persona, policies, and wow files give the AI that Agent-specific context BEFORE it reaches for any procedure.

### The relationship

Skills and the harness aren't competitors — they're complementary layers:

| Layer | Loaded when | Best for |
|---|---|---|
| **Agent identity** (persona/policies/wow) | Every session start, always-on | Who this Agent is, hard rules, way of working |
| **Universal conventions** (CLAUDE.md) | Every session start, always-on | Rules that apply to every Agent regardless of context |
| **Skills** | On-demand, by keyword trigger | Procedures for specific recurring tasks |
| **Retro flow** | Weekly + as learnings emerge | Keeping conventions alive and propagating wins |

Take any one of these layers out and the structure leaks. Stack them and the AI session arrives with Agent context already loaded, universal rules already in scope, the right procedure ready to fire when triggered, and a clean path for what it learns to compound into the next session.

That's the harness.

Shane runs a 6 hour workshop on the Information Product Canvas, a repeatable way to gather data requirements in 30 minutes. The approach he actually uses with stakeholders, not a textbook version that falls apart on contact with messy data and vague requirements.

He runs it regularly.

Latest dates : Gather data requirements in 30 minutes using a shared language

Workshop details : https://maven.com/agiledataguides/info-prod