“One Shot BI Apps” aren’t good enough for us

As I build and deploy Information Products in production, I have the benefit of the deployment and running of the Information Products being completely automated using the AgileData Information Platform templating system built by Nigel a wee while ago to enable this.

This means I can focus on what the Information Product front end looks like and what data and information it presents to our customers users, rather than how I deploy and manage them.

But as I build more of these Information Products in production I find the need to define and reuse ‘widgets’ in a repeatable way is becoming compelling.

For example I typically need a Date Picker in an Information Product.

But I don’t want Claude Code to randomly generate a new one with different code, different features and different styles every time, I want a default and repeatable version that I can choose change or replace if I need to:

Same with Page Intro’s, I want a certain style:

And with KPI Cards, I want to be able to define a standard style and have all new Information Products use that style:

And in the near future when I add to the KPI Card component the ability to see period on period change, which I know I will be doing soon, I want all my Information Products I have already deployed to inherit that change, where it makes sense.

Define Once, Reuse Often (DORO)

This behaviour is just following our DORO (Define Once, Reuse Often) principle.

Design System

Luckily there is a well proven Pattern that solves this reusability problem called a Design System.

I am out of Claude Code tokens as I write this so over to my other friend Perplexity to find the quick answer for me:

Claude Code of course has access to a lot of content about Design Systems and so with some simple prompts it can start to build one out as I am building Information Products.

There is a whole problem space on how you build these components in a way that can be inherited, but that is a problem and Pattern conversation I will write up in another article.

Making our Design System visible

One of the other challenges is we are not the only people reusing the AgileData Information Product templating capability.

Our AgileData Network partners also use it to build, deploy and manage Information Products for their customers.

So how do I let them know what Design System components are available so they can choose to reuse them, customise them, or ignore them and build their own?

Luckily again there is a proven Pattern that solves this problem.

Companies like Google solved it when they published their Design Systems like Material.

https://m3.material.io/

They created a website that let you explore and see the components that were available.

So that is the Pattern I reused.

First an overview of the Design System Context.

Then each ‘thing’ is visible.

And each Component is interactive so you can quickly see how it works:

And then provide visibility as things constantly change:

Dynamic Context not Static Documentation

In the past these Design System websites would be static, i.e they would be generated and then hosted so the AgileData Network partners could view them at will.

But that always took effort to keep it up to date.

Yes I could automate that refresh each time I get Claude Code to iterate the design system, but given how often I am iterating it at the moment just to get to “BI tablestakes” (that is a whole nother article) it will burn tokens constantly updating it, and there would always be those times where Claude Code will “forgot” to update it.

So applying our “Context not Code by Default” principle, I have embedded the capability to generate the Design System into the Design Systems / Information Product templating system itself.

What this means is when an AgileData Network partner wants to see the latest version of the Design System they just ask Claude Code to generate it. Claude will then read the Design System, generate a standalone localhost app and the partner can explore it to their hearts content.

That way it is always up to date, as it is generated when it is needed using the Context that is current.

(and it uses the AgileData Network partners Claude Code tokens, not mine …. )

Iteration One Done

This is just the first iteration of the Design System.

Now we need to test it for a while with our AgileData Network partners and prove it actually makes their lives easier and what we need to change.

As I have been working with Claude and Claude Code to do some “vibe coding” development over the last wee while I have also been working on a “harness” that is designed to make me more efficient.

Its following the DORO (Define Once, Reuse Often) principle I try and follow.

One of the surprising thing (to me anyway) is when I mention the concept of a “harness” to other people I sometimes get blank stares.

I actually did a LinkedIn poll and this feeling was reinforced by the reponses.

https://www.linkedin.com/posts/shagility_based-on-a-few-chats-and-a-bunch-of-reading-activity-7458771767482839041-cO_e/

No seriously what is an “AI Harness”

One of the problems I have is I can’t share the harness files we have developed within our AgileData platform and I don’t want to share the harness files from my PersonalOS version.

I thought about mocking some up, but then they would’t be real examples and so wouldn’t be that helpful in showing what they do, how they do it and their real value.

I actually started my harness journey based on playing with OpenClaw, listening to a few podcasts on the subject and chatting to . Nick did an in-person meetup on how he built his PersonalOS harness and how he uses it, I learnt a lot at that session.

You can watch a version of this talk online here:

There is also a great article by on LinkedIn that explains what an AI Harness is:

https://www.linkedin.com/pulse/i-learned-ai-harnesses-you-should-too-andreas-kretz-v2okf

Claude Write up on how my harness works

One way I thought might be useful to share this part of my journey is to get Claude to go through all the chats and code we have worked on together over the last we while to create my harness and explain it.

This has been based on iterating the harness for three specific use cases:

1. To support my PersonalOS

2. To support the vibe coding of the open source Pattern Template Standalone Apps I have been creating and sharing. (https://github.com/AgileDataGuides)

3. To support the creation of an Information Product templating system in the AgileData platform and using it to build and deploy Information Products for our customers.

The key is each time I work on one of these use cases, the learnings from doing that work is used to enhance the harness. And hopefully makes that work quicker and easier for me next time.

So here is what Claude wrote.

Universal harness     — applies to every Agent
   Family templates   — shared patterns for one Agent family
      Real Agents     — specific deployments

Weekly retro.md (per Agent)
   ↓ proven across sessions
"Retro learnings" in wow.md (per Agent)
   ↓ would benefit other Agents
Harness inbox (cross-Agent review)
   ↓ accepted by curator
Universal conventions / shared skills (every Agent)
   ↓ sync
Sister Agents pick it up

proposed_destination: agiledata     # universal Agile Data rule
# or: custom-app-template           # all Agile Data custom-app Agents
# or: demo-template                 # one Agent family
# or: shagility                     # personal preferences only

Markup version of the text at the bottom of the article if you want to copy and paste it into your own LLM.

Some More Context

I asked Claude to explain:

1. What the harness is

2. How it is inherited across “agents”

3. How it differs from a bunch of skills.

When it talks about “Agents” there is nothing fancy happening, its is just a different directory on my MacBook, that has separate code in it that I am working on for a specific Use Case.

Nothing I am doing is unique or magic, a lot of people are building these harnesses as they use these tools.

Why not just use somebody elses “harness”

There are a bunch of harnesses out there you can download and use.

I tried a few and found they didn’t help me as much as I had hoped.

It might have been the type of work I am trying to use them for.

It might have been my lack of coding skills.

Try one and see if they help you.

Part of this whole journey is learning as I am doing so I also naturally leaned towards craft my own.

My Agile Data Way of Working Language

You can see the Agile Data language I use coming through strongly in this harness:

• Persona

• Policies

• WoW

• Retro

• Defintion of Ready

• Definition of Done

• TODO / DOING / DONE

• Press Release

I have naturally being applying the Patterns and Pattern Templates I coach a human Data and Analytics team to use to my machine buddy.

Others use different terms for the same things (Identity, Soul etc). They may also include the things I hold in multiple files in a single file for their harness.

And I do worry that applying Human centric patterns to the Machine may not be the optimal approach.

A journey not a proven Pattern or Pattern Template

All the above is just a brain dump on part of my journey so far.

It is not a well formed Pattern or a tested Pattern Template.

But hopefully Sharing it in its half arsed state is still Caring.

Markdown Version of the Claude Content

# The Agile Data Way-of-Working Harness

A reusable convention package that gives every AI-assisted coding session in our repositories the same Way of Working — the disciplines, guardrails, and patterns we've evolved over months of working with AI coding assistants.

## The problem

AI coding assistants are stateless. Each session starts fresh, so the team re-explains conventions, hits the same gotchas, and watches the same mistakes recur across Agents. When a learning emerges in one Agent, there's no clean path to apply it everywhere it matters.

## The structure

Each repository carries a small, predictable bundle the AI reads at session start. Together this bundle defines a specific **Agent** — the AI is no longer a generic assistant; it has an identity, hard rules, a way of working, and a memory of what it has learned.

**Agent identity** — four short markdown files defining the Agent's character:

- `persona.md` — what this Agent is, its voice, who it serves
- `policies.md` — hard rules and guardrails (will not / will always / boundaries)
- `wow.md` — Agent-specific way of working + accumulated retro learnings
- `retro.md` — running weekly log of new learnings

**Universal conventions** in a single `CLAUDE.md` that the AI loads automatically:

- Task tracking (TODO / DOING / DONE files)
- Commit discipline (logical chunks, no work pileup)
- Definition of Ready / Definition of Done checklists
- Press release format for user-facing changes (working-backwards style)
- Estimation as t-shirt sizes (XS to XL), not minutes
- Pre-push verification (explicit checklist before push to auto-deploy branches)
- Design system compliance (every UI value traces to a design token)
- Scoped tool / data access per Agent (prevents wrong-environment bugs)

**A skills system** — discrete procedures the Agent invokes by name when triggered (e.g. *"draft a step-page"*, *"verify after change"*). Skills load on demand, so they don't bloat every session's context.

## The inheritance hierarchy

```
Universal harness     — applies to every Agent
   Family templates   — shared patterns for one Agent family
      Real Agents     — specific deployments
```

A new Agent cloned from a family template inherits all three layers and starts with a clean identity it customises. Every layer follows the same convention set.

## What it gives you

- New AI sessions reach productive work in minutes, without re-explaining basics
- Conventions stay consistent across Agents without anyone policing them
- Bugs caught once stay caught — they land as policies or guardrails
- AI estimates and pre-push verifications are predictable and comparable
- Cross-Agent learnings compound rather than re-occurring

## What it doesn't do

- Replace human judgement on architecture or product direction
- Generate code without context — good prompts still matter
- Work across tools that don't honour structured conventions

The harness is small, opinionated, and modular. Drop in what fits, leave what doesn't.

---

## The retro process — how learnings compound

The retro flow is the engine that keeps the harness alive. Without it, the conventions calcify and the harness becomes a relic. With it, every Agent that's harnessed benefits from every learning every other Agent surfaces.

### Three tiers, one direction

```
Weekly retro.md (per Agent)
   ↓ proven across sessions
"Retro learnings" in wow.md (per Agent)
   ↓ would benefit other Agents
Harness inbox (cross-Agent review)
   ↓ accepted by curator
Universal conventions / shared skills (every Agent)
   ↓ sync
Sister Agents pick it up
```

### Tier 1 — Weekly retro per Agent

Mid-session, when a learning emerges, it gets added to `retro.md` under the current week's heading. Examples of what lands here:

- *"Tooltips inside scrollable tables need to portal to `<body>` or they get clipped by overflow."*
- *"Don't fabricate descriptions when the catalog is silent — use an explicit `(no description in catalog)` fallback."*
- *"Status of an Information Product is about trust level, not dev stage. Code-complete + data-unvalidated is `experimental`, not `live`."*

The retro log is cheap, time-boxed, and read by humans during the weekly review.

### Tier 2 — Promote into wow.md when proven

If a retro entry has stuck across multiple sessions and continues to serve the Agent well, it's promoted into `wow.md` under "Retro learnings" — becoming a permanent Agent rule. The model reads `wow.md` at session start, so promoted entries shape future behaviour automatically.

### Tier 3 — Promote to the harness for universal value

A learning that would help every Agent, not just this one, gets sent to the harness inbox via the `retro-promote` skill. The promotion is a single markdown file with a frontmatter block declaring its proposed destination:

```yaml
proposed_destination: agiledata     # universal Agile Data rule
# or: custom-app-template           # all Agile Data custom-app Agents
# or: demo-template              # one Agent family
# or: shagility                     # personal preferences only
```

The curator reviews the inbox in a separate session, applies the universality test ("would another Agent benefit from this?"), and either accepts into the appropriate layer of the harness, rejects, or defers.

### The reverse path — sync down

When the curator accepts a learning into the universal harness, it doesn't magically appear in every Agent. Each Agent pulls it on the next `harness-sync` call. This explicit pull preserves Agent autonomy — a fork can choose not to sync a change it doesn't want.

### Why this matters

Without the retro loop, the harness becomes static reference material that quickly drifts out of date. With it, the harness is a living convention base that gets sharper every week. The cost of a hard-won learning is paid once; the value compounds across every Agent that follows.

---

## How this differs from "just a collection of skills"

It's tempting to imagine the harness is just a pile of skill files. It isn't. Confusing the two leads to a system that feels organised but doesn't actually change behaviour where it matters.

### Skills are tactical. The harness is strategic.

A **skill** is a discrete procedure for a specific task: *"draft a LinkedIn post from a published article"*, *"verify the changed code by running the test suite"*. Loaded on demand, by keyword match, when the user asks for the thing.

The **harness** is the identity, conventions, and rules of engagement that apply BEFORE any specific task starts. *"This is what this Agent is."* *"Here's what we never do."* *"Here's how we track work."* *"Here's the press release format every user-facing change must follow."*

A skill answers *how* to do a particular task. The harness answers *who is this Agent*, *what are the non-negotiables*, *how do we operate*.

### Skills load on demand. Harness conventions are always-on.

The model only loads a skill's full body when the user's task triggers its description ("write a press release" → load the press-release skill). Without that trigger, the skill sits silent.

Agent identity (persona, policies, wow) is loaded into the system prompt every session start. Every conversation, every response, the model is already operating inside that frame. That's why a stray "wrong tenancy" mistake gets caught — the policy is always in scope, not waiting for a keyword.

### Skills don't enforce discipline. Conventions do.

You can write a skill called *"do the Definition of Done"*, but it only fires if someone asks for it. The DoD checklist as a convention in `CLAUDE.md` shapes every response by default.

The pre-push verification rule that catches wrong-tenancy data, the press release format that protects brand voice, the t-shirt sizing rule that stops minute-level commitments — none of these would be reliable if they were skills the user has to invoke. They have to be the air the Agent breathes.

### Skills don't propagate. The harness has a sync path.

If you write a great skill in Agent A, it stays in Agent A. Agent B's session has no idea it exists. The retro flow + harness sync is what makes a learning in one Agent show up everywhere it matters.

A pile of skills with no propagation path gets reinvented in every repo. The harness pays the cost of a learning once.

### Skills don't carry Agent context. Identity files do.

A skill called *"build a marketplace page"* is the same procedure no matter which Agent it's in. But what a marketplace page *means* for an Information Product Agent with five live products is different to what it means for a single-product demo Agent. The persona, policies, and wow files give the AI that Agent-specific context BEFORE it reaches for any procedure.

### The relationship

Skills and the harness aren't competitors — they're complementary layers:

| Layer | Loaded when | Best for |
|---|---|---|
| **Agent identity** (persona/policies/wow) | Every session start, always-on | Who this Agent is, hard rules, way of working |
| **Universal conventions** (CLAUDE.md) | Every session start, always-on | Rules that apply to every Agent regardless of context |
| **Skills** | On-demand, by keyword trigger | Procedures for specific recurring tasks |
| **Retro flow** | Weekly + as learnings emerge | Keeping conventions alive and propagating wins |

Take any one of these layers out and the structure leaks. Stack them and the AI session arrives with Agent context already loaded, universal rules already in scope, the right procedure ready to fire when triggered, and a clean path for what it learns to compound into the next session.

That's the harness.

The reasoning behind this principles was creating a BI tool or BI platform had too many table stakes from day one to have a chance of succedding.

Too many expected features, too much competition, too much time, effort and moolah to build something that could compete with the established players.

So we used third party tools. Looker Studio, Power BI, whatever the customer had already paid for. We focused on the hard bit underneath, the data platform, the data, the business context, the patterns, the scaffolding.

All bound with the core principles of being able to scale at will, and rin on the smell of a oily rag ( well based on the cost of oil before the latest middle eastern “adventure”)

That was the right call and has lasted us well for the last seven years.

Then vibe coding turned up and we revisited our thinking.

TL;DR

We are betting the BI platforms of the previous waves become legacy and “one shot BI apps” become the norm.

Bold statement right?

Let me give you some context (after all without Context all we have is data)

Why we think “One Shot BI Apps” are the new black

Up until now if an organisation wanted a bespoke Information Product / BI App that did exactly what a consumer needed, they had to pay an expert development team to build it and then maintain it.

That took time, effort and money.

So instead they compromised, they bought a BI platform, a general purpose tool that kind of did what everyone needed but never quite did what anyone actually wanted.

Stakeholder don’t actually want a dashboard with 24 areas and 47 filters they have to squint at to find the one number they care about. That is just what they have had to live with until now.

They want an app that answers their specific question or even better tells them the next best action. They want an app that works the way they think and the way they work, with nothing else getting in the way.

Data teams doen’t want to spend three iterations building a Power BI report that gets used twice and then someone asks for “just one more chart”. They want to deliver something that actually gets used, that drives proven organisational outcomes and value and most of all and makes somebody’s life better.

Stakeholders and Data Teams want something that does one thing well, enables one specific job to be done, and gets out of the way.

The BI platform was always a compromise. It existed because bespoke was too slow and too expensive.

And that where vibe coding using tools like Claude Code have changed the promise.

They promise:

• Low cost :: Making bespoke cheap as chips.

• Speed to market :: Minutes and hours, not weeks and months.

• Democratisation :: Anybody who can describe what they need can build it.

• Cost :: Less than your annual BI platform licence, and your big server/services infrastructure costs by a long way.

With all these promises why would they keep compromising?

We have seen this movie before

We have seen promises like this before, promises that are early in the hype wave, and then eventually get grounded in reality

We have seen how every time we democratise access to something in the data domain, we eventually end up with sprawl.

• The OLAP wave democratised access to data. What did we get? A mess of cubes nobody could find or trust.

• The Tableau wave democratised access to dashboards. What did we get? Thousands of dashboards, most of them showing slightly different numbers for the same thing, or a slightly different number of data columns.

• The dbt/modern data stack wave democratised access to data transformation. What did we get? 5,000 dbt models with no actual data model, anybody?

Now we are about to democratise access to building BI apps. And the sprawl problem is going to make the previous waves look like a gentle ripple.

I have written thoughts about some of these anti-patterns in the new “AI” wave before:

But what if you already had the scaffolding?

Luckily we have spent the last seven years building out a data platform that is both opinionated and has all the scaffolding that should (in theory) help manage this inevitable sprawl.

So we started by creating a simple templating pattern in our platform. A way to quickly build and deploy a “one shot BI app” using your favourite LLM tool. Claude, ChatGPT, whatever you prefer.

Then we let it loose with one of our talented early adopters Network patterns, to see what he would actually do with it.

We were amazed with what he started creating.

Next we used it to build a prototype for a customer who wanted us to upgrade the capabilities they had been using with us for the last few years.

They loved it.

So once more into the breach, my friends.

“Can’t code, won’t code, don’t code” tries vibe coding

One of my frequent sayings is

But I wondered, with the latest tools like Claude Code and the latest models like Opus 4.6, is that still true?

I find I learn best by doing.

So I thought I would experiment with creating an app using Claude Code to see how the process would work for somebody like me. Somebody who understands the patterns and requirements inside out, but has spent decades successfully avoiding writing code.

Now I could have vibe coded a “one shot BI app” for a customer, or built something with public dummy data to experiment and learn.

But the Information Product Canvas was the obvious first cab off the rank.

I have been iterating that pattern template for over a decade. I know the 12 areas, how they relate to each other, what the user experience should feel like, and what the anti-patterns look like.

If Claude Code got something wrong, I would (hopefully) know immediately.

Plus I had “build an IPC app” gathering dust on my backlog for years. I never wanted to spend the moolah to pay somebody to build it.

Vibe coding supposedly changed the economics.

The result

One of my other common sayings (apart from #Whoot!) is

I published the resulting Information Product Canvas app as open source.

You can read about what it does and how to get it running over on the Information Product Canvas companion site:

Information Product Canvas

The Standalone Information Product Canvas App

Do you prefer the feel and joy of reading a physical book…

Read more

a month ago · Shagility

And grab the code from GitHub:

But of course I didn’t stop at one app

For those that know me know when I find something interesting, something that looks like a useful pattern, but I cant describe that pattern with clarity results in something I can’t leave well enough alone.

The Information Product Canvas app scratched one itch.

But it also made me realise how quickly I could experiment with ideas and templates that had been stuck on the backlog for years. Pattern Templates like the:

• Business Event Matrix;

• Concept Models;

• Business Glossary;

• Layered Data Architecture Checklist;

• Data Dictionary;

• Data Contracts;

• Data Asset Catalog;

Ideas I couldn’t justify spending the time or money on before, but could now explore by whispering sweet nothings to Claude Code in the background for a few hours, while still doing other more important work.

So I have kept building. More standalone apps, all connected by a shared backend.

Some of them are getting close to useful. Some of them I still hate.

I will publish each of them as open source standalone apps as they get to a stage where I don’t hate them.

If you have been following my Context Plane experiments you will start to see how these pieces fit together.

https://agiledata.info/t/context-plane

If you want to help build them with me, just sing out and let me know, the more the merrier.

The bit I haven’t been sharing

As part of my Sharing is Caring mantra I realised I haven’t been sharing my journey as I learn the process and pros and cons of vibe coding something that needs to be actually used, as a person in the data domain who can’t code.

So as I keep working, off and on, on my vibe coding process and building out apps, I will post articles on what I have experienced and my thoughts around it.

What worked. What went sideways. What surprised me. What I learned about building software through conversation rather than writing code.

If you are a data person curious about vibe coding, or a builder wondering what it is like when someone who “can’t code, won’t code, don’t code” picks up Claude Code and starts whispering sweet nothings to it, hopefully the series will be useful.

If not, maybe turn off substack notifications for this site for a little while ;-)

The thing I love about experimentation is it helps me coalesce some divergent ideas that have been floating in my head for a while, into some semblance of order.

A bit like writing does.

I wanted to get a handle on the latest state of “vibe coding” so decided to experiment with building an app using Claude Code and Opus 4.6.

I picked vibe coding an app for the Information Product Canvas, its an app I have wanted built for a while, but the cost to build it the old way never matched the value I found people were willing to pay for it.

Vibe coding in theory reduced the cost, the experiment was would it?

You can see the results of a few iterations of the canvas:

Plus a side experiment into the world of the Context Plane (couldn’t help myself).

But that not the point of this post.

Nick Pinfold is experimenting with his teams Agile Data Ways of Working and freely sharing his journey via LinkedIn comments.

In this comment

https://www.linkedin.com/feed/update/urn:li:ugcPost:7432018978505617408?commentUrn=urn%3Ali%3Acomment%3A%28ugcPost%3A7432018978505617408%2C7432267116197765120%29&dashCommentUrn=urn%3Ali%3Afsd_comment%3A%287432267116197765120%2Curn%3Ali%3AugcPost%3A7432018978505617408%29

He talks about how he is creating a Streamlit app that allows him to capture the IPC content as Context and use it to assist with the next steps in their Information Factory.

posted a comment on this Substack post:

On how she is seeing “datapacks” that provide the content of a book in a way it can be easily used in an LLM as having some value.

Both of these show that there is value in writing books and creating apps, but that the typical moat of both of those things has changed.

Nick can vibe code a IPC app as fast as I can, if not faster

Anna can take the ePUB version of my book and use it in a LLM as fast as I can. If she pays more for tokens and the latest models than I do, she can do it faster and better than I can.

So effort and expertise are no longer the moat.

Given I have always said I “Can’t Code, Don’t Code, Won’t Code” but now I can now create an App but just asking questions, Im pretty sure Effort and Expertise is not the moat it was anymore either.

But maybe experience is.

To create my app in a way that meant it was actually useful, I had to have experience:

• experience using the canvas

• experience working with multiple data teams on the problem the IPC solves

• experience using apps to know what UX features were needed

• experience to know that I needed to add Google Auth to provide an easy login

• experience to know not to store any API keys in a way that were public

• experience to know that I wanted to use Google Spanner as the backend data repository and Svelte as the front end language

• experience to know I wanted a API layer between the backend data repository and the front end app

• experience to know …..

I don’t have an answer to what the new moat is, but I have more clarity on what it isn’t after these experiments.

As Nigel and I keep testing out different use cases for where our Ask ADI capability can help data professionals reduce the cognition and effort required to do complex data work we stumbled on an interesting pattern we both use.

Building the House while living in it

As we are using ADI in anger to do the data work for our AgileData.team Fractional Data Service customers and we are also extending out the capabilities in our AgileData.cloud data platform at the same time, we found we would often use ADI to help us with a task and then ask her “How did you get that answer”.

This would often be triggered with ADI giving us one of three responses:

1. Something that was right

2. Something that was wrong

3. Something that was unexpected

Something that was right

For these ones we used the answer and moved on to the next step in the data work, sing with joy that she is doing what we designed her to.

Something that was wrong

She would give as an answer that could politely be called hallucinating, in the data world I typically think of it as just being wrong.

In this scenario we always want to understand the path she took to return the wrong results. So we can figure out what we would change to make her more accurate in the future.

So we would ask her “How did you get that answer”.

Something that was unexpected

She would reply with something that made us go, wow that was unexpected, but actually bloody good, how the hell did she get that response?

Which meant we would next want to understand the logic path that she took to get that response.

Hence the follow up “how did you get that answer?” question.

In hindsight the pattern were doing was a form of eval, which we were doing repeatably but manually.

And then we started scaling it

As we tweaked the Google Gemini models we used, we tweaked what and how we stored Context in the Context Plane, we tweaked the prompts and reinforcement objects we stored in the Context Plane and made accessible to ADI etc, we decided she was good enough to be out in the hands of our AgileData.network partners.

To be clear they were well aware that she was still in an “discovery” mode, and not to be let lose directly on their customers without the partner in the loop, and was not ready to be given directly to the customers themselves.

But as one of our core AgileData principles is co-design, its how we can scale what we do so fast with just two co-founders. We know that putting patterns and features into the hands of our talented partners is a much faster way to iterate and scale.

WTF did they just try to do?

And of course you can imagine what happens when you put a fairly permissive AskAI capability into a data platform that covers the complete gambit from data collection to data consumption, and enables every data task in-between, into the hands of data practitioners who are by default very early adopters, who are working across multiple customers, in multiple industries and are trying to be at the edge if not bleeding on that edge ….

They started doing things that made us say, why the fook did they try and do that, what were they trying to achieve?

And more importantly, how the hell did ADI come up with that answer, and was it right or wrong, or excitingly unexpected?

Log everything

One of the patterns we apply to the AgileData.cloud is we log everything.

So of course we were already logging the question they asked ADI and the response they got.

But this didn’t help us work out the rest.

Manual processes don’t scale

So of course the first thing we tried, was taking the question they asked and typing it into our own AgileData Tenancies.

You can imagine what happened.

• The data in our tenancies is different to theirs.

• You typically wont get exactly the same answer to the same question with a LLM based model.

Epic fail and not a lot of use to help iterate ADI’s behaviour.

Not just “how many sales were there”

The natural language and non-deterministic behaviour of the Google Gemini models we use under the covers for ADI is where a lot of the value resides.

And this means she can help with a lot more data tasks than the typical “Text to SQL” use case every data vendors is chasing as table stakes these days.

And so the questions being asked and data work being done by our talented partners was a lot more than the simple “how many sales were there”.

Which meant we couldn’t just implement something like a judge pattern to make sure ADI was returning the correct number to each question.

Iterate with simplicity

As part of our Way of Working we alway try and decompose the work to be done into the smallest chunk possible and to start off with simplicity and add complexity later.

Automatically log what path was used

First thing we did was to extend the logging to include the path ADI took to get the answer, so we could review it after the fact.

Originally this was just logging in the background, but we found that this logic was actually useful to the Data Practitioner, so we surfaced it as part of the ADI response.

You can see the path being returned here in the language of as set of assumptions.

Ask for feedback

We still don’t know if the path used was actually the best path and if the response helped the partner do the data work quicker and easier, so we reuse the feedback pattern from social media products so they can quickly give us an up or down thumb for each response.

As our partners know they are co-designing with us, this quick and easy feedback loop provides value, without slowing them down from doing the data work and delivering value to the customer.

Should I trust the response

We also decided to experiment with providing a confidence score on each ADI response, we find this useful when evaluating the responses after the fact, it will be interesting to see this helps our partners or not.

Another Use Case

Here is another use case where we Ask ADI to help us model the data from Google Analytics.

Again we are logging the path ADI took to respond, but with slightly different language to make it fit the Context of the question more.

And ADI is providing a suggest next step and asking if she can help.

And as mentioned before the user will always go somewhere we don’t expect.

The point of these is not that the Gemini LLM provided definitions for commonly used Google Analytics concepts, any LLM will do that, you can just use ChatGPT on its own etc.

But the fact is we end up with full visibility of what was asked, what the answer was and who that answered was derived.

And with that visibility we can decide the most valuable use cases to iterate Ask ADI for, based on the things our AgileData.Network partners actually need them to help them with.

And one more thing

And you can see where we will take this.

Given ADI is embedded in the data platform that the data practitioner is doing the actual data work in, we can increase the level of assistance over time.

ADI should probably respond to the Google Analytics data questions, with a response that is tailored on how you do this work in the AgileData.cloud and use the patterns and language we use in that platform (Information Product Canvas, Concepts instead of Entities etc)

ADI should probably create the Concept Model for those Concepts.

ADI should probably populate the Business Glossary with the default definitions for those Concepts.

ADI should probably use the Context we have defined before for Google Analytics to differentiate between a Pseudo User and a User.

ADI should probably rehydrate the Change Rules (data transformations) needed to populate those Pseudo Users from thw GA4 event data.

ADI should probably …. [insert use case we see our partners do, that we have never thought about here]

Patterns you can adopt.

Here are some simple patterns you can adopt as you continue your “AI” journey in your organisation:

• Log all questions asked and all responses given by your AskAI feature, somewhere you can see and query

• Log the path your LLM took to provide the response

• Find a way to gather feedback as you scale

• Put it in the hands of your early adopters as soon as possible and let them help you co-design the most valuable areas to iterate with next.

As we worked with an Agile Data Network partner this week to help them onboard a new Customer, we decided to use an ADI first approach for all the data work in the AgileData App and Platform and see what would happen.

And of course part of all data work is data modeling.

So we got ADI to look at the source system data that had been collected into History, it was data we had never modeled before, and got her to take us through the data modeling process.

Our Agile Data Network partner had already worked with the customer to understand their required Information Products and so had a good understanding of the Core Business Concepts, Core Business Processes, Facts, Measures and Metrics that would potentially meet their organisational needs.

This also stopped us letting ADI define a source system specific data model that would break on the first engagement with a Stakeholders chnaging requirements.

ADI did ok, but we always know we can do better, so time to McSpikey.

Agile Data Disco

Last year we did a raft of work for a Customer, where we reverse engineered 100’s of their Cognos Report definitions to help document their legacy data platform before they moved to a greenfield’s Modern Data Stack. This work removed the need for a team of Business / Data Analysts to spend months documenting the legacy system.

We ended up building this in public and semi-productising it, calling it Agile Data Disco.

You can read about that journey here:

We saw a potential market solving that data problem of understanding what a legacy data platform actually contains.

https://agiledata.team/data-problems/use-case/legacy-data-platform-discovery/

You can see an interactive demo of the final Agile Data Disco product we built to help us do that Fractional Data Work here:

https://agiledata.cloud/disco/#demo

One of the things we did as part of Disco was some interesting prompt engineering to take a single input (say a blob of SQL, a log from SQL execution or the defintion of a report) and from that create a series of useful populated Pattern Templates as the output.

The outputs we generate are:

• Information Product Canvas

• Event Model

• Conceptual Model

• Physical Model

• Reporting Model

• Business Glossary

• Data Dictionary

• Metric Definitions

• Bus Matrix

• Source Mapping

And we also did another McSpikey with Disco where I uploaded an image of a completed Information Product Canvas and had Disco generate all those object, from that one image.

You can read about that one or see the Interactive Demo for it here:

Can you guess where we went with this

Yup we took the core parts of Disco and did a McSpikey to see how much of the Disco prompt and reinforcement logic we could reuse to improve ADI.

Results of the McSpikey still to come

I still need to take the time to write up the results of the McSpikey properly, but the TL:DR is:

• Reusing the Disco prompt and reinforcement logic had massive value “time to value” wise;

• We ended up experimenting with breaking out a ADI sub agent - ADI the Agile Data Modeller, which had some real benefits in improvement in the ADI responses.

A word from our sponsor

One of the things we got asked in the early days of Disco was rather than just document what was in the legacy Data Platform, could we automagically migrate it to a new data platform.

When we thought about it that looked like a very complex problem, that would take a few years of development to make feasible and we would end up with a many to many problem before it was viable, we would need to read from many tools and technologies and also need to write to many tools and technologies. Until you had critical mass at both the read and writes ends the product wouldn’t be viable.

And we also saw data platform modernisation as a great catalyst for organisations to rethink the way their data teams worked, and to rearchitect a lot more than just their database, ETL tool and BI tool. So we weren’t fans of supporting a better “like for like” modernisation pattern.

But we also had in the back of our mind the idea that if we could take a Customers legacy data platform and automagically migrate it to the AgileData Platform with minimal human effort, that would be very valuable in removing one of the key points of friction for working with potential Fractional Data Service customers who had already had invested a shit ton of money in a data platform (legacy or modern).

Because our AgileData Platform is based on our very opinionated Ways of Working, Data Engineering and DataOps patterns, this wouldn’t be a “like for like” but an automated rebuild from new.

As we experiment with ADI the Agile Data Modeler we are wondering if that Agent on its own has some value. Upload your current data model, and get back a bunch of candidate data models to review.

This wouldn’t be just a LLM going text to text, we would extend the reinforcement model we used for Disco, and bring in our opinionated Business, Concept and Physical data modeling patterns to it as well.

If I have to place a bet (and I do) I am going to carry on with the Context Plane bet.

All the experimentation and development we do in that space is also immediately available in the AgileData App and Platform, so another form of DORO.

But if you think there is value in an ADI the Agile Data Modeler on her own, reach out and lets have a chat. I would be keen to understand the use case you have in mind.

An incoherent stream of Context

You can find all the previous articles with my train of thought listed in this thread:

https://agiledata.substack.com/t/context-plane

We are building the Context Plane while flying it, so always looking for early adopters to help us decide the final destination.

If you want a virtual chat grab a slot here:

https://contextplane.ai/contact-us/#bookemdanno

Different personas and use cases demand different ways of interacting with Context.

“Context” of this post

I often find writing helps me coalesce and refine my thoughts when new patterns start to emerge, but aren’t very clear yet.

So this article is a brain dump / train of thought continuation of the architecture needed to have one Context Plane to rule them all, as part of a proposed “AI Data Stack”.

This article provides an overview of the specific persona types / use cases I have identified so far that need to access the Context Plane and the typical UX patterns for some of those.

Updated Context Plane Architecture

This iteration in my thinking has a resulted in an update to the architecture diagram for the Context Plane:

The four personas / use cases

Here are the 4 persona types / use cases I have discovered so far:

• Human, GUI centric
  A person who wants to use a web based App, chat or Graphical User Interface to access/discover/explore/update the Context.

• Human, Code Centric
  Wants to use a Command Line Interface (CLI) or Code based App to access/discover/explore/update the Context.

• System
  Systems can access the Context directly, either querying the Context or creating to it programatically.

• Agent to Agent
  AI Agents can access/discover/explore/update Context autonomously, collaborating with other agents without a human in the loop.

The UX patterns

Here are the UX patterns I have experimented with that seem to make sense:

• Human, GUI centric

  • GUI centric Data Catalog

  • GUI centric Chatbot

• Human, Code Centric

  • GenAI App

  • CLI tool

• System

  • API’s

GUI centric Data Catalog

Your typical browser based Data Catalog interface.

Prefer a manual click through version?

https://guides.agiledata.io/demo/cmflomypg048l170irpj5zf6h

Click through demo

GUI centric Chatbot

Your typical browser Chatbot and Text to SQL interface.

GenAI App

Typical Claude “AI Agent” app tool interface.

CLI Tool

Typical Gemini CLI command line interface.

[Screenshot TBA]

API’s

Typical API endpoints.

Missing UX patterns

The key UX pattern I have yet to discover is how the Agent to Agent UX works.

I think we will need to do a McSpikey with the Google A2A to understand the options in that space a little more.

The Technology patterns

We already had a lot of the technology patterns in place before we started experimenting with the Context Plane.

Things like the browser based Data Catalog capability.

We also had a lot of ADI already built based on “AI Assisted” features we have experimented with over the last 6 odd years.

Our App and Platform architecture has always been based on API’s in the middle:

App > API > Context > Code / Data

The main iteration technology wise has been the addition of a MCP server. This has allowed the use of tools like Claude and Gemini CLI.

We have iterated ADI to use the MCP server to access the Context (well we actually use a hybrid access model but ill leave the diagram as simple as this for now.)

So many new questions

User?

UX stands for User Experience, but some of these persona types and use cases are machines not humans, should they still be referred to as Users?

BI Semantic Layer?

Where does the typical BI Tools and the “BI Semantic Layer” pattern fit into this?

For Context Plane we are only holding an Organisations Context not their Data, so we can’t execute any queries, like we can in the AgileData App and Platform.

Or do we want to look at generating the query the human can cut and paste into the data platform.

Or do we want a Context Agent to push the Query to a BI agent inside the Organisations agent ecosystem?

We don’t provide a caching layer or query rewrite patterns which is what the BI Semantic Layers / Metric Layers are doing these days. Im pretty sure we don’t want to go there.

When will BI Tools move to using MCP servers as a way of querying the data?

When will they all put Agents in from of their BI Semantic Layers"?

One step forward, but a raft of new uncertainties

So many new questions, so few answers.

Looks like there even more McSpikeys to add to the list!

Wood from the Trees

Still a way to go before I have a coherent set of Patterns that I can Coach / Mentor / Teach somebody else for the “Context Plane”, and the “AI Data Stack” or present as a robust Architecture map.

But as I have already said, writing my half formed ideas helps me think.

An incoherent stream of Context

You can find all the previous articles with my train of thought listed in this thread:

https://agiledata.substack.com/t/context-plane

We are building the Context Plane while flying it, so always looking for early adopters to help us decide the final destination

If you want a virtual chat grab a slot here:

https://contextplane.ai/contact-us/#bookemdanno

“Context” of this post

I often find writing helps me coalesce and refine my thoughts when new patterns start to emerge, but aren’t very clear yet.

So this article is a brain dump / train of thought continuation of the architecture needed to have one Context Plane to rule them all, as part of a proposed “AI Data Stack”.

This article provides an overview of a way to describe both the Object Types and the Object Categories that will hold the metadata I think should be included in the Context Plane as part of a new “AI Data Stack”.

The Context Plane Architecture

I have written some initial thoughts about the architecture of the Context Plane and the things that shoud be stored or federated in it before.

As a quick reminder my current thinking is the architecture diagram for the Context Plane should look something like this:

Object Types stored in the Context Plane

In that previous article I did a bit of a stocktake of all the Patterns and Pattern Templates I see in the Data Domain that hold what I would term as Context.

I have been mulling those over more as we do each McSpikey (experiment) to learn more by doing and to reduce some of the many uncertainties we have identified.

I am currently sitting with this list of Object Types that seem to have value being stored or federated in the Context Plane:

• Outcomes

• Actions

• Business Questions

• Business Glossary of terms, aliases and descriptions (use tags for Aliases)

• Conceptual Data Model

• Logical Data Model

• Physical Data Model

• Data Dictionary (with schema, fields types, field descriptions etc, also flags)

• Facts

• Transformation code

• Data Quality Rules

• Data Contract (Boundary of other Objects))

• Measures, and their formulas

• Metrics and their formulas

• Information Applications (Reports, dashboards, AI Agents etc)

• Information Products (Boundary of other Objects)

• Data Quality Scores

• Notifications

• Usage Statistic

• Data Sync Statistics

• Number of rows in tiles

• Principles

• Policies

• Patterns

• Personas

• Previous effort of change

Semantic Language is important

I often rail against the lack of clear Semantic definition in the Data Domain, but I am often as loose with my definitions as anybody else.

I had original used the following terms:

• Context Object

• Context Object Types

But as is the LinkedIn way Gabriel Tanase kindly reviewed my crap semantic definitions and provided a much better version.

https://www.linkedin.com/feed/update/urn:li:activity:7366194247613501441?commentUrn=urn%3Ali%3Acomment%3A%28activity%3A7366194247613501441%2C7366400089965129728%29&dashCommentUrn=urn%3Ali%3Afsd_comment%3A%287366400089965129728%2Curn%3Ali%3Aactivity%3A7366194247613501441%29

So based on that I have ended up with:

• Context Object
  A single instance of something in the Context Plane (e.g. the action “Approve Loan”).

• Context Object Type
  The class or category that defines what kind of object it is (e.g. Actions).

• Context Object Type Category
  A higher-level grouping of related object types that share a domain (e.g. Business Context).

Not reinventing the Wheel

Gabriel also pointed me at some excellent documentation from Collibra in that thread, for example:

https://productresources.collibra.com/docs/collibra/latest/Content/Settings/OperatingModel/to_operating-model-settings.htm

Im always keen to reuse Patterns from others, rather than reinventing the wheel so to speak, and so I went down a rabbit hole of the excellent Collibra documentation.

I came away with a feeling of an architecture that is founded on the Pattern of “thing is a thing of a thing”.

Which of course is an infinitely flexible Pattern, but to me has always been an Anti-Pattern and so not a Pattern I want to adopt.

I will need to spend more time looking into bth the Data Catalog and Information Science domains as I am pretty confident that the Patterns I am looking for already exist.

But there is also the value of trying to defining these patterns myself, as I learn by doing (and struggling). Plus when I finally think i have nutted something else and I spot that same pattern articulated in lots of other places then I knwo I am on to something repeatable and valuable.

Context Object Type Categories

Back on task, these are the Context Object Type Categories I have ended up with.

• Business Context
  Captures the intent, language, and needs that connect data work to business language.

• Structural Context
  Describes the technical metadata that defines how data is stored, shaped, and connected.

• Operational Context
  Provides the live signals and guardrails (trust scores, usage stats, policies, and access rules etc) that keep data reliable and governed in practice.

• Agent Context
  Provides the prompts and guardrails that guide AI agents in applying context.
  

Yes Structural Context is the same as what is commonly called “Technical Metadata”.

My definition for Operational Context has got examples, which means I do not have a clear semantic defintion for it yet, so I need to keep iterating that one.

Mapping Context Object Types to Categories

I stress tested my categorisation by trying to assign the Context Object Types to one and only one category.

Moving to the AssistedAI pattern and letting my ChatGPT friend expand my bullet point list to become richer text and write a more detailed story, ever so slightly edited by me ….

When people talk about data, they often focus on the raw tables, pipelines, and dashboards. But the real power comes from the Context that surrounds them, the language, the intent, the rules, and the patterns that let both Humans and AI agents understand what data means and how it should be used.

In the Context Plane, we capture that knowledge as a set of Object Types. These are building blocks that represent everything from business questions to transformation code, from policies to personas. Together, they create a shared layer of context that connects business, data, operational and agentic worlds.

To make sense of them, we group these Objects Types into four categories: Business Context, Structural Context, Operational Context, and Agent Context.

Business Context

Object Types in the Business Context category capture the why behind the data: the intent, the questions, and the people who care.

• Business Questions – The driving questions that stakeholders ask and want answered.

• Actions – What people (or processes) actually do with data once they have it.

• Outcomes – The results or changes in the business that happen because of those actions.

• Business Glossary – Shared terms, their aliases, and agreed definitions, so everyone speaks the same language.

• Conceptual Data Model – The high-level map of business concepts (customers, products, orders) and how they relate.

• Logical Data Model – A more detailed structure that shows how those concepts can be represented in data.

• Personas – Representations of the different types of people who use or consume data, each with their own needs.

• Information Products – The boundary objects that package up context into a consumable unit, such as a unified customer view, a churn model, or a financial performance pack.

Structural Context

Object Types in the Structural Context category describe the what of data: the way it’s stored, shaped, and transformed to become useful.

• Physical Data Model – The actual schema and structures in the database.

• Data Dictionary – A catalog of fields, data types, and descriptions, along with useful flags.

• Facts – Core, measurable events or counts in the data (e.g. sales transactions, logins).

• Transformation Code – The SQL, scripts, or pipelines that turn raw data into shaped, ready-to-use structures.

• Data Quality Rules – Checks and guardrails that ensure the data stays trustworthy.

• Measures – Defined calculations, such as revenue or average handling time.

• Metrics – Business-relevant measures with formulas and thresholds (e.g. Net Promoter Score, Churn Rate).

• Information Applications – The outputs where information is applied and consumed: reports, dashboards, AI agents, apps.

Operational Context

Object Types in the Operational Context category describe the how well and at what cost dimensions of data. These objects let us monitor and manage the reliability and change effort of the system.

• Usage Statistics – Insights into who is using which data, when, and how often.

• Data Quality Scores – Aggregated indicators that show how healthy the data is across rules and checks.

• Data Sync Statistics – Performance and freshness indicators of data movement.

• Number of Rows in Tiles – A quick proxy for data size and growth in a given slice of the platform.

• Notifications – Alerts that something requires attention, whether it’s a failed job or a threshold breach.

• Principles, Policies, Patterns – The guiding rules and repeatable approaches that shape how data is managed.

• Previous Effort of Change – A signal of how much time, cost, and disruption a past change required, helping estimate future impact.

Agent Context

Finally, Object Types in the Agent Context category deals with the prompts and instructions that guide AI agents in using all this context. These Objects define the boundaries and cues for generative systems to behave predictably and helpfully.

• Prompts – The structured inputs, templates, and guardrails that tell an AI agent how to use context objects to answer questions, generate insights, or perform actions.

Why it Matters

Each Object Type might seem less than valuable in isolation, but together they form a unified layer of meaning, that can be used by Humans and AI Agents alike to gain understanding.

By capturing both the why (business intent) and the what (structural detail), as well as the how well (operational health) and the how to guide agents (AI context), the Context Plane ensures that every stakeholder—whether a business leader, a data engineer, or an AI agent—works from the same playbook.

That’s how you move from disconnected metadata and data assets to a coherent ecosystem where context is always present, always available, and always shared.

And back to all me again ….

And once again with a map

Visual maps often give a better quick view than plain text, so here you go:

Far from Done Done

Based in my experience to date I will be iterating the Object Types and their categorisation for a while yet, but I think I have the the four Context Object Type Categories pretty stable.

Famous last words?

Wood from the Trees

Still a way to go before I have a coherent set of Patterns that I can Coach / Mentor / Teach somebody else for the “Context Plane”, and the “AI Data Stack” or present as a robust Architecture map.

But as I have already said, writing my half formed ideas helps me think.

An incoherent stream of Context

You can find all the previous articles with my train of thought listed in this thread:

https://agiledata.substack.com/t/context-plane

We are building the Context Plane while flying it, so always looking for early adopters to help us decide the final destination.

If you want a virtual chat grab a slot here:

https://contextplane.ai/contact-us/#bookemdanno

“Context” of this post

I often find writing helps me coalesce and refine my thoughts when new patterns start to emerge, but aren’t very clear yet.

So this article is a brain dump / train of thought continuation of the product and architecture needed to have one Context Plane to rule them all, as part of a proposed “AI Data Stack”.

This article provides an overview of adding the AI Agent capability to the AgileData Web App.

Not starting from scratch

We have been experimenting with an in App “Chatbot” helper in the the AgileData App for multiple years so I was lucky we already had the core UX needed for us to experiment with this feature.

And we had already integrated Gemini with the AgileData App a while ago. We already used it to assist or automate a few of the more complex bits of data work.

So it was ”just” a case of changing the Gemini Model we were using to switch over to use Gemini 2.5 Pro to get the reasoning capability and also adding the connector to expose it to our MCP Server.

So from a AgileData App UX point of view nothing changed.

Same Prompt

We have a simple set of prompts already defined in the AgileData Platform for ADI.

You are ADI, the AI assistant inside the AgileData.io product. Your purpose is to guide a data team of one—a data analyst—in configuring a data warehouse using the AgileData App and Platform.

Context & Capabilities:

You have access to:
	•	The AgileData user guides
	•	The screens and workflows within the AgileData app
	•	The APIs that power the AgileData app and platform
	•	The data stored in the AgileData platform

The Challenge:
	•	Every set of data has unique challenges.
	•	There is no single, linear process for working with data—users must “pick a path” or “choose their own adventure.”
	•	Each user will dynamically use the AgileData app in different ways based on their specific needs.

Who You Are Helping:
	•	The user is data-savvy but not a data professional, not a highly technical user, not a data expert.
	•	They may not always know exactly what data task they need to do—but they know the outcome they are aiming for.

How You Should Respond:
	•	Be clear and practical: Guide the user toward the best approach to complete their data task efficiently.
	•	Adapt to their needs: If they are unclear on what to do, ask clarifying questions and suggest possible paths forward.
	•	Use all available information: Leverage guides, app screens, APIs, and platform data to provide the most relevant answers.
	•	Encourage exploration: Offer multiple options for how they could proceed, allowing them to discover the best way forward.
	•	Think like a mentor, not a manual: Rather than just explaining how the platform works, help the user make decisions that will get them to their goal.
	•	Present them with the relevant url to the relevant screen if that will help them complete the task

When you have multiple choices:
	•	Pick the one you think is the most suitable and do that.
	•	Append the other options you had to your response and tell the Data Analyst which choice you selected, let them know they can tell you to pick another option.


Your goal is not just to answer questions but to help the user successfully build their data warehouse in the most effective way possible—no matter what challenges they face.

As you can see they are based on previous experiments where we were focussed on helping a Data Analyst build a Data Warehouse as a “team of one”.

I left this prompt in place for this experiment, but I should do another McSpikey where I tailor them for the Context Plane use cases.

In previous experiments where I used Gemini CLI and Claude to access the Context in the Context Plane, there was no “system” prompts in place at all so this should mean I get slightly different responses from Gemini 2.5 Pro in the AgileData App vs Gemini 2.5 Pro in the Gemini CLI.

ADI also has access to our Doco and also examples screens from the App, so again it has more Agent Context than the previous experiments with Gemini CLI had, as that only had access to the API’s via the MCP Server.

MCP Services

We limited the services available in the AgileData App to a subset of the available APIs.

When I use my typical question to get a list of Tiles and a list of Fields in a specific Tile then it pretty get the response I am expecting, no surprises there.

Except ……

When I use Claude to get the Blast Radius for a field change it gives me very rich Context back.

ADI in the AgileData App however…

She ain’t being that helpful!

Over to Gemini CLI to see if its her or Gemini.

Ah nope

We made a fundamental change to the MCP Server we are using since the last McSpikey. The previous version only had access to a subset of our API’s as tools. We changed it to have access to all API’s.

For Claude this didn’t really make much of a difference, but for Gemini CLI it seemed to have a major impact and not in a good way.

Listing the tiles always seems to work ok, simple task.

Listing the fields in a tile has worked well up until we made the MCP Server Tool change.

That is the LLM version of go F$#@ yourself lol.

I had a quick go to see if I could limit the Tools that Gemini CLI could access on the client side, as we limited them in both the AgileData App for ADI and I limited them in the Claude client.

So new entries in the Gemini CLI settings.json it is:

  "securityPolicy": {
    "mode": "configured",
    "allowedTools": [
      "get_business_glossary",
      "get_catalog_tiles",
      "get_ensemble_config",
      "get_change_rules"
    ]
  }

But that did nada, when I listed the tools in Gemini CLI using /mcp it could still see them all and there was no behaviour change.

If I was very specific with my language it would sometimes work:

But then it would go back to be a real dumb arse again.

So back to very specific language:

So theories from that rabbit hole:

• To many MCP Server Tools makes the AI Agent Tools job harder

• The previous Tools we exposed had some additional hints in the naming of the Tool that probably helped the AI Agent Tools

• Gemini 2.5 Pro is not as good as Claude Sonnet 4 for this use case

One more thing

I ran Gemini CLI against a different AgileData Tenancy, Newcastle, compared to the ADI version which was running in the Kapiti Tenancy (hence the change in Tile name I was asking questions about).

Lets go try ADI in the Newcastle Tenancy and see what happens.

Nope same problem.

So Im going to guess the problem could be:

• the Gemini 2.5 Pro model

• and / or the extra MCP Server Tools

• and / or the Prompts in the AgileData Platform

• and / or all of the above

that is causing the problems.

Out out time for this McSpikey so thats is an experiment for another day.

So in Summary

We can provide a web based interface in the AgileData App easily enough by reusing the current ADI capability.

But need to work out if it actually has any value.

And also validate the assumption that it would be a different personas using it compared to a AI Agent Tool and should that impact the type of response they receive or not. Gut feel is no, it should be “any Human, any AI tool, same response”

And I need to experiment with the combination of LLM models, MCP Server Tools and Prompts a lot more.

Claude Sonnet 4 seems more compotent than Gemini 2.5 Pro

Well for this use case anyway. I just seemed to get back better “Context” from my Context when I use Claude over Gemini.

As slow as a wet pig

One of the surprises was ADI was sooooo slow in the Kapiti Tenancy vs the Newcastle Tenancy.

Check it out.

Newcastle

Kapiti

Almost twice as long in Kapiti vs Newcastle.

One of these things is not like the other

After chatting with Nigel it seems were running Gemini 2.5 Flash in Newcastle and Gemini 2.5 Pro in Kapiti, hence the difference in response times.

So another McSpikey needed to see how different the responses are between the two models for our key use cases.

Its all about the reducing uncertainity

The key to a McSpikey is to reduce uncertainty and for this McSpikey I think we have increased the uncertainty.

But thats ok, better to know that now than later when we are much further down the Context Plane path.

Wood from the Trees

Still a way to go before I have a coherent set of Patterns that I can Coach / Mentor / Teach somebody else for the “Context Plane”, and the “AI Data Stack” or present as a robust Product Overview and / or Architecture map.

But as I have already said, writing my half formed ideas helps me think.

Context Plane Use Cases

You can find all the use cases where I think the Context Plane may have sone value over at:

ContextPlane.ai

An incoherent stream of Context

You can find all the previous articles with my train of thought listed in this thread:

https://agiledata.substack.com/t/context-plane

We are building the Context Plane while flying it, so always looking for early adopters to help us decide the final destination.

If you want a virtual chat grab a slot here:

https://contextplane.ai/contact-us/#bookemdanno

“Context” of this post

I often find writing helps me coalesce and refine my thoughts when new patterns start to emerge, but aren’t very clear yet.

So this article is a brain dump / train of thought continuation of the architecture needed to have one Context Plane to rule them all, as part of a proposed “AI Data Stack”.

This article provides an overview of creating and updating Context via a AI Agent Tools such as Claude and Gemini CLI within the “Context Plane” and via the use of a MCP Server.

Are they really AI Agents?

First thing first are Claude and Gemini CLI’s really “AI Agents” or are they just “Tools” that you can use to build agents?

I think of them as tools, other people I have asked think of them as Agents, so for now lets call them “AI Agent Tools”, to cover both Semantic versions.

Architecture Reminder

As a reminder of what we are currently working with we ended up with an architecture that looks like:

• Claude, Gemini CLI etc talks to MCP Service

• MCP Service calls applicable API (exposed as MCP tools)

• API talks to “Context Plane”
  

Connecting Claude to MCP server

Nigel uses Claude for his McSpikeys on the “Context Plane” and so I would join the fun.

First thing was download the Claude App for my Mac, and then find the option to set it to dark mode so I can pretend to look like a proper engineer.

Second thing is you need to pay for Claude if you want to use it with MCP Servers, so NZD $40 a month added to the ever growing AgileData McSpikey toolkit cost bucket.

Next thing hook it up to our “Context Plane” MCP Server via the custom connector:

A little bit of cheating on the auth side for this McSpikey but Claude makes it all pretty easy.

Update, Update baby

(Sung to the tune of Ice, Ice baby)

Nigel has done his plumbing magic and exposed the API that allows the creation or update of a Catalog Tile (think of Tiles as Conceptual or Physical tables).

New tool is enabled for Claude.

And its available for Gemini CLI.

Any AI Agent Tool

I can see a lot of value in the concept of a MCP Service and tools being exposed via that service.

People can bring what ever AI Agent Tool they prefer to do the work, and we can expose new tools at will and they all just talk nicely to each other.

Imagine if all our other data, analytics and BI tools had that level of interoperability!

Lets update a Tile

List Tiles

First lets list the Tiles that are available in the Context Plane.

Gemini CLI was way slower at processing this task than Claude!

Lets try an update

Its still easier for me to quickly check the details for a Tile in the AgileData App:

Claude first.

Then Gemini.

Ok simple example but proves it all works.

Lets create a new Tile

Claude first.

Hell no I ain’t spending time giving you all that “Context” (well not for this McSpikey anyway).

So minimal details you shall have.

Gemini’s turn.

Mmmmm well I know what I need to provide as the minimal amount of detail to create a Draft Tile, but most won’t, so Claude deffo wins this one so far.

But Gemini did do the job.

Update Tags

In the AgileData Platform we allow the same Tile Alias to be used (of course each Tile gets a unique GUID), so lets see if we can differentiate them using Tags.

Claude first

We hold Topics as an array for a Tile. Claude worked out that Tag was the same as Topic, but failed at adding one to the Tile. It did try a few different way to try and achieve its task.

Over to Gemini to shine (or not as the case maybe)

Mmmm but have you really Gemini?

Nope in fact you have not, you may call it Hallucination I call it damn lying!

So in Summary

I am not sure this is the fastest way for me to create and edit Tiles in the “Context Plane”. But that is because I know how to quickly do it via the AgileData App.

I can see for people that prefer code, this would be much faster for them.

Claude is deffo more helpful with its feedback loops as it does the work, and is also much faster in doing those tasks.

But I think the “use any tool” pattern has massive value, what I am thinking of as:

As does my other favourite saying:

The fact we can expose our API’s as Tools to the MCP Service and any “AI Agent Tool” gets access to it straight away, is one step closer to the data technology “Interoperability” that we have been striving to achieve for over the 3 decades I have being working in the data domain.

The “Golden Path” problem

This is a major problem for us in the AgileData App.

Nigel and I know how to use the AgileData App and Platform to do data work that takes hours in minutes. But nobody else does.

As Data Professionals already know, there is no “Golden Path” to doing data work. No linear steps of this then that, which apply everytime.

Every new set of data kicks your arse the first time you touch it.

So we designed the AgileData App and Platform so that it allowed you to do the work in any order.

But that made it confusing for people who have never used it before, or done data work very often.

We tried a bunch of things over the last 6 1/2 years, user guides, video examples, click through demos, and even a ChatBot, nothing really nailed it.

I can see how “AI Agent Tools” like Claude and Gemini CLI solve this “Golden Path” problem.

But as we experiment with the “Context Plane” I think we have stumbled across another bigger complex problem, at potentially the right time, that we can solve with simplicity.

So of course I had to create a website just in case I was right.

https://ContextPlane.ai

Time to design a new T-Shirt.

Actually if your reading this Nigel, of course i’m not! (um well …..)

Wood from the Trees

Still a way to go before I have a coherent set of Patterns that I can Coach / Mentor / Teach somebody else for the “Context Plane”, and the “AI Data Stack” or present as a robust Architecture map.

But as I have already said, writing my half formed ideas helps me think.

An incoherent stream of Context

You can find all the previous articles with my train of thought listed in this thread:

https://agiledata.substack.com/t/context-plane

We are building the Context Plane while flying it, so always looking for early adopters to help us decide the final destination.

If you want a virtual chat grab a slot here:

https://contextplane.ai/contact-us/#bookemdanno

“Context” of this post

I often find writing helps me coalesce and refine my thoughts when new patterns start to emerge, but aren’t very clear yet.

So this article is a brain dump / train of thought continuation of the architecture needed to have one Context Plane to rule them all, as part of a proposed “AI Data Stack”.

This article provides an overview of integrating and using Claude and Google Gemini CLI with the “Context Plane” via a MCP service.

MCP (Model Context Protocol)

The next thing I wanted to understand was how a MCP would work with the current version of the “Context Plane” we have in the current AgileData Platform.

McSpikey

So time to do a McSpikey (experiment) with Nigel. Success for this experiment was to be able to query the AgileData “Context Plane” with tools such as Claude and Gemini CLI.

MCP Server

We are a Google Cloud only platform so we thought that using this MCP server would make sense

https://github.com/googleapis/genai-toolbox

But after some investigation Nigel worked out it would mean a lot of rework, as we would have had to define all the tools and sql queries needed to query Google Spanner, where we store our Context.

Since we already have all those queries defined as part of the AgileData Platform API’s. Nigel discovered that we could just wrap our existing FastAPI endpoints with FastApiMCP and so we implemented that.

In the words of Nigel:

We always like minutes over hours at AgileData and we love the DORO principle (Define Once, Reuse Often)

There were some challenges deploying FastAPIMcCP via Google App Engine, if you want more details on that just ask and Nigel can write up a more technical post on how that went.

Architecture

So we ended up with an architecture that looks like:

• Gemini CLI talks to MCP Service

• MCP Service calls applicable API (exposed as tools)

• API talks to “Context Plane”
  

Connecting Gemini CLI to MCP server

Its always funny what takes the most time when doing a McSpikey.

When Nigel setup the MCP server he connected to it using Claude, next learning you need to have a paid version of Claude to provide MCP connectivity.

So I though I would use Google Gemini CLI for me.

Next learning, the paid version of Claude makes it very easy to define the MCP connection, you pretty much paste the url for it in.

Gemini CLI not so much, you need to frig around with JSON strings in a settings.json file.

And of course finding the exact structure for that JSON is a pain.

After a bit of trial and error got it working with this:

{
  “selectedAuthType”: “oauth-personal”,
  “theme”: “Dracula”,
“mcpServers”: {
  “MCPServer1": {
    “url”: "https://mcpserver1/mcp
  }
}
}

Lets Get Ready to Rumble

Does it connect?

So first test does it connect to our “Context Plane”?

/mcp list

Yup it can see the MCP server and it can see four API end points that we have exposed.

Does it see Context objects?

Next lets test if it can see objects stored in the “Context Plane”

list tile names

Its worked out the correct API to hit to get back a list of Tiles from the Catalog.

And its given me a list of Tile names.

Does it understand Context for those objects?

We run a layered data architecture and the Data Layer is held as Context for the Tiles, not in the tile name etc. So that why im getting multiple Tiles back with the same name.

Lets try to get some Context.

list tile names and their data layer

All good, it worked out what Data Layer the Tiles belong in.

Interestingly we never use the term “Data Layer” in the “Context Plane”, im guessing its picked up the Data Layer from the table prefix or the object type we hold against each tile.

I had assumed we would need to add extensive aliasing of terms in the “Context Plane” to make it find answers to people natural language questions, but given Gemini CLI is backed by the Gemini LLM maybe we don’t.

(This is the value of doing a McSpikey, you discover things you didn’t know, and of course end up with more questions to answer).

Real Life Use Cases

I have been asking people at the coal face for use cases they constantly hit where they think accessing the “Context Plane” may reduce the complexity of the data work they do.

Im looking for use cases that are more than the usual “text to SQL” use case everybody now has as table stakes in their product.

Get Data Catalog Tiles

We have already proven we can connect and get Catalog Tiles using Gemini CLI.

But I am a GUI by default kinda guy and so for that use case I would just use the Catalog search screen in the AgileData App.

A couple of clicks and i’m done, no need to type lots of words.

Ask a question, get an answer, Context provided for me

The challenge with the App approach is I know that I need to search for Consume Tiles if I want to use the data on a Last Mile report. A new user would not.

So lets try that scenario by asking:

I need to use Netflix data in the report I have to create for the boss

Again its using the concept of our Data Layers to understand that data in the Consume layer should be used in a report.

Given we have provided no RAG or Prompt reinforcement on our Data Layer Architecture I wonder if its because we use clear terms like “Consume” rather than “Gold” that is helping the model.

Its also come back with a follow up question, lets answer it and see what happens.

details about cast and directors

Looks like its checking the Data Dictionary for the Tiles to see what fields are available.

As an aside there is now finally a reason for us to populate a Description for each Field in the “Context Plane”.

And its come back with the Fields I would need to use for my report.

Its also asking if I want it to help me build that report.

show me the data

It recognises that it has no MCP service to access data or run a query.

One of the features we built in the AgileData App ages ago was the ability to click on a Catalog Tile and open the data directly in Looker Studio, as a very quick way to explore the data via a GUI.

Gemini CLI has understood that from the “Context Plane” and look like it found the relevant Looker Studio url to open the data I need in Looker Studio.

Lets cut and paste that url ….

Yup thats exactly what it has done that was unexpected but magical.

Wonder if I could get it to rewrite that url to only include the Fields I am after, that is a question for another day.

Show me the impact of a change

The second real life use case I got given was to see if it would help streamline the process of understanding the impact of the changing a Field somewhere in the data pipeline flow.

i need to update the field type for the director for the netflix tile what do I need to worry about impact wise

It did a pretty good job of understanding the flow of the data and transformation logic for that Field.

Because we store the Information Apps that are accessed by Information Consumers in the “Context Plane” and we link tose to the Consume Tiles they consume it also picked up the Last Mile objects that may break and impact our Information Consumers.

One question I have is how it determined the lineage for this data from the four API services we gave it access to, as we did not give it access to the Data Map (lineage) API service.

Im guessing it stitched it together from the Change Rules.

Another question to be answered. I am going to have to figure out how to get Gemini CLI to show me what MCP services it using to understand the process more. Task for another day.

So in Summary

Colour me impressed!

Wood from the Trees

Still a way to go before I have a coherent set of Patterns that I can Coach / Mentor / Teach somebody else for the “Context Plane”, and the “AI Data Stack” or present as a robust Architecture map.

But as I have already said, writing my half formed ideas helps me think.

An incoherent stream of Context

You can find all the previous articles with my train of thought listed in this thread:

https://agiledata.substack.com/t/context-plane

We are building the Context Plane while flying it, so always looking for early adopters to help us decide the final destination.

If you want a virtual chat grab a slot here:

https://contextplane.ai/contact-us/#bookemdanno

“Context” of this post

I often find writing helps me coalesce and refine my thoughts when new patterns start to emerge, but aren’t very clear yet.

So this article is a brain dump / train of thought continuation of the architecture needed to have one Context Plane to rule them all, as part of a proposed “AI Data Stack”.

This article provides an overview of the things or metadata I think should be included in the Context Plane as part of a new “AI Data Stack”.

The “Context Plane”

My current thinking is the architecture diagram for the “Context Plane” should look something like this:

Things stored in the Context Plane

So with that context for how I currently think of the “Context Plane” lets get into the meat of this article, of what I think should be stored in that plane.

I have worked in the data domain for over three decades. The data patterns and the data terms I regularly use I have learnt over those decades. I find them very difficult to unlearn.

As somebody mentioned to me lately I am haunted by “the Ghost of Data Past”.

But also there is a recurring pattern where people seem to invent a new term for a pattern that has been around for decades (medallion data layered architecture anyone), so the patterns of the past do have value.

(and yes I realise the dichotomy of me talking about “Context” not “Semantics” in the same article where I point the finger at the “Medallion Architecture”)

So with all that said, let me use the data components of the past to frame what we need to hold and surface via a “Context Plane” to support what a “AI Data Stack” future might look like.

Business Glossary

The Business Glossary stores a list of agreed-upon definitions for organisational terms.

This holds a list of plain-language descriptions and aliases that standardise terminology across stakeholders, ensuring consistent interpretation and usage of key terms such as ‘Customer’, ‘Order’, or ‘Churn’.

Core Business Concepts

Core Business Concepts stores the primary things (‘entities’) an organisation manages or counts. This includes concepts such as ‘Customer’, ‘Product’, ‘Order’, and ‘Payment’.

This holds a list of the definitions and identifiers for the Core Business Concepts.

It contains a subset of the terms that exist in the Business Glossary and forms the foundation of the organisations data design.

This describes a list of “things”.

Core Business Processes

Core Business Processes stores the major operational workflows, events or life cycles (‘relationships’) within an organisation.

This holds a list of the interactions between Core Business Concepts, representing relationships such as “Customer Places Order” and “Customer Pays for Order”

This also holds the sequence of activities or states that Core Business Concepts transition through, such as invoice approval or support resolution (which I call “Administration Processes”).

This describes ‘relationships’ between “things”.

Conceptual Data Model

The Conceptual Data Model stores a high-level map or diagram that provides an abstraction of the organisation’s data design.

This holds information that outlines the Core Business Concepts, Core Business Processes the other relationships between Core Business Concepts in a platform-independent format and without specifying implementation details.

The Conceptual Data Model can also hold the objects that generate the Diagram, so overlaps with the Core Business Concepts and Core Business Processes.

This describes “things” and the ‘relationships’ between “things”

Logical Data Model

The Logical Data Model stores a more detailed map or diagram that provides Details (‘attributes’) that are part of the organisation’s data design.

This holds information that outlines the Core Business Concepts, Core Business Processes / Relationship’s in a platform-independent format.

It also holds the Detail (‘attributes’), that are related to a Core Business Concept, in a platform-independent format

This describes the Details of “things”.

Physical Data Model

The Physical Data Model stores a technical implementation view of how the organisation’s data is stored and accessed within a specific system.

This holds information on the the physical structures such as tables, columns, data types, indexes, partitions, and naming conventions used in the data platform.

It is derived from the Logical Data Model but adds system-specific optimisations and constraints.

It overlaps with the Logical Data Model by representing the same Core Business Concepts and their Details but also includes physical performance and deployment details that are specific to the technology used.

This describes how to implement the “things”, their ‘relationships’, and their Details.

Data Contracts

Data Contracts store the expected structure, quality, and behaviour of data exchanged between systems, or components. They describe formal agreements between a data producer and a data consumer.

This holds information such as schema definitions, required fields, data types, validation rules, and guarantees such as update frequency, timeliness, or completeness.

It overlaps with the Logical Data Model and Physical Data Model by referencing the same attributes and structures, but focuses on expectations and enforcement rather than storage or modelling.

This describes the Rules that govern the movement of the data.

Data Dictionary

The Data Dictionary lists and describes the metadata for data fields used across the organisation’s data assets.

This holds a list and information for data fields, such as field names, data types, descriptions, allowable values, formats and default values.

It overlaps with the Business Glossary. Logical and Physical Data Models by documenting the same ‘attributes’.

This describes how “things” and their Details were implemented.

Data Profiles

Data Profiles provide statistical summaries and characteristics of the data physically stored.

This holds data such as minimum and maximum values, distribution ranges, cardinality, null counts, uniqueness, and data type conformity.

They overlap with the Data Dictionary (by describing the same fields) and with Facts (by profiling the raw values captured).

This describes the observed characteristics of the Details of “things” and ‘relationships’ between “things”.

Transformation Logic

Transformation Logic stores how raw or source data is converted into a trusted, usable form.

This holds the business rules, mappings, calculations, filters, joins and aggregation logic or code applied to reshape data.

It may reference fields defined in the Logical or Physical Data Models and rely on terms from the Business Glossary and Core Business Concepts to apply meaning.

It overlaps with Measures, Metrics, Actions, and Information Products by forming part of the logic used to answer questions or trigger downstream outcomes.

This describes how the raw “things”, Details of “things” and the ‘relationships’ between “things” were reshaped into something more useful.

Facts

Facts are raw numerical values stored or sourced directly from systems.

This holds a list of data such as quantities, amounts, durations.

Facts are the foundational inputs used in downstream calculations. They are not aggregated or transformed, but instead represent the atomic, immutable data from systems where that data is first captured.

They overlap with the Business Glossary, Physical Data Model and Data Dictionary where they are described and with Measures and Metrics, which are derived from them.

This describes the raw numerical values about “things”.

Measures

Measures are standardised aggregations of raw Facts.

This holds the logic or code for calculations such as “total revenue”, “count of orders”, or “average order amount”, typically derived using functions like sum, count, min, max, or average based on a defined Fact.

They overlap with Facts (as their source), Transformation Logic (which defines how they are calculated), and Metrics (which may use them as components).

This describes aggregated values about the Details of “things”.

Metrics

Metrics are calculated formulas that combine Measures, Facts, or other Metrics to express business performance or operational efficiency.

This holds the logic or code for calculations such as “average revenue per customer”, “conversion rate”, or “churn percentage”, typically involving arithmetic, ratios, or conditional logic.

They overlap with Measures (which they reference), Transformation Logic (which defines their formulas), and Business Questions (which they help answer).

This describes derived insights from aggregations of the Details of “things”.

Business Questions

Business Questions are predefined or commonly asked queries that reflect specific decision-making needs within the organisation.

This holds a list of the questions that have been asked before such as “What is our monthly churn rate?”, “Which products are underperforming?”, or “How many new customers joined last quarter?”

They overlap with Metrics (which may be the answer), Transformation Logic (which defines how to calculate the answer), and Information Apps (where answers are delivered).

This describes the repeatable questions asked about the “things”, Details of “things” and ‘relationships’ between “things” to support decisions to take action.

Actions

Actions define the decisions, tasks, or system behaviours triggered as a result of answering a Business Question.

This holds a list of actions that have been taken in the past or need to be taken int he future.

It also holds information of the mappings between specific Business Questions and the Actions they enable, such as “Identify outstanding support tickets” and “resolve outstanding support tickets,” or “Identify Customers who havent placed an order in the last 6 months” and “Offer discount to Customers to reduce the rate of customer churn.”

They overlap with Business Questions (which initiate them), Information Apps (where they may be launched), and Core Business Processes (which they may automate or influence).

This describes the decisions and actions we take based on what we know about the “things”, Details of “things” and ‘relationships’ between “things”

Information Apps

Information Apps are curated outputs that present data and information in a usable form to answer Business Questions and support Actions.

This holds lists and information about dashboards, reports, visualisations, datasets, APIs, data servies, and user interfaces that package answers to Business Questions and trigger related Actions.

They overlap with Business Questions (which they answer), Actions (which they enable), and Facts, Metrics and Measures (which they visualise or expose).

This describes the delivery mechanism that provides access to what we know about the “things”, Details of “things” and ‘relationships’ between “things”

Fragmentation and Overlaps

I realised as I wrote this one of the major problems is deffo “the Ghost of Data Past” and specifically the fragmented set of data technology and tools that I have used over the years. These set of data technology and tools helps define the architecture and language I use, and also influences the patterns I applied to list the things that should be stored in the ”Context Plane”.

I have ended up using these technologies and tools to define the things that are needed in the “Context Plane” and also to define boxes or boundary around things.

The problem I hit is when I use these standard languages / boundaries from the data domain for those things then I get overlaps, there are things that are in within / across multiple boundaries.

For example Core Business Concepts are held/described within both a Business Glossary and a Conceptual Model.

An example using a Metric

Metrics are a part of a Business Question, but also defined in both a Metrics Layer / Tool, Business Glossary and potentially a Data Dictionary.

Lets look at an example of the overlaps in more detail using a specific Metric ‘Active Users’.

Metrics:

'Active users' is the number of people who engaged with our site or app in the specified date range.

An active user is any user who has an engaged session or when Analytics collects:

* the first_visit event or engagement_time_msec parameter from a website
* the first_open event or engagement_time_msec parameter from an Android app
* the first_open or user_engagement event from an iOS app

The user is considered an active user as soon as the user_engagement event is detected within a second.

(Source: https://support.google.com/analytics/answer/12253918?hl=en)

Business Questions:

"The Number of Active Users who accessed our Website last month."

"The Number of Active Users who accessed our App last month."

The Metrics Layer often holds the relationship the Metric has with the Core Business Processes, the Business Glossary often does not.

Yaml Metric Definition in a Metrics Layer tool:

views:
  - name: active_users
    description: "14 days rolling count of active users"
    includes:
      # Measure
      - users.rolling_count
      # Dimensions
      - users.is_paying
      - users.signup_date
      - company.name

(Source: https://cube.dev/blog/introducing-views)

Business Glossary:

Active user

Active users are the people who currently use your product. A user who becomes inactive may have churned.

(Source: https://posthog.com/docs/glossary)

As you can see there are overlaps in what we define for the Metric of “Active Users”, depending on which traditional data component we define it in.

Removing the Boundaries and Overlaps

When I get stuck with a problem like this I find the key is to keep breaking it down into smaller and smaller things until I get to a list of atomic things that are unique.

Here is where I have ended up with two different languages I can use so far:

(compiled with some help from my ChatGPT friend using the above as the input, and then the human iterated it)

The list of Context we store

Or put another way the things we store.

1. Business Terms

   Agreed-upon definitions and aliases used across the organisation (from the Business Glossary).

2. Core Business Concepts (entities)

   The primary entities the organisation manages or counts (e.g. Customer, Product, Order).

3. Core Business Processes

   The workflows, events, or life cycles that define how Core Business Concepts interact and change over time.

4. Conceptual Relationships

   High-level, platform-independent relationships between Core Business Concepts.

5. Details (attributes)

   Descriptive details of Core Business Concepts, such as name, status, date of birth (from the Logical Model).

6. Field Metadata

   Technical descriptions of fields, such as data types, formats, default values, and allowable values (from Logical Data Model and Data Dictionary).

7. Physical Structures

   Tables, columns, data types, and indexes used to implement data in a specific platform (from the Physical Model and Data Dictionary).

8. Data Rules and Expectations

   Contractual schema definitions and guarantees that govern how data is exchanged and validated (from Data Contracts).

9. Statistical Profiles

   Observed characteristics of data, including distributions, cardinality, and null rates (from Data Profiles).

10. Transformation Logic

    Business rules and operations that convert raw data into usable outputs (e.g. mappings, joins, calculations).

11. Raw Numerical Values

    Source data captured as facts, such as quantity, amount, or duration (from Facts).

12. Aggregated Values

    Summarised calculations based on raw numerical values (from Measures).

13. Derived Metrics

    Formulas combining multiple measures or facts to express performance or ratios (from Metrics).

14. Business Questions

    Repeatable questions that drive insight and inform decisions (from Business Questions).

15. Actions

    Operational steps or decisions taken based on answers to Business Questions.

16. Delivery Interfaces

    Mechanisms that present data and enable interactions, such as dashboards, reports, datasets, and APIs (from Information Apps).

The types of Context we store

Or put it another way, how it is stored.

1. Lists

   Repeating sets of labelled items.
   

   Examples:

   – List of business terms (Business Glossary)

   – List of fields (Data Dictionary)

   – List of metrics, measures, concepts, questions, or actions

   – List of attributes, rules, or processes
   

2. Definitions

   Text-based descriptions that explain meaning, purpose, or intent.
   

   Examples:

   – Term definitions

   – Field descriptions

   – Action explanations

   – Business question phrasing
   

3. Identifiers

   Unique keys, codes, or labels used to reference or join data.
   

   Examples:

   – Concept identifiers

   – Field names

   – Relationship keys
   

4. Relationships

   Structured mappings between two or more things.
   

   Examples:

   – Concept A interacts with Concept B
   – Concept A, Concept B and Concept C are involved in Process A

   – Detail belongs to Concept (attribute belongs to entity)

   – Measure uses Fact
   – Metric uses Measure

   – Question → Metric → Action

   – Term → Concept → Field
   

5. Rules / Logic

   Outputs produced from raw data using logic or formulas or expressions that define behaviour or constraints.
   

   Examples:

   – Validation rules

   – Data contracts

   – Calculation logic

   – Transformation pipelines

   – Measures (aggregated)

   – Metrics (calculated)
   

6. Structures / Schemas

   Models that define how things are organised or composed.
   

   Examples:

   – Logical/physical data models

   – API schemas

   – Table definitions
   

7. Raw Data / Values

   Captured data points or measurements.
   

   Examples:

   – Facts (quantities, amounts)

   – Timestamps

   – Event logs

   – Data Profile results

   – Data Quality results

8. Interfaces / Outputs

   Representations used for delivery or consumption.
   

   Examples:

   – Information Apps
   – Dashboards

   – Reports

   – APIs

   – Data services

Time to Cross Domains

As I mentioned at the start, my experience and expertise is founded in the data domain.

I think its time to look for help from the other domains, and for this one specifically the Library and Information Sciences domain.

Juan Sequeda posted an LinkedIn article on how he is thinking about metadata/ontologies/knowledge graph/semantic layers. In that article he uses this framing:

I can probably simplify “the List of Context we store” and “the Type of Context we store” above to the Business/Technical/Mapping metadata categories in that diagram.

But Im not sure simplifying it will get me any closer to achieving my goal, I think I need to go down into the weeds a little more to get the clarity I need.

One option to do this safely I think I need to identify a number of use cases where “AI” will use the “Context Plane” and see what I need to provide it to be successful.

Another option is to try and collaborate with an expert from the Library and Information Sciences domain to help add their views and language to see if it gets me to the next step.

The other option is to follow a suggestion Joe Reis made on the Practical Data Modeling Discord,

“As a thought experiment, ask any AI what it needs. It’s very different from what we’ve devised so far”.

Time to have a little think about what the next step in my train of thought for the “AI Data Stack” will be.

Wood from the Trees

Still a way to go before I have a coherent set of Patterns that I can Coach / Mentor / Teach somebody else for the “Context Plane”, and the “AI Data Stack” or present as a robust Architecture map.

But as I have already said, writing my half formed ideas helps me think.

An incoherent stream of Context

You can find all the previous articles with my train of thought listed in this thread:

https://agiledata.substack.com/t/context-plane

We are building the Context Plane while flying it, so always looking for early adopters to help us decide the final destination.

If you want a virtual chat grab a slot here:

https://contextplane.ai/contact-us/#bookemdanno

“Context” of this post

I often find writing helps me coalesce and refine my thoughts when new patterns start to emerge, but aren’t very clear yet.

So this article is a brain dump / train of thought continuation of the architecture needed to have one Context Plane to rule them all, as part of a proposed “AI Data Stack”.

This article provides an overview of how I think the place we store context should be the equivalent of a ‘Control Plane’ and not a ‘Semantic BI Layer’ as part of a new “AI Data Stack”.

Plane not Layer

As I have been iterating and experimenting with my ideas and language for the “Context Plane” I initially started talking about the “Context Layer”.

New Semantic BI Layers for Old

But I found that as soon as I mentioned the term “layer” people from the data domain (and others from other domains) immediately thought like this:

Effectively the Context Layer was seen as the equivalent of a Semantic BI Layer of old. Something that provides Context of the data in the Centralised (or federated) Data Platform to the Last Mile, AI tools and AI Agents.

But to get to the data in the source systems it infers the AI Agents have to traverse through the Data Platform, which is not the pattern I envisage.

How I currently think of “Planes”

Here is a simplified version of the AgileData App and AgileData Platform architecture diagram for how we leverage the Google Cloud Platform infrastructure.

As you can see I tend to think of Control Planes as being horizontal rather than vertical objects on my diagrams.

The “Context Plane”

So my current thinking is the diagram should look something like this:

The “Context Plane” should provide the context for everything in the organisation that stores data. And it should allow AI Agents and GenAI tools.

The “Context Plane” should also be the thing a system or a set of tools/technologies accesses to communicate with another system or set of tools/technologies.

To be accurate I should I should have put the orange Context box between each of the Source Systems and Data Collection boxes but I didn’t want to complicate the diagram.

Wood from the Trees

Still a way to go before I have a coherent set of Patterns that I can Coach / Mentor / Teach somebody else for the “Context Plane”, and the “AI Data Stack” or present as a robust Architecture map.

But as I have already said, writing my half formed ideas helps me think.

An incoherent stream of Context

You can find all the previous articles with my train of thought listed in this thread:

https://agiledata.substack.com/t/context-plane

We are building the Context Plane while flying it, so always looking for early adopters to help us decide the final destination.

If you want a virtual chat grab a slot here:

https://contextplane.ai/contact-us/#bookemdanno

“Context” of this post

I often find writing helps me coalesce and refine my thoughts when new patterns start to emerge, but aren’t very clear yet.

So this article is a brain dump / train of thought continuation of the architecture needed to have one Context Plane to rule them all, as part of a proposed “AI Data Stack”.

This article provides an overview of how I think of the Context Plane in our AgileData Product as that anchors a lot of my thinking for what the Context Plane in a new “AI Data Stack” would look like.

First defining our current Context Plane pattern

When we started building the AgileData Platform and AgileData App, Nigel and I agreed on a set of core principles and patterns that we would align to as much as was practical.

One of these was the use of what we called Config (what I now refer to as Context) which should be at the center of everything we would build.

Context is a Pet not Cattle

The Context Plane is where we hold everything we care about that makes our AgileData Platform and AgileData App work, apart from our customers actual data.

We treat both the Context we hold and the Customers data we hold as Pets, and we treat everything else in our Platform as Cattle.

We generate, deploy, execute and destroy code at will. We automate the generation of all code from the Context we hold.

Context drives our App and our Platform

For example everything displayed on our Data Catalog screen is stored in our Context Plane, and then rendered in the AgileData App as needed.

The Concept, Conceptual, Logical and Physical data models are stored in the Context Plane and again rendered in the AgileData App, or used as part of our DataOps processes as and when required.

We define and hold Core Business Concepts, Detail for those Concepts and Core Business Events as the core “entities” in our Context Model.

Concepts, Details and Events are the semantic language we use as part of our Context Language.

We hold the relationships of how Core Business Concepts relate to Core Business Events in the Context Plane.

Which allows us to render and explore a “Graph” view of these relationships in our App.

For those old school of us, we also render these relationships as Bus Matrix in our App. Out Context Plane allows us to render the same Context in any visualisation or Map format that helps simplify complex data tasks.

Our version of Data Quality (Trust Rules) and Observability (Notifications) are again stored in the Context Plane and rendered / used as needed.

Context drives the generation of our code

Our version of Data Transformation logic (Change Rules) is you guessed it, stored in the Context Plane and then used as and when required.

If you perused that last series of screenshots you will see we actually hold surface the Change Rule logic in multiple languages, we surface it via the UI notebook style, we surface it as form of a Gherkin script and we surface the SQL that it is generating to load or transform data in Google BigQuery.

These are all dynamically generated from the Context stored in the Context Plane for each Change Rule we hold. We have defined it once and we can render it in many different ways, many different languages and for many different use cases.

Some people call this a table driven pattern, a metadata driven pattern or a model driven pattern etc.

Orchestrate via dynamic manifests based on Context

To orchestrate the execution of the Data Transformations (Change Rules) we of course build out a Directed Graph to run them in the order they need to be run, and to honour the dependencies they need to honour.

But this Directed Graph is dynamically generated.

If I go and add another Change Rule, it is stored in Context Plane and the next time I come into this screen it will be dynamically rendered including that rule.

When a new record turns up in a table on the left, it will trigger a bunch of steps.

• The Context Plane will be queried to see all the Change Rules that are dependent on that data.

• The Context Plane will be queried to find the dependencies between the Change Rules that will run, and tables they will load.

• A “Manifest” is created to describe what Change Rules need to run.

• The SQL code for each relevant Change Rule will be generated from the Context stored in the Context Plane about that Change Rule

• The code will be submitted to run using a Pub/Sub model, in parallel.

• When all the code has been successfully run, the code and the manifest will be destroyed. (We do hold a copy of the code that was executed in our Audit Vault, so we can always view what was run when).

• The latest Observability and Trust data will be stored in the Context Plane.

As we hold the Context of how every physical table (Tile in our language) relates to every other table (Tile), via these Data Models / Change Rules, we can easily show all the dependencies (Related Tiles) for a specific table (Tile), when viewing the Context of that table (Tile).

This includes the relationships for both the Data Model. i.e for this Concept of “customer” we hold these Details, and that Concept of “customer” is part of these four Events, as well as the relationships for all the Change and Trust and Consume rules that the “customer” Concept is part of, i.e this Detail tile is loaded by this Change Rule and validated by these Trust Rules.

Multiple languages to create Context

You can create new Context in the Context Plane via the AgileData App UI, or you can use the API’s to create the Context.

It would be relatively simple to allow the creation of the Context in yet another language, for example by uploading YAML files.

(Fun fact the first interface we had to create Context was Google Sheets!)

One set of Context to rule them all

So you can see how the Context we hold on in our Context Plane is the thing that drives everything our AgileData Platform and AgileData App does.

Context is first and powers everything that happens

I know what you thinking, this sounds a little bit like a data catalog, or a metadata repository.

But heres the thing, data catalogs run by sucking on the Data and Metadata exhaust.

We treat Context as the fuel that powers everything we do.

You cannot run anything if Context does not exist for that thing.

I cannot create a data catalog object, I cannot create a table, I cannot create code or run it, without it existing in the Context Plane.

I first create the Context and then the AgileData Platform then does all the other horrible and complex data work for me.

Its about more than just data and code

As you would expect of any mature data platform, we have added “Data Management” and “Data Governance” capabilities over the years where we found them useful.

One of those is the ability to store the Context needed to render a “Data Dictionary”.

But we are still missing a lot of useful Context

We only build things in our AgileData Platform and AgileData App, when they save us time or automate something we hate doing.

The Data Dictionary feature was built so we could find fields we needed to find by searching all the Context of all the data we hold for a customer.

There is some other Context we know will give us value that we have yet to add.

Things like Context typically related with Business Glossaries, Metric Layers/Engines and Semantic BI Layers to name three.

I believe they are all part of the Context that is needed in a Context Plane to make it useful for AI Agents.

Which will be the next article I write to help me with my thinking.

Wood from the Trees

Still a way to go before I have a coherent set of Patterns that I can Coach / Mentor / Teach somebody else for the “Context Plane”, and the “AI Data Stack” or present as a robust Architecture map.

But as I have already said, writing my half formed ideas helps me think.

An incoherent stream of Context

You can find all the previous articles with my train of thought listed in this thread:

https://agiledata.substack.com/t/context-plane

We are building the Context Plane while flying it, so always looking for early adopters to help us decide the final destination.

If you want a virtual chat grab a slot here:

https://contextplane.ai/contact-us/#bookemdanno

“Context” of this post

I often find writing helps me coalesce and refine my thoughts when new patterns start to emerge, but aren’t very clear yet.

This article is a brain dump / train of thought to help refine what I think the data stack looks like in the new “AI” domain.

This article provides an overview of how the combination of a “Context Plane”, AI Agents and MCP services might work together.

First defining the “AI” domain

First lets anchor the context of what I mean by the “AI” domain.

I am thinking about GenAI, Large Language Models (LLM’s), Agents, Agentic and MCP’s.

Lets pop over to my ChatGPT co-friend for definitions:

Don’t love the definitions it came up with, but don’t hate them enough to spend time rewriting them (yet).

(I have updated the definition of MCP after publishing the article, because as somebody kindly pointed out on LinkedIn, ChatGPT had “hallucinated” with the original definition I posted)

I’m not thinking about Data Mining, Statistics, Data Science, Machine Learning and all those patterns that I believe have been in the “Analytics” domain for many decades.

Second defining the current data stack

Context is key so lets understand how I see the current data stack pattern.

I tend to think of patterns in layers.

So for me there is a very high level view where I distill the patterns down into as few large boxes as possible:

And then I will break that down into a lot of much smaller boxes when I need to define more detailed patterns:

I have used both of these diagrams for a while now and they both need to be iterated for the “AI Data Stack” pattern.

I did an iteration a year or two ago where the core data “AI” pattern that was emerging was the Text to SQL pattern and so I added a box in the Consume areas to cover that.

But things have accelerated a lot in this space and I need to do another major iteration to my thinking and pattern library.

For this article I am going to call out a few core patterns that are helping my current train of thought on the patterns that need to be iterated the most.

Again when I am thinking about new patterns I find thinking in big boxes, not detailed ones helps, it stops me getting stuck in the weeds too early or stops me “boiling the ocean” as Juan Sequeda likes to say.

Centralised Data

The current data stack pattern is focussed on collecting data from source systems, storing that data in one place, changing the data to make it fit for purpose and then providing access to that data to any person or system who needs it.

Data platform(s) harvest the data from the places that create it. Last Mile tools consume the data from the data platform.

Catalogs harvest the Data / Metadata

The current data stacks are focussed on Catalog capabilities that extract the Metadata (and often data) from the various places that data is stored.

Catalog(s) harvest the metadata from the places that create it, after it has been created.

Third defining the “AI Data Stack”

This diagram / map is a brain dump, my focus is the number of boxes and what they do / don’t do. Not the flow, or layout etc (yet).

Key patterns I am thinking about (in no particular order)

• Source Systems are a source of Data and Context that the Agents need to understand and access;

• Data Platforms are a source of Data and Context that the Agents need to understand and access, but not the only one anymore;

• An Agent may invoke a Last Mile Tool, an action in an Application, or another Agent.

• Source Systems and Applications are one in the same, but keeping them as separate in this diagram helps with the thinking process right now.

• All the places that hold data, provide Context for that data to whatever needs that Context.

• All the “SYSTEMS” / places that hold Data or have a User Interface, will provide a MCP service that allows an agent to access it directly, removing the need to always access the centralised data platform (this feels very much like the data virtualisation patterns of old).

• If the source system cannot serve the data needed, for example historical data, then the Agent will need to use the data from the Data Platform.

• Source Systems will be forced to start storing Historical data to serve Agents directly.

• The Context Plane will provide a single pain of glass for all Data Context held by every system, so the Agents can talk to one place.

• The Context Plane will receive Context from the Systems, not harvest it. The Systems will Push the Context to the Context Plane. This will remove the need for the Context Plane to have to create an Pull pattern / adapter for every System in the world.

• Or the Context Plane will operate under a Federated / Virtualised pattern where it doesn’t hold any Context but can point the Agents to the Systems Context so they can access it directly.

• There needs to be a single language for Context, aka the equivalent of SQL, this is unlikely to happen as the data domain can never agree, and vendors like to make their own standards to create lock-in and a “moat”.

• The Context Plane will not provide Orchestration or Execution capabilities, Agents will talk directly with the MCP services for the relevant Systems to execute its task.

• Agents will either be Orchestrated following a “Direct Graph” pattern or a “Pub/Sub” / “Fire and Forget” / “Mesh” pattern.

Fourth “AI Data Stack” patterns

Next I start to think about some of the more detailed pattern diagrams to refine my thinking. I find thinking using maps helps me identify the underlying patterns.

Centralised Context Plane

Context is Pushed from each System to the Centralised Context Plane where it is stored.

Federated / Virtualised Context Plane

Context is Pulled from each System by the Federated Context Plane as and when it is needed.

Historical data at Source

Systems stores Historical data, either as part of their primary data store, or as a companion data store. This is managed by the the System Team/Owner/Vendor not by a seperate Data Team / Platform.

Agent Execution via Systems MCP Service

The Agent communicates with the Context Plane to find out where the data it needs lives, and to understand the Context of that data. It then executes directly via the Systems MCP Services.

I can see a raft of alternative patterns for this one, centralised MCP Services for execution for example.

Agent Execution via Centralised MCP Service

The Agent communicates with the Context Plane to find out where the data it needs lives, and to understand the Context of that data. It then executes via a centralised MCP Service.

Wood from the Trees

Still a way to go before I have a coherent set of Patterns that I can Coach / Mentor / Teach somebody else for the “AI Data Stack”, or present as a robust Architecture map.

But as I have already said, writing my half formed ideas helps me think.

An incoherent stream of Context

You can find all the previous articles with my train of thought listed in this thread:

https://agiledata.substack.com/t/context-plane

We are building the Context Plane while flying it, so always looking for early adopters to help us decide the final destination.

If you want a virtual chat grab a slot here:

https://contextplane.ai/contact-us/#bookemdanno

Thank you for your kind collaboration

Well not so much collaboration, but thank you to these people who’s thoughts and comments have helped me iterate my current thinking.

Chris Tabb for helping me settle on the term “Context Plane”

Juan Sequeda for both the term “boiling the ocean” and for starting the LinkedIn thread that Chris responded to about the “Context Plane”

Nigel Vining for the usual agile-tecture sessions as I try to get to grip with a new set of technical Patterns.

Tim Frazer for creating a variation of the “AI Data Stack” diagram that mirrored my own thinking at the same time.

Nick Pinfold for a LinkedIn comment that spurred me to pull finger and write down my thoughts today, rather than leaving it for another week, month …