As part of adopting GenAI in our Way of Working at AgileData, I have been experimenting with our Way of Working as much as I have been experiment with our GenAI tooling and actually doing the Data Work and Information Product building.
I have ended up with two separate “harnesses” for now, one called ‘AgileData Practi’ that assists me with the Data Work and one called ‘Information Product Builder’ that assists me in building and deploying the Information Products (yes I know ‘Builder’ is not the most creative name).
The Information Value Stream for “Builder’ looked something like this.
Upload completed Information Product Canvas and ask Builder to tell me what data in the Consume layer is missing.
Hand that off as a requirements brief to Practi to help do the Data Work.
Back to Builder to confirm its now has access to the data it needs.
Builder then builds a first cut of the Information Product.
I become the human in the loop and review it, ask for iterations until I don’t hate it.
Builder deploys it, and I ask our Customer Stakeholders to review it and tell me what they want iterated.
Builder and I work together to iterate it based on their feedback.
Rinse and repeat #7 until the Stakeholder is happy.
Deploy the Information Product as a ‘Live’ version.
There are a few hand-offs in that Value Stream at the moment, they are there on purpose, so there are natural breaks I can check things before we spend more time and tokens on the next step.
But thats an article for another day.
One of the things I have learnt in the current Value Stream is to recognise when I need to get the hell out of the way.
Originally step #7 would go something like this.
Stakeholder would email me the things they wanted iterated.
I would read it and summarise what I think they mean’t.
I would hand the summary to Builder to start the iteration.
I summarise the problem with this process as this.
The Stakeholder is speaking German
I translate that German to English
I give English to Builder and the underlying LLM translates it to French
There is no shared language
As a result a lot of context gets lots at each step of those language translations.
So I tested removing my language translation from that process. It made a big difference.
Now I just give Builder the content from the Stakeholders email and it goes from German to French with aplomb, hell thats what LLM’s are good at right, language.
And it got me thinking how much could I push that Builder work to the Stakeholder directly and let them iterate without me in the loop at all.
Given the manual hand-offs that I have in place at the moment, I didnt want to go full YOLO on this one in the first iteration.
While the dream is a magical box where the Stakeholder writes one line of text and the GenAI tooling builds them a trusted Information Product from the messy raw data in the source systems, we are far from that magic being reality yet.
So as is the Agile Data way I decided to do a McSpikey (research spike) to experiment with a small part of the process.
So I created an experimental Information Product called ‘Pioneer’.
This allows a person (can’t decide what exact Persona its targeting at the moment, think its a novice Data Practitioner rather than a Information Consumer) to do three things:
Dig down into the Consume data to see what that data looks like.
Prototype a page of an Information Product for that data
Share that prototype as a Brief I can put directly into Builder to build and deploy a supported Live Information Product.
A few key points on the Way of Working experiment:
They can only access trusted Consume data
They cant share the prototype with anybody, they can only share the generated brief
As I started working on it I ended up going down a rabbit whole on the style of graphs Pioneer was producing.
I have spent quite a bit of time working on Builders Design system for graphs to make them look attractive.
As they say sex sells and I personally didn’t want to produce graphs that looked like they were produced by a Data Analyst using R code.
The first versions were ok.
But not quite the same styles as we have in the Design System and our Information Products, and so down the rabbit hole I went, building out things to force Pioneer to comply with the Information Product Builder Design System.
And then I remembered a Pattern I have used for years.
When I start off the design process for an Information Product in the pre GenAI days I would do a rough wireframe sketch, typically by hand on a piece of paper or a whiteboard.
This let the Stakeholder know that it was a rough sketch, it was the beginning of an idea, it was ok to change it, the cost of change was very very low.
As soon as I created that sketch in a tool and it had an increased level of fidelity, a hogh level of resolution, then people were less likely to requests changes to it.
And so I stopped on the high fidelity resolution quest in Pioneer and embraced the idea of a sketch.
Of course when I asked Claude to make this change it asked me what I wanted to do about the Prospect page. Did I want the same sketch design applied to that as well?
The Prospect page allows the person to explore the Consume data to understand it more.
Data in the Consume layer is deterministic, it is trusted.
And so the person using that page had to feel like they were exploring data they could trust.
So high fidelity graphs it was.
I think if I do open Pioneer up to allow people to explore the data in the Designed or History layers in the AgileData Information Platform, I will have Pioneer change the style of these visualisations back to the sketch format, ie lower fidelity.
But that is a decision for the future.
For now its back to the original question the McSpikey was designed to answer.
Does Pioneer help to make our Way of Working better or worse.
One concern I have is now the Information Product Canvas is decoupled from this process. I have no line of sight how the Pioneer prototype links to an Action, an Outcome and Value,
I am worried it will just move us back into a Dashboard feature factory that has been a problem in the data domain for decades.
But again that is another problem for another day. And no doubt another McSpikey.