Agentic AI workflow: the architecture behind USU Knowledge

Creating content for a knowledge base takes time. A lot of time. It is the main obstacle for teams documenting their processes, and generative AI promised to remove it: describe what you want, and it writes the article.

Our AI agent dedicated to creating articles and diagnostics, USU Knowledge, started from that same promise. In this article, we share what we explored, what we ruled out, and the architecture we built to make it more effective.  

 The feedback that started it all

USU Knowledge - an AI-powered tool for creating knowledge base articles and diagnostics - was rolled out on the same promise the whole market was making at the time: describe what you want to document, and AI takes care of the writing. Technically, it worked: content was produced quickly and read well.

And yet, the feedback from our users all pointed in the same direction: 

• "The content is too generic, not precise enough."
• "The structure and format are never really consistent."
• "It's faster to do it myself, I have to rewrite everything.
• "I don't trust the result, so I don't use it."

What our users were describing was not a writing quality problem. It was a rework problem. The content came out complete, but it had to be reread in full to know what worked and what didn't. The structure didn't always match their article template. Some passages relied on generalities rather than on their own procedures. And above all, there was no way to tell where things had gone wrong in the process.

So the real user workflow wasn't "generate, then publish". It was "generate, reread line by line, re-prompt, regenerate, start over". Whatever time AI saved on writing, it took back on review.

"Usage was low because it was a one-way feature: the user prompts, the agent creates the content. But nobody manages to produce the perfect content on the first try, because an AI agent without context is very bad at creating relevant content." Théo Rosen, AI Product Manager at Mayday part of USU. We could have treated this feedback as a system prompt problem to fine-tune. We chose to treat it as an architecture problem.

Why single-prompt AI workflows have their limits  

...and why switching models doesn't change that. 

The natural reflex when a result falls short is to reach for a more powerful model. And it's a legitimate hypothesis: a newer model writes better, reasons over longer contexts, and makes fewer mistakes on the facts it knows. The content looks better, but it isn't any more usable. Because a more powerful model improves the quality of the text, not the way it is produced: it still does everything at once, and it is still impossible to know at what point it extrapolated.

What are the limits of generative AI, and what does agent orchestration change? We explain everything in this article.

The reason is structural. When you ask a single model to produce a complete article in one go, you are actually asking it to carry out four distinct cognitive tasks at the same time: identify reliable sources, build an outline, write, then check its own output.

These four tasks have conflicting objectives, and the conflict is sharpest when it comes to sources:

• Searching isn't just opening a document. It means deciding where to look (your knowledge base, your connected documents), reading what is there, and above all making judgment calls: is this procedure up to date? This article and this internal note contradict each other, which one is authoritative? Does this information exist in your sources, or only in what the model learned elsewhere, on the internet, from other companies, in a context that isn't yours? Research requires rigor, traceability, and the ability to conclude: "this information doesn't exist in your sources."
• Writing requires the opposite: fluency, continuity, a text that holds together from start to finish.

A model doing both at once will favor continuity. When a source is missing, contradictory or outdated, it doesn't stop to flag it: it fills the gap with what it knows from elsewhere. The result is an article about your industry in general, when you were expecting your specific procedure.

This is how hallucination happens in a knowledge base context. It isn't a model bug, it's the logical consequence of a poorly broken-down task.

And there is a second-order consequence, even more costly for a knowledge manager: a result produced in a single pass can only be evaluated as a whole. You can't validate the outline without validating the text. You can't say "the sources are right but the structure is off." You can only accept everything or rerun everything. This is exactly what our customers were dealing with every day, and why they stopped using this kind of model.

The role of a knowledge manager in AI content creation

"We broke down the content creation work by talking directly with knowledge managers." Théo Rosen, AI Product Manager at Mayday part of USU

Talking to our users isn't a reaction, it's what we do every day. What this feedback changed was the nature of the question we went to ask them. We were no longer trying to find out what was wrong with the tool; we already knew. We wanted to test what we thought we knew about their job against what they actually do. So we broke down the content creation process with knowledge managers, step by step.

Four findings came up consistently:

1. The work starts with research: before writing a single line, you need to understand the subject in depth. An article that duplicates existing content degrades the knowledge base instead of enriching it.
2. Structure is refined through iteration: a good knowledge article isn't just a good piece of writing. It's a text in which an agent can find the right information in seconds, under pressure, with a customer on the phone. That findability is decided in the outline, not in the writing.
3. You must take the reader's point of view: the writer knows the subject. The reader doesn't. The whole point of reviewing is to see the content through the eyes of someone discovering it.
4. Terminology must be rigorous: the same business action named three different ways in three articles gets in the way of understanding, and search stops returning anything.

Four tasks, four distinct objectives, carried out one after the other, never at the same time, and never without the outline being validated before writing. This was the sequence we wanted to recreate, and it's exactly the one a single pass flattens.

AI Workflow Architectures: Prompt Chaining, RAG and Fine-Tuning ?

We explored several options before building an architecture based on agent orchestration.

• Chaining calls manually (prompt chaining). Instead of a single request to the AI, you make several in a row, always in the same order: first search, then structure, then write. It's simple to set up, but rigid. The chain follows its script whatever it runs into along the way, and above all it has no fallback when a step fails: it just keeps going.
• Feeding the model better sources (enhanced RAG). You improve information retrieval upstream so the AI works from better material. It's necessary, and we did it, but it isn't enough on its own: better feeding a model that still has to complete four tasks at once doesn't change how it completes them.
• Training a model on each customer's formats (fine-tuning). The idea: teach the AI the document structure specific to each organization. It's a personalized solution, adapted to each customer's specifics. However, it would mean maintaining as many models as there are customers, and it still wouldn't solve the question of sources.

What these three approaches have in common: they all try to improve a single pass. None of them makes the process inspectable.

Agent Orchestration: Why a Promising Architecture Is Hard to Get Right 

Agent orchestration means assigning each task to a specialized agent, with the sequence driven by an agent whose sole role is to coordinate the others: which agent steps in, in what order, what gets passed from one step to the next, and what happens when a step fails.

→ The principle of agent orchestration, explained in detail.

But the number of agents isn't what makes the difference.

"The real architectural leap isn't replacing one big prompt with several small ones. It's making the creation process controllable: each agent has a clear responsibility, the user validates the structure before writing, and an error can be fixed at the step where it appears. It's this combination of agent specialization and human control that turns a generative AI demo into a product that works at scale." Olivier Maillot, Head of Data & AI at Mayday part of USU.

On paper, it's elegant. In production, it's another story, and that's precisely why few players really use it today. Moving from a single call to a coordinated chain brings up a whole class of problems that one-shot prompting doesn't have:

• Error propagation. A poorly qualified source at step 1 contaminates everything after it. You need checkpoints, not just steps.
• State management between agents. Each agent must receive exactly the context it needs: not too little, and not the whole history.
• Latency. Each request to a model is what's called a call: a request sent, processed, and answered. One-shot prompting makes one; an orchestrated chain makes as many as it has steps. It is therefore inherently slower. Yet speed is one of the main appeals of an LLM: a tool that takes several minutes to produce what the user thinks they could write themselves in ten will not be used, whatever the quality of the result. The challenge isn't only to reduce that time, but to make it visible: showing which agent is working, on what, and what has already been established, rather than a loading screen that says nothing.
• Cost. Several requests per piece of content. The architecture only makes sense if the rework time saved more than makes up for it.
• Behavior on failure. This is the hardest part. What does the chain do when an agent finds nothing? For us, the default answer isn't to keep going: it's to say so.

What we built with this new approach

USU Knowledgenow relies on a chain of specialized agents that reproduces the way an expert documents a process.

It all starts with a question, not a generation. The agent analyzes your intent and asks you questions when something is ambiguous, starting with the most structural one: are you looking to produce an article or a diagnostic? The two aren't built the same way, and it's the first thing an experienced writer determines before getting started. Then:

1. The sourcing agent identifies what is actually available in your knowledge base and connected documents, surfaces relevant related articles, and applies your writing preferences. Its job doesn't include writing, so it has no reason to fill a gap: when a piece of information doesn't exist, it says so.
2. The structuring agent builds an outline from those sources. It can rely on one of your existing templates, which you mention directly in your request; the article then follows your sections, your order and your format, with no reformatting. This outline is submitted to you before any writing begins, and you can adjust it with unlimited iterations. This is the tipping point of the whole architecture: you validate the structure when fixing it takes thirty seconds, not when you'd have to rewrite three thousand words.
3. The writing agent writes from the validated outline and the qualified sources, in the expected format.
4. The review agent goes back over the output and flags what deserves your attention: deviations from the validated outline, claims not backed by a source, points that need a decision.

The orchestrator drives the whole process: it decides the sequence, passes the relevant context to each step, and handles failures.

The key takeaway isn't the number of agents. It's that you no longer step in at the same moment. In a one-shot architecture, human control comes at the end, when everything is written and everything needs to be checked. Here, it comes at the step where your expertise adds the most value: validating the outline, and then deciding to publish.

"It's a paradigm shift: the user iterates with the agent across every phase of content creation, on finding context, on structuring the content, and on meeting quality standards." Théo Rosen. 

Diagnostics: Where validating the outline matters most

The same value chain produces your diagnostics, the decision trees that guide an agent step by step. And it's in this format that the architecture shows its value most clearly. A poorly structured article can be fixed during review: you move a section, you rephrase a transition. A poorly structured decision tree can't. If the first question isn't the right one, or if a branch is missing, it's not a correction, it's a redesign. The logic of the path determines everything that comes after.

That's exactly what single-pass generation makes impossible to control, and what validating the outline before writing solves. You decide the logic of the diagnostic (the questions, the branches, the outcomes) before a single step is written. You then view the complete structure in Mayflow, and keep iterating with the agent until it matches your business reality.

The result: one article, one conversation

The test we set was simple, and deliberately demanding: produce publishable knowledge content in a single conversation with the agent. No back-and-forth re-prompting. Structure, content and format respected from the very first output.

That's what USU Knowledge does today.

"Users spend less time writing content that meets their quality standards, while making sure answers are easier to find and the knowledge base stays high quality." Théo Rosen.

And the gain doesn't stop at writing time. Content that is sourced from your knowledge base, structured to your standards and consistent in its terminology is content that your search engine, your AI agents and your support agents can make better use of. The quality of the knowledge base determines the quality of everything connected to it.

What this says about the way we build 

We didn't choose agent orchestration because it was the most obvious answer to our customers' problem. We chose it because our customers described a problem to us, precisely, that one-shot prompting couldn't solve.

It's also, quite simply, how we work: listening to our users' feedback, seeking out the approaches that truly solve the problem, and putting them into production.

Want to see USU Knowledge in action?  What about booking a demo?