Case study · AI systems: agents, teams and results

Case: governance that signs. How an AI-agent operation decides what gets published

Sebastián Ocampo · 2026-07-25

This is the third production system documented in this lab, and the one holding up the other two: the rules deciding what an agent may publish alone, what requires a human signature and what happens when something goes wrong. It is not committee theory: every mechanism described below is running right now on this site's pages, including this one.

The problem: a production machine is, by default, a machine for being wrong at scale

This house runs on AI agents: a trilingual editorial operation and a market research system. That means the capacity to produce errors is automated too: an agent can generate more sourceless claims, broken links and style inconsistencies in an hour than a human would review in a week. And this house's product is precisely trust: a publication whose verdict system says on the front page whether a robot is real, teleoperated or staged cannot afford its own content breaking the rigor it demands of others. The public method behind those verdicts is in the methodology.

The risk also has an external name: search engines penalize scaled content abuse, and readers penalize sooner. The design question was not "how do we keep AI from being wrong" (you cannot) but: what system ensures no agent error reaches publication without passing through executable rules and a person who answers with their name? The useful answer for any company is that governance is not about slowing production: it is about deciding, with mechanisms rather than memory, what crosses the door.

The system: rules that are tests, gates that block, a signature that answers

The central piece is a displacement: the editorial rules do not live in a document agents are supposed to read, they live in code that runs against every piece. Today that is 34 automated tests over 417 content records in three languages, and publication stays blocked while a single one fails. Above the automated gates sit two more layers: real-browser verification (the built page is checked on mobile before publishing) and the final human review, which decides the only things tests cannot see: whether it is true, whether it is useful and whether it sounds like this house. The table details each gate and what it catches; the full flow:

  1. Written rule
  2. Rule turned into a test
  3. Agent produces
  4. Automated gates
  5. Browser verification
  6. Human review and signature
  7. Publication
  8. Failure log
  9. New or corrected rule
GateWhat it checksWhat it catches in practice
Structure validationEvery content piece validates against a strict schema: mandatory fields, formats, all three languages present.Half-translated pieces, sources without URLs, malformed dates: the structural error dies before existing as a page.
One query per pageNo search query may have two owner pages in the same language.Cannibalization, the silent failure mode of any high-output site: two pages competing against each other for the same thing.
Zero orphans and resolvable linksEvery page must be linked from the graph, and every internal link must resolve in all three languages.Pages nobody would find and links an agent wrote towards content that does not exist.
Clean answer fieldsTexts feeding structured data and extractable answers may not contain internal syntax.Caught a link inside an FAQ the very day this lab was being written: the error would have shipped in the page's structured data.
Verifiable styleThe house voice rules expressible as checks (banned characters, answer structure) get checked, not remembered.Broken again and again as instructions; never again as tests. The full story below, in the failures.
Real browser before publishingThe built page opens in a browser and gets checked at mobile size: overflows, broken elements, what the HTML does not tell.This gate exists because of a real failure that reached production: a broken video no test could see. The story, below.
Human review and signatureA person reads the final piece, answers for its truthfulness and signs it by name. Agents never publish alone, no exceptions.What no test can see: valid but false claims, misleading emphasis, promises the house should not make.

The decisions: why this and not that

As in every case from this house, the tools matter less than the whys. These are the five governance decisions that defined the system, with the alternative each displaced.

DecisionDiscarded alternativeWhy
Policy as blocking testsPolicy as a document plus goodwillWe lived it: the same rules broken as instructions were never broken again as tests. A manual informs; a mechanism governs.
Blocking gates before publishingAfter-the-fact audits of what already shippedAn error caught after publishing has already taken its price in trust. The gate costs seconds; the retraction costs credibility.
Public failure logFailures handled privatelyEvery case from this house publishes its failures section. It is the cheapest proof the rest is true, and it forces root fixes instead of cover-ups.
One person signs everything by nameSampling, or diffuse "team" responsibilityNamed accountability changes the signer's behavior and the reader's trust. "Reviewed by the team" hurts nobody.
Rules and data in owned, portable formatsGovernance configured inside the AI toolRules living inside a vendor's product leave with the vendor. Ours work the same if the model changes tomorrow: switching is reconfiguring, not rebuilding.

The accounting: what governing this way costs

Automated governance has a near-zero marginal cost: the 34 tests run in seconds on every site build, with no added platform cost, and the browser checks add a couple of minutes per publication. The real cost is human and it is the final review: reading, checking claims against sources and signing, the supervision line that dominates any agent operation (the full cost structure, with a calculator, is in the content operation blueprint).

The accounting on the other side is the one almost nobody does: what the gate prevents. Every failure in the table above caught before publishing cost minutes to fix; the one real failure that crossed into production (the video, told below) cost hours of diagnosis, an urgent fix and the reader-trust share that never gets measured. In an operation producing at agent speed, the ratio between those two numbers can only grow in the gates' favor: it is the best-yielding investment in the whole system.

What failed

Three real failures, chosen because each taught a different rule. The first is foundational: the house voice rules (from a banned typographic character to the structure of the first sentence) were broken again and again in drafts while they were written instructions. They stopped being broken the day they became tests that block publishing. That is where the principle behind this case's title comes from: if your style guide cannot reject a piece, it is a suggestion.

The second crossed into production and a reader found it: an embedded video rendering as a black box on mobile. No test could see it, because tests read data and structure, not pixels: the cause was the combination of a vertical video format and a style rule that could never apply to the dynamically created element. The technical fix took an afternoon; the governance fix was adding the missing gate, real-browser verification at mobile size before every publication. This system's gates did not come from a master plan: each one is the scar of a specific failure.

The third was a casting error and human review caught it, not a test: the first version of the author's profile credited him with verifying robots, when the publication verifies with its method and the person builds systems. Read cold, the sentence was false, and it is exactly the kind of error (legally harmless, reputationally lethal) that no structural test will ever see. That is why the last gate is a person, and why this case is named what it is: governance that signs.

Where this fits in the frameworks (NIST, ISO 42001, AI Act)

This system was not designed by filling in a framework, but it maps cleanly onto them, and that mapping is useful if you have to defend it before a board or an auditor. Against NIST's AI risk management framework: the written rules and named signature are the govern function; the failure log, measure; the blocking gates, manage. Against the spirit of ISO/IEC 42001: this is an AI management system in miniature, with policy, operational controls, failure-fed continuous improvement and assigned accountability. And against the direction of the EU AI Act, which pushes towards transparency and human oversight of content-producing systems: here human oversight is not a checkbox, it is the final gate with a first and last name. The CAIM certification signing this lab covers exactly that craft: aligning real systems with these frameworks without turning them into documentation theater.

The translation to your company is not copying these gates, it is copying the question that created them: for every rule in your AI policy, which mechanism executes it, and what exactly happens when it is broken? Wherever the answer is "we trust people to remember", there sits your next failure, waiting for a date.

How to build governance that signs in your company

Six steps, in the order that works. The pattern applies equally to content, financial reporting, customer service or any other flow where agents produce work that goes out into the world.

  1. Write the rules as checkable sentences Not "content will be high quality" but "every numeric claim carries a source with a URL" or "no piece ships without all three languages". If a rule cannot fail detectably, it is not yet a rule: it is an aspiration.
  2. Turn everything convertible into a mechanism Every checkable rule becomes an automated check that blocks, not warns. It takes no grand engineering: structure validation and banned-pattern lists cover most rules of any operation, and take days to build.
  3. Define what requires human signature, and make it named The clear line: agents prepare, gates filter, a named person signs what goes out into the world. Diffuse signing ("reviewed by the team") changes no behavior; named signing does, in both directions.
  4. Add the verification that sees what data cannot The final result, viewed as the recipient will see it: the page on a real phone, the report opened in the client's format, the email read in the inbox. Our only production failure lived exactly in that gap.
  5. Publish (at least internally) the failure log Every failure with cause, fix and new rule. A visible log turns errors into infrastructure and disarms the politics of concealment, which is the real continuity risk of any AI operation.
  6. Close the loop: every failure ends in a rule The ritual question after every incident: which gate would have caught this, and why did it not exist? That way the system improves at the pace of its errors, the only improvement pace an agent operation can honestly sustain.

Where this evolves (as read in July 2026)

Two curves are crossing. First: the cost of producing with agents keeps falling, so production stops being anyone's advantage and the difference moves entirely to what this case told: which system decides what gets published and who answers for it. Second: European regulation advances in phases towards demanding transparency and demonstrable human oversight, and the gap between "we have an AI policy" and "our policy can reject work and leave a trail" will become an auditable gap. Our operating bet for the coming quarters: agents themselves will take on more self-checking (proposing the test along with the piece), browser verification will get automated into the gates, and the failure log will gain dataset structure, like everything in this house. What will not change is the last gate: a person who signs. The more the machines produce, the more the signature is worth.

This house's full system, with its other two production cases and its quick answers, is in AI systems.

Frequently asked

Does governance not kill the speed you gain from agents?
The opposite: it is what lets you use the speed. The automated gates run in seconds and add no perceptible friction; what does consume time, the final human review, would exist anyway in any serious operation. Without gates, every fast piece is a bet; the day one goes wrong in public, the whole operation slams to a halt for far longer than the gates would have cost in a year.
How many rules do you need to start?
Fewer than you think: our gates were born from specific failures, not a master plan. An honest start is three mechanisms: structure validation of what agents produce (mandatory fields, sources with URLs), a short list of checkable prohibitions, and the named-signature rule. With that running, every new incident tells you which gate to add. Governance designed whole at a leadership retreat usually describes a company that does not exist.
What tools do the automated gates require?
Nothing exotic: ours are ordinary software tests (schema validation and text checks) running on every site build, plus an automated browser for visual verification. In an operation without developers, the same role is played by your CMS form validation, templates with mandatory fields and checklists someone actually runs. The tool matters little; what matters is the blocking property: if work can get published by skipping the check, you do not have a gate, you have a recommendation.
Does this satisfy the AI Act or the nFADP?
It is the operational foundation, not the whole compliance: which concrete obligations apply depends on your sector, systems and data, and that demands its own analysis. What this pattern gives you is what the frameworks presuppose and almost nobody has: demonstrable human oversight, controls that leave a trail and a living incident log. With that foundation, compliance work is mapping and documenting; without it, it is inventing on paper an operation that does not exist, and that fiction is what collapses at the first serious audit.

Sources

More cases and the method, in AI systems: agents, teams and results.