Building a Governed AI Analytics System with Verifiable Answers

Ask a language model for last quarter's revenue by region and it will give you a confident, nicely formatted answer. Whether the numbers are right is another matter. Language models are fluent with numbers and unreliable at producing them.

In a demo that is a small flaw. In front of a board, one invented figure can end the project.

The usual response is to check the output afterwards, with a reviewer or a second model. For analytics, I think that is the wrong design, because it is a bet that the reviewer is never wrong. When I built the Verifiable AI Analyst, a conversational analyst for a Saudi digital-services company, the goal was different: make a fabricated number un-writable.

This post explains the design, so you can apply it to your own system.

The central rule: the model never calculates

Split the work by what each part is good at.

The last row is the one that changes everything. The model does not write the figure and then have it checked. It writes the narrative with a placeholder, and the number is inserted from the engine's receipt. There is nothing for the model to get wrong, because the number never passes through it.

How a question flows through the system

1. The executive asks a business question in plain language.
2. The model plans over a registry of typed capabilities. It does not write SQL. It fills a grammar through constrained decoding, so it cannot invent a field, a number or a capability name.
3. The engine computes under PostgreSQL row-level security and column masking, through a read-only role that cannot bypass either.
4. The engine mints a receipt for each result: a content-hashed, tamper-evident record of what was computed, over what, and when.
5. The model narrates around those results, using placeholders.
6. A verifier checks every claim. A claim without a valid, matching receipt is blocked before the user sees it.
7. The answer is rendered, with figures taken from the receipts and each one linked to its evidence.

What a receipt needs to hold

Receipts hold references and hashes, not raw rows. That keeps sensitive data out of the evidence store. They persist in an append-only ledger, so a figure in an old conversation can be re-verified, and re-entitled when it is opened. If the person looking at it should no longer see it, the system can respect that.

Composite figures need composite receipts. A total across currencies, for example, cannot be written unless the engine minted that total. A token-level check, which we call a digit net, catches any number the model tries to smuggle into the prose.

Numbers are not the only risk

Once the figures are safe, the failure moves into the words. A model can quote every number correctly and still claim that one thing caused another.

So the evidence rule extends to prose. A claim-strength grammar and a deterministic causal-honesty guard stop the narrative from asserting more than the receipts support. "Consistent with a service issue" is allowed. "Caused by a service issue" needs evidence the data cannot provide.

Ambiguity is the same kind of problem. When a business term has two legitimate readings, the analyst answers the likely one and offers the alternative with its own number, instead of quietly choosing.

What this design does not solve

It helps to be precise about the guarantee. The receipt proves a figure came from the governed engine, computed as recorded. If the source table is wrong, or a metric is defined badly, the system will faithfully report the wrong thing, with a receipt.

So the design moves your trust. You stop trusting the model, and you put that trust in your data and metric definitions, which is where it can be audited. That still takes work. Someone has to own those definitions.

There is also a long tail of questions the certified capabilities will not cover. We handle those with clearly labelled surfaces, such as a gateway for model-drafted SQL and a Discovery mode for uncertified databases. Each is stamped for what it is, and none can mint a governed receipt. Being honest about which answers are certified is better than pretending they all are.

How to prove it works

A claim like "the analyst finds the true signal" needs an answer key, and real business data rarely has one.

So I built Data Invoker, a synthetic-data engine that generates a coherent person first, derives a behaviour contract from them, and deterministically simulates their orders, reviews and complaints. Because the persona causes every row, the tool knows the truth. It ships an answer key with planted segments and churn points, plus deliberate decoys.

A scoring contract then grades the analyst's typed findings against that key. It rewards a correct descriptive claim over a wrong causal one, and it penalises overclaiming. No LLM judge is involved in that grade.

If you take one thing from this section: build evaluation data where you know the answer, and include tempting false patterns. An analyst that finds the real insight but also reports the decoy is not ready.

If you are starting your own

1. Pick ten business questions executives ask, and define the metrics behind them with their owners.
2. Build those as typed, tested operations before adding any model.
3. Add row-level security through a read-only role.
4. Make every operation return a result with a receipt.
5. Only then connect a model, and only to the registry.
6. Render figures from receipts. Block anything without one.
7. Build a seeded dataset with an answer key, and test against it.

Notice how late the model arrives. Most of a trustworthy AI analyst is ordinary, careful data engineering. The model sits on top, doing the two things it is good at, planning and explaining, and it stays away from the two it should not be trusted with, arithmetic and entitlement.

Written by Eslam Hamed, a Technical Lead and AI Solutions Architect working on governed AI systems, agentic AI, and enterprise data.