When the Numbers Change: Toward Revision-Consistent AI Reporting
An independent research agenda for AI-generated financial reports that revise affected claims while preserving everything the evidence still supports.
Research Journal 01 · An idea, an architecture, and an experiment in progress
The report was accurate yesterday. What about today?
Imagine an analyst opens a credit committee report on Monday morning. It contains the expected ingredients: exposure by borrower, portfolio trends, a delinquency summary, several charts, and a concise narrative explaining what deserves attention.
Now imagine that on Tuesday a financial statement arrives with a corrected revenue figure.
The original report is no longer fully correct. Yet most of it may still be right. The exposure numbers have not moved. The delinquency rate is unchanged. Some charts are still valid, and several conclusions remain supported by the evidence.
What should an AI reporting system do next?
One obvious answer is to generate the entire report again. But that response hides an uncomfortable possibility: in repairing one fact, the system might quietly change five others. A claim supported on Monday might be paraphrased into something stronger on Tuesday. A previously correct number might disappear. A new conclusion might be introduced without a corresponding change in evidence.
This is a different reliability problem from producing a plausible first draft. It asks whether an AI system can remain faithful to a changing body of evidence across revisions.
That is the question behind my ongoing research direction: revision-consistent AI reporting.
A small correction can have a large footprint
Financial reporting is a useful environment in which to study this problem. Reports often depend on source systems that change after an initial publication: late-arriving entries, corrected statements, recalculated ratios, updated risk classifications, and historical restatements.
A single corrected field can propagate through several layers:
- A source value changes.
- A derived metric or ratio may change.
- A figure, table, or ranking may need updating.
- A written claim may become false or incomplete.
- A recommendation may need to be reassessed.
The difficult part is not simply detecting that a row changed. It is determining which claims are affected, which remain valid, and whether the revised report is internally consistent.
Consider a simplified, entirely synthetic example:
| Report item | Original | Revised source | Expected behavior |
|---|---|---|---|
| Quarterly revenue | $12.4M | $11.6M | Correct revenue claim and any dependent analysis |
| Credit exposure | $48.2M | $48.2M | Preserve exposure claim |
| Delinquency rate | 3.1% | 3.1% | Preserve delinquency claim |
The revenue change might also affect a growth calculation, a threshold-based rating, or a written assessment. Those dependencies cannot be inferred from the changed number alone; the reporting system needs a record of what each published claim was based on.
My working hypothesis
I want to test whether AI reporting improves when revisions are treated as a first-class engineering problem rather than another prompt to rewrite a document.
The proposed principle is simple:
Change everything that the new evidence requires. Preserve everything that the evidence does not invalidate.
A revision-consistent system would need to satisfy at least four properties.
Correctness: Every claim affected by the revision must reflect the latest authorized evidence.
Stability: Claims supported by unchanged evidence should not drift merely because the report was regenerated.
Traceability: Numbers, charts, and narrative claims should be linked to their inputs, calculations, and source versions.
Abstention: When a revised conclusion cannot be established from the available evidence, the system should flag it rather than invent a confident explanation.
Together, these properties connect data engineering, machine learning, report generation, and governance. Data versioning tells us what changed. Dependency tracking suggests what must be reconsidered. AI helps interpret and revise narrative text. Validation determines what may safely be published.
A proposed architecture
Figure 1. Proposed research architecture. This is a design hypothesis, not a validated production system.
The architecture begins with versioned source snapshots. A change detector identifies corrections at the relevant grain: for example, a loan, dealer group, statement period, or individual financial attribute.
A dependency model then connects source fields to derived metrics, charts, and narrative claims. For a changed input, the system builds an impact set. Only affected artifacts are selected for recomputation or reconsideration.
An AI component may revise narrative claims, but it should not be the sole authority for arithmetic or evidence selection. Deterministic calculations, evidence references, and explicit validation rules should constrain the resulting report.
Finally, a validation gate compares the old and new reports: Were all affected claims corrected? Did any unaffected claims change? Are figures and prose aligned? Can each conclusion be traced to supporting evidence?
This is what makes the challenge interesting. A useful system must be able to explain not only why something changed, but why something else did not.
Try the idea yourself
The demonstration below uses three synthetic financial metrics and a deliberately simplified dependency model. Choose a revised input and see which report claims should be updated.
Open the interactive simulator in its own page →
This is not an LLM benchmark. It illustrates the core correctness rule. The research challenge is making the dependency model, AI revisions, and validation robust when reports contain many interdependent metrics and nuanced claims.
The experiment I want to run
The first pilot will use 20 synthetic financial reporting packages, each subjected to four controlled revisions, producing 80 revision tasks. The initial scenarios will cover revenue corrections, delinquency changes, exposure changes, ranking shifts, missing-value updates, and historical corrections.
The goal is not to claim enterprise-level performance from a small pilot. It is to create a reproducible experimental scaffold where the expected impact of every revision is known.
I plan to compare five approaches:
- Full regeneration: regenerate the complete report using revised data.
- Prompt-based editing: ask an LLM to repair the existing report using the change description.
- Retrieval-augmented revision: retrieve relevant source excerpts and revise the report.
- Claim extraction and checking: identify individual claims, verify them, and replace failed ones.
- Dependency-aware revision: track claims to source versions and selectively update affected artifacts.
The evaluation will distinguish several failure modes instead of reporting one general accuracy score.
| Measure | What it asks |
|---|---|
| Affected-claim correction | Did the system fix every claim invalidated by the revision? |
| Unchanged-fact preservation | Did it leave evidence-supported claims intact? |
| Numeric accuracy | Do revised numbers match authoritative calculations? |
| Unsupported-claim rate | Did the new narrative introduce claims without sufficient support? |
| Fully correct report rate | Does the entire revised report pass all required checks? |
I also want to examine practical engineering questions: How much work must be recomputed? What happens when a revision crosses multiple reporting periods? Can we detect missing dependencies? How do failures change when the available evidence is incomplete or contradictory?
A strong outcome would be a method that achieves high correction accuracy and high factual stability at the same time. A useful negative result would identify precisely where selective revision breaks down.
Why I think this matters beyond financial reports
The financial domain makes the stakes and dependencies visible, but the underlying problem is broader. The same principle could apply to regulatory summaries, operational dashboards, healthcare analytics, scientific literature digests, or AI-generated management briefings.
AI systems are increasingly expected to maintain ongoing artifacts, not merely answer isolated questions. Once a system publishes a report and later receives corrected evidence, continuity becomes part of correctness.
That raises a more general research question:
Can we design AI systems that treat factual change as a controlled update to an evidence-backed artifact, rather than an invitation to create a new narrative from scratch?
What comes next
This is the first public entry in an ongoing research journal. The next steps are to specify a claim-and-evidence representation, release a reproducible synthetic-data generator, implement the five baselines, and publish an evaluation protocol before drawing performance conclusions.
The intent is to keep the work open, experimentally grounded, and useful to practitioners building AI-assisted reporting and decision systems.
No model performance results are claimed in this post. All example values are synthetic; the architecture and methodology are research proposals.
Follow the research: Explore Mukala Labs research · Try the revision simulator · Discuss a research collaboration