Abdullah Akbar Khalid
← All exhibits

Exhibit D · Provenance

AI drafting is cheap. Checking the draft is what costs you an hour.

An investment memo drafted only from source documents, where every figure cites its page, and the tool grades its own provenance before you read a word.

Status
Public
Stack
Python · PDF extraction · offline verifier
Source
github.com/akbar-33/cited-memo

The problem

AI writes a plausible investment memo in seconds. The problem is the next hour, when an associate re-checks every figure against the data room because nobody can tell which numbers came from the accounts and which the model invented.

That verification cost is why most AI drafting tools quietly stop being used around week three. The draft was never the expensive part.

The inversion

The tool drafts from source documents only, cites the exact page behind every figure, and then grades its own output. Two uncited claims surfaced by name means you check two sentences instead of twenty-six.

The self-grade: real output formatHover any line
traceable figures : 24/26 (92%) invalid citations : 0 unsupported claims: 2 - EBITDA margin of 18% is expected to hold through FY26. - Management estimates a further £400k of synergies.

How it operates

Three steps, and the third is the one that matters.

How it operates: drafting is cheap, verification is the productHover any stage
Ingest
Source PDFsthe data room
Extractevery page becomes a numbered passage, tagged document and page
Draft
The model writes from those passages only, and must follow each figure with the citation it came from
Verify
Offline checkerno API key, no model in the loop
Catches three failuresuncited figure · citation to a passage that does not exist · claim with no evidence
Graded memocoverage, invalid citations, unsupported claims

Decisions worth defending

The verifier is not a language model

It runs offline with no API key and no model in the loop. A model asked to grade its own homework can be argued out of a failure; a parser that looks for a citation and checks whether the passage exists cannot. Making verification dumber made it trustworthy.

Numbering the passages is the whole trick

A citation can only be checked if the thing it points at has an address. Turning every page into a numbered passage tagged with its document and page number is what makes the rest mechanically verifiable.

Name the unsupported claims in full

A coverage percentage on its own invites you to shrug. Printing the exact sentences that lack evidence turns a score into a two-minute task.

Three failure modes, caught separately

Verification catches a figure with no citation at all, a citation pointing at a passage that does not exist, and a claim asserted without evidence. They fail differently and are detected differently, which is why a single confidence score would hide more than it reveals.

Back to all exhibits