Canadian towns spent a century publishing engineering and public health reports on paper, and those volumes are now scans in the Internet Archive. Concordance reads them and writes structured records: a value, a place, a year, and a link to the exact page each number came from.
I built it around one rule. When a language model pulls a number out of a 1969 scan, Concordance does not trust that number until it can show where it came from. So every record keeps the sentence its number appeared in, and Concordance goes back to the page and checks that the sentence is really there.
Concordance is an MIT-licensed Python toolkit that reads scanned Canadian government publications and writes structured records. A local language model does the reading, and the toolkit verifies each record against the page before accepting it. For a prose record to be accepted it has to retain an exact quotation from the page it cites, and the verifier then checks that quotation, and the number inside it, against the page itself.
Open Source · MIT Python 3.11+ · zero required third-party runtime dependencies · 1,608 offline tests, all passing offline. The public instance at concordance.jonathancapone.com has read 24 towns of a located 102 so far, producing 6,554 source-linked records. · Source on GitHub
The Core Rule
Model answers are re-read against the page
Concordance saves a number together with its quotation and page reference, then checks that they agree. If the quotation or number cannot be found on the cited page, the record is rejected with a reason. That makes it possible to investigate a result instead of relying on the model's answer alone.
Working
A value must appear as its own token
Asking whether a page "contains" 5 by searching its text is the bug that lets fabrications through: 12 appears inside 3120, 1 inside 10, and 5 inside 53,549.66. The verifier compares complete numeric tokens, so a value has to appear as its own number to count.
Working
A fragment may not drop a sign
A quotation is allowed to be a sentence fragment, but it may not trim away an adjacent character that changes the meaning. 5 is not a faithful fragment of > 5 or 5%, even though the token 5 genuinely occurs there. The check reads what sits immediately either side of the match.
Working
Correct answers still fail
A value reached by arithmetic is refused even when the arithmetic is right, because the cited page never stated it. Letter-for-digit repair is allowed in the other direction: I5 units may resolve to 15, while a transposition or an invented figure still fails.
Abstention
It abstains when the text does not support an answer.
Publication-year proposals are deterministic, and they abstain when the available text does not support a date. A locator-only table contribution has no quotation to check, so it is kept and marked unchecked, because it cannot carry a verdict. Observations, design values, standards, and conclusions stay distinct from one another, and units that look alike are treated as incomparable across eras until shown otherwise.
The Assistant
Jay.
The research assistant is named for the Canada jay, which caches tens of thousands of items across several square kilometres of territory and remembers where it put them months later, under snow. That is the job exactly: the record is scattered through many documents, and the work is knowing where a thing is and going to get it.
Its tool-calling loop and pending-action model are forked from the agent built for OMEGA. What changed is what the tools reach. OMEGA's talk to a live sensor mesh, where a reading needs no further proof. These talk to a century of scanned paper, so every answer has to carry the page and sentence it came from.
Sharing Records
A Receiving Instance Rechecks Before It Merges
Records travel as contribution bundles that carry their sources with them. An instance receiving a bundle re-runs the evidence check on supported prose before merging anything, so no instance accepts a record on another's word. A bundle can be inspected offline, without contacting any provider.
Working
Offline by default
Checking stored evidence needs no API key and no model call. Local extraction runs against Ollama with a downloaded model; the optional Anthropic backend is used only when a caller selects it and supplies their own key.
Working
1,608 tests, no network
The suite runs offline against constructed records and temporary directories. Adversarial cases cover fabricated decimals, path traversal, unsafe identifiers, duplicate imports, and files interrupted partway through publication.
Working
Attributed gazetteer
Ontario place resolution uses a gazetteer derived from the Canadian Geographical Names Database, shipped with its source, transformation, checksum, and Open Government Licence – Canada attribution.
The Instance
What Has Been Read So Far.
102 title-derived place series covering 1961 to 1974. Orange means the scans have been
read; grey means the place is located and the reports survive, but nothing has been
extracted yet. Dot size is the number of surviving reports, so the map shows the size of
the remaining work as plainly as it shows the progress.
Accuracy
Scored Against Pages a Person Read First.
The live instance publishes its own scores, and publishes them including the misses. On 68 values a person had already read by hand: 96.8% precision (60 of 62 kept), 88.2% recall (60 of 68 present), 98.3% on telling a measurement from a design spec or a regulatory limit, and 88.9% on separating influent from effluent, where getting it backwards turns a working plant into a polluting one.
The sample is small, 4 pages across 2 documents, and the site says so on the same screen as the numbers. It also reports that it missed 8 values a person found and produced 2 the answer key does not have, so a reader can weigh the score against what it missed.
The first number was wrong, and the fault was in the scorer
The first scored run reported 49% precision. Auditing the records it had called wrong showed nearly all of them were right: the scorer could not convert "3.0 million gallons" to "3000000 gallons", and the hand-written ground truth was incomplete. Fixing the measurement, with no change at all to the extractor, moved precision from 49.1% to 96.8%. Publishing the first figure would have narrowed the project for no reason.
Known Gaps
What It Does Not Claim.
Experimental
Table extraction
A page, row, and column label does not by itself prove that a value came from a particular cell. Concordance preserves locator-only table contributions, and deliberately declines to mark them verified.
Small sample
A small sample so far
The public instance reads a corpus that is still small: 24 towns read of 102 located, and the accuracy figures come from 68 values across 4 pages of 2 documents. Those numbers describe the sample, and the site labels them that way. The repository ships the software, the offline tests, the interface assets, the attributed gazetteer, and provider declarations. It leaves out Internet Archive OCR, extracted measurements, hand-read gold data, derived vocabulary, and generated pages, because public access to the source material does not give me the right to relicense it.
