Source before summary
Keep the original file, source URL, published label, retrieved time, and content fingerprint.
How to trust the dashboard
Clear enough to use. Rigorous enough to challenge.
200 City-hub sources and 11 outside public feeds are preserved, normalized, connected, and tested in separate scopes. The latest audits passed 32 of 32 checks.
Never force a join
Budget, cash payments, and accounting actuals stay separate unless a record supports the relationship.
Names are not identities
Payee aliases require a disclosed rule; raw published names always remain recoverable.
Reject bad inputs visibly
Date conflicts, duplicate files, broken totals, and schema changes enter a public review queue.
Explain every signal
Pattern rules publish the calculation, denominator, exceptions, and non-allegation language.
Reproducible totals
Every headline can be rebuilt from accepted rows with a documented query and coverage window.
Data path
From city file to public question
Each step creates evidence that can be inspected later.
01DiscoverWatch the city page for new or changed files.
02PreserveStore the original URL, label, time, and fingerprint.
03ParseRead structured workbooks; extract PDFs with page references.
04ValidateCheck dates, totals, schemas, duplicates, and missing fields.
05NormalizeCreate auditable funds, payees, payments, accounts, and documents.
06ExplainPublish metrics, drill paths, limits, and review signals.
Automatic refresh contract
New data can arrive. Unchecked data cannot publish.
The scheduled collector runs every day, with a deeper source refresh each week. Each dataset keeps its own cadence, URL, fingerprint, record count, and coverage boundary.
Daily source checks; weekly full City-file retrieval; monthly and annual feeds only change when their publishers do.
All source adapters finish before a new public snapshot replaces the last trusted one.
City corpus, outside feeds, bond market, and rendered routes must each pass their audit.
Successful refreshes open a dated data change for review; partial or failed runs do not reach deployment.