Feature - Yesterday Audit
Every brief is followed by an audit of the previous brief.
Most research products in this space don’t look back. When yesterday’s call was wrong, it quietly disappears and tomorrow’s gets fresh attention. The reader has no structured way to know whether the service has been right or wrong over time.
We do the opposite. Every brief opens with a Yesterday Audit: per-bucket hit rate, calibration error per prediction type, drift flags, retrospective-vs-live disclosure.
What a populated audit section looks like
Each row pairs the bucket’s mean predicted probability against its actual hit rate. The calibration error is the difference. A small positive error means we were a touch over-confident; a small negative error means we under-priced the move. Errors larger than |0.08| trigger a drift flag on the public dashboard and a note in the next brief’s confidence section.
When the audit window includes retrospective replay
New customers see briefs days after we’ve started publishing them, which means the live-collected outcome log is thin at first. To fill the gap, we replay historical bundles through the same brief generator to produce retrospective predictions, then resolve them against actual bar data.
Retrospective replay is useful as a directional check on the engine’s calibration. It is not the same as a live track record - the bundle was fit on data that overlaps the replayed dates, which can encode hindsight in subtle ways.
When more than half of the resolved predictions in yesterday’s audit window came from retrospective replay, our renderer prepends an explicit disclosure line:
As live outcomes accumulate, the retrospective share drops. You can see the current share on the track record page, and the disclosure line will quietly disappear from your briefs once the live share crosses 50%.