Hackathon · Instalily × Google DeepMind · Toronto 26 Research and Forecasting Challenge · 22–30 September 2026
Ground Truth 2026
#7 · 0.7903 / 1.0
- What
- I built forecasters for 10 black-box simulators (power grid, hospital, epidemic, market, wildlife, traffic, reservoir, supply chain, ad auction, social contagion). Each predicts 4,000 steps ahead from one noisy observation and a schedule of interventions, with no feedback.
- Why it matters
- I raised the mean score from 0.655 to 0.790 in six days by rebuilding the models from the data instead of tuning them. The hardest systems gained the most: social contagion 0.34 → 0.66, wildlife 0.51 → 0.72.
- The hackathon
- Instalily × Google DeepMind, Toronto, 22–30 Sep 2026. In this individual research and forecasting challenge, each entrant probes hidden simulators with a metered step budget, models them, and forecasts under schedules never seen in development. The rank is the mean score over the ten systems.
Python · numpy · scipy · system identification · open-loop forecasting · solo entry
Score by system
The hardest systems gained the most. Social contagion 0.34 → 0.66, wildlife 0.51 → 0.72, power grid 0.66 → 0.83. I rebuilt these three models from the records instead of tuning them.
The task
Each simulator exposes a few observables and a few controls and has hidden state and hidden mechanisms. The effect of a control depends on the order and spacing of earlier actions as well as on the current setting. Development queries are metered. At evaluation the forecaster receives one noisy initial observation and a 4,000-step schedule of interventions and must return all 4,000 observations at once. The scorer compares them with the noiseless truth. A missing or crashed system scores zero, and the overall rank is the mean over the ten systems.
How I worked
Hypothesis before experiment. For every system I wrote a model from the brief alone, recorded its prediction, and only then compared it with a 200-step scout run. Where the prediction and the scout run disagreed, I fixed the model's structure instead of tuning a parameter.
Experiments that match the evaluation. I made the probe schedules piecewise-constant within the published bounds, with long holds at intermediate settings. I chose this design because the first leaderboard showed that my models were least accurate where a random setting was held for a long time.
A frozen base plus a learned residual. Once a system's structural model was right, a residual model learned its remaining error from the recorded runs. Each residual was validated outside the fitted interval and checked for drift over 3,000-step holds before any upload.
Every upload audited. A check script tested the output contract, determinism, the absence of state carried between calls, finite outputs at every bound, and the runtime limit in the exact scoring environment. I kept every submitted version, so I could assemble the final set from the best scored version of each system and verify it byte for byte.
I delegated the fitting and the audits to coding agents that worked from written specifications and a decision ledger. The hypotheses, the experiment plan, the step budget and the final selection were my own decisions.
Code
The repository is private for now. It includes the challenge's participant kit and briefs. Ask me if you want to see the models.