I tried to beat a prediction market with a forecasting model. It couldn’t. But the way it fell short taught me the difference between a signal and an edge.
Kalshi runs weekly markets on how many people the TSA screens at US airports. I picked them because the data is public and published daily, the answer is a single number, and the markets settle every week.
I built a forecaster from seven years of TSA data and tested it the way it would have run live, using only what was known at each moment, against every settled market since mid-2024.
On its own, the model couldn’t beat the market
Over 2025 and 2026, the two were roughly tied. My model was slightly better on Monday, before any data came in. From Tuesday on, once the first daily numbers were out, the market was better.
Trading the model by itself made $9 across 702 simulated trades. That’s statistically zero, and it disappeared if prices came in a cent or two worse than quoted. A model can look good on paper and still not beat the crowd. The only test that counts is the price.
My model alone vs. the market, by dayError compared with the market price (Brier score), 2025–26. Above the line, the model was better; below, the market was.
{{b.label}}
{{b.label}}
{{b.day}}
But it wasn’t useless
If my model only knew what the market already knew, mixing it in would just add noise. I tested that: I took the market’s price and nudged it toward my model’s forecast (65% market, 35% model). The result was more accurate than the market alone, so the model was adding something.
To make sure I wasn’t fooling myself, I picked the 65/35 mix using 2025 data, then checked it on 2026, which the model had never seen.
Model + market mix vs. the market alone, by dayLower error than the market price alone (Brier score), 2026, unseen data.
{{b.label}}
{{b.day}}
Almost all of the gain came early in the week: about 11% lower error on Monday and Tuesday, down to 2–3% by Friday.
My best guess at why: early in the week there’s no new data yet, so the market has less to go on. A forecast built on history, like holidays and the same weeks in past years, fills that gap. Once daily numbers arrive, the market has real information and the model stops adding anything. That’s a likely explanation, not a proven one, and 37 weeks is a small sample, so treat the exact numbers as rough.
A signal, not yet an edge
A signal means the model knows something useful. An edge means you can reliably make money from it.
To check, I simulated trading the mix on 2026 prices, only when it disagreed with the price and only when spreads were tight. It made 108 trades and $9.28, a 40% return on the money at risk, and still 34% with prices a cent worse. That looks like an edge, but it isn’t proven. Over 37 weeks, the result could still be luck. And even if it’s real, these markets are thin, so the dollars stay small.
That’s why there’s no real money in it yet. The next step is paper trading it going forward to see if it holds.
That’s the lesson I’d take into any market. Beating the price is the first test, not the last. A signal becomes an edge only when it survives the cost of trading and lasts long enough to rule out luck.
How the forecaster works
It’s deliberately simple. It only knows the calendar and recent travel trends.
01
Look back two yearsEach day still to come is matched to the same day in each of the past two years: a Monday to a Monday. In holiday weeks it matches by distance from the holiday instead, so the Sunday after Thanksgiving lines up with the Sunday after Thanksgiving.
02
Adjust for this yearIt compares the last 14 days of real counts with the same days in those past years. If travel is running 4% higher, the forecast goes up 4%.
03
Fill in the weekThe week’s forecast is the days TSA has already reported plus the forecast for the days left. That’s why it gets sharper as the week goes on.
04
Turn it into oddsKalshi asks yes-or-no questions, such as “Will the weekly average be above 2.5 million?” The model looks at how far off its own past forecasts were at the same point in the week, with holiday weeks tracked separately, and uses that spread to give a probability for each question.
What it can’t see is anything off the calendar. Storms, strikes and outages are where it misses most.