Which is not to say they’re never useful — I’ve long found them to be a valuable data point when analysing everything from outcomes of foreign elections, to energy production statistics.

Yet, so often, when I check prediction markets for an event I’m interested in, no market exists, or the one that does has fairly low volume, leading me to wonder how reliable the price is as a signal.

Prediction market proponents have long proffered expansive visions for what they could one day offer to society — from “info finance” for everything, to a governance device for society. Yet, years after prediction markets have entered the mainstream, we’re still a ways away from fully realising any of these visions.

So rather than relitigating the grand visions, I set out to ask three narrower questions:

• Are prediction markets actually accurate on the questions a policymaker, investor or planner might act on?

• Under what conditions is a given prediction market estimate likely to be accurate, and when should it be discounted?

• What are the factors constraining the creation of useful prediction markets, and what might it take to create more?

Are prediction markets really accurate?

If a market predicts an event has a 70% probability of happening, and it happens, how do we tell whether the market was right? For a lone event, we basically can’t. But if we take a group of, say, 100 events that each have a 70% probability, we should find that they occur approximately 70% of the time. Do this for each point from 0 to 100% probability, and you can produce a “calibration chart” like this, from a recent study released by prediction market operator Kalshi:

At a glance these charts, drawn from ~2 million Kalshi markets, seem to show reasonably good calibration. Most probability buckets actually do occur about as often as predicted, with the calibration unsurprisingly becoming more accurate as the event grows nearer (the 45-degree dotted line shows perfect calibration — note how much closer the market outcomes grow from three months out from the event, top left, to one day before the event, bottom right).

But we can put a specific number on this rather than just eyeballing a chart.

For every prediction, we take the difference between the predicted probability (say, 70% = 0.7) and the binary outcome (1 or 0 — depending on whether the event happened or didn’t), square it, and average the result across the full cohort of predictions. This gives us a “Brier score” — a number between 0 and 1, where 0 is perfect accuracy (always assigning 100% probability to the correct outcome) and 1 is perfect inaccuracy (always assigning 100% to the wrong outcome).

The same Kalshi study shows us that three months out, Kalshi markets earn a Brier score of 0.087, and these scores trend lower closer to the resolution date.

With this information, we can compare Kalshi markets to other forecasting methods. A good yardstick comes from the Good Judgment Project (GJP), a research program whose volunteer forecasters won the US Intelligence Advanced Research Projects Activity’s geopolitical forecasting tournaments, scoring a Brier of 0.130.1 Within that cohort, an elite group of “superforecasters,” who reportedly outperformed intelligence analysts with access to classified information, averaged 0.083. At face value, Kalshi performs comparably: better than the ordinary forecasters at every horizon, and comparably to the superforecasters at all horizons. (This result flatters the Kalshi markets somewhat in that most of the GJP questions had longer time horizons; we’ll dig further into this below.)

So prediction markets seem reasonably accurate in aggregate. But these stats include all kinds of markets — sports, elections, cryptocurrency, finance. How accurate are they on questions that can inform real-world decisions?

But are they accurate for useful real-world questions?

Not all prediction markets are useful for informing real-world decisions. Both Kalshi and Polymarket have started leaning heavily into sports, short-term crypto markets (eg. “Will Ethereum be above $x,xxx on y date?”), and mention markets (eg. “How many times will Trump say ‘Iran’ at his next press conference?”). These are great for attracting traders to the platform but are clearly less useful than, say, knowing the probability of an election outcome, or of the FDA approving a particular drug.

We can isolate the “real world useful” markets fairly easily by filtering by category. I used the Kalshi API to pull every market between July 2021 and August 2026, and filtered for the following categories: economics, politics, elections, climate, commodities, science and technology, plus a handful of tiny legacy categories. Then I further filtered it for markets that have a resolved yes/no outcome, and were designed to run for at least 30 days — long enough that a forecast is worth having — leaving us with a cohort of 27,016 markets, almost all of which are real-world decision-relevant.2

As a first heuristic, we can re-create the calibration charts from the Kalshi study for our filtered subset. Unsurprisingly forecasts get better calibrated as the event they forecast draws closer — the 180-day series is noisy and generally further away from the perfect calibration line, whereas the 14-day series is much closer.

Looks okay so far. Next, comparing the Brier scores, we find that even among our filtered subset of markets, Kalshi probabilities are competitive with expert human forecasters, even at far-out time horizons like 180 days from the event. The red line shows our full filtered cohort, and the blue line shows a fixed subset of our cohort, which only includes markets for which there is data at all time horizons.

We can also see these trends mostly hold by category. The three categories with fewer markets — climate, science and tech, and commodities — are much noisier.

But is Brier score truly a robust measure of accuracy? One criticism of the original Kalshi study points out that if you know how frequently markets resolve “yes” — called the “base rate” — you can improve your Brier by just naively guessing the base rate percentage. For instance, if I knew that Kalshi markets resolved “yes” 20% of the time historically, I could just guess 20% probability for all markets and my Brier score would improve from 0.25 to 0.16 without any skill on my part.

There is a simple way to check this. By finding the base rate for our cohort of markets, we can then measure how much of the improvement in Brier score is over and above just guessing the base rate. For our filtered cohort at 30 days before market resolution, the base rate is 29.9%, and naively guessing 29.9% for every event would give us a Brier score of 0.210. But at 30 days, the Brier score for our filtered cohort is 0.087 — a clear improvement over the base rate.

We can formally quantify this as the “Brier skill score,” or BSS, which measures the share of the naive base-rate forecaster’s Brier score that the forecast eliminates. In other words, if a forecaster just guesses the base rate, they will improve their Brier score but will receive a BSS of zero. A BSS above zero demonstrates actual forecasting skill above and beyond base-rate guessing.

And indeed, we can see that Kalshi markets outperform base-rate guessing. Consistent with Brier score observations, they increase in skill as the market’s resolution time grows nearer.

So, we know in aggregate that the probabilities predicted by Kalshi have real signal beyond naive guessing, and that they get more accurate closer to the event. But what causes markets to become more or less accurate? And most importantly, when should we trust them as an input for a real-world decision?

What makes prediction markets accurate?

It’s worth taking a moment to consider: why should we expect markets to produce accurate predictions at all? The literature proposes many different explanations, but the main mechanism relevant to our purposes is dispersed knowledge (see Hayek).

The key concept here is that many different people possess knowledge that might help value an asset. A market lets people trade on whatever fragment of information they hold, which causes the price to converge toward a figure which “prices in” these disparate data points. This is relevant to everything from pricing a stock (whose value is ultimately the net present value of the cash flows accruing to its owner), to a commodity futures contract, to a prediction market token (the price of which is the market-implied probability of the event happening).

For example, a classic 1984 study found that orange juice futures prices contained information about impending Florida weather that the National Weather Service’s own forecasts hadn’t yet captured — price moves predicted the errors in the official forecast. No one trader possessed a clairvoyant understanding of the weather — growers, distributors, and locals each held fragments (frost sightings, grove conditions) and by buying or selling futures contracts, they moved the marginal price of those contracts, causing the market to aggregate what they collectively knew.

So if this explanation holds, we should expect that the more traders who participate in a market, the more “information” aggregated by that market, and therefore the more accurate it will be. While the Kalshi API does not provide us a way to extract the number of traders participating in a market3, we can use total trading volume as a rough proxy.

And indeed, this mostly holds true. Per the charts below, which reflect our “real-world decision-relevant” subset, increased trading volume correlates with lower Brier scores and higher Brier skill scores. The effect becomes more pronounced as markets get closer to their resolution date: markets with ≥$100k trading volume marginally outperform lower-volume markets 180 days before the event, and this effect grows more pronounced as the event draws nearer. But most notably, the subset of markets with ≥$100k trading volume are very accurate, with 0.04–0.08 Brier scores. This result handily beats GJP’s ordinary forecasters at all time horizons up to 180 days and even edges out the superforecasters.

Another approach is to look at a market’s bid-ask spread, which is the gap between the highest price anyone is offering to buy at and the lowest price anyone is offering to sell at. A market might have buyers at 40c and sellers at 50c; the 10c gap is the spread. This reflects how confidently traders will commit to a number: anyone quoting a price risks being picked off by a trader who is better informed than them, so the less confident participants are about the true probability, the more room they leave themselves. A spread of a few cents or less is the market saying it has settled on a number, whereas a wider one indicates lower certainty. Typically markets with lower trading volumes will also have wider spreads; fewer traders are quoting trades.

If we run the numbers on our real-world-useful market subset, we can see that low bid-ask spreads are associated with substantially better Brier scores — and markets with <5c spreads outperform GJP’s superforecasters even at a 180-day time horizon!

In fact, if we calculate the R² — a measure of how much predictive power one variable has over another — we find that bid-ask spread correlates with better Brier scores and BSS far more than any other variable examined.4

Of course, there is a potential self-selection effect here: markets where the probability is “easier” to predict will more easily attract traders who are willing to provide liquidity. Therefore, markets that address harder or more uncertain questions will attract fewer traders/quotes, so this low-spread cohort is probably disproportionately made up of easier questions.

Nevertheless, it offers us a useful heuristic when looking at prediction markets: in general, the lower the bid-ask spread a market has, the more likely it is to be well-calibrated.

The missing markets problem

So we have our answer: if you want an independent estimate of the probability of an event, check a prediction market, adjust your confidence level based on the spread and amount of trading volume, and there you have it!

This works great, but for the fact that many potentially useful prediction markets do not exist.

In a sense this is inevitable, given the number of hypothetically-useful-to-someone markets is practically infinite. Every market added to Kalshi or Polymarket is an opportunity to generate fees from trading volumes and attract or retain new users; but on the other hand, each new market carries some risk of resolution disputes or reputational damage. Kalshi has to self-certify every new market it creates, including resolution criteria, with the US Commodity Futures Trading Commission, and Polymarket uses a blockchain-based resolution system which has not been without issues.

Looked at this way, it’s not surprising that both Polymarket and Kalshi have been heavily focussed on sports betting markets lately — the enormous potential trading volumes offer the best tradeoffs between these risk and reward elements. And Kalshi’s trading volumes tell the story — in the past 18 months, sports has swiftly gone from zero to Kalshi’s biggest category by far.

As for the cohort of real-world decision-relevant markets we care about, one way of looking at it would be to imagine that any event has an “addressable market” — the traders, or trading volume, that would show up were it to exist.

We might imagine that say, the upcoming US midterms, the number of SpaceX launches, or the release date of the next Anthropic model, would have plenty of traders interested in participating in related markets. But more obscure topics like how much electricity generation capacity China will add in 2026, or Armenia’s application to join the European Union, might have thinner trader interest. If I were a prediction market seeking to maximise trading volumes, one assumes I’d be trying to add the highest-addressable-market events first, then move down my list.

Yet this seems only partly the case in practice. For instance, who the hell asked for a market on whether the word “flywheel” or “frozen” will be mentioned in the Q2 2026 earnings call for ConAgra Brands, a large but relatively obscure consumer packaged goods company?

One reason for this is it’s easy to resolve these markets at scale, with minimal ambiguity. Simply run a program that pulls the earnings transcript, check for the word, and bang, done. What’s more, earnings calls recur every quarter, so Polymarket can just automatically create these markets on a rolling basis.

By contrast, a question like “Will Armenia formally apply for EU membership before 1 January 2028?” sounds simple, but actually creates a quagmire of ambiguity requiring very carefully written resolution criteria. In this case, there is no exact definition of at what point a state counts as having “applied” for membership, so you have to carefully define what document or process would count as an application. Then, what information source counts as confirmation Armenia has applied? Do you count press releases from the EU or the Armenian government that it has applied? What about the 2025 law Armenia’s parliament already passed which started the process of applying? And what happens if Armenia lodges some kind of application paperwork with the EU before 1 January 2028, but it’s only officially announced after the deadline?

These kinds of questions have tripped up markets many times, and require a non-trivial amount of effort to try to enumerate all the different possibilities in the market’s resolution criteria. All this is to say, these questions are much harder to add en masse than whether the ConAgra CEO will say the word “flywheel.”

We know from the above results that markets with even ~$1,000 of volume were, in aggregate, useful. But $1,000 — of which the prediction market earns a small percentage in fees — may not be enough for a market operator to deem it commercially worthwhile given the effort and risk factors mentioned above.

If we were to make a rough heuristic model of a market operator’s commercial incentives to create a given prediction market, it might look something like: marginal likelihood of market creation = trading volume potential × resolution simplicity.5

The ConAgra earnings market has fairly low trading volume potential, but the very simple resolution criterion probably means that those volumes are worth the (very small) marginal effort. Plus, it can be mechanically repeated every quarter. By contrast, our Armenia example probably has ~moderate volume potential (more than ConAgra but less than, say, the 2028 US presidential election), but is a one-off and carries a lot of resolution complexity.

Toward mass creation of useful markets?

So if we want to see more of these useful markets, the forces that might in theory justify their creation would be a) evidence that they can generate trading volumes, and b) reduced marginal risk and effort for market creation and outcome resolution.

On the latter, there are a few forces that could reduce the friction of market resolution. First, some of the issues highlighted above in our Armenia example could be sidestepped just by reformulating the question slightly. “Will the EU announce receipt of an Armenian accession application before January 1, 2028?” changes it from an ambiguous question about whether Armenia has applied, to a concrete question about whether a specific actor says so in a statement.

Second, AI should drive down the marginal cost of generating new markets by suggesting question formulations and predicting edge cases or resolution issues. I don’t know to what extent market operators are already using AI to generate markets, but the current generation of AI models is clearly capable of doing this job very competently, probably still with human review given the regulatory and reputational issues at play.

And it’s this issue — human review due to regulatory and reputational risk — that I suspect is actually the binding constraint on new market generation. Any market operator that wants to attract maximum trading volume has to be available to the widest array of traders possible, which means not falling totally afoul of various regulators, and not destroying their reputation by flippantly creating markets that create resolution issues for them down the road.

So if human review bandwidth and risk appetite pose the binding constraint on market creation, then market operators will focus their limited attention on the largest revenue opportunities. Which brings us right back to item a) above: demonstrating the trading volume potential of real-world-useful markets.

Is market trading volume elastic?

One way to address this question is to recall the addressable-market framing above. That is, there is a pre-existing number of traders who have a view on, say, the speed of China’s electrical grid build-out and might trade in the market if it were launched.

By this view, every new market is a data point which helps a market operator discover what the level of demand might be for a given topic. If my market for China’s electrical grid growth in 2026 attracts very few traders, I might not bother creating a new one for 2027. Conversely, if it attracts big volumes, I’ll definitely create another one for 2027, and maybe some other related markets in addition.

But I don’t think that viewing trading volume as a largely static number to be discovered is fully correct. In fact, I think that traders look for markets to trade, then decide to commit capital based on whether a) they can develop a confident view that a market is mispriced, and b) whether the amount of available liquidity makes the financial reward large enough.

There is some anecdotal evidence for this. For example, a parliamentary election in the Cook Islands, one of the world’s smallest states with a population of ~15,000, saw $73k in volume, and this sub-national election in Nigeria saw $90k in volume. I don’t think these markets were created with the expectation of being big money spinners, and yet they attracted enough trading volume to generate useful information.

Funnily enough, even the ConAgra market above generated several thousand dollars in trading volumes for some words — enough for the probability estimate to theoretically be meaningful. I doubt anyone got out of bed one morning and actively looked for a market to bet on whether the ConAgra CEO would say “omnichannel.”

And there’s one more group, beyond human traders, that I think these markets could increasingly attract...

Enter: AI forecasting agents

The best human forecasters aren’t clairvoyant — they follow a disciplined, structured process that can be learned. And if a reasonably smart human can learn to follow the process and produce good forecasts, why can’t a reasonably smart AI model?

This is the central premise behind an increasing number of AI forecasting tools, which scaffold an AI model through the superforecasting process. It’s early, but there are promising signs, if partly self-reported, about their accuracy, and these tools should continue to improve over time.

So, if AI tools let us produce good (or at least good-enough) forecasts for a range of events quickly and at scale, this should effectively lower the bar for the marginal trader to enter a particular market. Indeed, these AI forecasting tools are actively touting their records in finding profitable prediction market trades.

For now, these tools are either access-limited or fairly expensive (my rocket question above seems to have cost $5–6). But with the cost per unit of intelligence continuing to fall fast, the direction of travel is clear: decent AI forecasts will be get cheaper and more accessible. This means the number of traders who might take action on the basis of an AI forecast will grow. Net effect: more liquidity — the very thing we know correlates with more accurate markets.

An objection to this view might be that if these AI tools are really so accurate, couldn’t I just cut out the market and generate the AI estimate myself?

I think the market still generates much richer data than what even a very good AI may estimate on its own. Much like how prediction markets are accurate in aggregate but can vary in any individual case, this is almost certainly also the case for AI forecasting tools. Markets will reflect not just a single estimate, but the marginal price produced by many traders, some of whom might rely on various different AI estimates. This means that the market price still reflects a larger diversity of information than a single AI forecast.

Where to from here?

The question of whether prediction markets produce accurate forecasts, at least for events inside 180 days with sufficient volume and liquidity, is largely settled.

The question that remains open, though, is whether prediction market operators will truly embrace the “information markets for everything” concept that could make them a powerful tool for aiding forecasting across a whole range of domains.

This is possible. The explosion in sports prediction markets has drawn significant regulatory scrutiny. Perhaps increased regulation harms sports revenues and forces prediction market operators to turn elsewhere for trading volumes. And AI is attacking the problem from both the demand and supply sides — supplying traders with forecasts they can use to trade, and making it easier for prediction markets to create new markets and resolve existing ones.

But equally, the explosion in sports markets could continue, and decision-relevant markets remain a rounding error, leaving minimal financial incentive for prediction market operators to add them at scale. In this world, a host of other factors come to the fore, including the role of whales and market makers in providing trading volumes — but that’s a topic for another day.

For now, the next time you look at a prediction market: the price tells you what the market thinks, and the spread and trading volume tell you how much to believe it.