Release 1

Three AI models forecast which petitions the Supreme Court would take up at its 2026-09-28 conference; on the 97 scored petitions it has decided (91 of them pre-registered), the Court granted 3, two of them sent back for reconsideration, too few to measure a difference between the models.

What this is. Before the Supreme Court’s first conference of the term, three frontier AI models each forecast whether the Court would take up a set of pending cases. Every forecast for that pre-registered set was merged into a public ledger before the Court met, so anyone can check that it came first. Six further petitions in the scores were first forecast only after the conference had met, though before the Court acted on them (see What this does and doesn’t show). This is the first scored result.

Written by AI. This page, like the audit write-up behind it, was written by AI agents (Claude, from Anthropic) under the project maintainer’s direction: they drafted it before the conference and copied in each figure from the audit write-up and the release’s dataset. AI reviewer agents checked the figures, and the maintainer reviewed the page before it was published.

What was predicted, and when

• Cases forecast: 110 petitions distributed for the 2026-09-28 conference when the cohort was registered, plus 10 where the Court had asked for the Solicitor General’s views — the cohort registered before any outcome existed (section 5 of the audit write-up). Of those, 91 of the 110 distributed petitions and none of the 10 Solicitor General petitions had been decided and scored when this page was written.
• Models: claude-baseline ran claude-fable-5-1, codex-baseline ran gpt-6-astra and gemini-baseline ran gemini-3.1-pro-preview, under the same instructions and with the same case materials, though not under identical conditions (see The models did not run side by side below).
• Last forecast for the pre-registered set: merged into main at 2026-09-19 19:45:51 UTC (pull request #1921). The Court acted: 2026-10-01 (one grant) and 2026-10-05 (the order list carrying the other 90 decided pre-registered petitions, and the six late-forecast ones).
• Check it yourself: every forecast is a file in the public repo. The proof of timing is the time GitHub recorded when the forecast’s pull request merged into main, and the prereg/proc-v8 tag that fixed the rules beforehand; a commit’s own date is set by whoever made it and proves nothing. For example, the three models’ forecasts for No. 25-901 arrived in pull request #1845, merged 2026-09-16 18:43:21 UTC; the Court acted on that petition in its 2026-10-05 order list.

These were not all the petitions at the conference. The set was chosen by a pre-registered ranking that favours petitions showing signs of the Court’s interest — being relisted, or a request for the Solicitor General’s views — and petitions brought by the federal government, so it leans toward cases more likely to be granted. It is not a random sample, and the results below say nothing about the conference as a whole.

How it went

Every figure below is from forecasts made before their outcome existed.

Most petitions are denied, so a forecaster that always says “deny” is right most of the time. Each row shows what “always deny” scored on exactly the same petitions, and the lift is the difference, because beating that is the actual test.

The rows are the ranking’s bands. Baseline petitions had not been relisted when they were ranked; elevated petitions had been relisted. Each band has its own historical grant rate, and the skill column is measured against it.

Every petition in both bands carries a scored forecast from all three models: 28 of 28 elevated and 67 of 67 baseline. Among the elevated petitions the Court granted 2, where the band’s historical rate expected about 4.8; among the baseline petitions it granted none, where about 3.4 were expected.

Petitions the Court relisted or held are not scored yet, and relisted petitions are granted more often than others, so the petitions scored so far lean toward denials. That raises what “always deny” scores and lowers each band’s realized grant share below its historical rate, which moves the skill column too, in favour of lower forecasts.

A right call is a forecast whose named outcome matched the Court’s action exactly. So a “grant” call on a petition the Court sent back for reconsideration (a GVR) counts as a miss here, even though sent-back petitions count as grants in the probability scores and the calls below. In this set no model made a plain “grant” call on either GVR: the right calls on them (claude-baseline on both, codex-baseline on one) were forecasts that named a GVR. Skill vs. history compares each model’s probabilities with the band’s historical grant rate: above 0 means the forecasts beat that rate, and a model that just repeated the rate would score 0.

On the 67 baseline petitions the Court denied every one, so every model’s right calls equal “always deny” exactly, and only the skill column differs there: +0.50, +0.12 and −0.33. With every baseline petition denied, that column rewards forecasting low rather than telling grants from denials, and because the likelier-granted relisted and held petitions are still pending, the shortfall currently favours whichever model forecast lowest. On the 28 elevated petitions, one model’s right calls beat “always deny” by 3.6 points, one matched it and one fell 7.1 points below it — one or two petitions either way, out of 28.

Bands with only one petition (high and federal on distribution; the one state petition is still pending) are left out of the table because a single case cannot measure anything; each such petition appears in the calls below whatever its outcome.

Where the Court had asked for the Solicitor General’s views. These petitions are scored on their own and never mixed into the table above. They are read per band like the rest, so a row is one band. None of the 10 has been acted on yet, so none is scored and this table has no rows; they are counted under Still pending below.

The calls

Every probability here is copied from the release’s dataset export. Where a forecast was set aside, its probability is not shown: the cell reads “set aside” with the reason and a link to the ledger. A forecast is set aside when its record claims it was made before an outcome that had in fact already happened, or when every judge that graded it found it may have seen its own outcome. A forecast that some but not all of its judges flagged in that way keeps its probability, marked “flagged by one judge” (or two), and is scored only through the judges that did not flag it. Where a model produced no forecast for a petition, the cell reads “no forecast”.

Petitions the Court granted, in any form — granted, granted in part, sent back for reconsideration (GVR) or summarily reversed — with each model’s forecast probability of a grant:

Petitions are named by docket number, each linked to its ledger event.

Both GVRs sent the case back for reconsideration in light of the same intervening decision, Louisiana v. Callais. The one full grant, No. 25-1131, drew no model’s forecast above 0.30. No petition was sent back only because the case became moot.

The highest grant forecasts among denied petitions: each model’s three highest grant forecasts among the distribution-arm petitions the Court denied, merged into one list, with every petition tied at a model’s third place included. A 30% forecast is expected to be denied most of the time, so these are not errors on their own; they are shown so that a model’s confident misses are as visible as its hits.

A bold figure is one of that model’s top three; gemini-baseline’s third place is a tie at 0.40, so it has four. All seven petitions are in the elevated band and were denied on 2026-10-05. Two of gemini-baseline’s, No. 25-1208 at 0.82 and No. 25-1105 at 0.55, were calls that the petition would be granted.

Petitions left out of the table above: every petition in a band too small to score, and every Solicitor General petition not already listed, with its action and each model’s forecast:

No. 25-901 is also in the granted list above. No Solicitor General petition has been acted on, so none is listed here.

Still pending: 19 petitions on the distribution arm were relisted, held or rescheduled, and 10 petitions where the Court had asked for the Solicitor General’s views have not yet been acted on; none of them is scored yet. They will be scored when the Court acts. No petition in either group has been acted on without its forecasts being graded, and every one of the 120 carries a forecast from all three models.

What this does and doesn’t show

• It’s small. In each band the Court granted only a handful of the scored petitions (the counts are under the table), set against what the band’s historical rate over the eight Terms OT2017–OT2024 (a ten-Term lookback that the records available shorten to eight; OT2017–OT2025 for the seven petitions docketed in OT2026) expected. One or two cases can reorder the models, so any ordering points to a possible difference rather than measuring one.
• It’s a selected set, not the conference and not a random sample.
• Deciding which cases to hear is the warm-up. The project also forecasts how the Court decides the cases it takes — whether it leaves the lower court’s judgment standing — and those forecasts resolve as decisions come out through June 2027. Forecasts of the vote split and of who writes are pre-registered but not yet scored.
• The models did not run side by side. In every run the three models’ forecasts started in a fixed order, so the last model ran later and after the others had used part of a shared research allowance. That may have helped or hurt it, and nothing here corrects for it (Comparing the engines in section 3 of the audit write-up). How often each model’s research requests were turned away for lack of allowance cannot be measured from the logs, so no such figure is given.
• Some petitions asked for something rarer than review. Two of the petitions (Nos. 25-1252 and 25-1315, both baseline band, both denied) asked the Court to order a lower court to act (a writ of mandamus), which it almost never does; they are scored like the others, against the ordinary historical rate (section 5 of the audit write-up). Without them the baseline row covers 65 petitions, the right calls and “always deny” are unchanged at 100%, and the skill column reads +0.49, +0.12 and −0.37 instead of +0.50, +0.12 and −0.33.
• Some forecasts on the wider board came late. Six petitions (Nos. 26-66, 26-79, 26-80, 26-95, 26-121 and 26-183, all baseline band, all denied) were first forecast only on 2026-10-04, after their conference had met but the day before the order list that denied them; they are not among the cases this page lists, but they are in the scores (section 5 of the audit write-up). They are why the table counts 67 baseline petitions where the pre-registered set has 61 decided. Without them the baseline row covers 61 petitions, the right calls and “always deny” are unchanged at 100%, and the skill column reads +0.56, +0.13 and −0.38 instead of +0.50, +0.12 and −0.33; the elevated row does not change.
• The historical rates were worked out by the graders. Each grader worked out and wrote down the band’s historical rate it scored against; across all 873 gradings of the 291 scored forecasts none differs from the exact rate by more than 0.34% of that rate, and skill recomputed against the exact rates moves no row of the table by more than 0.001.
• Two of the three grants were sent back, not taken up. Both GVRs turned on one intervening decision, and the one petition the Court actually agreed to hear drew no model’s forecast above 0.30. So the right calls on granted petitions here measure whether a model spotted a send-back, not whether it spotted a case the Court would hear (section 3 of the audit write-up).
• Nothing was set aside. No forecast on this page was excluded or flagged by any judge as possibly having seen its own outcome.

Go deeper

• Full audit write-up, with every denominator: docs/release-ot2026-long-conference.md at results/ot2026-longconf
• The ledger: fedcourts.ai/ledger
• The exact data behind this page: 10.5281/zenodo.23263986 · the code that produced it: tag results/ot2026-longconf, in 10.5281/zenodo.22966596