New model watch
OpenAI has announced the GPT-5.6 family, and it comes with names instead of suffixes: Sol, Terra, and Luna. The community's working read — and it's a read, not a spec sheet — is that this is a three-tier lineup: a flagship, a mid-size workhorse, and a light model. That's the news. Everything else you've seen about it this week is somebody's guess wearing a headline.
We run a live coding leaderboard, which means we spend an unhealthy amount of time separating what a lab has actually shown from what a launch thread implies. So let's do that properly, and then talk about the more useful question: how do you evaluate a brand-new model family the day it lands without getting played?
What is actually confirmed
- The family exists. OpenAI has announced GPT-5.6 with three named models: Sol, Terra, and Luna.
- Three tiers is the shape. The naming strongly suggests flagship / mid / light — the same structure every major lab has converged on, because it maps to how people actually spend money.
- That's it. Genuinely. That is the complete list of confirmed facts as we write this.
What nobody knows yet
- Coding benchmarks. There are no public SWE-bench Verified, Terminal-Bench, or LiveCodeBench numbers for any of the three models. None. If you see a chart, check the fine print — it's either internal, projected, or invented.
- Pricing. Unknown for all three tiers.
- Context window. Unknown.
- Availability. The models are not broadly available. No general API access at time of writing.
- Which tier is which. Even the flagship/mid/light mapping of Sol, Terra, and Luna is the community's inference from the names, not an OpenAI statement.
We're not being coy — this is the honest state of public knowledge. When real numbers exist, they'll be on our leaderboard within days, scored the way we score everything: SWE-bench Verified at 40%, Terminal-Bench at 30%, LiveCodeBench at 30%. Until then, anyone telling you Sol "crushes" anything is doing fan fiction.
Why launch week is the worst week to form an opinion
Every frontier launch follows the same arc. Day one: cherry-picked demos and a benchmark chart with a suspiciously favorable baseline. Day three: influencers declare the previous king dead based on four prompts. Week two: the independent evals land and the picture gets complicated. Week four: everyone quietly settles on "it's great at some things."
The pattern isn't malicious — it's structural. Labs benchmark on what flatters the model. Early access goes to people incentivized to be excited. And coding, specifically, is where marketing numbers diverge hardest from lived reality, because "writes impressive code in a demo" and "resolves a gnarly issue in your repo without breaking two other things" are barely related skills.
There's real context for skepticism and optimism here. GPT-5.5 — the current OpenAI flagship on our board — scores 83.4 on Terminal-Bench, which beats every model we track except Claude Fable 5. Agentic shell work is a genuine OpenAI strength, and if the 5.6 family extends it, that matters. But that's a hypothesis to test, not a result to repeat.
The day-one eval loop we actually use
Here's the thing: you don't need to wait for anyone's benchmarks — ours included — to know whether a new model is good for you. Benchmarks tell you where a model sits in the pack. Only your own work tells you whether it earns a seat at your table. When Sol, Terra, and Luna get API access, this is the loop we'll run, and it's the same loop we run on every release:
- Pick one real task from your actual backlog. Not FizzBuzz, not "build a todo app." A real ticket: a bug with a reproduction, a feature with acceptance criteria, a refactor with a test suite standing guard. The task should be one you understand well enough to grade.
- Swap the new model into your existing harness. Same prompts, same project context, same tools. If you change five variables at once you learn nothing. The model is the only thing that changes.
- Let your test suite be the judge. Not the model's self-report, not your impression of the diff. Did the build pass? Did the tests pass? Did it touch files it had no business touching? A model that claims success is worthless; a model that survives verification is a candidate.
- Compare cost on your workload, not theirs. Per-token pricing is meaningless in isolation. A chatty model at half the price can cost more than a terse one. Measure what the task cost end to end — tokens burned, retries needed, human review time — against whatever currently does that job for you.
- Slot it by role, not by vibes. Maybe Luna turns out to be a brilliant cheap scout and a mediocre builder. Maybe Terra reviews better than it writes. New models rarely deserve your whole workflow on day one; they usually deserve one seat, on probation.
Run that loop and you'll have a defensible opinion about GPT-5.6 within an afternoon of API access — while the timeline is still arguing about a demo video.
The model-agnostic advantage, stated plainly
This is also the strongest practical argument for not welding your workflow to one lab. If your entire setup is built around a single provider's tool, "evaluating the new model" means migrating your workflow first — new CLI, new session format, new muscle memory — and by the time you've done that, you're emotionally invested in the answer. That's how people end up defending tools instead of choosing them.
In a model-agnostic harness, day-one evaluation is a dropdown. The moment OpenAI opens API access to the 5.6 family, you point an agent at it, run the loop above on a branch, and keep your current stack doing the real work in the meantime. If Sol is the new king, you promote it in minutes. If it's not, you've lost an afternoon, not a migration. We've made the longer version of this argument before: the model race is the labs' race. Your race is idea-to-shipped, and it's won by whoever can adopt the winner fastest without betting the workflow on the loser.
What we're watching for
Three specific questions, in order of how much they'd change our recommendations. First: does the flagship challenge Claude Fable 5's 95.0 on SWE-bench Verified — the real-repo-work crown that has stayed firmly in Anthropic's hands? Second: does the family extend OpenAI's Terminal-Bench edge, where GPT-5.5 already outpunches models that beat it everywhere else? Third: where does the light tier land on price — because the budget end of the market, where DeepSeek V4 Pro and GPT-5.3 Codex currently embarrass everyone per dollar, is where most real workloads live.
When the numbers exist, they'll be on the leaderboard and we'll say exactly what they mean — including if the answer is "less than the launch post promised." Until then: names confirmed, tiers probable, everything else unknown. Calibrate accordingly.