Gemini 4 Pro Leaked: What the Chart Says, and One Prompt to Test It Against Claude on Your Own Work
A leaked chart puts Google's next model ahead of Claude Opus 5.5 and GPT-6 Astra on a top coding test, at about half Claude's price. What it shows, what it doesn't, and the one prompt to paste into all three the day it ships.
A chart posted on X on September 28 shows a model called Gemini 4 Pro, Google's next AI, beating Claude Opus 5.5 and OpenAI's GPT-6 Astra on DeepSWE, a hard coding test: 88.7 against 74.2 and 74.1. The same chart prices it at about half of what Claude costs and gives it a 2 million token context window, twice Claude's.
Google hasn't announced Gemini 4 Pro, and nobody at Google has confirmed the chart. So treat every number below as a leak until the model ships. The post is here: https://x.com/rynorhn/status/2104497579463786839
What the leaked chart says
| Benchmark | Gemini 4 Pro (leak) | Claude Opus 5.5 | GPT-6 Astra |
|---|---|---|---|
| DeepSWE v1.1 (coding) | 88.7 | 74.2 | 74.1 |
| Terminal-Bench 2.1 | 95.3 | about 88 | about 88 |
| OSWorld 2.0 (using a computer) | 86.8 | 81.8 (partial) | 72.6 (offline, partial) |
| GDPval-AA (work tasks) | 2,064 (v2) | 1,846 (v2.1) | 1,542 (v2.1) |
| Context window | 2M tokens | 1M tokens | 1.1M tokens |
| Price per 1M tokens (in / out) | $2.25 / $11.25 | $4 / $20 | $10 / $50 |
Read it the way the chart itself asks you to. On DeepSWE the gap is big: about 15 points. Claude and Astra are 0.1 apart there, which is a tie. The price is 56% of Claude's for both input and output, so about half. Some cells come with notes: OSWorld is marked partial, and the GDPval-AA scores come from different versions of the test (v2 against v2.1), so those two rows don't compare like for like.
Why a benchmark won't pick your model
DeepSWE, Terminal-Bench and the rest are fixed sets of someone else's tasks. They tell you which model is stronger on average. They don't tell you which one is better at your codebase, your writing, your spreadsheets or your way of asking. Two models a few points apart on a chart can swap places on your own work, and a cheaper model that gets your task right is the better deal whatever it scored.
So the day a new model drops, give the same real task to each model you might use and compare the answers. The prompt below turns that into a five-minute test: same task, same context, same rules, and the answers come back in a shape you can score side by side.
The one test prompt
Pick a task you already know the answer to: a bug you fixed last week, a feature you shipped, an email or a report you already wrote. Knowing what good looks like is what makes the scores fair. Fill in the two brackets and paste the whole thing into each model.
I'm testing which AI model is best at my real work. Treat this as a real task, not a demo. THE TASK [One real task from your work, and what "done" looks like. Example: "Users on Safari can't upload a profile photo. Done = the upload works on Safari and Chrome, with a test that proves it."] THE CONTEXT [Paste the files, data or notes a new teammate would need for this task. Nothing else.] HOW TO ANSWER 1. In two sentences, say what you think I'm asking for and what "done" means. List any assumptions you're making. 2. If something truly blocks you, ask me at most 3 questions and stop. Otherwise don't ask anything; carry on. 3. Do the whole task. Give complete files or the full text: no placeholders, no "the rest stays the same". 4. Show me how to check it's right: the tests to run or the steps to follow, and what I should see if it worked. 5. End with a short list of anything you guessed, couldn't check or aren't sure about. Don't invent functions, libraries, settings or facts. If you need something I didn't give you, say so instead of assuming it.
Free, for your email
Get the full setup steps + the copy-paste setup prompt
The rest of this guide (4 more sections, 1 copy-paste prompt) opens right here. The setup prompt, one paste into Claude that sets it all up, comes by email only. Use your real address.
Unlocks: How to run it fairly · Score the answers · For developers: the same test in the terminal · Before you rely on it
50,000+ follow where these guides come from. No spam, reply “unsubscribe” any time.
