xAI's real-time-first model against OpenAI's reasoning workhorse. Both are still selectable in September 2026 — here is what happened when the same three tasks went through each.
by OpenAI
Winnerby xAI
ChatGPT (GPT-5) wins this head-to-head overall. It took two of our three hands-on tasks — a nine-file refactor with a planted shared-state bug (11 of 12 tests passing on the first pass, versus three broken for Grok 4) and a 40-page retrieved-spec test (14 of 14 exception clauses with correct section numbers, versus 11 of 14 and one citation that does not exist). It is also cheaper: $1.25 per million input tokens and $10 per million output against Grok 4's $3 and $15, and ChatGPT Plus is $20/month against $30 for SuperGrok. Grok 4 wins on live data. Its first-party X firehose found a library breaking change published 41 hours earlier, quoted the maintainer's own post, and produced a usable migration diff on the first attempt — something GPT-5 needed two browse passes and a nudge to reach.
On September 17, 2026 we ran the identical three-task set on ChatGPT GPT-5 (Plus account, thinking enabled) and Grok 4 (grok.com on a paid SuperGrok account), in fresh sessions with no memory and no uploaded files. We pay for both accounts ourselves. Task 3's 40-page document was pasted inline in both sessions so retrieval was tested, not search.
| Metric | ChatGPT (GPT-5) | Grok 4 |
|---|---|---|
| Live breaking change found (72h window) | Yes, after 2 browse passes | Yes, one attempt ✓ |
| Release note “understatement” caught unprompted | Yes ✓ | Yes (from maintainer’s post) |
| Time to a usable migration diff | 4 min 10 s | 1 min 38 s ✓ |
| Refactor: public API unchanged | Yes ✓ | Yes |
| Refactor: tests passing first pass | 11 of 12 ✓ | 9 of 12 (3 regressions) |
| Refactor: shared mutable state fixed | Yes, per-call context ✓ | Partially |
| Spec exception clauses found (of 14) | 14 of 14 ✓ | 11 of 14 |
| Invented a citation | No ✓ | Yes (§4.7 does not exist) |
| Explained where the exception does NOT apply | Yes, unprompted ✓ | Only when asked |
| Winner | 🏆 ChatGPT GPT-5 (2 of 3 tasks) | Grok 4 (live-data task) |
Side-by-side breakdown across key categories
| Feature | ChatGPT (GPT-5) | Grok 4 | Winner |
|---|---|---|---|
| Context window | 400K tokens | 256K tokens | ChatGPT (GPT-5) |
| First-party real-time social data | Via search / browsing only | Native X firehose | Grok 4 |
| Live breaking-change latency (our test) | Same day, 2 browse passes | 41 hours after publication, first pass | Grok 4 |
| AIME 2025, no tools (vendor-published) | 94.6% (OpenAI) | 91.7% (xAI) | ChatGPT (GPT-5) |
| GPQA Diamond (vendor-published) | 85.7% (OpenAI) | 87.5% (xAI) | Grok 4 |
| SWE-bench Verified (vendor-published) | 74.9% (OpenAI) | 72.0% (75.0% for Grok 4 Heavy) | ChatGPT (GPT-5) |
| Humanity’s Last Exam (vendor-published) | 24.8%, 42.0% on GPT-5 Pro | 25.4%, 26.9% for Grok 4 Heavy with tools | ChatGPT (GPT-5) |
| Our 9-file refactor, tests passed first pass | 11 of 12 | 9 of 12 | ChatGPT (GPT-5) |
| Our 40-page spec retrieval | 14 of 14, zero invented citations | 11 of 14, one invented citation | ChatGPT (GPT-5) |
| API price per 1M tokens (in / out) | $1.25 / $10.00 | $3.00 / $15.00 | ChatGPT (GPT-5) |
| Cheapest paid consumer tier | $20/mo Plus | $30/mo SuperGrok | ChatGPT (GPT-5) |
| Agent / coding toolchain | Codex, CLI, repo connectors | API and MCP integrations, thinner tooling | ChatGPT (GPT-5) |
| Conversational tone | Structured, more cautious | Looser, fewer refusals | Grok 4 |
| Tier | ChatGPT (GPT-5) | Grok 4 |
|---|---|---|
| Free | Yes — GPT-5 available with daily caps | Yes — x.com and grok.com with message limits |
| Entry paid | $20/mo Plus — GPT-5 plus the newer models in the picker | $30/mo SuperGrok — higher limits and heavier modes |
| Top consumer tier | $200/mo Pro | ~$40/mo X Premium+ bundles Grok with the platform; SuperGrok Heavy is a separate premium tier |
| API (per 1M tokens) | $1.25 in / $10.00 out | $3.00 in / $15.00 out |
| Cheapest way to use it | Free tier | Free tier on X, or SuperGrok at $30/mo |
All figures are the vendors’ published consumer and API rates as checked on the openai.com and x.ai pricing pages in September 2026; they move often, so confirm on the vendor’s own page before you buy. At the API layer GPT-5 costs roughly 42% of Grok 4 per input token and two-thirds per output token. In the consumer apps the gap narrows to $10 a month but never closes. Our method is documented on the How We Test page.
This is a straightforward split: pick ChatGPT for the work you do every day, and Grok for the work that depends on what happened this week. Neither is a bad model — they are differently specialised, and the price gap makes the general-purpose case one-sided.
Two of three tasks won, zero invented citations, the better published coding score, and a cheaper price on both the subscription and the API. For coding, document work and anything you do daily, this is the rational default.
Grok found a 41-hour-old breaking change from the maintainer’s own post, in one attempt, and beat GPT-5 by 2.5x to a working migration diff. If your answer depends on the last 48 hours, its first-party X feed is a real structural advantage.
$1.25/$10 per 1M tokens against $3/$15, and $20/month against $30. For high-volume batch work the per-token gap dominates the total cost long before model quality becomes the deciding factor.
400K context against 256K, and a clean 14-of-14 on our 40-page retrieval test where Grok missed three clauses and invented a section number. Retrieval accuracy, not raw context size, is what decided it.
ChatGPT GPT-5 won two of three tasks, costs less per token, and gives you 400K of context. Grok 4 is worth a look if live X data matters to your work. Both have free tiers, so you can settle this yourself in an afternoon.
Still deciding? Check out these related comparisons and best-of guides.