Claude 3.5 Sonnet: When the Mid-Tier Beats the Flagship
Claude 3.5 Sonnet, released on June 20, 2024, outperforms Anthropic’s flagship Claude 3 Opus at twice the speed and one fifth of the price. We assess the benchmark data and the falling cost of quality since GPT-4, then explain why Artifacts marks a shift from chat transcripts to inspectable work products — including the limits.
The Flagship Premium Is Collapsing
In March 2023, frontier quality was inseparable from premium pricing. GPT-4 launched at $30 per million input tokens and $60 per million output tokens in its 8K-context variant. Teams that needed the strongest model paid a flagship premium of an order of magnitude. Just fifteen months later, that coupling no longer holds.
On June 20, 2024 Anthropic released Claude 3.5 Sonnet. It is explicitly a mid-tier model, priced like one — and it outperforms Claude 3 Opus, Anthropic's flagship from March 2024, on every benchmark in Anthropic's published table. The relevant question for engineering teams is no longer which model is best. It is which point on the cost-quality curve a given workload actually requires.
What Anthropic Shipped on June 20
The facts first. Claude 3.5 Sonnet is the first release in the Claude 3.5 family. It costs $3 per million input tokens and $15 per million output tokens, has a 200K-token context window, and runs at twice the speed of Claude 3 Opus at one fifth of its price. It is available for free on Claude.ai and the Claude iOS app, and via the Anthropic API, Amazon Bedrock, and Google Cloud Vertex AI.
The pricing is identical to Claude 3 Sonnet from March 2024; Claude Pro and Team subscribers get significantly higher rate limits. Vision improves measurably: 68.3% on MMMU versus 59.4% for Opus. Anthropic has announced Claude 3.5 Haiku and Claude 3.5 Opus for later in 2024, completing the family.
Benchmarks Against Two Generations of Flagships
Anthropic's model card addendum reports 59.4% on GPQA Diamond (0-shot CoT) against 50.4% for Claude 3 Opus, 88.7% on MMLU (5-shot CoT) against 86.8%, and 92.0% on HumanEval against 84.9%. In an internal agentic coding evaluation, Claude 3.5 Sonnet solved 64% of problems; Opus solved 38%. GPT-4o, released five weeks earlier, reports 53.6% on GPQA and 90.2% on HumanEval — between the two Claude generations on most axes.
One caveat belongs next to every such table: these are vendor-reported numbers, prompting regimes differ per cell, and the GPT-4 figures are the March 2023 launch values as compiled in Anthropic's model cards. Benchmarks rank models. They do not predict behavior on your data.
| Model | Released | Input ($/M tokens) | Output ($/M tokens) | GPQA Diamond | HumanEval |
|---|---|---|---|---|---|
| Claude 3.5 Sonnet | Jun 2024 | 3 | 15 | 59.4% | 92.0% |
| GPT-4o | May 2024 | 5 | 15 | 53.6% | 90.2% |
| Claude 3 Opus | Mar 2024 | 15 | 75 | 50.4% | 84.9% |
| GPT-4 (8K) | Mar 2023 | 30 | 60 | 35.7% | 67.0% |
Cost per Quality Is the Metric That Matters
Read the price column against the quality columns. GPT-4 in March 2023: $30 input, $60 output. GPT-4o in May 2024: $5 and $15, half the price of GPT-4 Turbo. Claude 3.5 Sonnet in June 2024: $3 and $15, one fifth of Opus. Within fifteen months the input price for above-GPT-4 quality fell by roughly a factor of ten while benchmark scores rose across the board.
The numbers are concrete. A pipeline processing 100 million input tokens and 20 million output tokens per month cost $4,200 at GPT-4 launch prices. On Claude 3.5 Sonnet the same volume costs $600 — a sevenfold reduction, at higher measured quality.
For system design this changes the default. Model selection becomes a routing decision revisited every quarter, not a platform commitment made once a year. In our projects at Blue IT Systems we now specify the required quality per workload and select the cheapest model that clears the bar — and that model is increasingly a mid-tier one.
Artifacts and the Shift to Working Outputs
The second announcement of June 20 is an interface, not a model. Artifacts is a preview feature on Claude.ai: when Claude generates content such as code snippets, text documents, or website designs, the result appears in a dedicated window next to the conversation. Users see it, edit it, and iterate on it in place.
The important change is the unit of interaction. A chat transcript records the exchange; an artifact is the work product itself. Output no longer has to be copied from a message but becomes an object users can continue to develop. Anthropic describes its longer-term direction in the same terms: teams bringing knowledge, documents, and ongoing work into one shared space where Claude works alongside them.
We read Artifacts as a general pattern rather than a product detail. The same shift applies to internal tooling: generation is cheap, review is the bottleneck, and review needs a surface. A diff, a rendered component, a runnable document — anything inspectable beats prose about code. Interfaces that make outputs inspectable convert model quality into delivered work.
What This Release Does Not Solve
The scoping matters. A 64% score on an internal agentic coding evaluation means roughly one in three tasks still fails; unsupervised end-to-end coding is not what this model delivers. The context window stays at 200K tokens, unchanged since Claude 3. Hallucination is reduced, not eliminated. And vendor benchmarks, however consistent, do not replace evaluations on your own data before production traffic switches models.
Artifacts has limits of its own. It is a preview in one consumer interface. It does not execute arbitrary code, the announced team collaboration and shared knowledge bases have not shipped yet, and API users get none of it. Anyone building products must implement the working-output pattern themselves.
Outlook From June 2024
From where we stand in June 2024, two trajectories look stable. First: Claude 3.5 Haiku and Claude 3.5 Opus are due later this year, and the cost-per-quality curve has been halving on a cadence of months, not years. We expect today's frontier quality to be available at mid-tier prices within roughly a year. The last-year-flagship comparison will become routine.
Second, interface design may matter as much as the models themselves. Once outputs become editable, persistent work products, differentiation moves from benchmark tables to reliable integration with real toolchains — versioned, reviewed, tested, and deployed. That integration is engineering, not prompting. It is where we intend to invest our time over the coming quarters.