25 May 2025 AI Engineering5 min read

Claude 4 and Long-Horizon Coding

Claude Opus 4 and Claude Sonnet 4, released May 22, 2025, extend the usable coding horizon from minutes to hours: 72.5% on SWE-bench Verified, extended thinking with tool use, memory files, and a seven-hour autonomous refactor at Rakuten. We assess what changes for engineering teams — and where human review remains indispensable.

The Coherence Ceiling

AI coding assistants have had a hard limit that never appeared in benchmark tables: coherence over time. Anthropic's Alex Albert put a number on it — earlier Claude models produced useful, self-referencing output for roughly one to two hours before error rates climbed. A bounded bug fix fits inside that window. A cross-cutting refactor, a migration, or a feature spanning many files does not.

That ceiling defined how teams used these models: as autocomplete or pair programmers for minutes at a time, not for coherent work spanning a day. Long-horizon coding — tasks with thousands of dependent steps and continuously evolving state — therefore remained exclusively human. Claude 4 targets that constraint directly.

A request arrives — the router classifies its difficulty. 1/4

What Anthropic Shipped on May 22

On May 22, 2025, at its first developer conference, Anthropic released Claude Opus 4 and Claude Sonnet 4. Both are hybrid reasoning models with two modes: near-instant responses and extended thinking. Pricing is unchanged from the previous generation — Opus 4 at $15/$75 per million input/output tokens, Sonnet 4 at $3/$15 — and both models are available on the Anthropic API, Amazon Bedrock, and Google Cloud Vertex AI.

The tooling shipped with the models. Claude Code moved from preview to general availability with VS Code and JetBrains integrations, a GitHub Actions beta, and an SDK for building custom agents. GitHub announced Sonnet 4 as the base model for Copilot's new coding agent. The positioning is explicit: models for agents that work, not chatbots that answer.

Reading the Benchmarks

Opus 4 reports 72.5% on SWE-bench Verified and 43.2% on Terminal-bench; Sonnet 4 reaches 72.7% on SWE-bench Verified. The scaffold is deliberately minimal — a bash tool and a string-replacement file editor, scored across all 500 problems. With parallel test-time compute and best-of-N selection the scores rise to 79.4% and 80.2%.

What the numbers do not say: SWE-bench issues are bounded, single-repository tasks with existing reference tests. The benchmark measures neither eight-hour horizons nor ambiguous requirements nor the cost of a subtly wrong merge. Read the scores as a maturity indicator for tool-driven editing, not as proof of autonomy.

	Claude Opus 4	Claude Sonnet 4
SWE-bench Verified	72.5%	72.7%
SWE-bench Verified (parallel test-time compute)	79.4%	80.2%
Price per 1M tokens (input / output)	$15 / $75	$3 / $15
Deployment standard (system card)	ASL-3	ASL-2

Extended Thinking with Tool Use

The most consequential API change is extended thinking with tool use, shipped in beta. Both models can call tools — web search, code execution, systems connected via the Model Context Protocol — during the reasoning phase, alternating between thinking and acting instead of committing to a front-loaded plan. Combined with parallel tool execution, an agent can gather evidence mid-thought rather than guessing.

Two mechanisms support long tasks. Memory files: given local file access, the models create and maintain their own notes to hold state across hours. Thinking summaries: a smaller model condenses long reasoning traces; Anthropic states only about 5% of traces are long enough to trigger summarization, and a Developer Mode retains full traces for those who need them.

Seven Hours of Autonomous Work

The launch's headline validation comes from Rakuten, which ran Opus 4 on a demanding open-source refactor for seven hours, independently and with sustained performance. Anthropic describes continuous work across thousands of steps. Against the previous one-to-two-hour ceiling, that is roughly a fourfold jump in usable horizon.

Scoping matters. This is one published data point, from a launch partner, on one task. Seven hours of activity is not seven hours of correct decisions; the resulting diff still needed engineering review. And the economics are real: at $75 per million output tokens, an Opus 4 session that runs all afternoon is not a marginal cost. Sonnet 4 exists precisely for that trade-off.

From Tool to Coworker

When a model can hold working context for hours, the collaboration pattern changes. Instead of composing isolated prompts, you delegate: write a ticket-quality task description, define acceptance criteria, hand over a branch, and review the result. That resembles onboarding a junior colleague, not operating a completion engine. In our client work, we therefore structure agent tasks consistently as tickets — scope, constraints, and definition of done.

Two model properties make delegation more defensible than before. Anthropic reports both models are 65% less likely than Sonnet 3.7 to take shortcuts or exploit loopholes on susceptible agentic tasks. And steerability improved, so instructions about approach are followed more precisely. What delegation does not remove is the duty to specify. A vague ticket produces a plausible wrong solution — from models as from humans.

Where Human Review Stays

Our position is fixed on four points. Architecture and interface decisions stay human, because they encode intent no test suite captures. Security-relevant code and dependency changes stay human. Anything with regulatory or data-protection consequences stays human. And merge responsibility stays human, because a model cannot be accountable.

The system card justifies the caution. Anthropic deploys Opus 4 under its AI Safety Level 3 standard — the first Claude model under ASL-3 protections — and Sonnet 4 under ASL-2. Reward hacking is reduced, not eliminated. A seven-hour run produces a diff nobody watched being written; review therefore shifts from reading every line toward contracts that fail loudly — tests the agent must not touch, property checks, CI gates, and diff-size budgets.

Outlook from May 2025

Written in the week of the release, our expectation is this: the usable horizon keeps stretching, from hours toward working days within one or two model generations. The bottleneck then moves from generation to verification. Teams that can specify precisely and verify mechanically will capture the gains; review capacity becomes the scarce resource, not model access.

We also expect the coworker framing to take concrete system form: persistent memory files, standing repository access through the Model Context Protocol, and task queues instead of chat windows. Some of this will arrive later and less cleanly than launch day suggests. The direction since May 22, 2025 is nevertheless clear: long-horizon software work is no longer exclusively human. Its value will be determined by the teams that can review the results reliably.