21 Mar 2024 AI Engineering5 min read

Agentic Workflow Patterns

Reflection, tool use, planning, and multi-agent collaboration promise more than single-pass prompting. We examine the measured gains and documented limits of each pattern and derive a simple rule: evaluation first, then the smallest loop that demonstrably helps.

Why One-Shot Prompting Hits a Ceiling

In March 2024, most production language-model applications still run in a single pass: a prompt goes in, tokens stream out, and the first draft becomes the final answer. The model does not run its code, check a claim against a source, or revise. It is equivalent to asking a developer to type a feature from top to bottom, never press backspace, and ship the first version immediately.

An agentic workflow replaces that one-shot pass with a controlled loop. The model drafts, checks, gathers missing information, and revises before the result reaches the user. The published gains are substantial, but they are not free. The relevant engineering question is therefore which loop justifies its added latency, token use, and failure modes. Four patterns recur in the literature.

Four Patterns for Agentic Workflows

Between October 2022 and today a consistent structure has emerged from the research: reflection (the model critiques its own output), tool use (the model calls external functions), planning (the model decomposes a goal into steps), and multi-agent collaboration (several role-prompted instances divide the work). Andrew Ng's letter of 6 March 2024 in The Batch describes the same landscape: rapid month-over-month progress, writer-critic pairs, and frameworks such as Microsoft's AutoGen, CrewAI, and LangGraph that make these loops programmable.

Those four patterns are not equally mature. In our projects, reflection and tool use are predictable enough for production. Planning and multi-agent collaboration produce occasional strong results, but not yet reliably repeatable ones. The sections below therefore pair each measured gain with a documented limitation.

Reflection: The Cheapest Gain

Reflection prompts the model to criticize its own output and then revise it. Self-Refine (Madaan et al., March 2023) reported roughly 20 percentage points of absolute improvement averaged over seven tasks, using the same model for drafting, feedback, and revision. Reflexion (Shinn et al., 2023) added memory of past failures and an external signal — unit tests — and reached 91% pass@1 on HumanEval, against 80% for its GPT-4 baseline. For scale: OpenAI's zero-shot evaluation of March 2023 put GPT-3.5 at 48.1% and GPT-4 at 67.0% on the same benchmark — baseline protocols differ, the direction does not.

The limitation is documented just as clearly. Huang et al. showed in October 2023 that intrinsic self-correction — reflection without external feedback — can reduce reasoning accuracy rather than improve it. The pattern earns its cost only when an objective check exists: a failing test, a schema validator, or a retrieved source. Without such a signal, reflection quickly becomes paraphrase.

Tool Use: Grounding the Model

Tool use gives the model functions to call: search, code execution, database queries. ReAct (Yao et al., October 2022) interleaved reasoning traces with actions and beat imitation- and reinforcement-learning baselines by 34 percentage points absolute on ALFWorld and 10 on WebShop, while reducing hallucination on knowledge tasks. OpenAI's function-calling API of June 2023 turned the idea into a mainstream interface, and every serious agent framework now builds on some variant of it.

Tool use does not automatically make a model reliable at deciding when or which tool to call. Every tool also expands the attack surface: inputs the model constructs from untrusted text reach real systems. Schemas, timeouts, retries, and permission boundaries remain ordinary engineering work. The pattern does not remove those requirements; it creates them.

Planning: Powerful but Brittle

Planning has the model decompose a goal into steps before executing them. HuggingGPT (Shen et al., March 2023) demonstrated the pattern by having an LLM plan task graphs across specialized models. On 12 March 2024 Cognition announced Devin, an agent with its own shell, editor, and browser, reporting 13.86% of real GitHub issues resolved unassisted on a random 25% subset of SWE-bench — against 1.96% for the previous best.

Both readings of that number are correct. It is a sevenfold improvement, and it leaves roughly 86% of issues unresolved. Long plans drift, errors compound across steps, and a forty-minute trajectory is far harder to debug than a single completion. Planning pays off when steps are checkable in isolation; it degrades when only the final state can be judged.

Multi-Agent Collaboration

Multi-agent systems assign roles to several model instances — often the same base model with different prompts — and let them converse. ChatDev and MetaGPT (both 2023) staffed a virtual software company this way; AutoGen (Wu et al., 2023) generalized the idea into a framework for programmable agent conversations. The simplest useful configuration remains a writer and a critic, which is reflection implemented as two agents.

In multi-agent systems, costs multiply visibly. Every additional instance increases token use and latency, conversation state needs separate management, and several agents can confidently converge on the same wrong answer. Splitting one model across five role prompts adds structure, not intelligence. We therefore treat multi-agent designs as a later optimization, not a starting point.

Measured Gains Versus Added Complexity

The evidence available in March 2024 shows the same trade for every pattern: higher accuracy costs additional tokens, latency, and failure modes. A gain is real only when an evaluation harness reproduces it. That is why we build the measurement before the loop.

Start with the smallest useful form: one model, one generate-critique-revise pass, and one objective check such as a unit test or validator. Add tools only when the task requires external facts or actions. Add planning or more agents only when measurements show that the simple loop has reached a plateau. Every escalation must justify itself against the same evaluation as the stage before it.

Outlook: Where Agents Go From Here

Writing on 18 March 2024, we expect workflow design to rival model upgrades as a source of capability gains this year. The 11-point difference Reflexion measured between GPT-4 alone and GPT-4 in a loop is on the scale of a model-generation jump. Benchmarks such as SWE-bench are likely to become standard yardsticks for agents, while falling inference prices make loops economical where they remain marginal today.

The open problems will not disappear with lower prices. Agents are harder to evaluate than individual completions, cost across long trajectories remains difficult to control, and tool-wielding systems create security questions the industry has only begun to answer. We expect frameworks to consolidate while the underlying patterns endure. Teams that learn to measure loops now will be able to adopt new models quickly — not by rebuilding, but by placing them inside a workflow that is already verified.

Sources

• ReAct: Synergizing Reasoning and Acting in Language Models (Yao et al., 6 Oct 2022)
• GPT-4 Technical Report (OpenAI, 15 Mar 2023)
• Reflexion: Language Agents with Verbal Reinforcement Learning (Shinn et al., 20 Mar 2023)
• Large Language Models Cannot Self-Correct Reasoning Yet (Huang et al., 3 Oct 2023)
• The Dawning Age of Agents — The Batch (Andrew Ng, 6 Mar 2024)
• Introducing Devin the first AI software engineer (Cognition, 12 Mar 2024)