16 Jan 2025 AI Engineering5 min read

Deep Research Agents: What They Do and What They Do Not

Deep research agents do not answer in one shot: they plan, browse, and synthesize over several minutes. We define the pattern, examine Google’s Gemini Deep Research from December 2024, and quantify its central weakness, citation quality. Measured baselines show why verification remains the reader's responsibility and how engineering teams should respond.

The Limits of Single-Shot Search

Most research questions do not fit into a single search query. A question such as “Which vector database fits our latency budget and licensing constraints?” breaks down into a dozen sub-questions, each answered by different sources and each answer reshaping the next question. Classic chatbots with retrieval run one search round and summarize the first results. Retrieval-augmented generation answers the question you asked, not the one you should have asked next. The result reads well but stops one analytical step too soon.

The GAIA benchmark (Mialon et al., November 2023) makes this gap measurable. Its 466 questions require reasoning, web browsing, and tool use. Human respondents reach 92 percent; GPT-4 with plugins reached 15 percent. Humans needed 6 to 17 minutes per question. The bottleneck is therefore not knowledge alone, but sustained process: searching, reading, searching again, and preserving intermediate state.

A task arrives — the agent plans its next step. 1/4

What Defines a Deep Research Agent

A deep research agent is a system that turns a question into a research plan, executes that plan over many browsing steps, and condenses the collected evidence into a cited report. Three properties separate it from chat with retrieval: an explicit plan the user can inspect, a horizon of minutes and dozens of tool calls instead of one round trip, and a synthesis pass over the full accumulated state rather than over the last search result.

Just as important is what such an agent does not do. It does not validate its sources or distinguish a peer-reviewed paper from a vendor blog unless instructed to. Nor does it know when it has read enough; it stops when its plan is exhausted. The result is a draft with references, not a verified document.

Gemini Deep Research in Practice

Google shipped the first mainstream implementation on December 11, 2024. Deep Research is part of Gemini Advanced, gated behind the Google One AI Premium plan at 20 US dollars per month. The user selects "Gemini 1.5 Pro with Deep Research", enters a question, and receives a multi-step research plan to revise or approve. Only then does the agent start browsing.

A run takes a few minutes. The agent searches, reads, and starts new searches based on what it has learned, holding intermediate results in a context window of one million tokens. The output is a structured report with links to the original sources, exportable to Google Docs. Since December 20, 2024 the feature has been available in over 45 languages and more than 150 countries.

The launch scope is instructive: no API access, no confidence scores, availability only inside the Gemini interface. The report carries links to original sources and a list of researched websites. Google describes the feature as working "under your supervision" — that clause carries more weight than the marketing tone suggests.

Long-Horizon Browsing and Synthesis

The underlying pattern is general and older than the product: plan decomposition, a loop of searching, reading and refining, state accumulation in a long context, and a final synthesis pass. Open-source projects such as GPT Researcher have implemented variants of it since 2023. The plan-approval step is the most important interface decision: it moves user control to the point where steering is still cheap. In our own agent work we see the same loop regardless of framework; the pattern, not the product, is what transfers.

The pattern carries structural trade-offs. Latency rises from seconds to minutes, while cost scales with pages fetched and tokens held in context. An early misreading steers every subsequent search, so errors compound instead of averaging out. Coverage also ends where a crawler cannot reach: paywalled journals, internal wikis, and documents behind logins remain invisible. Breadth therefore tends to win over depth — ten skimmed pages rather than one read carefully.

Citation Quality in Numbers

Citation quality has measured baselines. Liu, Zhang and Liang audited four generative search engines with human raters in 2023. On average only 51.5 percent of generated sentences were fully supported by their citations, and only 74.5 percent of citations supported their associated sentence. Worse, citation precision correlated negatively with perceived utility (r = −0.96): the answers that looked most helpful cited worst.

The study predates deep research agents, and longer pipelines may score better or worse: more sources per claim, but also more paraphrase steps between source and sentence. As of January 2025 no comparable public audit of Gemini Deep Research exists. Until one does, a link in a generated report is a claim, not proof.

System	Citation recall (%)	Citation precision (%)
Bing Chat	58.7	89.5
perplexity.ai	68.7	72.7
NeevaAI	67.6	72.0
YouChat	11.1	63.6
Average	51.5	74.5

Verification Stays With the Reader

Verification therefore remains the reader's responsibility. A cited report shifts work but does not remove it. Our working rule at Blue IT Systems is to treat the report as an annotated bibliography with a draft attached. Check every load-bearing claim — numbers, dates, version compatibilities, and legal statements — against its primary source before it informs a decision.

Fluency is not evidence. The Stanford numbers show that polish and citation accuracy diverge. Reading the report costs minutes; reading the sources behind the three claims your decision rests on costs a few more. That ratio is the real productivity gain — not skipping verification, but concentrating it where it matters.

Outlook From January 2025

As of January 16, 2025 we expect three developments. First, competition: OpenAI announced its o3 reasoning model in December 2024, and pairing reasoning models with agentic browsing is the obvious next product. We expect every major lab to ship a deep research feature during 2025. Google has already announced that Deep Research will reach the Gemini mobile app and Workspace accounts in early 2025. Second, benchmarks: GAIA-style scores will rise quickly once long-horizon agents are tuned for them.

Third, citation precision will become a marketed metric because it is measurable and currently weak. What we do not expect in 2025 is automated verification good enough to relieve the reader of review. Deep research agents reduce research from hours to minutes; judgment does not compress at the same rate. The workflow should reflect that distinction.