05 Jul 2024 AI Engineering4 min read

GraphRAG: Answering Global Questions with Knowledge Graphs

Microsoft's GraphRAG uses an LLM to extract an entity graph from text, partitions it with the Leiden algorithm, and summarizes every community in advance. We show why this index can answer global questions that vector RAG cannot, how strong the evaluation is, and why indexing cost is the central trade-off.

Questions That Span an Entire Corpus

Retrieval-augmented generation, in its common form, is a vector pipeline: documents are split into chunks and embedded, then the top-k most similar chunks enter the context window at query time. This works well for local questions because the answer sits in a handful of passages that semantic similarity can find.

The approach reaches its limits on global questions. There is no single answer chunk for "What are the main themes in this dataset?" Top-k retrieval returns passages that merely resemble the question, then produces a misleadingly confident summary of that sample. The GraphRAG paper classifies such questions as query-focused summarization, not retrieval — and prior summarization methods do not scale to corpora of RAG size.

Entities and relations are extracted from every document. 1/4

What Microsoft Has Released

Microsoft Research introduced GraphRAG in a blog post on 13 February 2024, demonstrated on thousands of Russian and Ukrainian news articles from June 2023. On 24 April 2024 the team around Darren Edge and Jonathan Larson published the accompanying paper (arXiv:2404.16130), which announced an open-source Python implementation as forthcoming.

The implementation is now publicly accessible: the microsoft/graphrag repository is on GitHub under the MIT license and includes documentation, an indexing engine, and global and local search modes. There is still no dedicated announcement post; the code is simply available. The key distinction is that GraphRAG does not replace a vector database. It adds a more expensive index alongside it.

An LLM Extracts the Entity Graph

Indexing starts conventionally: source texts are split into chunks of around 600 tokens. Each chunk then passes through a multipart extraction prompt that returns entities, relationships between them, and short natural-language descriptions of both. Because a single pass misses items, the pipeline runs additional "gleaning" rounds, asking the model whether entities were overlooked, up to a configured maximum.

Chunk size is not cosmetic. The paper reports that 600-token chunks yielded almost twice as many entity references as 2400-token chunks on the same corpus. Duplicate descriptions of the same element are summarized into one. The result is a weighted, described entity graph — built without a predefined ontology and therefore domain-agnostic.

Leiden Communities as Index Structure

GraphRAG partitions this graph with the Leiden algorithm (Traag et al., 2019), a refinement of Louvain that guarantees well-connected communities. The partition is hierarchical and recursive: coarse root communities at level C0, progressively finer sub-communities at C1 through C3. Every node belongs to exactly one community per level.

This is the structural difference from vector search. A community partition covers all nodes, so every input text has influenced the index. Top-k retrieval samples the corpus; the community hierarchy describes it in full, at several resolutions.

Community Summaries and Map Reduce

During indexing, the LLM writes a report for every community — from element summaries at the leaf level, and from sub-community reports higher up. These reports exist before any question is asked. They function as a pre-computed, hierarchical summary of the corpus.

At query time, global search uses map-reduce. In the map phase, community reports from a chosen level are distributed across context windows and turned into partial answers with helpfulness scores; the reduce phase condenses the highest-scoring contributions into the final answer. Local search takes the complementary path by resolving entities in the question and combining their graph neighborhoods with the underlying text.

What the Paper Measured

The evaluation used two corpora in the one-million-token range: podcast transcripts (about 1 million tokens) and news articles (about 1.7 million tokens), with 125 GPT-4-generated sensemaking questions per dataset and an LLM judge scoring comprehensiveness, diversity, empowerment, and directness in head-to-head comparisons.

GraphRAG won roughly 70–80% of comparisons against naive vector RAG on comprehensiveness and diversity. Naive RAG won on directness — its answers are terser. Against direct source-text summarization, root-level community summaries were competitive in quality while requiring over 97% fewer context tokens per query.

Approach	Global questions	Tokens per query	Index construction
Vector RAG (top-k)	fails	low	embeddings only — cheap
Source-text summarization	good answers	entire corpus — very high	none
GraphRAG (root level C0)	competitive quality	>97% below summarization	LLM extraction — expensive

The Bill Arrives at Indexing Time

The trade-off is stated plainly in the repository itself: indexing is warned to be an expensive operation. Mechanically, every 600-token chunk passes through the LLM at least once, gleaning rounds multiply that, element descriptions are summarized, and a report is generated for every community on every hierarchy level. For a million-token corpus this means thousands of LLM calls — orders of magnitude above the cost of embedding the same text.

Equally important is the negative scope. Simple fact lookups do not improve; naive RAG answers them more directly and far more cheaply. The released pipeline has no incremental updates, so corpus changes require re-indexing, while extraction errors propagate silently into the graph. The evidence base is also narrow: one LLM judge, two datasets, and a single question class.

Outlook From June 2024

We expect the indexing cost to fall faster than skepticism suggests. Microsoft is already working on automatic tuning of extraction prompts, and smaller models specialized for extraction are an obvious next step. Ports into the LangChain and LlamaIndex ecosystems and managed graph-RAG offerings seem likely within months. The hierarchical community summary itself — a corpus that writes its own table of contents — is a pattern we expect to outlive this particular implementation.

Our working position at Blue IT Systems is therefore clear: the graph remains a derived index and never becomes a second source of truth. Local questions still belong in vector search; a graph index earns its cost only when global questions recur against a stable corpus. Whether that scope expands depends chiefly on incremental indexing and more rigorous evaluation standards.