Deep dive · 03

Knowledge-graph RAG — reasoning baked into the datastore

In a product that must never improvise clinical facts, every answer has to trace back to an exact retrieval path. That requirement pushed me away from a plain vector store and toward a knowledge graph where the reasoning is built at ingestion time.

Where should the reasoning live?

Retrieve grounded clinical knowledge for a turn

A · Thin index, heavy agent

Vector store is just an index. The agent does the reasoning on the fly — extracting and connecting facts at query time. Probabilistic: a fresh chance of failure on every turn.

rejected

B · Smart datastore, thin agent

Move the hard work into ingestion. The graph itself encodes the reasoning — concepts as nodes, relationships as edges. The agent only has to ask the right question; traversal supplies the connected facts.

chosen

Option B buys deterministic, traceable reasoning — the graph is built once and inspected, instead of re-derived probabilistically on every query.

Detail — why determinism mattered here

On-the-fly generative reasoning carries a failure probability every single time it runs. By contrast, a graph traversal over a curated structure is repeatable and auditable: we can point at the exact nodes and edges behind any answer. For a clinical assistant where traceability is the whole premise, that property was worth the heavier ingestion investment.

Ingestion — turning specialist PDFs into a reasoning graph

Curated PDFs Specialists pick documents & papers per chronic condition.

Text → facts LLM extracts text, then atomic facts from the text.

Concepts → nodes Fixed concept set — medication, symptom, therapy — wrapped as nodes.

Relations → edges e.g. medication X treats symptom Y → Neo4j graph.

Built with Graphiti (Neo4j-backed). It was created for bitemporal agent memory; we repurposed it to build curated clinical knowledge graphs from specialist-selected documents.

A concrete slice — menopause

hot flashessymptom

treats / treated by

hormone replacement therapytreatment

One of five conditions. Two typed nodes, one directional relationship — multiply this across a curated corpus and the graph becomes a traversable reasoning engine.

Retrieval — seed by similarity, reason by traversal

Agent question The one job left to the agent: ask the right question.

Hybrid search BM25 + semantic similarity, fused with RRF reranking, lands on seed nodes.

Traverse n-hops Walk to neighboring concepts — implicit, pre-built reasoning.

Grounded answer The agent composes on top of a connected, traceable fact set.

We used Graphiti's combined hybrid search with RRF reranking — semantic + lexical to find entry points, then the graph structure carries the reasoning.

How we knew retrieval was good

recall@k

did we fetch the right facts?

precision@k

how much noise came with them?

auto + HITL

automated metrics + doctor review

Retrieval quality fed straight into the Knowledge eval track — relevance and accuracy were judged on the retrieved facts, not just the final answer.

(Specific recall/precision figures aren't on hand here.) See Evaluations → for how the doctor panel scored retrieval.

What we tried and dropped

Naive chunking

Fixed-size splits cut across the facts that mattered.

Contextual chunking

Better boundaries, still no guarantee the needed fact lived in one chunk.

Sliding window

More coverage, more redundancy and noise.

With large source documents, the information you need is scattered across the text — so chunked vector search dragged in irrelevant passages and leaned on LLM reranking to clean up. The graph removed that noise at the source.