Finds Patient A's record and a similar Patient B record. If the nearest text wins, the answer can cross the identity boundary.
Retrieval finds. Elvena checks.
A retrieval system can be useful and still unsafe for high-stakes work. It can find related text, but it does not inherently know which record is current, which patient it belongs to, which correction controls, or whether a record was slipped into the store outside the trail.
Elvena adds the missing step: decide what the AI is allowed to use.
Elvena adds the missing step: decide what the AI is allowed to use.
Start with the wrong patient.
This is the failure people remember. A normal search layer can surface a clinically similar record from another patient because it looks close. Elvena applies the identity rule before ranking, so the wrong-patient candidate never becomes model context.
Rejects Patient B before scoring, then returns Patient A's active record with the proof checks that allowed it.
The 5,000-question result
The Sayer benchmark used 99,455 public article records and 5,000 sealed questions. It scored whether each system returned the expected source-backed answer. A nearby article did not count.
| System | Right | Not answered correctly | Plain-English meaning |
|---|---|---|---|
| Elvena gate | 5,000/5,000 | 0/5,000 | Returned the expected source-backed result for every scored Sayer question. |
| Haystack BM25 setup | 325/5,000 | 4,675/5,000 | BM25 over the full corpus often found related documents, not the required source answer. |
| Basic RAG setup | 247/5,000 | 4,753/5,000 | Nearest-record retrieval found nearby articles but missed the expected source answer. |
| LangGraph/LangMem setup | 247/5,000 | 4,753/5,000 | LangGraph/LangMem was present, but the run still answered through deterministic retrieval. |
| LlamaIndex setup | 247/5,000 | 4,753/5,000 | LlamaIndex documents were loaded, but the run still answered through deterministic retrieval. |
Benchmark basis: Sayer benchmark over a public article corpus, with separate comparison runs for Basic RAG, LangGraph/LangMem, Haystack BM25, and LlamaIndex. Rows compare the specific test setups used for this run. They do not evaluate every product feature, vendor best practice, later version, or custom deployment.
Why the others fail
The problem is not that retrieval is useless. Retrieval is doing what it was built to do: find related material. The failure starts when "related" gets treated as "ready to act on."
Fictional fixture: identity
Wrong patient
For Example Patient A, what blood-pressure record controls this workflow?
Plain retrieval returned
Search surfaced Patient A's 118/76 note beside a similar Patient B 178/104 note.
Elvena returned
Only Patient A's 118/76 record is eligible. Patient B is rejected before ranking.
Looks similar is not enough. The identity has to match first.
Fictional fixture: correction
Old correction
What contact method is active for Example Contact A?
Plain retrieval returned
The old phone record is still searchable, so the answer returns phone.
Elvena returned
The correction overlay selects the case mailbox and suppresses the old phone record.
The old record stays in the audit trail, but it cannot win the live answer.
Fictional fixture: time state
Expired state
What authorization state applies to Example Authorization A as of May 1, 2026?
Plain retrieval returned
A stale pending record ranks near the question.
Elvena returned
The current approved state is selected because the read is bounded to May 1, 2026.
Current state beats a stale match.
Fictional fixture: healthcare workflow
Medication-safety conflict
For Example Patient A, which medication-safety record controls the workflow flag today?
Plain retrieval returned
One old synthetic note says no conflict. A later synthetic medication list says interaction risk.
Elvena returned
The authoritative synthetic medication list flags a conflict and routes the item for clinician review.
Conflicting records are resolved by configured authority, not paragraph similarity.
Fictional fixture: legal workflow
Stale legal draft
Which deadline source is marked authoritative in Example Case B?
Plain retrieval returned
Search returns the draft memo date and the court-order date together.
Elvena returned
The court-order source is marked authoritative: June 15.
A draft can be searchable without being selected for workflow use.
What Elvena adds that ordinary retrieval does not
These controls apply when records, retrieval, and actions are routed through Elvena policies.
The record must belong to the right person, tenant, case, device, or account before it can rank.
Old answers can stay preserved for audit while being blocked from the live answer.
A verified record wins over a closer-looking unverified one when they conflict.
The system can show the source, the checks applied, and why the answer was allowed.
A record inserted around the gate does not silently become truth. The chain check catches the mismatch.
For routed tools and actions, the same gate can block an action that depends on stale, wrong, or unproved context.
Adoption context for the named frameworks
The benchmark rows represent tools and patterns people actually build with. Public end-user counts are not available for these developer frameworks, so this section uses adoption proxies and keeps them separate from the correctness score.
| System or category | Adoption proxy | What it represents | Safe mainstream framing | Source snapshot |
|---|---|---|---|---|
| Haystack | ~25.8k GitHub stars | Production RAG framework | Developer adoption proxy, not an end-user count. | deepset-ai/haystack: 25,777 stars, checked 2026-06-29 13:30 UTC. |
| RAG | No single owner or user count | The general pattern behind AI document search | Used as a broad baseline category, not a named product. | Pattern category from the benchmark framing. |
| LangGraph/LangMem | ~36.0k + ~1.5k GitHub stars | Agent workflow plus memory tooling | Developer adoption proxy for the LangChain ecosystem tools tested. | langchain-ai/langgraph: 36,023 stars; langchain-ai/langmem: 1,528 stars, checked 2026-06-29 13:30 UTC. |
| LlamaIndex | ~50.5k GitHub stars | LLM data framework | Meta Developer Docs publish a LlamaIndex integration guide for Meta Llama. That is documentation evidence only, not production-stack evidence. | run-llama/llama_index: 50,497 stars, plus Meta Developer Docs LlamaIndex integration guide, checked 2026-06-29. |
| Consumer AI assistants | Scale context only | The kind of answer surface where bad retrieval scales fast | Meta announced Meta AI availability across Facebook, Instagram, WhatsApp, and Messenger in 2024, with rollout limits. This is not a benchmark row. | Meta Newsroom 2024 Meta AI assistant announcement; Meta 2026 update describes later rollout across Meta apps. |
Meta using Llama or Meta AI is separate from LlamaIndex adoption. The safe claim is documentation-level: Meta publishes a LlamaIndex integration guide for Meta Llama. It is not evidence that Meta AI uses LlamaIndex in production.
Technical design notes for the visualization
The charts use a small normalized table with one row per system and the fields system, correct, failed, questions, and family. The visual grammar is a horizontal bar chart because the reader compares systems on one measure at a time. The shipped page renders static SVG from the same table so it has zero dependencies and zero network requests. The Vega-Lite specs are embedded below so the same data can be reused in another declarative renderer. Bar domains are pinned to 0 through 5,000.
Instance count is intentionally tiny: two charts, five rows, no runtime chart library, no hover-only information. Mobile behavior uses horizontal chart scrolling so labels remain readable instead of shrinking into illegible text.
Data
[
{
"system": "Elvena gate",
"correct": 5000,
"failed": 0,
"questions": 5000,
"family": "source-of-truth gate",
"plain": "Returned the expected source-backed result for every scored Sayer question."
},
{
"system": "Haystack BM25 setup",
"correct": 325,
"failed": 4675,
"questions": 5000,
"family": "retrieval baseline",
"plain": "BM25 over the full corpus often found related documents, not the required source answer."
},
{
"system": "Basic RAG setup",
"correct": 247,
"failed": 4753,
"questions": 5000,
"family": "retrieval baseline",
"plain": "Nearest-record retrieval found nearby articles but missed the expected source answer."
},
{
"system": "LangGraph/LangMem setup",
"correct": 247,
"failed": 4753,
"questions": 5000,
"family": "agent-memory baseline",
"plain": "LangGraph/LangMem was present, but the run still answered through deterministic retrieval."
},
{
"system": "LlamaIndex setup",
"correct": 247,
"failed": 4753,
"questions": 5000,
"family": "retrieval baseline",
"plain": "LlamaIndex documents were loaded, but the run still answered through deterministic retrieval."
}
]
Failure chart spec
{
"$schema": "vega-lite-v5",
"description": "Questions not answered correctly",
"data": {
"values": [
{
"system": "Elvena gate",
"correct": 5000,
"failed": 0,
"questions": 5000,
"family": "source-of-truth gate",
"plain": "Returned the expected source-backed result for every scored Sayer question."
},
{
"system": "Haystack BM25 setup",
"correct": 325,
"failed": 4675,
"questions": 5000,
"family": "retrieval baseline",
"plain": "BM25 over the full corpus often found related documents, not the required source answer."
},
{
"system": "Basic RAG setup",
"correct": 247,
"failed": 4753,
"questions": 5000,
"family": "retrieval baseline",
"plain": "Nearest-record retrieval found nearby articles but missed the expected source answer."
},
{
"system": "LangGraph/LangMem setup",
"correct": 247,
"failed": 4753,
"questions": 5000,
"family": "agent-memory baseline",
"plain": "LangGraph/LangMem was present, but the run still answered through deterministic retrieval."
},
{
"system": "LlamaIndex setup",
"correct": 247,
"failed": 4753,
"questions": 5000,
"family": "retrieval baseline",
"plain": "LlamaIndex documents were loaded, but the run still answered through deterministic retrieval."
}
]
},
"mark": {
"type": "bar",
"cornerRadiusEnd": 4
},
"encoding": {
"y": {
"field": "system",
"type": "nominal",
"sort": null,
"title": null
},
"x": {
"field": "failed",
"type": "quantitative",
"title": "Questions not answered correctly out of 5,000",
"scale": {
"domain": [
0,
5000
]
}
},
"color": {
"field": "family",
"type": "nominal",
"scale": {
"domain": [
"source-of-truth gate",
"retrieval baseline",
"agent-memory baseline"
],
"range": [
"#0f766e",
"#b91c1c",
"#7c2d12"
]
},
"title": "System type"
},
"tooltip": [
{
"field": "system",
"type": "nominal"
},
{
"field": "correct",
"type": "quantitative"
},
{
"field": "failed",
"type": "quantitative"
},
{
"field": "plain",
"type": "nominal"
}
]
}
}
Correct-answer chart spec
{
"$schema": "vega-lite-v5",
"description": "Right answers",
"data": {
"values": [
{
"system": "Elvena gate",
"correct": 5000,
"failed": 0,
"questions": 5000,
"family": "source-of-truth gate",
"plain": "Returned the expected source-backed result for every scored Sayer question."
},
{
"system": "Haystack BM25 setup",
"correct": 325,
"failed": 4675,
"questions": 5000,
"family": "retrieval baseline",
"plain": "BM25 over the full corpus often found related documents, not the required source answer."
},
{
"system": "Basic RAG setup",
"correct": 247,
"failed": 4753,
"questions": 5000,
"family": "retrieval baseline",
"plain": "Nearest-record retrieval found nearby articles but missed the expected source answer."
},
{
"system": "LangGraph/LangMem setup",
"correct": 247,
"failed": 4753,
"questions": 5000,
"family": "agent-memory baseline",
"plain": "LangGraph/LangMem was present, but the run still answered through deterministic retrieval."
},
{
"system": "LlamaIndex setup",
"correct": 247,
"failed": 4753,
"questions": 5000,
"family": "retrieval baseline",
"plain": "LlamaIndex documents were loaded, but the run still answered through deterministic retrieval."
}
]
},
"mark": {
"type": "bar",
"cornerRadiusEnd": 4
},
"encoding": {
"y": {
"field": "system",
"type": "nominal",
"sort": null,
"title": null
},
"x": {
"field": "correct",
"type": "quantitative",
"title": "Correct answers out of 5,000",
"scale": {
"domain": [
0,
5000
]
}
},
"color": {
"field": "family",
"type": "nominal",
"scale": {
"domain": [
"source-of-truth gate",
"retrieval baseline",
"agent-memory baseline"
],
"range": [
"#0f766e",
"#b91c1c",
"#7c2d12"
]
},
"title": "System type"
},
"tooltip": [
{
"field": "system",
"type": "nominal"
},
{
"field": "correct",
"type": "quantitative"
},
{
"field": "failed",
"type": "quantitative"
},
{
"field": "plain",
"type": "nominal"
}
]
}
}