Multi-Agent · RAG · MCP · 2026

OpenXR Spec Agents

A supervisor-pattern multi-agent assistant for OpenXR developers: spec RAG, SQL over the extension registry, live web through MCP, and a hybrid router whose every decision is visible.

Type: Multi-Agent System · RAG · Evaluation
Stack: LangGraph · FastAPI · Chroma · NVIDIA NIM · DeepSeek · Tavily MCP · Cloud Run
Corpus: OpenXR 1.1 spec · 2,855 chunks · 999 registry extensions
Date: October 2026

Overview

OpenXR is a large, strict specification, and the questions developers ask about it come in different shapes: what must happen before xrBeginSession, how many released extensions a vendor has, what changed in the latest SDK. A LangGraph supervisor classifies each question and dispatches it to specialist agents. A Research agent runs RAG over the OpenXR 1.1 spec and reaches the live web through the Tavily remote MCP server. A Data agent writes read-only SQL over the official extension registry. A Summary agent reconciles their outputs into one answer that keeps its citations.

Routing you can see

Two providers share the work. NVIDIA NIM's free Nemotron model handles the planner, SQL generation and short answers; DeepSeek writes long multi-source syntheses and takes over when NIM stalls. Each LLM call records which provider it went to, on which dimension (cost, latency, capability or fallback) and why, and the UI shows that next to a timestamped trace of every plan, retrieval, MCP call and SQL query. A long RAG answer costs about $0.0016 on DeepSeek, and many cost nothing.

Retrieval

The spec is split into 2,855 structure-aware chunks, each prefixed with its section breadcrumb, embedded with NIM and stored in an embedded Chroma index (k = 6). Every claim in an answer carries a label that links to the exact chunk, with its similarity score. An ablation showed that explicitly pinning identifiers into the results added nothing: all 16 test queries already ranked the right chunk first, because the breadcrumb prefix puts identifiers in the chunk headings.

Measured, not assumed

The test set has 18 spec questions with reference answers and 8 routing checks. Over six RAGAS rounds the deployed routing reached faithfulness 0.904, answer relevancy 0.892, context precision 0.792 and context recall 0.963. All 8 plans were routed correctly and the registry counts matched gold SQL. Targeted probes lifted planner routing from 10/20 to 30/30 and NIM-written SQL to 24/24 against gold queries. Five injected failures, from a Tavily outage to an invented citation, all degrade gracefully.

What evaluation found

Evaluation kept finding real defects. A routing check that passed had actually failed during a transient Tavily outage, which led to a stricter check and an MCP retry. Failure injection exposed a streaming deadline that fast chunks could never trip, and a degraded path that guessed “12” for a count whose true value was 19. The free model's throughput swung between 33 and 95 tokens per second, so routing follows measured latency and failure modes rather than price alone.