Ground an SRE Agent in Your Own Runbooks

We asked an AURA agent about an error code that doesn't exist anywhere on the internet. It exists only in our internal docs.

A general-purpose AI assistant has nothing to go on there. It either guesses or gives up. This agent searched our knowledge base, found the entry, and answered correctly. A second question returned our own thresholds and our own escalation path: 250 failed transactions a minute, with rollback going through the release captain.

The model didn't know those things. Our runbooks did. This page shows how to connect them.

TL;DR

• What it is: An open-source AURA agent team that answers operations questions from your own runbooks and documentation, and cites which source each fact came from.
• Two knowledge stores in one config:
  • A Qdrant vector collection for internal runbooks and postmortems.
  • An Amazon Bedrock managed knowledge base for product documentation.
• What it does when it doesn't know: It says the stores have no relevant content, instead of guessing.
• What you need: A model provider API key, a Qdrant instance with a populated runbooks collection, and AWS credentials with a Bedrock knowledge base ID.
• Cost: AURA is free and open source (Apache 2.0). You pay only for your model, embedding, and AWS usage.
• Source: examples/complete/rag-knowledge-base.toml

What this example does

The config defines two knowledge stores and a three-agent team:

1. Coordinator. Breaks the question into lookups, assigns them, and writes the final answer with a source for each fact. It can search the runbooks store itself while planning.
2. Knowledge specialist. Searches both stores, quotes the most relevant passages word for word, and returns the store name and document identifiers with each one.
3. General analyst. Reasons only over material already retrieved. It has no store access and no tools.

Each store becomes its own search tool for the agent, such as vector_search_runbooks. Access is scoped per agent: a specialist sees only the stores listed for it, so retrieval stays limited to the agent whose job needs it.

Who it's for

Jobs to be done

• On-call engineer: When I'm paged at 3 a.m. for something I've never seen, I want the agent to find our runbook for it, so I follow our process instead of a generic one.
• SRE lead: When we've invested in runbooks, I want the agent to check them whenever it answers an incident question, so that knowledge isn't stuck in a wiki nobody opens.
• Platform engineer: When I add knowledge to an agent, I want each source scoped to the agent that needs it, so retrieval stays predictable.
• Security reviewer: When internal docs feed an AI, I want to control where the documents and queries go, so sensitive procedures don't leave our environment.

Pain points it removes

• Generic answers. Without your docs, an AI answers from public knowledge. It doesn't know your thresholds, owners, or rollback rules.
• Runbook rot from disuse. Runbooks that nobody reads during incidents go stale. An agent that reads them gives them a job.
• Untraceable answers. Every fact is tied to the store it came from, with passages quoted word for word.
• Confident guessing. The coordinator is told to say when the stores have nothing relevant.
• Data residency objections. A Bedrock knowledge base keeps documents and retrieval inside your AWS account.

What it replaces, and why it's better

The value it drives

• Your team's process, applied consistently. The agent follows the documented procedure, not a plausible-sounding alternative.
• Faster answers for less experienced on-call engineers. Institutional knowledge reaches whoever is paged.
• More value from documentation you already wrote. Runbooks become live inputs instead of reference material.
• Answers you can check. Quoted passages and store names make verification quick.

What to measure in a pilot: how often answers cite a runbook, how often the cited runbook was correct, and how often the agent reports "no relevant content" (a signal of documentation gaps).

Why AURA

• Production-tested. AURA started as Mezmo's internal harness for operating our own SaaS. Our engineering and SRE teams still use it in production, and it powers thousands of agent sessions each month in Mezmo's hosted platform.
• Retrieval as config. Qdrant collections and Bedrock knowledge bases are declared in TOML and become tools automatically.
• Scoped retrieval. Each agent sees only the stores listed for it.
• Runs inside your security boundary. Deploy in your own infrastructure, including air-gapped environments.
• Model-agnostic. OpenAI, Anthropic, Bedrock, Gemini, Ollama, or OpenRouter.
• Traceable end to end. Every retrieval and model turn can be exported as OpenTelemetry traces.

Start from where you are

• Runbooks in Notion? Connect the Notion MCP server and give one specialist read-only access to your runbook database. No vector store required.
• Runbooks in a Git repo? Connect the GitHub MCP server and point a specialist at the repo.
• Need everything inside AWS? Use the Bedrock provider for the model and a managed Bedrock knowledge base over runbooks in S3, as shown in the video on this page.
• Already running Qdrant? Point the runbooks store at your collection. Its embedding model must match how the collection was embedded.

Look inside the config

Each knowledge store is a short block:

[[vector_stores]]
name = "runbooks"
type = "qdrant"
context_prefix = "Internal SRE runbooks and postmortems"
collection_name = "runbooks"

[[vector_stores]]
name = "customer_kb"
type = "bedrock_kb"
context_prefix = "Customer-facing product documentation"
knowledge_base_id = "{{ env.BEDROCK_KB_ID }}"
managed = true

Access is scoped per agent:

[orchestration]
coordinator_vector_stores = ["runbooks"]

[orchestration.worker.knowledge]
preamble = """
You are a research specialist. Search the vector stores available to you,
quote the most relevant passages verbatim, and return them with the store
name and any document identifiers. Do not summarize away specifics.
"""
vector_stores = ["runbooks", "customer_kb"]
mcp_filter = []

Try it in four steps

1. Install AURA (Linux or macOS):

curl -fsSL https://raw.githubusercontent.com/mezmo/aura/main/scripts/install.sh | bash

2. Start Qdrant and load your runbooks into a runbooks collection (any pipeline that writes 1536-dimension vectors from text-embedding-3-small):

docker run -p 6333:6333 -p 6334:6334 qdrant/qdrant

3. Set credentials:

export OPENAI_API_KEY="sk-..."
export AWS_REGION="us-east-1"
export AWS_PROFILE="default"
export BEDROCK_KB_ID="ABCDEFGHIJ"

4. Download the example and run it:

curl -fsSLO https://raw.githubusercontent.com/mezmo/aura/main/examples/complete/rag-knowledge-base.toml
aura --config rag-knowledge-base.toml

More detail: AURA configuration reference, [[vector_stores]] section.

Then ask a question only your docs can answer.

FAQ

Do I need both Qdrant and Bedrock?

No. The example shows both to demonstrate multiple stores. Use one, either, or add more. Each store becomes its own search tool.

Does my documentation leave my environment?

It depends on your stores and model. In this example, Qdrant runs where you deploy it, and Qdrant embeddings use the OpenAI API. With the Bedrock provider and a managed Bedrock knowledge base, the model, retrieval, and documents all stay in your AWS account.

What if the runbook is wrong or missing?

The coordinator is told to say when the stores have no relevant content. Quoted passages make a wrong or outdated runbook visible quickly.

Can the agent update runbooks?

Not in this example. Retrieval is read-only. To let an agent propose runbook updates, give it GitHub MCP tools and gate pull request creation behind human approval.

Next steps

• Engineers: Clone the example, load five real runbooks, and ask the questions your on-call team asks most.
• Leads: Pick the ten runbooks that matter most during incidents and use them as the pilot set.
• Everyone: Join the AURA Slack community.