Run an AI SRE agent with a local LLM: from model endpoint to first investigation
TL;DR
- AURA is an SRE Agent for Production. Its open-source repository lets you inspect the agent that runs investigations.
- Choose a locally served model with a compatible endpoint. AURA’s documentation lists Ollama support, while the demonstration uses an OpenAI-compatible endpoint.
- Point AURA at the endpoint and run a read-only investigation. The video shows AURA inspecting a running AI stack without changing it.
Why run an SRE agent against a model you host yourself
A locally hosted model lets you run an AI SRE agent’s inference on infrastructure you control, while a read-only investigation lets you inspect its behavior before granting it permission to make changes. You can check where model requests go, then review the agent’s tool calls and findings.
Open model weights do not reveal how an agent handles production data. The agent’s orchestration layer determines which tools it can call and what information it sends to the model. Mezmo built AURA as an SRE Agent for Production, and its open-source repository lets you inspect that layer. When you evaluate a self-hosted AI agent, review the agent’s configuration and code alongside the local model endpoint.
What AURA is and why endpoint compatibility matters
AURA’s documented provider options let you evaluate a locally hosted model without changing agents. Mezmo built AURA as an SRE Agent for Production to help SRE teams manage production environments. Its documentation describes workers with their own prompts and tool lists, and its open-source repository lets you inspect how the agent uses those tools.
The AURA documentation lists provider options including Ollama and describes how to change providers without replacing the agent. Check the current provider guidance before selecting an endpoint for your local model.
Choosing a compatible local model endpoint
Choose a local model for its tool-calling behavior, not its general chat quality. AURA needs the model to select an investigation tool and supply valid arguments before it can use the tool’s response. Test representative read-only requests and inspect which tools the model calls, what arguments it sends, and whether it uses the returned information.
Evaluate candidate models by whether they make valid tool calls and use the returned evidence in representative read-only requests. The local-LLM walkthrough shows one model choice, not a model-size requirement for AURA.
The serving layer must expose an endpoint AURA can use. The walkthrough demonstrates a locally served model through an OpenAI-compatible endpoint. The AURA documentation also lists Ollama as a provider option. Before connecting AURA, confirm that your server exposes a compatible endpoint and passes through the tool calls your model produces.
Pointing AURA at your local endpoint
Configure AURA’s OpenAI-compatible provider to use the host and port where your local model server accepts requests. In the Mezmo walkthrough, AURA points to a local server and names the model by an alias exposed by that server. Check that the alias matches the model you intend to run before starting an investigation.
Keep the model connection separate from the investigation’s tool access. Use the walkthrough to see the connection in context, and check the AURA documentation for current configuration guidance rather than copying another server’s settings.
Running a first read-only investigation
Start the first investigation with a question AURA can answer by reading the running AI stack. In the local-LLM demonstration, the presenter points AURA at a local homelab server and asks what it can tell them about the AI stack. The video describes AURA returning a diagnosis without acting on the environment.
Keep that boundary explicit when you repeat the test. AURA’s documentation describes separate inspection and action roles, along with approval gates for tools that change an environment. Give the first investigation read-only access, then check which resources AURA inspected and whether its findings follow from the evidence it reports.
The demonstration shows AURA investigating a running AI stack with a locally served model. It does not establish the accuracy of its findings or provide performance measurements.
Local inference is not a verified air gap
Local inference does not verify an air gap. Other enabled components may make network connections, so review AURA’s security policy and verify your deployment’s network boundary before describing it as isolated.
Next steps
Watch the local-LLM walkthrough to see AURA connected to a local model and used for a read-only investigation. Consult the AURA documentation for provider and configuration guidance. Browse the open-source repository to inspect the agent before trying it in your environment.
