Build an SRE Agent Harness for AIOps Without Context Blowout

How to Build Your SRE Agent

The missing runtime layer for AI in operations

PlatformCon 2026 · Chip Johnson and Mike Shearer, Mezmo · 33 minutes

Why Not Just Use Claude Code With an MCP Server?

[00:03] Chip Johnson: Okay, so we are up and running. We are here at How to Build Your SRE Agent. I am Chip Johnson. With me also is Mike Shearer, who is currently labeled as "Other Chip" by Zoom, but he's really Mike.

[00:21] So we are here today to talk to you about our SRE agent harness, which is called Aura. Let's go ahead and get into it.

[00:36] The first question, of course, is: if we are here today to talk about a specialized project for building SRE agents, the natural first question is why? Why don't you just use Claude Code with an MCP server?

[00:52] The answer is that you need a harness — or we found we needed a harness — that was more oriented toward day-to-day operations work rather than software development. Claude Code's a fantastic software development harness. I have it open in the background right now on the computer I'm presenting to you from. I use it every day. Fantastic tool.

[01:20] The big issues that we run into are around Claude Code not really having a structured schema for setting up a planned workflow. There's kind of an impedance mismatch in the guardrails, right? Everybody's hit yes — except, shift-tab their way into auto-accept mode — and let things run. It's really meant for local machine use, and not necessarily for something you daemonize as an agent out in your infrastructure. It's really good at making sure it doesn't screw up your git history, but it'll dump your secrets all day long.

[02:00] It also doesn't emit any events by default, so it's hard to debug it or review the decisions that a standalone development agent makes, whether it's Claude or Cursor. It writes its own trail and leaves out a lot of its internal reasoning dialogue. And it's really eager to ask for human input generally.

[02:30] So we found that if we had it crawling files to correlate an investigation, or a lot of other use cases, then it would just sit there and ask us: Can I access this? Can I access that? Be throwing perms every day. So we really wanted to build a harness that was custom-purpose for SRE and operations work. And thus Aura was born.

What an Agent Harness Actually Does

[02:53] Chip: So let's talk a little bit about agent harnesses. A harness — again, I have already alluded to this — but Claude Code is itself a harness. It wraps around the Anthropic models and makes them suitable for doing software development work. And it excels at this. Cursor is similar.

[03:20] They do that by bringing in behavioral tools: structured access to skills, for instance. Built-in prompts that can get loaded on the fly, depending on what you're doing at the moment.

[03:35] They give you an execution environment — the sandbox, plugin interfaces — and they have a bunch of opinionated defaults. There's all those prompts under the hood to manage memory and context, and really give you permission sets that all work out of the box.

[03:50] So the idea is that the harness is the control plane around your model, so that you don't have to keep rebuilding those pieces from scratch.

[03:59] I'm going to kick off to Mike now, because he was actually there for this, to talk about how we actually developed Aura.

Building Aura at a Telemetry Company

[04:09] Mike Shearer: Yeah, thanks a lot, Chip. First off, I'm Mike. I'm not "Other Chip." Aura, and building this harness for Mezmo, has kind of been my work life for the past year.

[04:25] When we started, there weren't a lot of good server-side options. As Chip mentioned, just using a coding agent wasn't quite there. Models weren't quite as strong as they are now.

[04:40] So we had this interesting situation. Mezmo is a telemetry company. We handle logs. We've got some great pipeline products. We figured out a really interesting way to serve a much more targeted context via MCP. And that was all great, but we didn't really have something amazing to run it in — other than signal from a few customers that it's a very useful addition to Claude Code, but they weren't running it server-side.

[05:05] So we really had no choice but to dive head first into how we would tackle this and make it work for us.

[05:20] One of our first times connecting, we found that even with filtered logs — we're a company that manages everybody's logs, we have a lot of them — we ended up in this situation where just a 15-minute span in one of our environments was like 10 million tokens. And models at the time were like 200K max-ish. So there was a lot of engineering that needed to happen to make that work well.

[05:50] We made some interesting choices around how we tackled this. I definitely don't regret them; it worked out really well. We decided to start in production.

[06:05] And no, that doesn't mean that we just let our first pass at an agent go wild and do remediations and all this stuff. It was very staged, but everything was very goal-driven from the beginning. So we started with what we had, tried to see how much we could take on, and we had this really interesting development cycle of just charging forward and figuring out: okay, this breaks — even if it's kind of an edge case, we don't want that. We're a company that supports a lot of very large companies, important logs. We want to be reliable.

Production First, Edge Cases Early

[06:38] Mike: There wasn't really a playbook. A lot of what we found in the literature really didn't apply to our situation. It was very obvious that MCP, initially, was really developed to serve your happy little buddy on your machine — which is your coding agent or your chat agent. There weren't the right considerations for it to really play well in production. So we had to solve all of that ourselves.

[07:05] One thing that we found — I alluded to this a little bit earlier — was that fixing edge cases now was a really interesting and helpful philosophy for us. You can do a lot more cautious product development, where that's an edge case, we're not going to hit it as much. Well, the stakes were too high.

[07:25] Edge cases around how to solve saturation from too many logs. How to split tasks up and figure out how to break up this context window. And then just insane amounts of coverage for various edge cases, whether with the provider — we had a lot of problems with MCP schemas up front — but really going production-first and just head-first into it was a really good way, until we could get to the point where we were confident that the agent was going to make a good decision.

[07:45] We weren't remediating early on, but it took a lot to get us to the point where we even trusted it to not over-query our system to death. So: solve your use cases, dive in, remove blockers as they come. That's probably the right way to do it. Compared to a lot of traditional API development, there's a lot of unpredictability with agents. So you just have to do an insane amount of manual testing. It's not something that is writing integration tests.

The Lossiness Trap

[07:57] Mike: When we started getting some signal, we got into this worse situation. We were actually pretty confident with things, and answers were starting to look right. And we realized this scary thing with AI: sometimes actually having it fail hard is better than when it gets to the point where the answers start to look right.

[08:25] So for handling these large log volumes, not only were we doing log reduction techniques and other methods, but we eventually split off into an orchestration framework so that we could say: this worker handles this part of digging into telemetry, this worker handles this other part. So it's more focused. There's an isolated context window that can't blow up.

[08:50] Everything was looking good. But a colleague and I pointed out at some point — we were getting these answers that were kind of needle-in-a-haystack. Find all these things that are broken. You look at it at a glance, and you started to trust the system. It was interesting, because the answers looked very right. But under the surface, we actually started cataloging things from a couple of very specific production events, and we were actually having a large amount of lossiness. Basically 16% of the findings were gone. What was there looked right.

[09:04] We had to re-architect and keep moving forward on edge cases to make sure that we weren't losing data. So as you dive into building your agents, that's where this relentless testing comes in. You have to get a feel for it. There's a lot of these holes you'll get into where things start to look right — and just when you start to trust it is when you need to dive in and look for some edge cases and vetting.

Orchestrator and Worker Architecture

[09:35] Mike: This is kind of the design we landed on. The coordinator can route and plan. That's where your query goes. That's your main buddy, if you were thinking about it like Claude Code. This is actually where you're chatting — or in our case, this is where unattended prompts, maybe if something's running in the background, would come through.

[09:55] It's the coordinator's job to split up tasks as a DAG and run things really wide. There's just a huge amount of context that can come out of any of these tools or MCPs.

[10:10] For example, the Prometheus MCP or the Kubernetes MCP — even in a trivial environment, a lot of those can produce, on a simple call, more data than any LLM can handle on its own. So you really have to be splitting things into smaller pieces.

[10:25] This gives a buffer between the main coordinator and the MCP tools. If it overestimates how much a worker can handle, no big deal. You haven't blown out your context window. You haven't killed your worker. It can recover gracefully.

Intent Disclosure as an Audit Trail

[10:41] Mike: Something interesting we did with this whole process, that we thought was really missing in the coding harnesses: at every step of this fan-out — the DAG, and then solving problems that you see here — the LLM actually discloses its intent, and the reason why, all the way down to the point of why it called certain arguments.

[11:05] We did that just for debugging, because writing multi-agent orchestrations is actually incredibly hard and really time-consuming testing-wise. So we wanted to be able to see: hey, single agent, we've got a black box here. You just have to kind of guess, or have experience, as to why the LLM did something. At least when it's broken up into small pieces, you can see where things are split up.

[11:30] And we realized later that we had created an audit trail for the agents as well. That wasn't the intent, but that's one of the things that came out of it.

[11:45] But you really need those things when you're not attending an agent — you're not watching it, you're not watching it through a client, you're not re-prompting it, you're not able to very easily cancel things as they come up. So you have to think a little bit more proactively about this.

[12:00] There's also a little bit of tuning that Chip mentioned. Your coding agents are very eager to work with you. This is changing a little bit — they're getting a little bit better at it. But we had to do some specific tuning to get it to at least run a long enough task to bring something back. You kick off something in an incident, and if it's just coming back asking questions every two seconds before it's even dug in, it's just not very useful.

Setup Walkthrough

[12:21] Chip: Let me switch my share real quick. First of course is the repository: github.com/mezmo/aura. You can clone from here.

[12:45] If you have the correct environment variables right now, you can just get it up and running in a very basic mode, without any config in it, by running docker compose up straight from this repo. But that's not how we expect most people will be picking it up. We think most people will want a straight binary.

[13:10] So you can see the project's in Rust. And getting going from the binary is really easy. We have just a basic aura init. It'll take you through a quick setup path to write its own basic config.

[13:35] Since I have OpenAI environment variables exported already, it detected that. But you can see right now we can do Anthropic, Bedrock, Gemini, fully local if you want to air-gap.

[14:00] I know we had a question about air-gapping with OpenClaw in the chat. The question was: have we considered using an air-gapped, self-hosted OpenClaw agent to achieve this? They've seen good results for it with a similar pattern.

[14:20] The answer is yeah, we can absolutely air-gap, and you can run this either locally or against a remote model wherever it's hosted. The "why not OpenClaw" is similar to the question of why not Claude Code. Again, you really want that full decision traceability when you've got something sitting in your ops environment that's making choices. You're going to want to be able to audit every single choice that's made, and why it made those choices.

[15:00] So I'm going to run with OpenAI here and just let it pick up my environment. And then it's going to suggest, for my orchestrator — the main coordinator — that we use GPT-5.5. Although I can pick any of those hundred-odd models there.

[15:30] Let me cat my config file here. This is our out-of-the-box config. By default, it's going to set you up an orchestrator and a couple of workers.

[15:50] The orchestrator has a very basic prompt out of the box, and then tells you which workers we have. By default, we have the incident responder. Its job is triage — it'll fetch alerts, check on-calls, handle escalation. The metrics analyst is going to go through various alerts you might have hooked up via those MCPs. And the log analyst's job, similarly, is to go through your log analysis: look for error patterns, correlated events, that kind of thing.

[16:30] It also is going to pop up and say that if any of your worker configs don't have any MCP tools yet, we should tell them that the worker can't do any live query — but it can still be useful for offering you general advice. But really this is designed to be used with tools, and hooking up into whatever you already have in your environment.

[17:00] So we've got the basic routing guidance. How it should handle parallelism, delegation, and the red/green/yellow for when responding back to users.

[17:30] Then we get down to our basic setup: OpenAI with GPT-5.5. Default context window. The scratchpad is an internal communication tool for the different agents to be speaking with each other and passing information back and forth.

[17:50] Out of the box this doesn't have any MCPs. Moving down, we're in orchestration mode — number of planning cycles. And really I want to get down into: here's the incident responder. Again, it has its own context here.

[18:10] Also, we don't have it set, but your log reader might not need GPT-5.5. If we were on an Anthropic setup, maybe you want to have your log reader running on Haiku so it can be faster and use fewer tokens. You aren't going to necessarily need the latest frontier model for each worker. You can go ahead and start tuning those for use, and for cost optimization.

[18:30] We also have a couple of workers that are commented out by default. We have the ops engineer. The idea of the ops engineer is that it is the worker that knows how to operate the software that's actually being supported by this infrastructure. So if there are tools inside of your bespoke tooling for upping, downing, restarting clusters, hopping — that sort of thing — the ops engineer worker's job is to take care of that.

[18:45] And we also have the idea of the runbook engineer, who can talk to your runbook store and pull runbook scenarios, and also write new ones if we have an update to an existing scenario that we need to push out for use next time. Or if there is a totally new scenario that we ran into that we've never seen before, we can automatically amend the runbook set.

[18:50] They aren't created by default, but we do leave them scaffolded so that people can figure out what to do by example. And of course, you can build whatever workers you want, depending on how you want them to go.

Chat Mode and Standalone Server

[18:51] Chip: So that's the init process. Very simple. And then you can kick off in a very familiar chat mode with just a local one running on your laptop, or you can decide to run it as a standalone web server. I'm going to pass off to Mike here to continue.

[19:20] Mike: Thanks, Chip. I'm going to pull a few things up. Chip mentioned a couple of different modes of running this. I'm running the CLI detached right now, so I've got a local server set up. It's actually connected to one of our real environments. We're going to do some real-time diagnosis. Wish me luck with the demo gods.

[20:00] On your left we have a more fleshed-out config. This is not terribly far off from the defaults that Chip was mentioning there from the bootstrap, but this has our own provider. This has a little more opinionated setup with the orchestration itself. And this has full MCP and everything. Like I said, this is going to a real environment here.

Live Demo: Diagnosing a Real Incident

[20:25] Mike: So this is running on the machine. Logs are on the left. On the right-hand side, we'll go ahead and kick off the CLI so you can take a look at what it looks like. Nothing too unexpected if you've used any sort of chat-based agent.

[20:50] What we're going to do here is actually take an excerpt from a real incident. And it's kind of funny in the dev process, right? We have Aura, and we spend so much time thinking about and developing Aura, sometimes we forget that we can use it.

[21:05] There's this particular thing that happened with me and one of our head SREs as we were going through things, and it was a failure on Aura in one of our environments. We spent about an hour debugging it. It wasn't a big deal. We fixed it. But later I was thinking: well, why didn't we just use Aura to debug that and dogfood it? Sometimes I forget it's there.

[21:18] So let's take a look and let's do some really sloppy debugging and see how it goes. We're actually going to take this whole thing and just sloppy-paste the whole conversation out of chat. We're not even going to give the agent much of an idea as to any additional context, and let's see how it does.

[21:50] We're not using the LLM the way you should. We're not going to be super specific. We're not going to help it out a lot. We're just going to let it figure it out.

[22:10] Now, this is actually hitting one of our reasonably loaded-up environments. It's not our live customer environment, but it's loaded down a little bit more in some ways, so it ends up being a great test for us. There's a lot of experimental stuff that happens there, or things that are still being developed. So they've got more errors, more flood. It's an interesting environment to develop against, but for the most part, infrastructure-wise, not too much different than what we'd run in a more interesting environment.

[22:53] Got lucky today, because it's actually going pretty good. We essentially pasted in a bug. What we had happen is a downstream timeout from an MCP wasn't being handled well. So we pasted in pieces from both of the logs on that, and we actually get a complete diagnosis based on the logs that are there, because it's just essentially tracing some of our Rust logs. It's even providing remediation and code as to how we might figure out how to overlap these timeouts.

[23:30] So it doesn't always work out this way. It's generally pretty good at finding things. But that was an interesting one, given that we spent an hour on this internally — and we saw how fast that came back.

[23:50] That's running Bedrock Sonnet 4.6 under the hood, so it's not running a dumb model or anything, but it's also not running anything top-tier. And the results are pretty interesting.

[24:10] As we go, we just keep trying to load it more and more. It's handling bigger and bigger problems all the time. It's kind of crazy to think about this development process — nine months ago, a vague query like that, there's no way we would have been able to handle it context-wise. That would have just been cascading worker failures, and the agent coming back saying it can't help you. So a lot of engineering goes into making sure that we can handle those big things, and we'll keep pushing to handle bigger things after that.

Human in the Loop

[24:23] Chip: We buffered in a bunch of time in case the LLM was slow here, so we might have some extra question time at the end. Are we planning to support a Slack interface? That's a great question. Let's pause and talk about that real quick.

[24:45] Mike: So that's actually our next release. Not a direct Slack interface, but let me back up a little bit. What we're getting ready to release is a public human-in-the-loop integration. This supports arbitrary webhooks.

[25:10] We don't have a public Slack integration. We have one internally, but we don't have a public Slack plugin for that. That's not off the table — it may come out at some point. But we're actually dogfooding this internally right now. That'd be interesting, I'll show you the config, since we have that set up here.

[25:30] Chip: And by "just released," we mean like yesterday.

[25:35] Mike: Yeah, we just started testing the final version yesterday. It's not the PR submitted and everything — there's a little bit of work to follow that up — but it'll be public here pretty soon.

[26:00] So you can arbitrarily configure human-in-the-loop. This allows us to say: here's your destructive tools. In our case, we set up for this demo some specific log tools that can get called that are actually heavy enough that you want to be a little bit careful about how much you're pulling — especially in one of our test environments, where there's limited bandwidth on some of that stuff.

[26:30] It supports both conversational as well as webhook. What you saw here, we're running it in conversational mode. So if it were to run into one of those tools specifically, it would have popped up a little dialogue in the CLI asking you if you can go forward.

[26:50] The way we're running it internally right now is that it calls a webhook. So if it hits these very specific tools that the agent is gated on, the agent actually can't execute those tools at all. It's deferred to the webhook. You respond to the webhook with what you would expect from turning an agent: approval, denial, or denial with reason. And that gets folded back into the agent's decisions.

[27:15] But it sandboxes out. It legitimately can't run these tools without approval, and we support both modes on that. Right now, it's just our schema that we posted for how that webhook works. We'll figure out what we want to do for integrations in the future. But internally, we have an internal app that translates this right now to be able to run this in Slack. So these things are posting in Slack right now.

[27:35] We've got a poor logging agent that I let sit there as a demo. It's spinning, finding stuff all day, and I just leave it to die. I never approve any of them. But it's been a fun exercise to watch it run.

[27:50] So that's new as of this week, and we have a little bit more fun stuff planned here soon for other use cases: production notifications, and being able to have a little bit more confidence in your agent running essentially headless. You're not monitoring. Your fingers aren't on the chat. There's a lot more that has to go into making sure that we feel comfortable running that, especially when you tie in remediations into the equation.

Core Capabilities and Guiding Principles

[28:05] Chip: So here we are talking through the walkthrough. Clone it, build it. We are working on the binary distribution right now, but currently you need to clone it and build it, or just docker compose up — we do have the Docker images done.

[28:30] The core capabilities we're working into Aura are investigation, root cause analysis, and diagnosis. We want to be able to talk to anything you've got, always share its reasoning, always give you its findings and any decision it made.

[28:55] And then, being able to work on remediation is terrifying. You don't want to do remediation out of the box the first time with one of these tools, obviously. That would be a lot of trust to put in something in a production environment that your livelihood depends on. But over time, as it becomes more and more reliable in the probabilistic system, you want it to be able to start handling things once you know that it's probably going to do the right thing. I mean, maybe you don't. Some people do. So you'll be able to make those decisions.

[29:15] And then we want to move into prevention and hardening use cases, cost optimization, change validation. The idea is that since these tools are inherently capable of being self-improving by building better context, then we want to build better context and feed it back.

[29:30] As we're building forward, we're committed to four guiding principles for how this thing gets worked out.

[29:45] 1. Self-improving. Each meaningful finding needs to improve the next one. We want it to continue to be self-improving.

[30:10] 2. Open. We are Apache 2.0 licensed. We are staying open source. If you want to take this and build some competition, that would be sweet. I am in the business of making a de facto standard, not of trying to do a commercial bait-and-switch. We absolutely want this to be a completely free product for anyone to pick up, and all the pieces you need to be successful will be in the kit.

[30:40] 3. Composable. The tool stack you already have — everybody's already got Slack and monitoring and a whole toolkit they've put together. You don't need to adopt a whole new toolkit in order to use this. It should be something that can plug into whatever you've got today. Right now, that interface is MCP, but as new interfaces emerge and new standards come out, you should be able to hook this up to whatever you're using.

[31:10] 4. Progressive trust. Out of the box, I don't expect you to trust an LLM with production work. I expect you to ask it to look at things and give you ideas, and be kind of a sidecar for you. Something that you can use as an advisor at first, and then over time maybe hand it lower-consequence decisions. And as it becomes more trustworthy, you can tune the kind of oversight you have.

[31:30] So these are the things that guide what we decide to put in the product and how we'll build them. Anything that doesn't align with this will probably be closed and merged.

Q&A

[31:38] Chip: So that's our wrap-up. We can get into Q&A. If you want to try it, it's up on GitHub today. We have a community Slack — QR codes right there. We are looking to build this together in the open. I want to hear about new use cases, new things people are doing with it. I hope you'll join us, and I hope you'll give it a try in your own environments and tell us what you find.

[32:13] Can I use it with a LiteLLM endpoint? I don't know, actually. Mike, do you know?

[32:25] Mike: I guess it depends on the use case. Let me make sure I'm reading this right.

[32:40] Chip: I don't know the answer to that one off the top of my head. I might be conflating it with a different product. I'm thinking of a particular router and interface over the top, which should play fine, because we support the OpenAI standard as a default if you want it. If we're talking about something else, then I'm off base.

[33:02] But absolutely, I would love an issue requesting LiteLLM support. If you find that it's something we need to add, I absolutely want to add it for you.

‍