Keep Large Tool Outputs Out of Your AI Agent's Context Window

Ask an AI agent to investigate a busy Kubernetes namespace, and one tool call can return an event list or resource dump longer than everything else in the conversation combined.

Most agent frameworks pass that whole result to the model. The context window fills, costs climb, and the agent loses track of what it was doing, or the call fails outright.

AURA does something different. The full output goes to disk. The model gets a pointer and a set of tools to read only the part it needs, the way an engineer would grep a file instead of reading it top to bottom.

TL;DR

• What it is: AURA Scratchpad, a built-in feature that intercepts oversized MCP tool results, saves them to disk, and gives the model structure-aware tools to extract what it needs.
• What this example does: A single Kubernetes investigation agent with Scratchpad turned on and per-tool size thresholds.
• What you control: Token thresholds for each tool, a cap on how much any one extraction can return, and how much of the context window stays reserved for reasoning.
• What you see: The CLI reports tokens intercepted versus tokens extracted.
• What you need: kubectl access to a cluster, Node.js, and a model provider API key.
• Cost: AURA is free and open source (Apache 2.0). You pay only for your model usage, and Scratchpad is designed to reduce it.
• Source: examples/complete/scratchpad-single-agent.toml

What this example does

1. Count. AURA measures every tool result in real tokens, using the tokenizer matched to your model.
2. Intercept. A result over that tool's threshold is written to a file. The model receives a short pointer with the file name, size, line count, and format.
3. Explore. The model uses eight exploration tools to pull out only what it needs:‍
   
   1. schema maps keys and types to line ranges.‍
   2. item_schema maps fields across an array.‍
   3. iterate_over selects across items.‍
   4. get_in, grep, head, slice, and read handle targeted reads.‍
4. Cap. Every extraction is token-counted before it's returned. If it's too large, the tool refuses and suggests narrowing the range, paginating, or requesting fewer fields.

Errors always pass through inline, so the model can react to them immediately.

This example sets thresholds per tool, and the longest matching pattern wins:

• Lower thresholds for pod logs (1,500 tokens), event lists (1,500 tokens), and resource lists (2,500 tokens). These outputs are large and rarely needed in full.
• A catch-all default of 5,120 tokens for everything else on the server.

Who it's for

Jobs to be done

• On-call engineer: When the agent queries something that returns a huge result, I want it to keep investigating, so I don't have to restart with a narrower question.
• Platform engineer: When I connect MCP servers that can return unbounded output, I want a guardrail on context usage, so one tool can't break every run.
• Engineering leader: When we scale agent usage, I want token cost tied to what the agent needs, not to how big the tool output happens to be.

Pain points it removes

• Context window overflow. Large results stay on disk instead of filling the window.
• Cost driven by data size. Token cost follows what the model extracts, not what the tool returned.
• Lost reasoning. A reserved share of the window (20% in this example) is held back for reasoning and the final answer.
• Guessing spirals. An oversized extraction is refused with suggestions, costing one turn instead of derailing the run.
• Hand-written truncation. No custom code per tool to trim outputs.

What it replaces, and why it's better

The value it drives

• Reliability on real data. Agents keep working on large clusters and verbose tools.
• Lower token spend. The intercepted-versus-extracted ratio is reported for every run, so savings are visible.
• Smaller models stay viable. Bounded context makes non-frontier and local models more practical for multi-step investigations.
• Exact values, not summaries. The model reads the specific fields it needs.

What to measure in a pilot: tokens intercepted vs extracted per run, runs completed vs failed on context limits, and cost per investigation before and after.

Why AURA

• Production-tested. AURA started as Mezmo's internal harness for operating our own SaaS. Our engineering and SRE teams still use it in production, and it powers thousands of agent sessions each month in Mezmo's hosted platform.
• Works across providers. Exact tokenizers for OpenAI and Gemini, a calibrated approximation for Claude, and a safe fallback for everything else.
• Built into orchestration. Specialists in an agent team can each have their own Scratchpad settings.
• Runs inside your security boundary. Deploy in your own infrastructure, including air-gapped environments.
• Traceable end to end. Token usage is reported in the CLI and in streaming events.

Start from where you are

• Running an agent team? Scratchpad settings can be inherited or set per specialist. See orchestrated-worker-overrides.toml.
• Deploying in Kubernetes with Helm? The Kubernetes SRE quickstart values file enables Scratchpad by default.
• Running a local model with a small context window? Scratchpad pairs well with local and air-gapped setups.
• Not sure which tools to gate? Turn it on for any tool that can return unbounded output, such as list-everything endpoints and log queries. Skip small, bounded tools. Interception there only adds a turn.

Look inside the config

Turning Scratchpad on takes three settings, and the config won't load without all three:

memory_dir = "{{ env.HOME }}/.aura/scratchpad"

[agent.llm]
context_window = 400_000

[agent.scratchpad]
enabled = true
context_safety_margin = 0.20
max_extraction_tokens = 10_000
turn_depth_bonus = 6

Thresholds are set per tool, per MCP server:

[mcp.servers.kubernetes.scratchpad]
"*" = { min_tokens = 5_120 }
"pods_log" = { min_tokens = 1_500 }
"events_list" = { min_tokens = 1_500 }
"resources_list" = { min_tokens = 2_500 }

turn_depth_bonus gives the agent extra turns, so exploration calls don't use up its investigation budget.

Try it in three steps

1. Install AURA (Linux or macOS):

curl -fsSL https://raw.githubusercontent.com/mezmo/aura/main/scripts/install.sh | bash

2. Start the Kubernetes MCP server in read-only mode:

npx -y kubernetes-mcp-server@latest --port 8081 --read-only

3. Set your model key, download the example, and run it:

export OPENAI_API_KEY="sk-..."
curl -fsSLO https://raw.githubusercontent.com/mezmo/aura/main/examples/complete/scratchpad-single-agent.toml
aura --config scratchpad-single-agent.toml

More detail: AURA Scratchpad docs.

Then ask for something broad, like "Find anything unhealthy in the cluster over the last five minutes," and watch the intercepted and extracted token counts.

FAQ

Does Scratchpad change what the agent can do?

No. It changes how large results reach the model. This example's Kubernetes tools are read-only.

Will every run save a lot of tokens?

No. Savings depend on how large the tool outputs are. Runs with small results see little difference. Runs with large results see the most.

Where is intercepted output stored, and is it secure?

In memory_dir, as plaintext. AURA doesn't tighten permissions on what it creates, so use a directory only your user can read. Avoid shared locations like /tmp.

Does it work with Claude, Gemini, and local models?

Yes. Token counting is exact for OpenAI and Gemini, calibrated for Claude, and falls back safely for other providers.

Next steps

• Engineers: Clone the example and compare a run with Scratchpad off and on.
• Leads: Track tokens intercepted vs extracted over a week of agent use to estimate savings.
• Everyone: Join the AURA Slack community.