Headroom review: Compression ratios and retrieval limits

Testing Headroom 0.43.1 with logs, typed JSON and real catalog data, including minification baselines, lossless fallback and original retrieval after MCP restarts.

AgentIndex · Published 2026-11-07 · Updated 2026-11-07

Headroom compresses context before a model receives it. The useful selection question is how much it saves on your input format, which details remain available and what happens when original content must be recovered later.

Our offline tests found large savings on repetitive logs and uniform JSON. They also found a less marketable but useful boundary: an already minified sample of 80 real catalog entries saved nothing in either tested mode. Lossless mode restored a log that its compressor had shortened, and MCP retrieval depended on the selected storage backend and entry lifetime.

Fixed release and actual inputs

On November 7, 2026, we installed PyPI headroom-ai[mcp]==0.43.1. The x86_64 macOS wheel's SHA-256 was a35c3bbbe13b8f46abb5e7007d710ce6916c4f028d72fe9e57cba62c7fc9367a; its installed native binary matched the verified wheel. Nine selected Python files also matched release commit 2d3ee51846435ef5b320eea4806af14e602b0d8c. Fixed release

Execution used macOS 26.6.2 on Apple M1 Pro, with x86_64 Python 3.13.1, tiktoken 0.14.0, LiteLLM 1.103.10 and MCP SDK 2.3.0. The initial unrestricted dependency install tried building cryptography 50.0.2, whose Rust requirement exceeded our installed 1.82 compiler. A binary-only install resolved cryptography 48.0.1 successfully, and pip check passed. This was an installation boundary of our environment, not evidence that installation fails everywhere.

We ran 18 library cases and 12 actual tool calls across four stdio MCP processes. Inputs included artificial logs and typed arrays, the first 80 public AgentIndex catalog records projected to six fields, and a frozen explanatory article. The fixtures, lock file and full results retain their provenance and outputs.

Tokenizer vocabularies were prepared before the offline run. During execution, HEADROOM_OFFLINE=1 was set, the ML compressor was disabled and Python socket auditing recorded no TCP connection attempts across five processes. There were no model or provider calls. These measurements establish neither answer quality nor an API bill reduction.

Compare against JSON minification, not just pretty printing

The table counts content text with o200k_base, excluding message envelopes and retrieval/tool-call overhead. The library selected gpt-4o for accounting; that parameter did not cause a model request. We also recounted its complete returned messages with the same published counter. MCP's own model-accounting path is separate and is not mixed into this table.

InputOriginal content tokensDefault resultAgent/lossless result
1,002 artificial log lines15,02210915,022
500 uniform JSON rows, formatted20,5145,0295,057
Same rows, already minified11,0155,0315,067
120 rows with type/quoting edge cases9,3722,8212,351
80 real catalog entries, formatted7,3634,6264,491
Same catalog, already minified4,5014,5014,501
Explanatory article, ML disabled1,6571,6571,657

The uniform array retained a substantial advantage over ordinary minification. The heterogeneous real catalog did not: plain minification already reached 4,501 content tokens. When fed that minified input directly, both modes triggered the inflation guard and returned the original. A large percentage measured against formatted JSON therefore does not establish an equally large benefit over a compact payload you already send.

This catalog was a fixed sample, not the whole directory or a representative workload benchmark. Its decoded records preserved all 80 entries, including the seven NOASSERTION licenses. The six selected field values were copied unchanged.

Lossless mode means checking the final output

Default log compression kept the FATAL message, incident identifier, Chinese text and traceback while omitting 997 INFO lines. The shorter representation carried a retrieval hash. That is useful reduction with omitted information, not a copy of every original line.

Running the same log through densify() returned the original content, reported lossless=True, and counted one reverted message. Its transform list still described the attempted log compression, so the transform name alone was not a valid savings metric. The final text and final counts mattered.

Structured inputs behaved differently. We decoded the returned compact representation and compared canonical JSON, retaining distinctions between integers, floats, booleans, null, missing keys, empty strings, the literal string null, quoted commas, newlines and nested values. Selected outputs from both modes matched their original structured data. Lossless mode additionally reported successful verification. Compression entry points, compact-format decoder

The compact output was not always a JSON array. A consumer expecting json.loads(output) to return a list needs the appropriate expansion path or a different integration contract. Exact programmatic round trips also do not prove that a model will interpret the denser notation correctly.

Two controls clarified configuration. A short message stayed unchanged. A large user message was protected by default; explicitly enabling user-message compression and setting protect_recent=0 reduced it. Our 1,657-token explanatory text also stayed unchanged with the ML compressor disabled. None of these is evidence that prose compression with the optional model succeeds or fails.

Original retrieval worked, with storage boundaries

We launched the unmodified published HeadroomMCPServer over actual stdio, disabling optional local-proxy probing. It exposed headroom_compress, headroom_retrieve and headroom_stats; all three were called. Compression returned a hash, and retrieval returned the complete Unicode log with an identical SHA-256.

With the default SQLite backend, a second MCP process using the same dedicated workspace recovered that original. With HEADROOM_CCR_BACKEND=memory, retrieval worked inside the first process but missed after restart. This verifies the tested local restart behavior, not cross-host durability or an unlimited archive.

A separate store-level test used a one-second TTL: retrieval initially worked and expired after 1.2 seconds. The tested MCP compression code sets its entries to one hour; we did not wait an hour to test that expiry path. Store implementation, MCP implementation

An unknown hash and empty compression content both produced an error field inside the returned JSON while the MCP envelope had isError=false. A caller checking only the protocol flag would miss these application-level failures. Inspect the parsed result before assuming compression or recovery succeeded.

Where it fits

Headroom is worth evaluating when your actual tool payloads contain repetitive logs or uniform arrays and you can choose an acceptable retention mode. Start with your real serialized input, compare against simple minification, and validate recovery as well as the compressed view. The release requires Python 3.11 or newer; our platform and dependency lock cover the environment above.

Already compact heterogeneous JSON may offer little benefit. Lossless verification may preserve everything by returning the original, and optional ML, proxy routing, agent wrapping, TypeScript integration, provider prompt caching and model answer quality were not tested. Recoverability only helps when the caller requests the right original while it remains available.

For selected browser workflows that generate tool results, see our Playwright MCP form and session tests. For a different retrieval task—finding previous coding-agent conversations—see the ctx history-search review.

For building and checking the server-side contracts behind MCP tools, see our FastMCP validation, timeout and scope tests.