Suggestion: Native token-efficient serialization for agent messages to reduce context overhead

[makroumi]

Problem

AutoGen agent messages are serialized as JSON by default. In multi-agent pipelines running at scale, JSON syntax overhead silently consumes a significant portion of the context window before any reasoning begins.

I measured this extensively on production workloads:

~44% of tokens in typical AutoGen agent payloads are pure syntax noise.

Brackets. Repeated key names. Whitespace.
None of it is intelligence. All of it costs money.

At scale:

• 10M agent loops = ~$59K wasted on GPT-4o
• Context window degraded before agent reasons
• Silent state failures pass through undetected

Why This Matters For AutoGen Specifically

AutoGen's GroupChat and nested agent patterns are particularly affected because:

1. Message history grows with every turn
2. Every agent sees the full serialized history
3. JSON key repetition compounds across turns
4. No validation layer catches broken states
   before they reach the LLM

The result: agents reasoning on degraded context, higher hallucination rates, and ballooning inference costs.

Proposed Solution

An optional pluggable serialization layer that lets users swap JSON for a more token-efficient format.

I built ULMEN specifically for this problem.

Benchmarks on NVIDIA Tesla T4:

Beyond compression, ULMEN adds a Semantic Firewall that validates agent state transitions before they reach the LLM:

• Rejects orphaned tool calls
• Catches backwards step transitions
• Validates enum states
• Raises structured errors vs silent failures

This directly addresses a common AutoGen failure mode where broken inter-agent state passes silently through JSON and triggers downstream hallucinations.

Implementation Sketch

Non-breaking. Opt-in.

# Current
assistant = AssistantAgent(
    name="assistant",
    llm_config=llm_config
)

# Proposed
assistant = AssistantAgent(
    name="assistant",
    llm_config=llm_config,
    serializer="ulmen"  # opt-in
)

ULMEN is a drop-in Python/Rust library.
No schema compilation required.
Pure Python fallback if Rust unavailable.
BSL license - free under $10M revenue.

Reproducible Benchmarks

Live notebook in the repo - run on your own data to verify:

github.com/makroumi/ulmen

Happy to submit a PR implementing theserializer interface if there's interest from maintainers.

Questions For Maintainers

Is there an existing serializer interface I should hook into?
Would a plugin approach be preferred over a native integration?
Are there specific AutoGen message types that should be prioritized?