One of the recurring problems with long-running AI agents is context management.

An agent starts with a clear instruction, accesses documents, invokes tools, makes decisions, and gradually accumulates a large conversation history.

Eventually, the history becomes expensive to process or approaches the model's context limit.

Developers have several ways to deal with this: truncate older messages, summarize them, move information into external memory, or let an orchestration framework manage the process.

Each approach introduces trade-offs.

On September 14, Anthropic introduced on-demand conversation compaction in the Claude Messages API.

The new beta feature allows an application to ask Claude to summarize an existing conversation and return a signed summary block that can replace the original message history in subsequent requests.

It also supports retaining recent turns verbatim and preparing summaries while the conversation continues.

For developers building agents that operate over many steps or extended sessions, this is a useful addition.

But compressing a conversation introduces an engineering question that deserves more attention: which information can safely be summarized, and which must remain authoritative outside the model?

What is on-demand compaction?

On-demand compaction allows the application to decide when conversation history should be summarized.

The process begins with a normal sequence of messages containing user requests, assistant responses, and potentially tool results.

When the application decides the history is too large, it sends a separate compaction request.

Claude returns a summary of the conversation in a special content block.

The application then uses that block in place of the summarized messages when continuing the conversation.

As of the September 14 release, the feature is in beta and uses the following header:

compact-2026-09-04

The request includes a top-level compaction parameter:

"compaction": {"type": "summarize"}

A successful compaction response contains a signed summary block and uses the stop reason compaction.

It does not produce an ordinary assistant reply to the user.

The application sends the returned block on subsequent requests, preserving the signature and keeping it at the beginning of the message history.

Source: Anthropic, Claude Platform release notes.

The intended benefit is straightforward.

Instead of repeatedly sending the full conversation, the application sends a shorter representation of the earlier context.

This can reduce prompt size and make it easier to continue long-running tasks.

However, the summary is generated by a language model, and a summary is not a lossless representation of the original conversation.

That distinction shapes how I would use the feature.

How compaction differs from a larger context window

A model with a one-million-token context window can accept a very large conversation.

That does not mean the application should always send the full history.

Large prompts have costs in processing time, memory requirements, and token consumption.

They can also introduce irrelevant information that distracts the model from its current task.

Compaction attempts to reduce that burden by keeping a condensed representation of earlier messages.

But the two capabilities solve different problems.

A large context window increases the maximum information a model can receive at once.

Compaction decides which information from earlier interactions remains available in a shorter representation.

The second operation requires judgment about what to preserve.

Imagine an engineering agent that has spent several hours investigating a production incident.

During the conversation, it discovers:

• An API request fails only for users belonging to a certain tenant.
• The cause is related to a missing authorization filter.
• A proposed database migration has been explicitly rejected.
• The permitted fix must preserve existing public interfaces.
• Several alternative approaches have already failed.

A useful summary should retain those facts.

If it drops the rejected migration or the authorization constraint, the agent might revisit an unsafe solution later.

The problem is no longer context capacity. It is preservation of decisions and constraints.

Keeping recent turns helps, but it doesn't solve everything

Anthropic's implementation allows developers to keep recent conversation turns in their original form.

The application selects a point in its conversation history.

Older messages are summarized, while newer messages are retained unchanged after the summary block.

This is useful because recent instructions, tool responses, and unfinished work are often sensitive to small wording differences.

The feature also supports preparing the summary in the background.

The application can continue operating on the existing conversation while the compaction request runs, then replace the older history when the summary becomes available.

According to Anthropic's documentation, developers must preserve the messages generated after the compaction request. Those newer turns are not included in the earlier summary.

This sounds like an implementation detail, but it matters.

If an agent continues working while a summary is generated, the application needs to distinguish between:

1. The conversation state that was summarized.
2. New messages and tool results created afterward.
3. The current task state when the summary is finally applied.

Incorrectly combining those states can lose work, repeat an operation, or reintroduce outdated information.

The underlying application still needs to manage state transitions correctly.

The hidden risk: a summary can change the meaning of a conversation

Consider a customer-support agent working with subscription records.

Early in the conversation, the user says:

I have two accounts. Only make changes to my personal account, not the company account.

After many messages, the agent compacts the conversation.

Suppose the summary retains that the user has two accounts but omits the restriction on which account can be modified.

Later, the user requests a subscription change.

The agent may now lack the information needed to select the correct account safely.

This is a hypothetical failure, but the risk follows directly from replacing detailed history with generated text.

The same problem can affect:

• Approved and rejected actions.
• Business rules and exception conditions.
• Security restrictions.
• User preferences that affect financial decisions.
• References to earlier documents or tool results.
• Decisions made during a multi-step workflow.

For low-risk conversations, losing a small detail might be inconvenient.

For an agent with permission to change records, trigger payments, or modify infrastructure, losing a restriction can be costly.

I would therefore separate conversational context from authoritative application state.

The conversation summary can help the model understand what happened.

It should not be the sole record of what the agent is allowed to do.

Signed summaries are not verified facts

Anthropic's compaction blocks include signatures.

The API expects the application to preserve the signed block exactly when sending it in subsequent requests.

This protects the structure and integrity of the compaction artifact as required by the API.

But a signed summary should not be confused with a verified factual record.

The signature doesn't establish that every statement in the summary is correct, complete, or still current.

For example, a compacted conversation might contain:

The user's monthly revenue was $240,000.

Even if that text is included in a valid signed block, it doesn't establish where the figure came from, whether the user was authorized to see it, or whether it remains accurate.

Those questions require separate evidence.

The same distinction applies to agent decisions.

A summary stating that an action was approved is not necessarily equivalent to an approval recorded by the application's authorization system.

This is why I would avoid treating model-generated summaries as a source of truth for sensitive operations.

An architectural example: keeping authoritative facts outside the LLM

I faced a closely related design challenge while building a verifiable AI analyst for a Saudi digital-services company.

The system allows executives to ask questions about operational data in natural language.

The difficult requirement was guaranteeing that numerical claims were supported by real data and that users could not access information outside their permissions.

I designed the application so the language model plans the analysis and explains the results, but never computes or invents the numerical facts.

A deterministic engine executes approved analytical operations and produces evidence records.

Every numerical claim must reference a valid receipt, and the application blocks unsupported figures before presenting the answer.

The data layer also enforces permissions through PostgreSQL row-level security and constrained data-access capabilities.

The evidence records are preserved independently of the model's conversation history, allowing an earlier figure to be reverified under the user's current permissions.

This architecture was not built around Anthropic's new compaction feature.

However, it illustrates how I would approach the same problem in any long-running agent.

If the conversation is summarized, the model may retain the general context of an earlier analysis. But when a user asks to revisit a number, the application should retrieve and validate the underlying evidence again.

It should never rely solely on the summarized text.

That separation makes context management safer because the authoritative facts survive independently of what the language model remembers.

The right way to organize memory in an enterprise agent

For a production agent, I would divide information according to how it needs to be managed.

This design makes a distinction between information that helps the model continue a conversation and information the application must be able to prove.

Consider an agent tasked with preparing a financial report.

The compacted summary might say:

The user requested a regional sales analysis, and the system identified three regions requiring further investigation.

That is useful conversational context.

But the underlying sales figures, calculation definitions, data-access permissions, and report-generation state should remain available through independent systems.

If the model needs the actual figures again, it retrieves them.

If an action has already been completed, the execution journal can prevent it from being repeated.

If the user's permissions change, the authorization layer evaluates the current state rather than relying on an older summary.

This approach reduces dependence on the model's memory accuracy.

Compaction isn't automatically cheaper

A shorter conversation history can reduce input-token consumption in subsequent requests.

However, compaction itself requires an additional model operation.

Anthropic's documentation treats that summarization request as a billable API call.

Therefore, the economic benefit depends on how much history is removed and how many future requests reuse the shorter summary.

Consider an agent that compacts its history and then ends after one additional message.

The cost of generating the summary might outweigh the input tokens saved on that final request.

Now consider another agent that continues for hundreds of steps.

Reducing the repeated context size could save considerably more over the remaining execution.

Prompt caching also changes the calculation.

If an application already receives heavily discounted cache reads for repeated context, compaction may provide less financial benefit than the raw token reduction suggests.

And if compaction causes the model to forget information that must be retrieved again, some savings can disappear through additional tool calls.

I would measure:

• Total tokens consumed before and after compaction.
• Cost of generating the summary.
• Cached versus uncached input expenditure.
• Task completion time.
• Additional retrieval or recovery operations.
• Quality and consistency of completed tasks.

There is no universal percentage of cost reduction that can be assumed from enabling the feature.

The value needs to be measured over a complete workload.

How I would test conversation compaction before production

I would begin with long conversations that have known, verifiable outcomes.

The evaluation should include different kinds of information that the agent needs to preserve.

Constraint retention

Provide a conversation containing explicit restrictions, rejected actions, and approved alternatives.

Compact the conversation, then test whether the agent continues respecting those restrictions.

Task continuity

Give the agent a multi-step task with completed, pending, and failed operations.

After compaction, verify that it resumes from the correct state and doesn't repeat completed operations.

Evidence preservation

Include references to documents, source records, and numerical claims.

Test whether the agent can recover and verify the original evidence rather than treating the summary as authoritative.

Permission changes

Change the user's permissions after compaction and confirm that the application enforces the new authorization state.

Long-term consistency

Run multiple compaction cycles and test whether decisions, task state, and references degrade over time.

For each scenario, I would compare the complete-history baseline with the compacted configuration.

The evaluation should measure the final task outcome, not simply whether the generated summary appears accurate.

A summary can read perfectly while omitting the one constraint that prevents a dangerous action.

Should you use on-demand compaction?

For long-running conversational agents, coding assistants, and multi-step automation systems, Anthropic's on-demand compaction is worth evaluating.

It offers more control over when history is condensed, allows recent turns to remain intact, and provides a structured way to continue from a signed summary.

I would start by using it for conversational information that can tolerate summarization.

For sensitive state, I would retain independent records of approvals, permissions, task progress, external actions, and authoritative facts.

I would also test repeated compaction, because information lost during an earlier summary cannot be reconstructed from the summary alone.

The larger architectural lesson is that an AI agent needs different kinds of memory for different responsibilities.

A model-generated summary is useful for continuity.

Reliable execution depends on knowing which information can be summarized and which must remain verifiable.

Sources

• Anthropic, Claude Platform release notes, September 14, 2026
• Anthropic, Compaction on demand documentation
• Anthropic, Keeping recent conversation turns
• Anthropic, Background compaction

Written by Eslam Hamed, a Technical Lead and AI Solutions Architect specializing in production agentic AI, enterprise RAG, governed data access, and verifiable AI systems.