Skip to content
← FrontpagearXiv hallmark logoarXivAnalysis

Researchers introduce ARCBench evaluating AI agent rule compliance under context compaction

Topographic elevation contour card for Researchers introduce ARCBench evaluating AI agent rule compliance under context compaction
VISUAL ARTWORK:Original editorial artwork. Third-party brand marks used under nominative fair use.

What happened.AI researchers introduced ARCBench, an empirical evaluation suite measuring whether autonomous execution-loop agents maintain adherence to operating guidelines and safety boundaries after context compaction.

Why it matters.Autonomous agents running multi-hour workflows rely on periodic context compaction to avoid exhausting prompt token limits. Demonstrating that summarization filters out negative constraints proves software teams cannot rely on conversational memory to enforce organizational safety boundaries.

The technical specifications:

  • Evaluation scale: Tested 14 frontier language model agents across 2,800 multi-turn interactive execution loops containing strict negative constraints.
  • Compaction degradation: Discovered that agent rule compliance drops by 42 percent on average when long conversation histories are summarized into compact memories.
  • Constraint erasure: Showed that automated compaction routines systematically prioritize immediate task progress while discarding implicit negative constraints like rate limits.
  • Safety drift: Revealed that compacted agents violate environment access policies three times more often than uncompacted baselines on long-horizon jobs.
  • Architectural fixes: Demonstrates that pinning immutable rule manifests into system prompts prevents safety drift during repeated context summarization passes.