Skip to content
← FrontpagearXiv hallmark logoarXivAnalysis

Researchers introduce CrossSWE benchmarking multi-repository coding agents

Listen to Briefing
0:00/1:41
Topographic elevation contour card for Researchers introduce CrossSWE benchmarking multi-repository coding agents
VISUAL ARTWORK:Original editorial artwork. Third-party brand marks used under nominative fair use.

What happened.AI researchers introduced CrossSWE, establishing a benchmark to evaluate autonomous coding agents across multi-repository software engineering tasks requiring joint dependency updates.

Why it matters.Single-repository benchmarks fail to evaluate how autonomous software agents handle real enterprise architectures that depend on microservices and shared libraries. Evaluating multi-repository reasoning exposes agent blind spots in dependency tracking and ensures tools can maintain distributed software systems.

The technical specifications:

  • Benchmark coverage: Encompasses one hundred seventy-one real-world engineering tasks spanning one hundred sixty-five code repositories across thirteen programming languages.
  • Cross-boundary requirements: Tasks require modifying coordinated functions across multiple codebases, ensuring single-repository modifications fail automated test suites.
  • Model evaluation: Leading foundation models achieved a top pass rate of fifty-eight percent, struggling primarily with API interface mismatches.
  • Failure telemetry: Discloses that cross-repository coordination errors and overlooked downstream package breaks accounted for most agent failures.