Researchers introduce CrossSWE benchmarking multi-repository coding agents
Listen to Briefing
0:00/1:41

What happened.AI researchers introduced CrossSWE, establishing a benchmark to evaluate autonomous coding agents across multi-repository software engineering tasks requiring joint dependency updates.
Why it matters.Single-repository benchmarks fail to evaluate how autonomous software agents handle real enterprise architectures that depend on microservices and shared libraries. Evaluating multi-repository reasoning exposes agent blind spots in dependency tracking and ensures tools can maintain distributed software systems.
The technical specifications:
- Benchmark coverage: Encompasses one hundred seventy-one real-world engineering tasks spanning one hundred sixty-five code repositories across thirteen programming languages.
- Cross-boundary requirements: Tasks require modifying coordinated functions across multiple codebases, ensuring single-repository modifications fail automated test suites.
- Model evaluation: Leading foundation models achieved a top pass rate of fifty-eight percent, struggling primarily with API interface mismatches.
- Failure telemetry: Discloses that cross-repository coordination errors and overlooked downstream package breaks accounted for most agent failures.