01
Functional correctness
Software & terminal
SWE-Marathon
Long-horizon software tasks spanning system implementation, integration and recovery.
Benchmark / task domain
The work
Build and repair systems with connected requirements.
Agents work across connected components to implement requirements, integrate changes and maintain reliable system behavior. Tasks cover both building systems and resolving failures.
Delivery can include
- 01
- Harbor-format task packages with Markdown instructions and task.toml configuration
- 02
- Docker environment definitions and task inputs
- 03
- Reference solutions and executable acceptance tests
How we evaluate the task
Define how to evaluate the result.
02
System behavior and integration
03
Completeness of task outputs
Request a sample
Review a representative task.
Leave your work email and we’ll follow up about this program, sample availability and your delivery requirements.
Email us directlyBenchmark names indicate reference directions. Task scope and delivery specifications are agreed for each project.