01
Functional correctness
Software & terminal
Terminal-Bench 3.0
Executable software tasks for repository repair, migration and recovery.
Benchmark / task domain
Terminal-Bench 3.0 format
The work
Measure software work inside executable environments.
Tasks cover configuration migration, transaction recovery and persistent application state. Agents must complete changes that preserve the required behavior of the surrounding system.
Delivery can include
- 01
- Harbor-format task packages with Markdown instructions and task.toml configuration
- 02
- Docker environment definitions, code and task inputs
- 03
- Reference solutions and executable evaluation code
How we evaluate the task
Define how to evaluate the result.
02
System behavior and integration
03
Completeness of task outputs
WORK WITH ASTRAFLUX
What should your
model be able to do?
Share the capability, task types and delivery requirements. We’ll discuss the right program and representative tasks.
Contact us norman383@astraflux.spaceBenchmark names indicate reference directions. Task scope and delivery specifications are agreed for each project.