Skip to content

Add an integration ​

Start with the guide for the component you are adding. Use the closest existing implementation as a reference and keep your changes within that component's responsibilities.

ComponentWhat it ownsStart here
AlgorithmProposal, evaluation budget, selection, and the resources it may editAdd an algorithm, using OPRO
BenchmarkTasks, splits, task environment, and gradingAdd a benchmark, using SearchQA and Terminal-Bench
HarnessNative agent execution and the resources that make up a SnapshotAdd a harness, using MiniAgent

The runner connects these components. The algorithm selects a Snapshot, and the runner evaluates that version on held-out test tasks. A Snapshot carries harness resources. The destination benchmark supplies its own task data, tools, and verifier when the Snapshot is evaluated elsewhere. See the framework overview and the MVP contract for the interfaces.

For a contribution, include a configuration example, the supported capabilities and prerequisites, and the validation you performed. Distinguish an offline check, successful initialization, and a real run that reached the grader.

See the algorithm inventory for the implemented methods, editable resources, selection rules, and Worker requirements.

Released under the Apache License 2.0.