Project  ·  Results browserGitHub ↗
FlashRSI · Benchmark atlas

See what an agent change does to the score.

Browse the shared experiment setting, then open the docs for the method, harness, and evidence behind each row.

Browse results →Run a quickstart
12RSI methods
30Benchmark suites
3Native harnesses
1Run interface
Results / 01

Results from the shared setting

The summary rows below cover the source-task scores in the public result matrix. Filter by harness to compare methods; the full table includes near-domain and far-OOD transfers.

Percent scores · baseline delta
MethodHarnessSearchQAHotpotQATerminal-Bench 2.0AppWorld
RRSIClaude Code82.79+8.8660.98+3.6676.67+13.3492.86+27.98
RRSIMini-Agent81.64+5.2159.76+1.2276.67+10.0084.52+19.04
Meta-HarnessClaude Code85.07+11.1462.20+4.8870.00+6.6790.48+25.60
Meta-HarnessMini-Agent79.14+2.7159.76+1.2273.33+6.6688.10+22.62
ACEClaude Code83.93+10.0063.41+6.0966.67+3.3491.07+26.19
ACEMini-Agent84.71+8.2863.41+4.8770.00+3.3391.67+26.19
SkillOptClaude Code83.14+9.2160.98+3.6670.00+6.6784.52+19.64
SkillOptMini-Agent82.93+6.5059.76+1.2266.670.0091.67+26.19
Complete matrix · 28 rows → Use the horizontal scrollbar to inspect every transfer route.
Results / 02

Full source and transfer matrix

Each cell shows the final score and its change from the corresponding initial harness. Bold marks the highest final score within a benchmark and harness, including ties.

Surface / 03

Choose the parts of a run

Each card names the part of the experiment it controls. Open the docs when you need the exact settings.

Algorithms

How candidates are searched

These methods propose, compare, and select edits inside a declared resource scope.

OPROGEPAACESkillOptRRSI
Show all 12 methods
OPROGEPAEvoPromptACESkillOptEvo-HarnessAHEADASDarwinXMeta-HarnessModularRSIRRSI
Benchmarks

What gets measured

The suites cover question answering, reasoning, math, coding, web research, documents, tool use, and interactive tasks.

SearchQAHotpotQAAppWorldTerminal-Bench
Show all 30 suites
SearchQAHotpotQAMuSiQueTriviaQANatural Questions2WikiMultiHopQABBEHSuperGPQAHLEMATHAIMELiveMathBenchSWE-bench VerifiedSWE-bench Pro V2DeepSWETerminal-BenchBrowseCompDeepSearchQAWideSearchOfficeQA-ProHarvey LABJobBenchSpreadsheetBench 2AppWorldBFCL V4WorkBenchTravelPlannerτ²-benchτ³ BankingALFWorld
Harnesses

Where candidates run

Native execution keeps the edited resources in the context that makes them useful.

MiniAgentClaude CodeCodex
Show integration status
MiniAgent · availableClaude Code · availableCodex · availablePi · in progressHermes · in progressOpenCode · in progress
Evidence / 04

Every row has a run behind it

FlashRSI keeps the configuration, Snapshot lineage, trajectories, metrics, usage, and task artifacts together.

  1. Capture the baseline

    Build the initial AgentSnapshot and record the resources available to the method.

  2. Search inside scope

    Use Target feedback or Worker edits to produce candidate resource versions.

  3. Run the complete candidate

    Execute the candidate through its native harness and keep the trajectory.

  4. Score and select

    Grade the trajectory, select a Snapshot, and evaluate it on the destination split.

Snapshot lineageParent, changed resources, roles, and version identity.
Target and WorkerTask answers, proposals, reflections, and selection decisions.
Run artifactsRedacted config, metrics, usage, logs, and task files.
Analysis / next

Open the run when the logs are ready.

This page has a place for task outcomes, Worker proposals, Snapshot lineage, usage, and failure clusters. Add raw logs to turn the summary into a searchable explorer.

Run / 05

Start from a saved command

The docs quickstart explains installation, role-specific model settings, and the output directory layout.

# from the repository root
flashrsi run \
  --config examples/quickstart.yaml \
  --algorithm skillopt \
  --benchmark searchqa