Task overview
Summary JSON ↗| Environment | Coverage | Native success | Input tokens | Output tokens | Usage coverage |
|---|
Current recorded result per task · Seed 0 · Native success criteria. Each task shows its latest result and the instruction used for that recording. Rates include valid completed results only; pending tasks are excluded. No pooled score across environments. Input includes cached tokens; output includes reasoning tokens.
No tasks match this search.
The evaluation protocol
Modified instructions are labeled, with changes highlighted. Each recording retains the instruction it used; native success predicates are unchanged.
One independent episode, seed 0. GPT-6 Astra with high reasoning effort. Stock Codex CLI 0.160.0; each task starts with a fresh workspace and no inherited trajectory.
RoboCasa and LIBERO: 6,000 steps at 20 Hz. RoboTwin and RoboDojo: 7,500 frames at 25 Hz. SIMPLE G1: 10,000 or 15,000 control frames at 50 Hz; see each task protocol. Stepped simulation, with an eight-hour wall-clock limit.