The paper starts from a practical bottleneck in world-model research: the
hard part is not only scaling a backbone, but reliably turning a model
failure, a code change, and a noisy experiment into a defensible research
conclusion.
Figure 1 from the paper: comparison between VerdiWM and other research agents.
01
Research-Loop Reliability
A world-model improvement may require diagnosing long-horizon failure,
editing objectives or conditioning paths, running distributed jobs, and
deciding whether the measurements really support the intended conclusion.
02
Claim-Licensed Repair
VerdiWM separates the authority to execute experiments from the authority
to update claims. Actions change models; only fenced evidence, receipts,
and frozen verification can move the claim frontier.
03
Measured Reuse
Verified repair experience is not copied by name or intuition. It moves
to a new backbone only when probe responses, support, sign agreement,
and effect bounds license transfer; otherwise the system abstains.
From Code Agents to Learning Agents
General software agents can edit files, run scripts, and report scores.
VerdiWM targets a stricter control surface: execution can propose evidence,
but frozen claim gates decide whether a repair becomes reusable knowledge.
Intervention Level
Existing Agent Example
A Devin-style workflow sees that Ctrl-World lacks an action module,
opens models/unet.py, inserts an ActionEncoder with an LLM,
and restarts python train.py. The intervention stays at
the file-system and code-text layer.
VerdiWM Example
In the Ctrl-World campaign, VerdiWM diagnoses copy-current-frame
behavior as weak action-driven latent transition, then adds a LaMo
motion prior and drift regularization under the original backbone interface.
Core Difference
Existing agents act like external programmers. VerdiWM changes the
measured repair geometry while preserving hook contracts and verifier scope.
Loop Depth
Existing Agent Example
An AutoGPT-style workflow receives "optimize FID", changes the learning
rate from 1e-4 to 1e-5, reruns training,
prints a new FID, and stops.
VerdiWM Example
In the Cosmos-Predict2 campaign, REP improves depth, flow, and
motion smoothness, but background and subject consistency fall. The
claim is narrowed to structure-related coordinates instead of reported
as an unconditional win.
Core Difference
Existing agents run and read a score. VerdiWM runs, diagnoses,
attributes, freezes verification, updates a bounded claim, or abstains.
Diagnosis Granularity
Existing Agent Example
An MLAgentBench-style report may observe loss = 2.3 and
recommend lowering the learning rate. The diagnosis remains at the scalar
metric level.
VerdiWM Example
VerdiWM applies paired intervention probes around the frozen model:
action scaling, context retention, first-frame anchoring, and sampler
stress form a response fingerprint before any expensive transfer.
Core Difference
Existing agents diagnose symptoms. VerdiWM measures whether the target
model has the deficiency that a stored repair actually addresses.
VerdiWM Pipeline
A campaign starts from a goal and validity gates, fingerprints the model with
paired probes, compiles typed repairs, verifies effects under a frozen evaluator,
and updates the claim DAG only when evidence obligations are satisfied.
Online Decision Loop
Pipeline overview.
The loop separates operational side effects from epistemic claim updates.
Every trial produces a receipt; every reusable repair needs a gate-passing
settlement vector rather than a single favorable score.
The experiments test whether VerdiWM can diagnose a world-model failure,
select a localized repair, preserve validity gates, and transfer only when
the target backbone passes a certificate.
Campaign-level gates prevent score-only claims by tracking action sensitivity,
rollout fidelity, protected trade-off metrics, and policy-level outcomes.
Experiment 1 / Ctrl-World Physical Consistency
LaMo + drift repair from the paper.
VerdiWM diagnoses the Ctrl-World baseline as over-preserving static appearance
and under-using action-conditioned motion. The repair adds a LaMo latent
motion prior and drift regularization while keeping the backbone interface
fixed. Motion smoothness rises from 68.75 to 81.53; LPIPS and FID also improve.
Three-row replay for the Ctrl-World physical-consistency campaign.
The comparison follows the paper-style layout: ground truth, VerdiWM repair,
then the original baseline.
4 samples / 960x576
Sample 01
Sample 02
Sample 03
Sample 04
Top row
GT reference.
Middle row
VerdiWM-improved output.
Bottom row
Ctrl-World baseline.
Paper figure for the same Ctrl-World campaign: VerdiWM improves two views
while the paper records the remaining view-specific trade-offs.
Experiment 2 / Cosmos-Predict2 REP
DINO-REP repair from the paper.
VerdiWM identifies unstable object boundaries, scene layout, and semantic
relations in Cosmos-Predict2, then injects DINO features as higher-level
representation targets. The repair improves semantic alignment, depth,
flow, and motion smoothness, while background and subject consistency remain
explicit claim-boundary coordinates.
Qualitative replay for the REP campaign: baseline is shown on the right, while the
VerdiWM REP prediction is compared under the same context as the paper figure above.
WorldArena Metrics
WorldArena-style scoring gives a broader view of the REP rollout than
PSNR and SSIM alone. The table compares Cosmos-Predict2 with REP on semantic,
geometric, aesthetic, background, dynamics, flow, smoothness, and subject
consistency metrics.
6 / 8 LeadsREP leads on most WorldArena entries.
REP is higher on semantic alignment, depth accuracy, aesthetic quality,
dynamic degree, flow score, and motion smoothness.
Depth + MotionThe largest gains are in geometry and temporal behavior.
Depth accuracy increases from 95.35 to 97.05, and motion smoothness
increases from 78.93 to 81.31.
Remaining GapThe baseline still keeps stronger identity and background consistency.
Background consistency and subject consistency remain lower for REP,
which is logged as a boundary of the REP repair rather than hidden
inside a single aggregate score.
Paper Table 3 screenshot.
Scores are reported on a 0-100 scale where higher is better. The image is
a direct crop from the VerdiWM PDF, not a redrawn table.
This experiment follows the paper's automatic-adaptation setting: given a new
robot embodiment, action interface, camera layout, and task distribution,
VerdiWM constructs the training and replay contract without a hand-written
adapter. The target dataset is
RoboCOIN-DataManager.
The robot embodiment is Agilex Cobot Magic. From the dataset
structure, embodiment metadata, camera viewpoint, and action interface, the
system selects a compatible base model and continues training
DreamDojo/Cosmos Predict2 action-conditioned video-to-world
from the AgiBot post-train 2B weights.
Dataset Parsing
RoboCOIN introduces Agilex Cobot Magic rollouts, a new task distribution,
object layout, and camera setup; the agent maps those signals into the
training and replay contract.
Base Model Choice
The selected structure keeps Cosmos Predict2 video-to-world generation and
adds action conditioning so predicted future frames respond to robot actions.
Control Check
Each video includes a no-action prediction. This checks that the model
is using action input rather than performing unconditional visual completion.
Open the Green Folder
Chinese task name: 打开绿色的文件夹. The rollout compares ground truth,
the trained action-conditioned world model, and a no-action prediction.
Open the Red Folder
Chinese task name: 打开红色的文件夹. The same three-column comparison is
used to verify that action injection changes the predicted trajectory.
Three-View Consistency
This mosaic replay stresses cross-view agreement. VerdiWM keeps the
object interaction and motion progression coherent across three views,
showing that the adaptation is not limited to a single camera stream.
Left columnGT reference.Middle columnAction-conditioned world-model prediction.Right columnNo-action-injection prediction, used as an overfitting control.
Experiment 4 / Action-Quality Rollouts
The paper tests whether video-level repair translates into action quality:
the repaired Cosmos3 policy is evaluated as a closed-loop action generator.
This block-stacking overview adds a RoboLab120 rollout view, with baseline
and repaired/RL-updated behavior shown as paired clips.
Signal Source
Terminal success comes from RoboLab, and progress comes only from verified
subtask/progress fields, matching the paper's rule that video scores do
not substitute for environment success.
Claim Boundary
Failed trajectories are capped below successful trajectories; safety events
enter the final quality gate; video/VLM checks remain audit-only evidence.
Block Stacking: Original vs RL
Four paired rollouts compare the original Cosmos3 behavior with the
RL-trained policy under the same RoboLab120 task family.
Spatial Re-layout Generalization
The scene is kept the same, but the block positions are swapped. The
RL-trained policy preserves the block-stacking behavior under this
object re-layout, showing generalization beyond the original placement.
Experiment 5 / FRANKA Real-Robot Comparison
Real-robot FRANKA comparisons provide qualitative evidence for the action-quality
claim. All clips are shown at 4x speed. The stacking task tests interference
handling around a blue bowl, the tablecloth task tests path quality, and the
drawer task tests occlusion-aware grasping from inside a bowl.
Task Set
Stack three blocks, fold the white tablecloth, and move a polyhedron
from a bowl into a drawer before closing it.
Original COSMOS3
The stacking rollout is affected by the blue bowl, the folding rollout
takes a longer path, and the drawer task fails under bowl occlusion.
Improved COSMOS3
The improved policy completes the stack, shortens the folding path, and
finishes the occluded pick-place-close sequence. These clips complement
the paper's closed-loop success-rate comparison rather than replacing it.
Stack Three Blocks / Original COSMOS3
Baseline rollout: the policy reaches the late-stage interaction, but
bowl interference prevents a clean blue-block grasp.
Stack Three Blocks / Improved COSMOS3
Improved rollout: the policy keeps the grasp sequence stable and
completes the three-block stacking task.
Fold the White Tablecloth / Original COSMOS3
Baseline rollout: the task is eventually approached through a long,
inefficient trajectory, exposing weak path planning on deformable-object
manipulation.
Fold the White Tablecloth / Improved COSMOS3
Improved rollout: the motion plan is shorter and cleaner, showing a
clear gain in path planning after the COSMOS3 update.
Bowl-to-Drawer Polyhedron / Original COSMOS3
Baseline rollout: the polyhedron is occluded inside the bowl, and the
policy cannot establish a reliable grasp before the drawer step.
Bowl-to-Drawer Polyhedron / Improved COSMOS3
Improved rollout: the policy grasps the occluded object, places it
into the drawer, and closes the drawer to complete the task.
