Benchmark Atlas

Atari is valid for RL-proxy claims because the score ultimately asks whether a policy trained with limited environment data performs well in a standardized game suite.

Limitation: Does not test robotics executability

Crafter adds a longer-horizon survival and crafting domain where policy progress depends on coherent state and resource dynamics.

Limitation: WM Arena currently treats it as a pilot surface

VBench-2.0 directly operationalizes intrinsic video faithfulness across physical, commonsense, controllability, human, and composition dimensions.

Limitation: Does not prove embodied closed-loop utility

Physics-IQ is valid for physical reasoning claims because it checks whether video models preserve spatial and temporal object dynamics rather than only visual appeal.

Limitation: Some WM Arena pilot cells currently fail due to local model availability

Object-contact and violation-of-expectation probes isolate intuitive physics failures that broad video quality scores can miss.

Limitation: Probe-style evidence does not imply downstream policy utility

WorldModelBench is relevant because it judges video generation as world modeling in application-driven domains, not only as image aesthetics.

Limitation: Requires reproduction and judge audit before WM Arena ranking

Conservation-law probes test a stronger form of physical reasoning than generic video realism.

Limitation: Not yet integrated

Text-to-video physical realism benchmarks directly target whether generated scenes obey physical constraints implied by the prompt.

Limitation: Not a closed-loop benchmark

Physical commonsense generation probes expose impossible-object and impossible-motion failures that aggregate video scores often hide.

Limitation: External code must be cloned and smoke-tested before adapter integration

World-in-World is valid for embodied utility because it evaluates whether world-model rollouts improve closed-loop task success rather than only visual quality.

Limitation: IGNav and AEQA formal evaluations are recorded; AR still requires licensed MP3D Habitat-ready assets

WorldArena bridges perception and functional utility by evaluating video metrics plus data-engine, policy-evaluator, and action-planner roles.

Limitation: EWMScore is useful but should not be mixed with WM Arena self-measured rankings until reproduced

RoboWM-Bench tests whether generated manipulation behavior can be translated into executable robot actions, which directly matches the robotics world-model construct.

Limitation: Does not replace broad policy transfer benchmarks

LIBERO-X is relevant when robotics claims involve robust task execution under realistic perturbations rather than only in-distribution demonstrations.

Limitation: Need official artifact and smoke test before adapter

RoboCasa365 is a large standardized household manipulation benchmark, so it is useful evidence for broad robotics task success and generalization claims.

Limitation: Policy leaderboard is not automatically a world-model evaluator

ManiSkill3 is not itself a world-model score, but it is a practical substrate for repeatable contact-rich embodied evaluation.

Limitation: Needs WM-specific protocol layered on top

DrivingGen is domain-valid for driving world models because it targets scene generation under driving-specific conditions and metrics.

Limitation: Driving simulation evidence should be separated from robotics and video-generation evidence

WorldLens is relevant because it evaluates driving world models from appearance fidelity through behavioral and 4D consistency.

Limitation: Public repo must be cloned and smoke-tested before adapter integration

The Waymo reference is useful market evidence that driving world models are being used for simulation, but it is not an open benchmark.

Limitation: Not open for WM Arena scoring

WorldScore decomposes world generation into next-scene generation under camera trajectory specifications, directly testing spatial-temporal consistency.

Limitation: Not a closed-loop control benchmark

4DWorldBench broadens 3D/4D evaluation across input modalities and physics-centric generation prompts.

Limitation: Not yet integrated

InSpatio-World is relevant as an open 4D world model reference tied to WorldScore-style dynamic consistency claims.

Limitation: Model repo must be cloned and smoke-tested before any adapter

MIND directly measures two core interactive world-model constructs: remembering the world across revisits and following action controls.

Limitation: Not yet integrated