Dual Exam: Controlled Evaluation of Multimodal Agents and World Models
Zhefan Rao, Meng Chu, Xuan Zhang, Jingfan Chen, Guohao Cai, Runyi Li and 77 more
Collected · snapshot shipped with the site build, not a live feed
Abstract · as written by the authors
World Models (WMs) can support Multimodal Agents (MAs) through simulated experience, while Agent trajectories can inform World Model refinement. Evaluating this interaction requires separate evidence about action selection and state evolution. We introduce Dual Exam, which tests Agents under fixed environment response mechanisms and World Models under fixed action interventions across physical, digital, social, and scientific domains. The Agent track compares 10 configurations on 1,088 items in 15 reporting categories. The World Model track contains 400 cases across eight scenarios; 15 configurations have full coverage and one covers six scenarios. GPT 6 Astra leads the case-weighted Agent Index at 31.3, and GPT-Image-2 leads the full-coverage WM Overall at 21.7. These scores use different tasks and metrics and are not directly comparable. A 150-case pilot of one coupled pairing records zero task successes and motivates component-level diagnosis. Results reveal uneven task performance and failures to preserve task-relevant state. Their value for predicting coupled success or sustained reciprocal learning remains to be established.
This is the original abstract from arXiv. It has not been summarised, translated or rewritten.
Terms in plain language
AutomaticMade by matching the words of the title and abstract against a fixed list of definitions. It does not read the paper and does not change the abstract above.
- Multimodal
- able to work with more than one kind of data, such as text together with images, audio or video.
- Agents
- an AI system that plans and takes several steps, often by calling tools, towards a goal.
- World Models
- a model that predicts how an environment changes, so it can imagine the result of an action.