Interactive world models are progressing beyond passive simulation toward active world generation and agency. However, deliberately editing an existing, running executable world while preserving unedited systemic invariants remains an underexplored frontier. An international research consortium introduces IGMWorld and IGMBench (arXiv:2610.02331), framing world editing through industry-grade game modding across Minecraft (Java) and Terraria (C#/.NET). Spanning 110 tasks evaluated against 1,100+ deterministic executable criteria across property, entity, dynamics, and system-level intervention depths, IGMBench rigorously assesses coding agent precision. Frontier agents achieve 78.2% task-level success and 94.8% criterion-level pass rates. Crucially, failure rates scale monotonically with intervention depth—most failed edits build and load without syntax errors but diverge in runtime execution logic. Project assets are accessible at vinesmsuic.github.io/IGMWorld.
Key Takeaways
- ✓Introduces IGMWorld and IGMBench, the first benchmark for executable world editing using Minecraft and Terraria game modding
- ✓Defines four hierarchical intervention depths (property, entity, dynamics, system) with 110 tasks and 1,100+ deterministic executable criteria
- ✓Frontier coding agents achieve 78.2% task-level and 94.8% criterion-level accuracy, identifying sharp drops in deep system interventions
Key Decision Metrics at a Glance
Turn your technical choice into a development budget
Compare 40 dev plans & simulate token costs vs $20/mo subscriptions
Project Links & Resources
Direct AccessIn-Depth Technical Analysis
Background and the Problem
Interactive world models increasingly generate digital scenes and navigate virtual spaces. However, practical simulation and game engineering demands world editing: modifying operational mechanics, game balance, or entity dynamics in an existing world while guaranteeing systemic invariance elsewhere. Standard code benchmarks evaluate localized function correctness without assessing complex multi-system physical simulations.
Architecture and How It Works
IGMWorld and IGMBench (arXiv:2610.02331) establish a programmatic framework for executable world editing grounded in real-world game modding:
- Intervention Depth Taxonomy: Classifies editing across four depth tiers: Property changes, Entity additions, Dynamics alterations, and System-level restructuring.
- Concrete Sandboxed Execution: Evaluates 110 real tasks against 1,100+ criteria using Minecraft (Java) and Terraria (C#/.NET) ecosystems.
- Four-Part Deterministic Verification: Enforces strict criteria including compile-time executability, runtime behavioral checks, property preservation of unedited systems, and multi-modal visual consistency.
Benchmarks and Measured Results
Benchmarked across frontier coding agent configurations:
- High Aggregate Accuracy: State-of-the-art agent pipelines resolve 78.2% of tasks under strict task-level constraints and 94.8% at criterion level.
- Depth-Induced Reliability Degradation: Success rates drop sharply as interventions advance from localized properties to multi-entity dynamics and cross-cutting systems.
- Silent Logic Failures: Failed edits typically build without compiler warnings, but fail behavioral assertions during gameplay. Joint visual consistency rates remain under 50% across evaluated architectures.
Getting Started for Developers
The IGMWorld framework and benchmarks are published at vinesmsuic.github.io/IGMWorld. Developers building simulation agents or modding copilots should integrate invariant preservation assertions to ensure agent edits maintain runtime coherence without breaking peripheral systems.
Benchmark side-by-side against alternatives, or calculate monthly token cost vs subscription break-even.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.