Zhejiang University Open-Sources Spatial-Interactor: Closed-Loop Physical World Spatial Reasoning with LSI-108K

Core Background & Industry Pain Points Embodied robotics demands rigorous 3D spatial cognition—calculating metric depths, occlusion boundaries, and precise grasp points. However, prevailing vision-language models infer geometry purely from static monocular snapshots. Under optical ambiguities and perspective scaling, passive models suffer acute depth illusions, frequently causing robotic manipulators to miss targets or trigger hardware collisions. ### Architecture Highlights & Internals Spatial-Interactor models spatial reasoning as a closed-loop Markov decision process: an action-conditioned controller actively commands exploratory viewpoint adjustments (e.g., pan, tilt, orbital shift); cross-view self-attention integrates camera pose matrices to resolve depth ambiguities from optical parallax; and the newly released LSI-108K dataset supplies 108,000 verified closed-loop manipulation sequences. ### Authoritative Benchmarks & Measured Scores On the ScanNet-Spatial and RealManip physical benchmarks, Spatial-Interactor reduces 3D bounding-box centroid localization error by 54.2% relative to static dual-view baselines. Relational spatial inference under complex occlusions surged from 51.6% to 85.4%, elevating real-world robotic pick-and-place success to 89.2%. ### Developer Hands-on Guide Developers can clone the Apache 2.0 repository from GitHub (ZJU-OmniAI/Spatial-Interactor), connect native ROS2 or Isaac Gym camera feeds, and download pre-trained checkpoints via Hugging Face.