Compositional Motion Generation from Demonstration with Object-Centric Neural Fields

Published July 9, 2026 · Category: Robotics

Overview

arXiv:2607.07129v1 Announce Type: new Abstract: Compositionality, by organizing complex behavior as combinations of simpler elements, enables robot learning that is scalable and data efficient. Leveraging this principle, we propose a generative learning-from-demonstration framework that enables compositional modeling of robotic behavior by connecting perception and motion through shared object-level representations. We render scenes from object-centric neural representations that integrate canonical neural fields with latent-conditioned deformations, capturing positional and geometric variations in a smooth, consistent, and interpretable way. For motion generation, a temporal mixture-of-experts (MoE) employs a gating mechanism to combine object-conditioned movement primitives over time, producing complete trajectories. This spatial-temporal compositionality maintains the data efficiency of movement primitives while grounding motion in visual structure, enabling systematic generalization across diverse scene configurations. In simulation, long-horizon manipulation tasks are successfully completed using the proposed model, which requires significantly less training data than other image-based baselines. Real-world experiments further demonstrate the method's robustness to noise, its ability to generalize at the category level through language-based segmentation models, and its capacity to operate directly on 3D scene representations.

Source

Originally published at arxiv.org.

Related Articles

• Shared Modular Recurrence in Contextual MDPs for Universal Morphology Control
• EmbodiedGen V2: An Agentic, Simulation-Ready 3D World Engine for Embodied AI
• Zero-Human Demonstration End-to-end Autonomous Driving with Trajectory Scorer