AERMANI-VLM: Structured Prompting and Reasoning for Aerial Manipulation with Vision Language Models

Published June 30, 2026 · Category: Robotics

Overview

arXiv:2511.01472v2 Announce Type: replace Abstract: The rapid progress of vision--language models (VLMs) has sparked growing interest in robotic control, where natural language can express the operation goals while visual feedback links perception to action. However, directly deploying VLM-driven policies on aerial manipulators remains unsafe and unreliable since the generated actions are often inconsistent, hallucination-prone, and dynamically infeasible for flight. In this work, we present AERMANI-VLM, the first framework to adapt pretrained VLMs for aerial manipulation by separating high-level reasoning from low-level control, without any task-specific fine-tuning. Our framework encodes natural language instructions, task context, and safety constraints into a structured prompt that guides the model to generate a step-by-step reasoning trace in natural language. This reasoning output is used to select from a predefined library of discrete, flight-safe skills, ensuring interpretable and temporally consistent execution. By decoupling symbolic reasoning from physical action, AERMANI-VLM mitigates hallucinated commands and prevents unsafe behavior, enabling robust task completion. We validate the framework in both simulation and hardware on diverse multi-step pick-and-place tasks, demonstrating strong generalization to previously unseen commands, objects, and environments.

Source

Originally published at arxiv.org.

Related Articles

• You Only Touch Once: 6-DoF Object Pose Estimation from Single Tactile Contact
• The Speedup Paradox: Rethinking Inference Speed-Quality Trade-off in Embodied Tasks
• X-Morph: Human Motion Priors for Scalable Robot Learning Across Morphologies