OP-Follower: Onboard Vision-based Obstacle-Aware Predictive Person Following for Quadruped Robots via IBVS-NMPC Control

This paper presents OP-Follower, a fully onboard, pure vision-based, obstacle-aware person-following framework for quadruped robots. Unlike existing approaches that rely on LiDAR, external infrastructure, or offboard computation, OP-Follower operates exclusively on visual inputs and runs entirely on a Unitree Go2 equipped with a Jetson Orin NX. The proposed framework bridges the gap between low-frequency, delay-prone visual perception and safety-aware high-rate control by integrating YOLO-World-based open-vocabulary person detection, BoxMOT multi-object tracking, and Kalman filter-based delay-aware state prediction within a unified planning-control architecture that tightly couples image-based visual servoing with nonlinear model predictive control. A decoupled artificial potential field-based visual obstacle avoidance strategy is further employed to ensure collision-free navigation without interfering with target tracking objectives. Comprehensive evaluations in high-fidelity Gazebo simulations and real-world indoor and outdoor environments, including long-distance following, dynamic pursuit-evasion interactions, and multi-person scenarios, demonstrate robust real-time performance with fully onboard processing at a 50Hz control rate. Compared with a non-predictive baseline, OP-Follower reduces pixel-space and distance tracking errors by up to 57.2% and 50.5%, respectively, achieving representative root-mean-square errors of approximately 13-16 pixels and 0.09-0.11m under sharp turning motions, while maintaining stable outdoor following over trajectories exceeding 600m.