Real-time behavior recognition using a legged robot for animal-robot interaction

Animal-robot interaction is an emerging interdisciplinary field that explores the dynamics between animals and robotic systems, as well as the design principles for effective engagement. While previous approaches have investigated animal responses to robotic stimuli, they have yet to integrate artificial intelligence (AI) for real-time behavioral analysis during the interaction. This paper addresses this gap by introducing an AI-driven framework that enables a robotic dog to autonomously monitor and analyze livestock behavior, specifically in cows and chickens. Our system processes real-time camera observations using deep-learning models to detect animal presence and recognize actions. It integrates three neural networks: YOLO-Chicken and YOLO-Cows, for accurate detection of chickens and cows, respectively, and DARTEMIS, a novel, distilled unimodal variant of a state-of-the-art Animal Action Recognition model. The networks communicate efficiently via Redis in a lightweight manner, with all processing conducted onboard the robot. We trained YOLO-Cow and YOLO-Chicken on a subset of the COCO data set for cows and a public data set for chickens, achieving mAP@50-95 scores of 0.67 and 0.56, respectively. DARTEMIS, trained on the Animal Kingdom data set like ARTEMIS, reached an mAP of 77.3. With these models, we tested our system in real-world conditions through field trials, evaluating its ability to accurately detect animals and classify their behaviors. This study presents the first successful integration of efficient deep- learning models into a robotic platform for real-time animal behavior analysis. The proposed framework paves the way for continuous automated livestock monitoring, with potential applications in improving animal welfare and farm management. The full implementation is publicly available and designed to be adaptable to various robotic platforms and related challenges.