Interaction-via-Actions: Cattle Interaction Detection with Joint Learning of Action-Interaction Latent Space

This paper introduces a method and application for automatic detection of behavioral interaction between grazing cattle from a single image, which is essential for smart livestock management in the cattle industry, such as for detecting estrus. Although the interaction detection for humans has been actively studied, a non-trivial challenge lies in cattle interaction detection, i.e., the lack of a comprehensive behavioral dataset including interaction, since the interactions of grazing cattle are rare events. We, therefore, propose CattleAct, a data-efficient method for interaction detection by decomposing interactions into the combinations of actions by individual cattle. Specifically, we first learn an action latent space from a large-scale cattle action dataset, then embed rare interactions via the fine-tuning of the pre-trained latent space using contrastive learning, constructing a unified latent space of action and interactions. On top of the proposed method, we develop a practical working system integrating video and GPS inputs. Experiments in a commercial-scale pasture show the accurate interaction detection by our method compared to the baselines.