Studying animal behaviour without markers

Newswise — Recent advancements in machine learning and computer vision are transforming the study of animal behavior, leaving researchers intrigued about how animal collectives function. Social learning and collective vigilance, among other complex behaviors, can now be unraveled thanks to these innovative techniques.

A pioneering interdisciplinary research team from the Cluster of Excellence Centre for the Advanced Study of Collective Behaviour (CASCB) at the University of Konstanz and the Max Planck Institute of Animal Behavior has successfully introduced a revolutionary markerless approach for tracking bird postures in 3D, solely based on video recordings. Gone are the days of needing to attach position or movement transmitters to the animals. This novel method, named 3D-POP (3D posture of pigeons), allows researchers to observe a group of pigeons and analyze the gaze and fine-scaled behaviors of each individual bird, along with its interactions in space with other birds. According to Alex Chan, a PhD student at CASCB, the dataset provided by 3D-POP enables scientists to study collective bird behavior using just two video cameras, even in natural and wild settings.

For more information and videos demonstrating the new method 3D-POP, you can visit this link: https://www.campus.uni-konstanz.de/en/science/studying-animal-behaviour-without-markers

The dataset generated using 3D-POP was released at the prestigious Conference on Computer Vision and Pattern Recognition (CVPR) in June 2023, and it is available through open access, encouraging other researchers to reuse it for their investigations. The creators of 3D-POP, Hemal Naik and Alex Chan, envision two potential application areas: firstly, scientists studying pigeons can directly utilize the dataset to observe and analyze the behavior of multiple freely moving pigeons by employing at least two cameras. Secondly, the annotation method can be adapted for use with other bird species or even different animals, opening doors for researchers to decode the behavior of various creatures in the near future.

Key Facts:

• An interdisciplinary team of computer scientists, biologists, and comparative psychologists from CASCB and the Max Planck Institute of Animal Behavior developed this innovative method, enabling the creation of large-scale datasets with multiple animals.
• The publication of this research can be found under the title: "3D-POP - An Automated Annotation Approach to Facilitate Markerless 2D-3D" in the Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, pages 21274-21284.
• To access the paper, please follow this link: https://openaccess.thecvf.com/content/CVPR2023/html/Naik_3D-POP_-_An_Automated_Annotation_Approach_to_Facilitate_Markerless_2D-3D_CVPR_2023_paper.html
• Additionally, the dataset and the code to apply the annotation method to other birds can be found at: https://github.com/alexhang212/Dataset-3DPOP
• The study was generously funded by the Cluster of Excellence Centre for the Advanced Study of Collective Behaviour at the University of Konstanz.