DeepFly3D, a deep learning-based approach for 3D limb and appendage tracking in tethered, adult Drosophila
- EPFL, Switzerland
- Cited 5
- Views 2,956
- Annotations
Abstract
Studying how neural circuits orchestrate limbed behaviors requires the precise measurement of the positions of each appendage in 3-dimensional (3D) space. Deep neural networks can estimate 2-dimensional (2D) pose in freely behaving and tethered animals. However, the unique challenges associated with transforming these 2D measurements into reliable and precise 3D poses have not been addressed for small animals including the fly, Drosophila melanogaster. Here we present DeepFly3D, a software that infers the 3D pose of tethered, adult Drosophila using multiple camera images. DeepFly3D does not require manual calibration, uses pictorial structures to automatically detect and correct pose estimation errors, and uses active learning to iteratively improve performance. We demonstrate more accurate unsupervised behavioral embedding using 3D joint angles rather than commonly used 2D pose data. Thus, DeepFly3D enables the automated acquisition of Drosophila behavioral measurements at an unprecedented level of detail for a variety of biological applications.
Article and author information
Author details
Funding
Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (175667)
- Daniel Morales
- Pavan Ramdya
Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (181239)
- Daniel Morales
- Pavan Ramdya
EPFL (iPhD)
- Semih Günel
Microsoft Research (JRC Project)
- Helge Rhodin
Swiss Government Excellence Postdoctoral Scholarship (2018.0483)
- Daniel Morales
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
Reviewing Editor
- Timothy O'Leary, University of Cambridge, United Kingdom
Publication history
- Received: May 18, 2019
- Accepted: September 28, 2019
- Accepted Manuscript published: October 4, 2019 (version 1)
- Version of Record published: November 4, 2019 (version 2)
Copyright
© 2019, Günel et al.
This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.
Metrics
-
- 2,956
- Page views
-
- 419
- Downloads
-
- 5
- Citations
Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.
Download links
Downloads (link to download the article as PDF)
Download citations (links to download the citations from this article in formats compatible with various reference manager tools)
Open citations (links to open the citations from this article in various online reference manager services)
Further reading
-
- Neuroscience
- Physics of Living Systems
-
- Neuroscience
