DeepFly3D, a deep learning-based approach for 3D limb and appendage tracking in tethered, adult Drosophila
- EPFL, Switzerland
Abstract
Studying how neural circuits orchestrate limbed behaviors requires the precise measurement of the positions of each appendage in 3-dimensional (3D) space. Deep neural networks can estimate 2-dimensional (2D) pose in freely behaving and tethered animals. However, the unique challenges associated with transforming these 2D measurements into reliable and precise 3D poses have not been addressed for small animals including the fly, Drosophila melanogaster. Here we present DeepFly3D, a software that infers the 3D pose of tethered, adult Drosophila using multiple camera images. DeepFly3D does not require manual calibration, uses pictorial structures to automatically detect and correct pose estimation errors, and uses active learning to iteratively improve performance. We demonstrate more accurate unsupervised behavioral embedding using 3D joint angles rather than commonly used 2D pose data. Thus, DeepFly3D enables the automated acquisition of Drosophila behavioral measurements at an unprecedented level of detail for a variety of biological applications.
Data availability
All data generated and analyzed during this study are included in the DeepFly3D GitHub site: https://github.com/NeLy-EPFL/DeepFly3D and in the Harvard Dataverse.
-
aDN-GAL4 UAS-CsChrimsonHarvard Dataverse, doi:10.7910/DVN/S4L4KX.
-
MDN-GAL4 UAS-CsChrimsonHarvard Dataverse, doi:10.7910/DVN/8SUC9U.
Article and author information
Author details
Funding
Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (175667)
- Daniel Morales
- Pavan Ramdya
Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (181239)
- Daniel Morales
- Pavan Ramdya
EPFL (iPhD)
- Semih Günel
Microsoft Research (JRC Project)
- Helge Rhodin
Swiss Government Excellence Postdoctoral Scholarship (2018.0483)
- Daniel Morales
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
Copyright
© 2019, Günel et al.
This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.
Metrics
-
- 7,991
- views
-
- 880
- downloads
-
- 126
- citations
Views, downloads and citations are aggregated across all versions of this paper published by eLife.
Download links
A two-part list of links to download the article, or parts of the article, in various formats.
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
DeepFly3D, a deep learning-based approach for 3D limb and appendage tracking in tethered, adult Drosophila
eLife 8:e48571.
https://doi.org/10.7554/eLife.48571
Further reading
-
- Neuroscience
Combining individual actions into sequences is a hallmark of everyday activities. Classical theories propose that the motor system forms a single specification of the sequence as a whole, leading to the coarticulation of the different elements. In contrast, recent neural recordings challenge this idea and suggest independent execution of each element specified separately. Here, we show that separate or coarticulated sequences can result from the same task-dependent controller, without implying different representations in the brain. Simulations show that planning for multiple reaches simultaneously allows separate or coarticulated sequences depending on instructions about intermediate goals. Human experiments in a two-reach sequence task validated this model. Furthermore, in co-articulated sequences, the second goal influenced long-latency stretch responses to external loads applied during the first reach, demonstrating the involvement of the sensorimotor network supporting fast feedback control. Overall, our study establishes a computational framework for sequence production that highlights the importance of feedback control in this essential motor skill.
-
- Neuroscience
Neuropathic pain (NP) is caused by a lesion or disease of the somatosensory system and is characterized by abnormal hypersensitivity to stimuli and nociceptive responses to non-noxious stimuli, affecting approximately 7–10% of the general population. However, current first-line drugs like non-steroidal anti-inflammatory agents and opioids have limitations, including dose-limiting side effects, dependence, and tolerability issues. Therefore, developing new interventions for the management of NP is urgent. In this study, we discovered that the high-frequency terahertz stimulation (HFTS) at approximately 36 THz effectively alleviates NP symptoms in mice with spared nerve injury. Computational simulation suggests that the frequency resonates with the carbonyl group in the filter region of Kv1.2 channels, facilitating the translocation of potassium ions. In vivo and in vitro results demonstrate that HFTS reduces the excitability of pyramidal neurons in the anterior cingulate cortex likely through enhancing the voltage-gated K+ and also the leak K+ conductance. This research presents a novel optical intervention strategy with terahertz waves for the treatment of NP and holds promising applications in other nervous system diseases.
