1. Neuroscience

DeepFly3D, a deep learning-based approach for 3D limb and appendage tracking in tethered, adult Drosophila

  1. Semih Günel  Is a corresponding author
  2. Helge Rhodin
  3. Daniel Morales
  4. João H Campagnolo
  5. Pavan Ramdya  Is a corresponding author
  6. Pascal Fua
  1. EPFL, Switzerland
https://doi.org/10.7554/eLife.48571
Share this article
Cite this article
  1. Semih Günel
  2. Helge Rhodin
  3. Daniel Morales
  4. João H Campagnolo
  5. Pavan Ramdya
  6. Pascal Fua
(2019)
DeepFly3D, a deep learning-based approach for 3D limb and appendage tracking in tethered, adult Drosophila
eLife 8:e48571.
https://doi.org/10.7554/eLife.48571
  2. Download BibTeX
  3. Download .RIS

Abstract

Studying how neural circuits orchestrate limbed behaviors requires the precise measurement of the positions of each appendage in 3-dimensional (3D) space. Deep neural networks can estimate 2-dimensional (2D) pose in freely behaving and tethered animals. However, the unique challenges associated with transforming these 2D measurements into reliable and precise 3D poses have not been addressed for small animals including the fly, Drosophila melanogaster. Here we present DeepFly3D, a software that infers the 3D pose of tethered, adult Drosophila using multiple camera images. DeepFly3D does not require manual calibration, uses pictorial structures to automatically detect and correct pose estimation errors, and uses active learning to iteratively improve performance. We demonstrate more accurate unsupervised behavioral embedding using 3D joint angles rather than commonly used 2D pose data. Thus, DeepFly3D enables the automated acquisition of Drosophila behavioral measurements at an unprecedented level of detail for a variety of biological applications.

Data availability

All data generated and analyzed during this study are included in the DeepFly3D GitHub site: https://github.com/NeLy-EPFL/DeepFly3D and in the Harvard Dataverse.

The following data sets were generated

Article and author information

Author details

  1. Semih Günel

    School of Computer and Communication Sciences, Computer Vision Laboratory, EPFL, Lausanne, Switzerland
    For correspondence
    semih.gunel@epfl.ch
    Competing interests
    The authors declare that no competing interests exist.

  2. Helge Rhodin

    School of Computer and Communication Sciences, Computer Vision Laboratory, EPFL, Lausanne, Switzerland
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-2692-0801

  3. Daniel Morales

    School of Life Sciences, Brain Mind Institute and Interfaculty Institute of Bioengineering, Neuroengineering Laboratory, EPFL, Lausanne, Switzerland
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-7469-0898

  4. João H Campagnolo

    School of Life Sciences, Brain Mind Institute and Interfaculty Institute of Bioengineering, Neuroengineering Laboratory, EPFL, Lausanne, Switzerland
    Competing interests
    The authors declare that no competing interests exist.

  5. Pavan Ramdya

    School of Life Sciences, Brain Mind Institute and Interfaculty Institute of Bioengineering, Neuroengineering Laboratory, EPFL, Lausanne, Switzerland
    For correspondence
    pavan.ramdya@epfl.ch
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0001-5425-4610

  6. Pascal Fua

    School of Computer and Communication Sciences, Computer Vision Laboratory, EPFL, Lausanne, Switzerland
    Competing interests
    The authors declare that no competing interests exist.

Funding

Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (175667)

  • Daniel Morales
  • Pavan Ramdya

Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (181239)

  • Daniel Morales
  • Pavan Ramdya

EPFL (iPhD)

  • Semih Günel

Microsoft Research (JRC Project)

  • Helge Rhodin

Swiss Government Excellence Postdoctoral Scholarship (2018.0483)

  • Daniel Morales

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Copyright

© 2019, Günel et al.

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 8,239
    views
  • 900
    downloads
  • 133
    citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Semih Günel
  2. Helge Rhodin
  3. Daniel Morales
  4. João H Campagnolo
  5. Pavan Ramdya
  6. Pascal Fua
(2019)
DeepFly3D, a deep learning-based approach for 3D limb and appendage tracking in tethered, adult Drosophila
eLife 8:e48571.
https://doi.org/10.7554/eLife.48571

Share this article

https://doi.org/10.7554/eLife.48571

Categories and tags

Research organism

Further reading

    1. Neuroscience

    Short-term social isolation acts on hypothalamic neurons to promote social behavior in a sex- and context-dependent manner

    Xin Zhao, Yurim Chae ... Katherine Tschida
    Research Article

    Social animals, including both humans and mice, are highly motivated to engage in social interactions. Short-term social isolation promotes social behavior, but the neural circuits through which it does so remain incompletely understood. Here, we sought to identify neurons that promote social behavior in single-housed female mice, which exhibit increased rates of social investigation, social ultrasonic vocalizations (USVs), and mounting during same-sex interactions that follow a period of short-term (3 days) isolation. We first used immunostaining for the immediate early gene Fos to identify a population of neurons in the preoptic hypothalamus (POA) that increase their activity in single-housed females following same-sex interactions (POAsocial neurons) but not in single-housed females that did not engage in social interactions. TRAP2-mediated chemogenetic silencing of POAsocial neurons in single-housed females significantly attenuates the effects of short-term isolation on social investigation, USV production, and mounting. In contrast, caspase-mediated ablation of POAsocial neurons in single-housed females robustly attenuates mounting but does not decrease social investigation or USV production. Optogenetic activation of POAsocial neurons in group-housed females promotes social investigation and USV production but does not recapitulate the effects of short-term isolation on mounting. To understand whether a similar population of POAsocial neurons promotes social behavior in single-housed males, we performed Fos immunostaining in single-housed males following either same-sex or opposite-sex social interactions. These experiments revealed a population of POA neurons that increase Fos expression in single-housed males following opposite-sex, but not same-sex, interactions. Chemogenetic silencing of POAsocial neurons in single-housed males during interactions with females reduces mounting but does not affect social investigation or USV production. These experiments identify a population of hypothalamic neurons that promote social behavior following short-term isolation in a sex- and social context-dependent manner.

    1. Neuroscience

    Reevaluating the neural noise in dyslexia using biomarkers from electroencephalography and high-resolution magnetic resonance spectroscopy

    Agnieszka Glica, Katarzyna Wasilewska ... Katarzyna Jednoróg
    Research Article

    The neural noise hypothesis of dyslexia posits an imbalance between excitatory and inhibitory (E/I) brain activity as an underlying mechanism of reading difficulties. This study provides the first direct test of this hypothesis using both electroencephalography (EEG) power spectrum measures in 120 Polish adolescents and young adults (60 with dyslexia, 60 controls) and glutamate (Glu) and gamma-aminobutyric acid (GABA) concentrations from magnetic resonance spectroscopy (MRS) at 7T MRI scanner in half of the sample. Our results, supported by Bayesian statistics, show no evidence of E/I balance differences between groups, challenging the hypothesis that cortical hyperexcitability underlies dyslexia. These findings suggest that alternative mechanisms must be explored and highlight the need for further research into the E/I balance and its role in neurodevelopmental disorders.

    1. Neuroscience

    Ethograms predict visual fear conditioning status in rats

    David C Williams, Amanda Chu ... Michael A McDannald
    Research Advance

    Recognizing and responding to threat cues is essential to survival. Freezing is a predominant threat behavior in rats. We have recently shown that a threat cue can organize diverse behaviors beyond freezing, including locomotion (Chu et al., 2024). However, that experimental design was complex, required many sessions, and had rats receive many foot shock presentations. Moreover, the findings were descriptive. Here, we gave female and male Long Evans rats cue light illumination paired or unpaired with foot shock (8 total) in a conditioned suppression setting, using a range of shock intensities (0.15, 0.25, 0.35, or 0.50 mA). We found that conditioned suppression was only observed at higher foot shock intensities (0.35 mA and 0.50 mA). We constructed comprehensive temporal ethograms by scoring 22,272 frames across 12 behavior categories in 200-ms intervals around cue light illumination. The 0.50 mA and 0.35 mA shock-paired visual cues suppressed reward seeking, rearing, and scaling, as well as light-directed rearing and light-directed scaling. The shock-paired visual cue further elicited locomotion and freezing. Linear discriminant analyses showed that ethogram data could accurately classify rats into paired and unpaired groups. Using complete ethogram data produced superior classification compared to behavior subsets, including an Immobility subset featuring freezing. The results demonstrate diverse threat behaviors – in a short and simple procedure – containing sufficient information to distinguish the visual fear conditioning status of individual rats.