1. Neuroscience
Download icon

DeepFly3D, a deep learning-based approach for 3D limb and appendage tracking in tethered, adult Drosophila

  1. Semih Günel  Is a corresponding author
  2. Helge Rhodin
  3. Daniel Morales
  4. João H Campagnolo
  5. Pavan Ramdya  Is a corresponding author
  6. Pascal Fua
  1. EPFL, Switzerland
Tools and Resources
  • Cited 33
  • Views 5,562
  • Annotations
Cite this article as: eLife 2019;8:e48571 doi: 10.7554/eLife.48571

Abstract

Studying how neural circuits orchestrate limbed behaviors requires the precise measurement of the positions of each appendage in 3-dimensional (3D) space. Deep neural networks can estimate 2-dimensional (2D) pose in freely behaving and tethered animals. However, the unique challenges associated with transforming these 2D measurements into reliable and precise 3D poses have not been addressed for small animals including the fly, Drosophila melanogaster. Here we present DeepFly3D, a software that infers the 3D pose of tethered, adult Drosophila using multiple camera images. DeepFly3D does not require manual calibration, uses pictorial structures to automatically detect and correct pose estimation errors, and uses active learning to iteratively improve performance. We demonstrate more accurate unsupervised behavioral embedding using 3D joint angles rather than commonly used 2D pose data. Thus, DeepFly3D enables the automated acquisition of Drosophila behavioral measurements at an unprecedented level of detail for a variety of biological applications.

Data availability

All data generated and analyzed during this study are included in the DeepFly3D GitHub site: https://github.com/NeLy-EPFL/DeepFly3D and in the Harvard Dataverse.

The following data sets were generated

Article and author information

Author details

  1. Semih Günel

    School of Computer and Communication Sciences, Computer Vision Laboratory, EPFL, Lausanne, Switzerland
    For correspondence
    semih.gunel@epfl.ch
    Competing interests
    The authors declare that no competing interests exist.

  2. Helge Rhodin

    School of Computer and Communication Sciences, Computer Vision Laboratory, EPFL, Lausanne, Switzerland
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-2692-0801

  3. Daniel Morales

    School of Life Sciences, Brain Mind Institute and Interfaculty Institute of Bioengineering, Neuroengineering Laboratory, EPFL, Lausanne, Switzerland
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-7469-0898

  4. João H Campagnolo

    School of Life Sciences, Brain Mind Institute and Interfaculty Institute of Bioengineering, Neuroengineering Laboratory, EPFL, Lausanne, Switzerland
    Competing interests
    The authors declare that no competing interests exist.

  5. Pavan Ramdya

    School of Life Sciences, Brain Mind Institute and Interfaculty Institute of Bioengineering, Neuroengineering Laboratory, EPFL, Lausanne, Switzerland
    For correspondence
    pavan.ramdya@epfl.ch
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0001-5425-4610

  6. Pascal Fua

    School of Computer and Communication Sciences, Computer Vision Laboratory, EPFL, Lausanne, Switzerland
    Competing interests
    The authors declare that no competing interests exist.

Funding

Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (175667)

  • Daniel Morales
  • Pavan Ramdya

Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (181239)

  • Daniel Morales
  • Pavan Ramdya

EPFL (iPhD)

  • Semih Günel

Microsoft Research (JRC Project)

  • Helge Rhodin

Swiss Government Excellence Postdoctoral Scholarship (2018.0483)

  • Daniel Morales

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Reviewing Editor

  1. Timothy O'Leary, University of Cambridge, United Kingdom

Publication history

  1. Received: May 18, 2019
  2. Accepted: September 28, 2019
  3. Accepted Manuscript published: October 4, 2019 (version 1)
  4. Version of Record published: November 4, 2019 (version 2)

Copyright

© 2019, Günel et al.

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 5,562
    Page views
  • 601
    Downloads
  • 33
    Citations

Article citation count generated by polling the highest count across the following sources: Scopus, Crossref, PubMed Central.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Download citations (links to download the citations from this article in formats compatible with various reference manager tools)

Open citations (links to open the citations from this article in various online reference manager services)

Categories and tags

Research organism

Further reading

    1. Neuroscience

    Maturation of persistent and hyperpolarization-activated inward currents shapes the differential activation of motoneuron subtypes during postnatal development

    Simon A Sharples, Gareth B Miles
    Research Article Updated

    The size principle underlies the orderly recruitment of motor units; however, motoneuron size is a poor predictor of recruitment amongst functionally defined motoneuron subtypes. Whilst intrinsic properties are key regulators of motoneuron recruitment, the underlying currents involved are not well defined. Whole-cell patch-clamp electrophysiology was deployed to study intrinsic properties, and the underlying currents, that contribute to the differential activation of delayed and immediate firing motoneuron subtypes. Motoneurons were studied during the first three postnatal weeks in mice to identify key properties that contribute to rheobase and may be important to establish orderly recruitment. We find that delayed and immediate firing motoneurons are functionally homogeneous during the first postnatal week and are activated based on size, irrespective of subtype. The rheobase of motoneuron subtypes becomes staggered during the second postnatal week, which coincides with the differential maturation of passive and active properties, particularly persistent inward currents. Rheobase of delayed firing motoneurons increases further in the third postnatal week due to the development of a prominent resting hyperpolarization-activated inward current. Our results suggest that motoneuron recruitment is multifactorial, with recruitment order established during postnatal development through the differential maturation of passive properties and sequential integration of persistent and hyperpolarization-activated inward currents.

    1. Neuroscience

    Population receptive fields in nonhuman primates from whole-brain fMRI and large-scale neurophysiology in visual cortex

    P Christiaan Klink et al.
    Research Article Updated

    Population receptive field (pRF) modeling is a popular fMRI method to map the retinotopic organization of the human brain. While fMRI-based pRF maps are qualitatively similar to invasively recorded single-cell receptive fields in animals, it remains unclear what neuronal signal they represent. We addressed this question in awake nonhuman primates comparing whole-brain fMRI and large-scale neurophysiological recordings in areas V1 and V4 of the visual cortex. We examined the fits of several pRF models based on the fMRI blood-oxygen-level-dependent (BOLD) signal, multi-unit spiking activity (MUA), and local field potential (LFP) power in different frequency bands. We found that pRFs derived from BOLD-fMRI were most similar to MUA-pRFs in V1 and V4, while pRFs based on LFP gamma power also gave a good approximation. fMRI-based pRFs thus reliably reflect neuronal receptive field properties in the primate brain. In addition to our results in V1 and V4, the whole-brain fMRI measurements revealed retinotopic tuning in many other cortical and subcortical areas with a consistent increase in pRF size with increasing eccentricity, as well as a retinotopically specific deactivation of default mode network nodes similar to previous observations in humans.

    1. Developmental Biology
    2. Neuroscience

    Constitutive deficiency of the neurogenic hippocampal modulator AP2γ promotes anxiety-like behavior and cumulative memory deficits in mice from juvenile to adult periods

    Eduardo Loureiro-Campos et al.
    Research Article

    The transcription factor activating protein two gamma (AP2γ) is an important regulator of neurogenesis both during embryonic development as well as in the postnatal brain, but its role for neurophysiology and behavior at distinct postnatal periods is still unclear. In this work, we explored the neurogenic, behavioral, and functional impact of a constitutive and heterozygous AP2γ deletion in mice from early postnatal development until adulthood. AP2γ deficiency promotes downregulation of hippocampal glutamatergic neurogenesis, altering the ontogeny of emotional and memory behaviors associated with hippocampus formation. The impairments induced by AP2γ constitutive deletion since early development leads to an anxious-like phenotype and memory impairments as early as the juvenile phase. These behavioral impairments either persist from the juvenile phase to adulthood or emerge in adult mice with deficits in behavioral flexibility and object location recognition. Collectively, we observed a progressive and cumulative impact of constitutive AP2γ deficiency on the hippocampal glutamatergic neurogenic process, as well as alterations on limbic-cortical connectivity, together with functional behavioral impairments. The results herein presented demonstrate the modulatory role exerted by the AP2γ transcription factor and the relevance of hippocampal neurogenesis in the development of emotional states and memory processes.