1. Neuroscience

DeepFly3D, a deep learning-based approach for 3D limb and appendage tracking in tethered, adult Drosophila

  1. Semih Günel  Is a corresponding author
  2. Helge Rhodin
  3. Daniel Morales
  4. João H Campagnolo
  5. Pavan Ramdya  Is a corresponding author
  6. Pascal Fua
  1. EPFL, Switzerland
https://doi.org/10.7554/eLife.48571
Share this article
Cite this article
  1. Semih Günel
  2. Helge Rhodin
  3. Daniel Morales
  4. João H Campagnolo
  5. Pavan Ramdya
  6. Pascal Fua
(2019)
DeepFly3D, a deep learning-based approach for 3D limb and appendage tracking in tethered, adult Drosophila
eLife 8:e48571.
https://doi.org/10.7554/eLife.48571
  2. Download BibTeX
  3. Download .RIS

Abstract

Studying how neural circuits orchestrate limbed behaviors requires the precise measurement of the positions of each appendage in 3-dimensional (3D) space. Deep neural networks can estimate 2-dimensional (2D) pose in freely behaving and tethered animals. However, the unique challenges associated with transforming these 2D measurements into reliable and precise 3D poses have not been addressed for small animals including the fly, Drosophila melanogaster. Here we present DeepFly3D, a software that infers the 3D pose of tethered, adult Drosophila using multiple camera images. DeepFly3D does not require manual calibration, uses pictorial structures to automatically detect and correct pose estimation errors, and uses active learning to iteratively improve performance. We demonstrate more accurate unsupervised behavioral embedding using 3D joint angles rather than commonly used 2D pose data. Thus, DeepFly3D enables the automated acquisition of Drosophila behavioral measurements at an unprecedented level of detail for a variety of biological applications.

Data availability

All data generated and analyzed during this study are included in the DeepFly3D GitHub site: https://github.com/NeLy-EPFL/DeepFly3D and in the Harvard Dataverse.

The following data sets were generated

Article and author information

Author details

  1. Semih Günel

    School of Computer and Communication Sciences, Computer Vision Laboratory, EPFL, Lausanne, Switzerland
    For correspondence
    semih.gunel@epfl.ch
    Competing interests
    The authors declare that no competing interests exist.

  2. Helge Rhodin

    School of Computer and Communication Sciences, Computer Vision Laboratory, EPFL, Lausanne, Switzerland
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-2692-0801

  3. Daniel Morales

    School of Life Sciences, Brain Mind Institute and Interfaculty Institute of Bioengineering, Neuroengineering Laboratory, EPFL, Lausanne, Switzerland
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-7469-0898

  4. João H Campagnolo

    School of Life Sciences, Brain Mind Institute and Interfaculty Institute of Bioengineering, Neuroengineering Laboratory, EPFL, Lausanne, Switzerland
    Competing interests
    The authors declare that no competing interests exist.

  5. Pavan Ramdya

    School of Life Sciences, Brain Mind Institute and Interfaculty Institute of Bioengineering, Neuroengineering Laboratory, EPFL, Lausanne, Switzerland
    For correspondence
    pavan.ramdya@epfl.ch
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0001-5425-4610

  6. Pascal Fua

    School of Computer and Communication Sciences, Computer Vision Laboratory, EPFL, Lausanne, Switzerland
    Competing interests
    The authors declare that no competing interests exist.

Funding

Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (175667)

  • Daniel Morales
  • Pavan Ramdya

Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (181239)

  • Daniel Morales
  • Pavan Ramdya

EPFL (iPhD)

  • Semih Günel

Microsoft Research (JRC Project)

  • Helge Rhodin

Swiss Government Excellence Postdoctoral Scholarship (2018.0483)

  • Daniel Morales

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Reviewing Editor

  1. Timothy O'Leary, University of Cambridge, United Kingdom

Publication history

  1. Received: May 18, 2019
  2. Accepted: September 28, 2019
  3. Accepted Manuscript published: October 4, 2019 (version 1)
  4. Version of Record published: November 4, 2019 (version 2)

Copyright

© 2019, Günel et al.

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 6,443
    Page views
  • 730
    Downloads
  • 58
    Citations

Article citation count generated by polling the highest count across the following sources: Scopus, Crossref, PubMed Central.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Semih Günel
  2. Helge Rhodin
  3. Daniel Morales
  4. João H Campagnolo
  5. Pavan Ramdya
  6. Pascal Fua
(2019)
DeepFly3D, a deep learning-based approach for 3D limb and appendage tracking in tethered, adult Drosophila
eLife 8:e48571.
https://doi.org/10.7554/eLife.48571

Categories and tags

Research organism

Further reading

    1. Neuroscience

    7-Dehydrocholesterol-derived oxysterols cause neurogenic defects in Smith-Lemli-Opitz syndrome

    Hideaki Tomita, Kelly M Hines ... Libin Xu
    Research Article Updated

    Defective 3β-hydroxysterol-Δ7 -reductase (DHCR7) in the developmental disorder, Smith-Lemli-Opitz syndrome (SLOS), results in a deficiency in cholesterol and accumulation of its precursor, 7-dehydrocholesterol (7-DHC). Here, we show that loss of DHCR7 causes accumulation of 7-DHC-derived oxysterol metabolites, premature neurogenesis from murine or human cortical neural precursors, and depletion of the cortical precursor pool, both in vitro and in vivo. We found that a major oxysterol, 3β,5α-dihydroxycholest-7-en-6-one (DHCEO), mediates these effects by initiating crosstalk between glucocorticoid receptor (GR) and neurotrophin receptor kinase TrkB. Either loss of DHCR7 or direct exposure to DHCEO causes hyperactivation of GR and TrkB and their downstream MEK-ERK-C/EBP signaling pathway in cortical neural precursors. Moreover, direct inhibition of GR activation with an antagonist or inhibition of DHCEO accumulation with antioxidants rescues the premature neurogenesis phenotype caused by the loss of DHCR7. These results suggest that GR could be a new therapeutic target against the neurological defects observed in SLOS.

    1. Neuroscience

    Distinct population and single-neuron selectivity for executive and episodic processing in human dorsal posterior cingulate

    Lyndsey Aponik-Gremillion, Yvonne Y Chen ... Brett L Foster
    Research Article

    Posterior cingulate cortex (PCC) is an enigmatic region implicated in psychiatric and neurological disease, yet its role in cognition remains unclear. Human studies link PCC to episodic memory and default mode network (DMN), while findings from the non-human primate emphasize executive processes more associated with the cognitive control network (CCN) in humans. We hypothesized this difference reflects an important functional division between dorsal (executive) and ventral (episodic) PCC. To test this, we utilized human intracranial recordings of population and single unit activity targeting dorsal PCC during an alternated executive/episodic processing task. Dorsal PCC population responses were significantly enhanced for executive, compared to episodic, task conditions, consistent with the CCN. Single unit recordings, however, revealed four distinct functional types with unique executive (CCN) or episodic (DMN) response profiles. Our findings provide critical electrophysiological data from human PCC, bridging incongruent views within and across species, furthering our understanding of PCC function.

    1. Neuroscience

    Flexing the principal gradient of the cerebral cortex to suit changing semantic task demands

    Zhiyao Gao, Li Zheng ... Elizabeth Jefferies
    Research Article

    Understanding how thought emerges from the topographical structure of the cerebral cortex is a primary goal of cognitive neuroscience. Recent work has revealed a principal gradient of intrinsic connectivity capturing the separation of sensory-motor cortex from transmodal regions of the default mode network (DMN); this is thought to facilitate memory-guided cognition. However, studies have not explored how this dimension of connectivity changes when conceptual retrieval is controlled to suit the context. We used gradient decomposition of informational connectivity in a semantic association task to establish how the similarity in connectivity across brain regions changes during familiar and more original patterns of retrieval. Multivoxel activation patterns at opposite ends of the principal gradient were more divergent when participants retrieved stronger associations; therefore, when long-term semantic information is sufficient for ongoing cognition, regions supporting heteromodal memory are functionally separated from sensory-motor experience. In contrast, when less related concepts were linked, this dimension of connectivity was reduced in strength as semantic control regions separated from the DMN to generate more flexible and original responses. We also observed fewer dimensions within the neural response towards the apex of the principal gradient when strong associations were retrieved, reflecting less complex or varied neural coding across trials and participants. In this way, the principal gradient explains how semantic cognition is organised in the human cerebral cortex: the separation of DMN from sensory-motor systems is a hallmark of the retrieval of strong conceptual links that are culturally shared.