1. Neuroscience
Download icon

DeepFly3D, a deep learning-based approach for 3D limb and appendage tracking in tethered, adult Drosophila

  1. Semih Günel  Is a corresponding author
  2. Helge Rhodin
  3. Daniel Morales
  4. João H Campagnolo
  5. Pavan Ramdya  Is a corresponding author
  6. Pascal Fua
  1. EPFL, Switzerland
Tools and Resources
  • Cited 8
  • Views 3,335
  • Annotations
Cite this article as: eLife 2019;8:e48571 doi: 10.7554/eLife.48571

Abstract

Studying how neural circuits orchestrate limbed behaviors requires the precise measurement of the positions of each appendage in 3-dimensional (3D) space. Deep neural networks can estimate 2-dimensional (2D) pose in freely behaving and tethered animals. However, the unique challenges associated with transforming these 2D measurements into reliable and precise 3D poses have not been addressed for small animals including the fly, Drosophila melanogaster. Here we present DeepFly3D, a software that infers the 3D pose of tethered, adult Drosophila using multiple camera images. DeepFly3D does not require manual calibration, uses pictorial structures to automatically detect and correct pose estimation errors, and uses active learning to iteratively improve performance. We demonstrate more accurate unsupervised behavioral embedding using 3D joint angles rather than commonly used 2D pose data. Thus, DeepFly3D enables the automated acquisition of Drosophila behavioral measurements at an unprecedented level of detail for a variety of biological applications.

Article and author information

Author details

  1. Semih Günel

    School of Computer and Communication Sciences, Computer Vision Laboratory, EPFL, Lausanne, Switzerland
    For correspondence
    semih.gunel@epfl.ch
    Competing interests
    The authors declare that no competing interests exist.

  2. Helge Rhodin

    School of Computer and Communication Sciences, Computer Vision Laboratory, EPFL, Lausanne, Switzerland
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-2692-0801

  3. Daniel Morales

    School of Life Sciences, Brain Mind Institute and Interfaculty Institute of Bioengineering, Neuroengineering Laboratory, EPFL, Lausanne, Switzerland
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-7469-0898

  4. João H Campagnolo

    School of Life Sciences, Brain Mind Institute and Interfaculty Institute of Bioengineering, Neuroengineering Laboratory, EPFL, Lausanne, Switzerland
    Competing interests
    The authors declare that no competing interests exist.

  5. Pavan Ramdya

    School of Life Sciences, Brain Mind Institute and Interfaculty Institute of Bioengineering, Neuroengineering Laboratory, EPFL, Lausanne, Switzerland
    For correspondence
    pavan.ramdya@epfl.ch
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0001-5425-4610

  6. Pascal Fua

    School of Computer and Communication Sciences, Computer Vision Laboratory, EPFL, Lausanne, Switzerland
    Competing interests
    The authors declare that no competing interests exist.

Funding

Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (175667)

  • Daniel Morales
  • Pavan Ramdya

Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (181239)

  • Daniel Morales
  • Pavan Ramdya

EPFL (iPhD)

  • Semih Günel

Microsoft Research (JRC Project)

  • Helge Rhodin

Swiss Government Excellence Postdoctoral Scholarship (2018.0483)

  • Daniel Morales

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Reviewing Editor

  1. Timothy O'Leary, University of Cambridge, United Kingdom

Publication history

  1. Received: May 18, 2019
  2. Accepted: September 28, 2019
  3. Accepted Manuscript published: October 4, 2019 (version 1)
  4. Version of Record published: November 4, 2019 (version 2)

Copyright

© 2019, Günel et al.

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 3,335
    Page views
  • 442
    Downloads
  • 8
    Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Download citations (links to download the citations from this article in formats compatible with various reference manager tools)

Open citations (links to open the citations from this article in various online reference manager services)

Categories and tags

Research organism

Further reading

    1. Neuroscience

    Empathic pain evoked by sensory and emotional-communicative cues share common and process-specific neural representations

    Feng Zhou et al.
    Research Article Updated

    Pain empathy can be evoked by multiple cues, particularly observation of acute pain inflictions or facial expressions of pain. Previous studies suggest that these cues commonly activate the insula and anterior cingulate, yet vicarious pain encompasses pain-specific responses as well as unspecific processes (e.g. arousal) and overlapping activations are not sufficient to determine process-specific shared neural representations. We employed multivariate pattern analyses to fMRI data acquired during observation of noxious stimulation of body limbs (NS) and painful facial expressions (FE) and found spatially and functionally similar cross-modality (NS versus FE) whole-brain vicarious pain-predictive patterns. Further analyses consistently identified shared neural representations in the bilateral mid-insula. The vicarious pain patterns were not sensitive to respond to non-painful high-arousal negative stimuli but predicted self-experienced thermal pain. Finally, a domain-general vicarious pain pattern predictive of self-experienced pain but not arousal was developed. Our findings demonstrate shared pain-associated neural representations of vicarious pain.

    1. Neuroscience

    Activation of astrocytes in hippocampus decreases fear memory through adenosine A1 receptors

    Yulan Li et al.
    Research Article Updated

    Astrocytes respond to and regulate neuronal activity, yet their role in mammalian behavior remains incompletely understood. Especially unclear is whether, and if so how, astrocyte activity regulates contextual fear memory, the dysregulation of which leads to pathological fear-related disorders. We generated GFAP-ChR2-EYFP rats to allow the specific activation of astrocytes in vivo by optogenetics. We found that after memory acquisition within a temporal window, astrocyte activation disrupted memory consolidation and persistently decreased contextual but not cued fear memory accompanied by reduced fear-related anxiety behavior. In vivo microdialysis experiments showed astrocyte photoactivation increased extracellular ATP and adenosine concentrations. Intracerebral blockade of adenosine A1 receptors (A1Rs) reversed the attenuation of fear memory. Furthermore, intracerebral or intraperitoneal injection of A1R agonist mimicked the effects of astrocyte activation. Therefore, our findings provide a deeper understanding of the astrocyte-mediated regulation of fear memory and suggest a new and important therapeutic strategy against pathological fear-related disorders.

    1. Neuroscience

    Chronic postnatal chemogenetic activation of forebrain excitatory neurons evokes persistent changes in mood behavior

    Sthitapranjya Pati et al.
    Research Article

    Early adversity is a risk factor for the development of adult psychopathology. Common across multiple rodent models of early adversity is increased signaling via forebrain Gq-coupled neurotransmitter receptors. We addressed whether enhanced Gq-mediated signaling in forebrain excitatory neurons during postnatal life can evoke persistent mood-related behavioral changes. Excitatory hM3Dq DREADD-mediated chemogenetic activation of forebrain excitatory neurons during postnatal life (P2-14), but not in juvenile or adult windows, increased anxiety-, despair-, and schizophrenia-like behavior in adulthood. This was accompanied by an enhanced metabolic rate of cortical and hippocampal glutamatergic and GABAergic neurons. Furthermore, we observed reduced activity and plasticity-associated marker expression, and perturbed excitatory/inhibitory currents in the hippocampus. These results indicate that Gq signaling mediated activation of forebrain excitatory neurons during the critical postnatal window is sufficient to program altered mood-related behavior, as well as functional changes in forebrain glutamate and GABA systems, recapitulating aspects of the consequences of early adversity.