1. Neuroscience

DeepFly3D, a deep learning-based approach for 3D limb and appendage tracking in tethered, adult Drosophila

  1. Semih Günel  Is a corresponding author
  2. Helge Rhodin
  3. Daniel Morales
  4. João H Campagnolo
  5. Pavan Ramdya  Is a corresponding author
  6. Pascal Fua
  1. EPFL, Switzerland
https://doi.org/10.7554/eLife.48571
Share this article
Cite this article
  1. Semih Günel
  2. Helge Rhodin
  3. Daniel Morales
  4. João H Campagnolo
  5. Pavan Ramdya
  6. Pascal Fua
(2019)
DeepFly3D, a deep learning-based approach for 3D limb and appendage tracking in tethered, adult Drosophila
eLife 8:e48571.
https://doi.org/10.7554/eLife.48571
  2. Download BibTeX
  3. Download .RIS

Abstract

Studying how neural circuits orchestrate limbed behaviors requires the precise measurement of the positions of each appendage in 3-dimensional (3D) space. Deep neural networks can estimate 2-dimensional (2D) pose in freely behaving and tethered animals. However, the unique challenges associated with transforming these 2D measurements into reliable and precise 3D poses have not been addressed for small animals including the fly, Drosophila melanogaster. Here we present DeepFly3D, a software that infers the 3D pose of tethered, adult Drosophila using multiple camera images. DeepFly3D does not require manual calibration, uses pictorial structures to automatically detect and correct pose estimation errors, and uses active learning to iteratively improve performance. We demonstrate more accurate unsupervised behavioral embedding using 3D joint angles rather than commonly used 2D pose data. Thus, DeepFly3D enables the automated acquisition of Drosophila behavioral measurements at an unprecedented level of detail for a variety of biological applications.

Data availability

All data generated and analyzed during this study are included in the DeepFly3D GitHub site: https://github.com/NeLy-EPFL/DeepFly3D and in the Harvard Dataverse.

The following data sets were generated

Article and author information

Author details

  1. Semih Günel

    School of Computer and Communication Sciences, Computer Vision Laboratory, EPFL, Lausanne, Switzerland
    For correspondence
    semih.gunel@epfl.ch
    Competing interests
    The authors declare that no competing interests exist.

  2. Helge Rhodin

    School of Computer and Communication Sciences, Computer Vision Laboratory, EPFL, Lausanne, Switzerland
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-2692-0801

  3. Daniel Morales

    School of Life Sciences, Brain Mind Institute and Interfaculty Institute of Bioengineering, Neuroengineering Laboratory, EPFL, Lausanne, Switzerland
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-7469-0898

  4. João H Campagnolo

    School of Life Sciences, Brain Mind Institute and Interfaculty Institute of Bioengineering, Neuroengineering Laboratory, EPFL, Lausanne, Switzerland
    Competing interests
    The authors declare that no competing interests exist.

  5. Pavan Ramdya

    School of Life Sciences, Brain Mind Institute and Interfaculty Institute of Bioengineering, Neuroengineering Laboratory, EPFL, Lausanne, Switzerland
    For correspondence
    pavan.ramdya@epfl.ch
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0001-5425-4610

  6. Pascal Fua

    School of Computer and Communication Sciences, Computer Vision Laboratory, EPFL, Lausanne, Switzerland
    Competing interests
    The authors declare that no competing interests exist.

Funding

Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (175667)

  • Daniel Morales
  • Pavan Ramdya

Schweizerischer Nationalfonds zur Förderung der Wissenschaftlichen Forschung (181239)

  • Daniel Morales
  • Pavan Ramdya

EPFL (iPhD)

  • Semih Günel

Microsoft Research (JRC Project)

  • Helge Rhodin

Swiss Government Excellence Postdoctoral Scholarship (2018.0483)

  • Daniel Morales

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Reviewing Editor

  1. Timothy O'Leary, University of Cambridge, United Kingdom

Publication history

  1. Received: May 18, 2019
  2. Accepted: September 28, 2019
  3. Accepted Manuscript published: October 4, 2019 (version 1)
  4. Version of Record published: November 4, 2019 (version 2)

Copyright

© 2019, Günel et al.

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 6,115
    Page views
  • 670
    Downloads
  • 46
    Citations

Article citation count generated by polling the highest count across the following sources: Crossref, Scopus, PubMed Central.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Semih Günel
  2. Helge Rhodin
  3. Daniel Morales
  4. João H Campagnolo
  5. Pavan Ramdya
  6. Pascal Fua
(2019)
DeepFly3D, a deep learning-based approach for 3D limb and appendage tracking in tethered, adult Drosophila
eLife 8:e48571.
https://doi.org/10.7554/eLife.48571

Categories and tags

Research organism

Further reading

    1. Neuroscience

    A tonic nicotinic brake controls spike timing in striatal spiny projection neurons

    Lior Matityahu et al.
    Research Article Updated

    Striatal spiny projection neurons (SPNs) transform convergent excitatory corticostriatal inputs into an inhibitory signal that shapes basal ganglia output. This process is fine-tuned by striatal GABAergic interneurons (GINs), which receive overlapping cortical inputs and mediate rapid corticostriatal feedforward inhibition of SPNs. Adding another level of control, cholinergic interneurons (CINs), which are also vigorously activated by corticostriatal excitation, can disynaptically inhibit SPNs by activating α4β2 nicotinic acetylcholine receptors (nAChRs) on various GINs. Measurements of this disynaptic inhibitory pathway, however, indicate that it is too slow to compete with direct GIN-mediated feedforward inhibition. Moreover, functional nAChRs are also present on populations of GINs that respond only weakly to phasic activation of CINs, such as parvalbumin-positive fast-spiking interneurons (PV-FSIs), making the overall role of nAChRs in shaping striatal synaptic integration unclear. Using acute striatal slices from mice we show that upon synchronous optogenetic activation of corticostriatal projections blockade of α4β2 nAChRs shortened SPN spike latencies and increased postsynaptic depolarizations. The nAChR-dependent inhibition was mediated by downstream GABA release, and data suggest that the GABA source was not limited to GINs that respond strongly to phasic CIN activation. In particular, the observed decrease in spike latency caused by nAChR blockade was associated with a diminished frequency of spontaneous inhibitory postsynaptic currents in SPNs, a parallel hyperpolarization of PV-FSIs, and was occluded by pharmacologically preventing cortical activation of PV-FSIs. Taken together, we describe a role for tonic (as opposed to phasic) activation of nAChRs in striatal function. We conclude that tonic activation of nAChRs by CINs maintains a GABAergic brake on cortically-driven striatal output by ‘priming’ feedforward inhibition, a process that may shape SPN spike timing, striatal processing, and synaptic plasticity.

    1. Neuroscience

    Higher-order olfactory neurons in the lateral horn support odor valence and odor identity coding in Drosophila

    Sudeshna Das Chakraborty et al.
    Research Article

    Understanding neuronal representations of odor-evoked activities and their progressive transformation from the sensory level to higher brain centers features one of the major aims in olfactory neuroscience. Here, we investigated how odor information is transformed and represented in higher-order neurons of the lateral horn, one of the higher olfactory centers implicated in determining innate behavior, using Drosophila melanogaster. We focused on a subset of third-order glutamatergic lateral horn neurons (LHNs) and characterized their odor coding properties in relation to their presynaptic partner neurons, the projection neurons (PNs) by two-photon functional imaging. We show that odors evoke reproducible, stereotypic, and odor-specific response patterns in LHNs. Notably, odor-evoked responses in these neurons are valence-specific in a way that their response amplitude is positively correlated with innate odor preferences. We postulate that this valence-specific activity is the result of integrating inputs from multiple olfactory channels through second-order neurons. GRASP and micro-lesioning experiments provide evidence that glutamatergic LHNs obtain their major excitatory input from uniglomerular PNs, while they receive an odor-specific inhibition through inhibitory multiglomerular PNs. In summary, our study indicates that odor representations in glutamatergic LHNs encode hedonic valence and odor identity and primarily retain the odor coding properties of second-order neurons.

    1. Neuroscience

    Motor cortex activity across movement speeds is predicted by network-level strategies for generating muscle activity

    Shreya Saxena et al.
    Research Article

    Learned movements can be skillfully performed at different paces. What neural strategies produce this flexibility? Can they be predicted and understood by network modeling? We trained monkeys to perform a cycling task at different speeds, and trained artificial recurrent networks to generate the empirical muscle-activity patterns. Network solutions reflected the principle that smooth well-behaved dynamics require low trajectory tangling. Network solutions had a consistent form, which yielded quantitative and qualitative predictions. To evaluate predictions, we analyzed motor cortex activity recorded during the same task. Responses supported the hypothesis that the dominant neural signals reflect not muscle activity, but network-level strategies for generating muscle activity. Single-neuron responses were better accounted for by network activity than by muscle activity. Similarly, neural population trajectories shared their organization not with muscle trajectories, but with network solutions. Thus, cortical activity could be understood based on the need to generate muscle activity via dynamics that allow smooth, robust control over movement speed.