1. Neuroscience
Download icon

DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning

  1. Jacob M Graving  Is a corresponding author
  2. Daniel Chae
  3. Hemal Naik
  4. Liang Li
  5. Benjamin Koger
  6. Blair R Costelloe
  7. Iain D Couzin
  1. Max Planck Institute of Animal Behavior, Germany
  2. Princeton University, United States
Tools and Resources
  • Cited 26
  • Views 10,765
  • Annotations
Cite this article as: eLife 2019;8:e47994 doi: 10.7554/eLife.47994

Abstract

Quantitative behavioral measurements are important for answering questions across scientific disciplines-from neuroscience to ecology. State-of-the-art deep-learning methods offer major advances in data quality and detail by allowing researchers to automatically estimate locations of an animal's body parts directly from images or videos. However, currently-available animal pose estimation methods have limitations in speed and robustness. Here we introduce a new easy-to-use software toolkit, DeepPoseKit, that addresses these problems using an efficient multi-scale deep-learning model, called Stacked DenseNet, and a fast GPU-based peak-detection algorithm for estimating keypoint locations with subpixel precision. These advances improve processing speed >2× with no loss in accuracy compared to currently-available methods. We demonstrate the versatility of our methods with multiple challenging animal pose estimation tasks in laboratory and field settings-including groups of interacting individuals. Our work reduces barriers to using advanced tools for measuring behavior and has broad applicability across the behavioral sciences.

Article and author information

Author details

  1. Jacob M Graving

    Department of Collective Behaviour, Max Planck Institute of Animal Behavior, Konstanz, Germany
    For correspondence
    jgraving@gmail.com
    Competing interests
    No competing interests declared.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-5826-467X
  2. Daniel Chae

    Department of Computer Science, Princeton University, Princeton, United States
    Competing interests
    No competing interests declared.
  3. Hemal Naik

    Department for Collective Behaviour, Max Planck Institute of Animal Behavior, Konstanz, Germany
    Competing interests
    No competing interests declared.
  4. Liang Li

    Department for Collective Behaviour, Max Planck Institute of Animal Behavior, Konstanz, Germany
    Competing interests
    No competing interests declared.
  5. Benjamin Koger

    Department for Collective Behaviour, Max Planck Institute of Animal Behavior, Konstanz, Germany
    Competing interests
    No competing interests declared.
  6. Blair R Costelloe

    Department for Collective Behaviour, Max Planck Institute of Animal Behavior, Konstanz, Germany
    Competing interests
    No competing interests declared.
  7. Iain D Couzin

    Department of Collective Behaviour, Max Planck Institute of Animal Behavior, Konstanz, Germany
    Competing interests
    Iain D Couzin, Reviewing editor, eLife.

Funding

National Science Foundation (IOS-1355061)

  • Iain D Couzin

Horizon 2020 Framework Programme (Marie Sklodowska-Curie grant agreement No. 748549)

  • Blair R Costelloe

Nvidia (GPU Grant)

  • Blair R Costelloe

Office of Naval Research (N00014-09-1-1074)

  • Iain D Couzin

Office of Naval Research (N00014-14-1-0635)

  • Iain D Couzin

Army Research Office (W911NG-11-1-0385)

  • Iain D Couzin

Army Research Office (W911NF14-1-0431)

  • Iain D Couzin

Deutsche Forschungsgemeinschaft (DFG Centre of Excellence 2117)

  • Iain D Couzin

University of Konstanz (Zukunftskolleg Investment Grant)

  • Blair R Costelloe

The Strukture-und Innovations fonds fur die Forschung of the State of Baden-Wurttemberg

  • Iain D Couzin

Max Planck Society

  • Iain D Couzin

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Ethics

Animal experimentation: All procedures for collecting the zebra (E. grevyi) dataset were reviewed and approved by Ethikrat, the independent Ethics Council of the Max Planck Society. The zebra dataset was collected with the permission of Kenya's National Commission for Science, Technology and Innovation (NACOSTI/P/17/59088/15489 and NACOSTI/P/18/59088/21567) using drones operated by B.R.C. with the permission of the Kenya Civil Aviation Authority (authorization numbers: KCAA/OPS/2117/4 Vol. 2 (80), KCAA/OPS/2117/4 Vol. 2 (81), KCAA/OPS/2117/5 (86) and KCAA/OPS/2117/5 (87); RPAS Operator Certificate numbers: RPA/TP/0005 AND RPA/TP/000-0009).

Reviewing Editor

  1. Josh W Shaevitz, Princeton University, United States

Publication history

  1. Received: April 26, 2019
  2. Accepted: September 18, 2019
  3. Accepted Manuscript published: October 1, 2019 (version 1)
  4. Version of Record published: December 6, 2019 (version 2)

Copyright

© 2019, Graving et al.

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 10,765
    Page views
  • 1,246
    Downloads
  • 26
    Citations

Article citation count generated by polling the highest count across the following sources: Crossref, Scopus, PubMed Central.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Download citations (links to download the citations from this article in formats compatible with various reference manager tools)

Open citations (links to open the citations from this article in various online reference manager services)

Further reading

    1. Neuroscience
    Shenghong He et al.
    Research Article Updated

    Previous studies have explored neurofeedback training for Parkinsonian patients to suppress beta oscillations in the subthalamic nucleus (STN). However, its impacts on movements and Parkinsonian tremor are unclear. We developed a neurofeedback paradigm targeting STN beta bursts and investigated whether neurofeedback training could improve motor initiation in Parkinson’s disease compared to passive observation. Our task additionally allowed us to test which endogenous changes in oscillatory STN activities are associated with trial-to-trial motor performance. Neurofeedback training reduced beta synchrony and increased gamma activity within the STN, and reduced beta band coupling between the STN and motor cortex. These changes were accompanied by reduced reaction times in subsequently cued movements. However, in Parkinsonian patients with pre-existing symptoms of tremor, successful volitional beta suppression was associated with an amplification of tremor which correlated with theta band activity in STN local field potentials, suggesting an additional cross-frequency interaction between STN beta and theta activities.

    1. Neuroscience
    2. Stem Cells and Regenerative Medicine
    Julien G Roth et al.
    Tools and Resources Updated

    Microdeletions and microduplications of the 16p11.2 chromosomal locus are associated with syndromic neurodevelopmental disorders and reciprocal physiological conditions such as macro/microcephaly and high/low body mass index. To facilitate cellular and molecular investigations into these phenotypes, 65 clones of human induced pluripotent stem cells (hiPSCs) were generated from 13 individuals with 16p11.2 copy number variations (CNVs). To ensure these cell lines were suitable for downstream mechanistic investigations, a customizable bioinformatic strategy for the detection of random integration and expression of reprogramming vectors was developed and leveraged towards identifying a subset of ‘footprint’-free hiPSC clones. Transcriptomic profiling of cortical neural progenitor cells derived from these hiPSCs identified alterations in gene expression patterns which precede morphological abnormalities reported at later neurodevelopmental stages. Interpreting clinical information—available with the cell lines by request from the Simons Foundation Autism Research Initiative—with this transcriptional data revealed disruptions in gene programs related to both nervous system function and cellular metabolism. As demonstrated by these analyses, this publicly available resource has the potential to serve as a powerful medium for probing the etiology of developmental disorders associated with 16p11.2 CNVs.