1. Computational and Systems Biology
  2. Neuroscience
Download icon

Real-time, low-latency closed-loop feedback using markerless posture tracking

  1. Gary A Kane
  2. Gonçalo Lopes
  3. Jonny L Sanders
  4. Alexander Mathis
  5. Mackenzie Mathis  Is a corresponding author
  1. Princeton University, United States
  2. NeuroGears, United Kingdom
  3. University of Oregon, United States
  4. EPFL, Switzerland
Tools and Resources
  • Cited 2
  • Views 2,753
  • Annotations
Cite this article as: eLife 2020;9:e61909 doi: 10.7554/eLife.61909


The ability to control a behavioral task or stimulate neural activity based on animal behavior in real-time is an important tool for experimental neuroscientists. Ideally, such tools are noninvasive, low-latency, and provide interfaces to trigger external hardware based on posture. Recent advances in pose estimation with deep learning allows researchers to train deep neural networks to accurately quantify a wide variety of animal behaviors. Here we provide a new DeepLabCut-Live! package that achieves low-latency real-time pose estimation (within 15 ms, >100 FPS), with an additional forward-prediction module that achieves zero-latency feedback, and a dynamic-cropping mode that allows for higher inference speeds. We also provide three options for using this tool with ease: (1) a stand-alone GUI (called DLC-Live! GUI, and integration into (2) Bonsai and (3) AutoPilot. Lastly, we benchmarked performance on a wide range of systems so that experimentalists can easily decide what hardware is required for their needs.

Article and author information

Author details

  1. Gary A Kane

    Department of Psychology, Princeton University, Princeton, United States
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-7703-5055
  2. Gonçalo Lopes

    NeuroGears, London, United Kingdom
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-0731-4945
  3. Jonny L Sanders

    Institute of Neuroscience, Department of Psychology, University of Oregon, Eugene, United States
    Competing interests
    The authors declare that no competing interests exist.
  4. Alexander Mathis

    Life Sciences, EPFL, Geneva, Switzerland
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-3777-2202
  5. Mackenzie Mathis

    Brain Mind Institute, EPFL, Genève, Switzerland
    For correspondence
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0001-7368-4456


Chan Zuckerberg Initiative (EOSS)

  • Alexander Mathis
  • Mackenzie Mathis

National Science Foundation (1309047)

  • Jonny L Sanders

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.


Animal experimentation: All mouse work was carried out under the permission of the IACUC at Harvard University (#17-07-309). Dog videos and feedback was exempt from IACUC approval (with conformation with IACUC).

Reviewing Editor

  1. Gordon J Berman, Emory University, United States

Publication history

  1. Received: August 8, 2020
  2. Accepted: December 6, 2020
  3. Accepted Manuscript published: December 8, 2020 (version 1)
  4. Accepted Manuscript updated: December 9, 2020 (version 2)
  5. Version of Record published: January 4, 2021 (version 3)


© 2020, Kane et al.

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.


  • 2,753
    Page views
  • 443
  • 2

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Download citations (links to download the citations from this article in formats compatible with various reference manager tools)

Open citations (links to open the citations from this article in various online reference manager services)

Further reading

    1. Cell Biology
    2. Computational and Systems Biology
    Taraneh Zarin et al.
    Research Advance Updated

    In previous work, we showed that intrinsically disordered regions (IDRs) of proteins contain sequence-distributed molecular features that are conserved over evolution, despite little sequence similarity that can be detected in alignments (Zarin et al., 2019). Here, we aim to use these molecular features to predict specific biological functions for individual IDRs and identify the molecular features within them that are associated with these functions. We find that the predictable functions are diverse. Examining the associated molecular features, we note some that are consistent with previous reports and identify others that were previously unknown. We experimentally confirm that elevated isoelectric point and hydrophobicity, features that are positively associated with mitochondrial localization, are necessary for mitochondrial targeting function. Remarkably, increasing isoelectric point in a synthetic IDR restores weak mitochondrial targeting. We believe feature analysis represents a new systematic approach to understand how biological functions of IDRs are specified by their protein sequences.

    1. Cancer Biology
    2. Computational and Systems Biology
    Kenichi Shimada et al.
    Tools and Resources Updated

    Individual cancers rely on distinct essential genes for their survival. The Cancer Dependency Map (DepMap) is an ongoing project to uncover these gene dependencies in hundreds of cancer cell lines. To make this drug discovery resource more accessible to the scientific community, we built an easy-to-use browser, shinyDepMap (https://labsyspharm.shinyapps.io/depmap). shinyDepMap combines CRISPR and shRNA data to determine, for each gene, the growth reduction caused by knockout/knockdown and the selectivity of this effect across cell lines. The tool also clusters genes with similar dependencies, revealing functional relationships. shinyDepMap can be used to (1) predict the efficacy and selectivity of drugs targeting particular genes; (2) identify maximally sensitive cell lines for testing a drug; (3) target hop, that is, navigate from an undruggable protein with the desired selectivity profile, such as an activated oncogene, to more druggable targets with a similar profile; and (4) identify novel pathways driving cancer cell growth and survival.