1. Computational and Systems Biology
  2. Neuroscience
Download icon

Real-time, low-latency closed-loop feedback using markerless posture tracking

  1. Gary A Kane
  2. Gonçalo Lopes
  3. Jonny L Saunders
  4. Alexander Mathis
  5. Mackenzie W Mathis  Is a corresponding author
  1. Harvard University, United States
  2. NeuroGears, United Kingdom
  3. University of Oregon, United States
  4. EPFL, Switzerland
Tools and Resources
  • Cited 2
  • Views 2,766
  • Annotations
Cite this article as: eLife 2020;9:e61909 doi: 10.7554/eLife.61909


The ability to control a behavioral task or stimulate neural activity based on animal behavior in real-time is an important tool for experimental neuroscientists. Ideally, such tools are noninvasive, low-latency, and provide interfaces to trigger external hardware based on posture. Recent advances in pose estimation with deep learning allows researchers to train deep neural networks to accurately quantify a wide variety of animal behaviors. Here we provide a new DeepLabCut-Live! package that achieves low-latency real-time pose estimation (within 15 ms, >100 FPS), with an additional forward-prediction module that achieves zero-latency feedback, and a dynamic-cropping mode that allows for higher inference speeds. We also provide three options for using this tool with ease: (1) a stand-alone GUI (called DLC-Live! GUI, and integration into (2) Bonsai and (3) AutoPilot. Lastly, we benchmarked performance on a wide range of systems so that experimentalists can easily decide what hardware is required for their needs.

Article and author information

Author details

  1. Gary A Kane

    The Rowland Institute at Harvard, Harvard University, Cambridge, United States
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-7703-5055
  2. Gonçalo Lopes

    NeuroGears, London, United Kingdom
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-0731-4945
  3. Jonny L Saunders

    Institute of Neuroscience, Department of Psychology, University of Oregon, Eugene, United States
    Competing interests
    The authors declare that no competing interests exist.
  4. Alexander Mathis

    Life Sciences, EPFL, Geneva, Switzerland
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-3777-2202
  5. Mackenzie W Mathis

    Brain Mind Institute, EPFL, Genève, Switzerland
    For correspondence
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0001-7368-4456


Chan Zuckerberg Initiative (EOSS)

  • Alexander Mathis
  • Mackenzie W Mathis

National Science Foundation (1309047)

  • Jonny L Sanders

The Rowland Institute at Harvard, Harvard University

  • Gary A Kane
  • Alexander Mathis
  • Mackenzie W Mathis

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.


Animal experimentation: All mouse work was carried out under the permission of the IACUC at Harvard University (#17-07-309). Dog videos and feedback was exempt from IACUC approval (with conformation with IACUC).

Reviewing Editor

  1. Gordon J Berman, Emory University, United States

Publication history

  1. Received: August 8, 2020
  2. Accepted: December 6, 2020
  3. Accepted Manuscript published: December 8, 2020 (version 1)
  4. Accepted Manuscript updated: December 9, 2020 (version 2)
  5. Version of Record published: January 4, 2021 (version 3)


© 2020, Kane et al.

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.


  • 2,766
    Page views
  • 443
  • 2

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Download citations (links to download the citations from this article in formats compatible with various reference manager tools)

Open citations (links to open the citations from this article in various online reference manager services)

Further reading

    1. Chromosomes and Gene Expression
    2. Computational and Systems Biology
    Michael Roland Wolff et al.
    Research Article

    Chromatin dynamics are mediated by remodeling enzymes and play crucial roles in gene regulation, as established in a paradigmatic model, the S. cerevisiae PHO5 promoter. However, effective nucleosome dynamics, i.e. trajectories of promoter nucleosome configurations, remain elusive. Here, we infer such dynamics from the integration of published single-molecule data capturing multi-nucleosome configurations for repressed to fully active PHO5 promoter states with other existing histone turnover and new chromatin accessibility data. We devised and systematically investigated a new class of 'regulated on-off-slide' models simulating global and local nucleosome (dis)assembly and sliding. Only seven of 68145 models agreed well with all data. All seven models involve sliding and the known central role of the N-2 nucleosome, but regulate promoter state transitions by modulating just one assembly rather than disassembly process. This is consistent with but challenges common interpretations of previous observations at the PHO5 promoter and suggests chromatin opening by binding competition.

    1. Cell Biology
    2. Computational and Systems Biology
    Taraneh Zarin et al.
    Research Advance Updated

    In previous work, we showed that intrinsically disordered regions (IDRs) of proteins contain sequence-distributed molecular features that are conserved over evolution, despite little sequence similarity that can be detected in alignments (Zarin et al., 2019). Here, we aim to use these molecular features to predict specific biological functions for individual IDRs and identify the molecular features within them that are associated with these functions. We find that the predictable functions are diverse. Examining the associated molecular features, we note some that are consistent with previous reports and identify others that were previously unknown. We experimentally confirm that elevated isoelectric point and hydrophobicity, features that are positively associated with mitochondrial localization, are necessary for mitochondrial targeting function. Remarkably, increasing isoelectric point in a synthetic IDR restores weak mitochondrial targeting. We believe feature analysis represents a new systematic approach to understand how biological functions of IDRs are specified by their protein sequences.