1. Neuroscience

Selfee, self-supervised features extraction of animal behaviors

  1. Yinjun Jia  Is a corresponding author
  2. Shuaishuai Li
  3. Xuan Guo
  4. Bo Lei
  5. Junqiang Hu
  6. Xiao-Hong Xu
  7. Wei Zhang  Is a corresponding author
  1. Tsinghua University, China
  2. Chinese Academy of Sciences, China
https://doi.org/10.7554/eLife.76218
Share this article
Cite this article
  1. Yinjun Jia
  2. Shuaishuai Li
  3. Xuan Guo
  4. Bo Lei
  5. Junqiang Hu
  6. Xiao-Hong Xu
  7. Wei Zhang
(2022)
Selfee, self-supervised features extraction of animal behaviors
eLife 11:e76218.
https://doi.org/10.7554/eLife.76218
  2. Download BibTeX
  3. Download .RIS

Abstract

Fast and accurately characterizing animal behaviors is crucial for neuroscience research. Deep learning models are efficiently used in laboratories for behavior analysis. However, it has not been achieved to use an end-to-end unsupervised neural network to extract comprehensive and discriminative features directly from social behavior video frames for annotation and analysis purposes. Here, we report a self-supervised feature extraction (Selfee) convolutional neural network with multiple downstream applications to process video frames of animal behavior in an end-to-end way. Visualization and classification of the extracted features (Meta-representations) validate that Selfee processes animal behaviors in a way similar to human perception. We demonstrate that Meta-representations can be efficiently used to detect anomalous behaviors that are indiscernible to human observation and hint in-depth analysis. Furthermore, time-series analyses of Meta-representations reveal the temporal dynamics of animal behaviors. In conclusion, we present a self-supervised learning approach to extract comprehensive and discriminative features directly from raw video recordings of animal behaviors and demonstrate its potential usage for various downstream applications.

Data availability

Major data used in this study were uploaded to Dryad, including pretrained weights. Data could be accessed via:https://doi.org/10.5061/dryad.brv15dvb8.With the uploaded dataset and pretrained weights, our experiments could be replicated. However, due to its huge size and the limited internet service resources, we are currently not able to share our full training dataset. The full dataset is as large as 400GB, which is hard to upload to a public server and will be difficult for others users to download.For training dataset, it would be available from the corresponding author upon reasonable request (wei_zhang@mail.tsinghua.edu.cn), and then we can discuss how to transfer the dataset. No project proposal is needed as long as the dataset is not used for any commercial purpose.Our Python scripts could be accessed on GitHub: https://github.com/EBGU/SelfeeOther software used in our project include ImageJ(https://imagej.net/software/fiji/) and GraphPad Prism(https://www.graphpad.com/).All data used to plot graphs and charts in the manuscript can be fully accessed on Dryad (DOI 10.5061/dryad.brv15dvb8).

The following data sets were generated
The following previously published data sets were used

Article and author information

Author details

  1. Yinjun Jia

    IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing, China
    For correspondence
    jyj20@mails.tsinghua.edu.cn
    Competing interests
    The authors declare that no competing interests exist.

  2. Shuaishuai Li

    Institute of Neuroscience, Chinese Academy of Sciences, Beijing, China
    Competing interests
    The authors declare that no competing interests exist.

  3. Xuan Guo

    IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing, China
    Competing interests
    The authors declare that no competing interests exist.

  4. Bo Lei

    IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing, China
    Competing interests
    The authors declare that no competing interests exist.

  5. Junqiang Hu

    IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing, China
    Competing interests
    The authors declare that no competing interests exist.

  6. Xiao-Hong Xu

    Institute of Neuroscience, Chinese Academy of Sciences, Beijing, China
    Competing interests
    The authors declare that no competing interests exist.

  7. Wei Zhang

    IDG/McGovern Institute for Brain Research, Tsinghua University, Beijing, China
    For correspondence
    wei_zhang@mail.tsinghua.edu.cn
    Competing interests
    The authors declare that no competing interests exist.
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-0512-3096

Funding

National Natural Science Foundation of China (32022029)

  • Wei Zhang

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Ethics

Animal experimentation: All mating experiments were approved by the Animal Care and Use Committee of the Institute of Neuroscience, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, China (IACUC No. NA-016-2016)All studies and experimental protocols of CIS and OFT were approved by Institutional Animal Care and Use Committee (IACUC) at Tsinghua University (No. 19-ZY1). Experiments were performed using the principles outlined in the Guide for the Care and Use of Laboratory Animals of Tsinghua University.

Copyright

© 2022, Jia et al.

This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 4,083
    views
  • 762
    downloads
  • 15
    citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Yinjun Jia
  2. Shuaishuai Li
  3. Xuan Guo
  4. Bo Lei
  5. Junqiang Hu
  6. Xiao-Hong Xu
  7. Wei Zhang
(2022)
Selfee, self-supervised features extraction of animal behaviors
eLife 11:e76218.
https://doi.org/10.7554/eLife.76218

Share this article

https://doi.org/10.7554/eLife.76218

Categories and tags

Research organisms

Further reading

    1. Neuroscience

    Leveraging place field repetition to understand positional versus nonpositional inputs to hippocampal field CA1

    William Hockeimer, Ruo-Yah Lai ... James J Knierim
    Research Article

    The hippocampus is believed to encode episodic memory by binding information about the content of experience within a spatiotemporal framework encoding the location and temporal context of that experience. Previous work implies a distinction between positional inputs to the hippocampus from upstream brain regions that provide information about an animal’s location and nonpositional inputs which provide information about the content of experience, both sensory and navigational. Here, we leverage the phenomenon of ‘place field repetition’ to better understand the functional dissociation between positional and nonpositional information encoded in CA1. Rats navigated freely on a novel maze consisting of linear segments arranged in a rectilinear, city-block configuration, which combined elements of open-field foraging and linear-track tasks. Unlike typical results in open-field foraging, place fields were directionally tuned on the maze, even though the animal’s behavior was not constrained to extended, one-dimensional (1D) trajectories. Repeating fields from the same cell tended to have the same directional preference when the fields were aligned along a linear corridor of the maze, but they showed uncorrelated directional preferences when they were unaligned across different corridors. Lastly, individual fields displayed complex time dynamics which resulted in the population activity changing gradually over the course of minutes. These temporal dynamics were evident across repeating fields of the same cell. These results demonstrate that the positional inputs that drive a cell to fire in similar locations across the maze can be behaviorally and temporally dissociated from the nonpositional inputs that alter the firing rates of the cell within its place fields, offering a potential mechanism to increase the flexibility of the system to encode episodic variables within a spatiotemporal framework provided by place cells.

    1. Neuroscience

    Olfactory ensheathing cells from adult female rats are hybrid glia that promote neural repair

    Patricia E Phelps, Sung Min Ha ... Xia Yang
    Research Article

    Olfactory ensheathing cells (OECs) are unique glial cells found in both central and peripheral nervous systems where they support continuous axonal outgrowth of olfactory sensory neurons to their targets. Previously, we reported that following severe spinal cord injury, OECs transplanted near the injury site modify the inhibitory glial scar and facilitate axon regeneration past the scar border and into the lesion. To better understand the mechanisms underlying the reparative properties of OECs, we used single-cell RNA-sequencing of OECs from adult rats to study their gene expression programs. Our analyses revealed five diverse OEC subtypes, each expressing novel marker genes and pathways indicative of progenitor, axonal regeneration, secreted molecules, or microglia-like functions. We found substantial overlap of OEC genes with those of Schwann cells, but also with microglia, astrocytes, and oligodendrocytes. We confirmed established markers on cultured OECs, and localized select top genes of OEC subtypes in olfactory bulb tissue. We also show that OECs secrete Reelin and Connective tissue growth factor, extracellular matrix molecules which are important for neural repair and axonal outgrowth. Our results support that OECs are a unique hybrid glia, some with progenitor characteristics, and that their gene expression patterns indicate functions related to wound healing, injury repair, and axonal regeneration.

    1. Immunology and Inflammation
    2. Neuroscience

    Microglia aging in the hippocampus advances through intermediate states that drive activation and cognitive decline

    Jeremy M Shea, Saul A Villeda
    Research Article

    During aging, microglia – the resident macrophages of the brain – exhibit altered phenotypes and contribute to age-related neuroinflammation. While numerous hallmarks of age-related microglia have been elucidated, the progression from homeostasis to dysfunction during the aging process remains unresolved. To bridge this gap in knowledge, we undertook complementary cellular and molecular analyses of microglia in the mouse hippocampus across the adult lifespan and in the experimental aging model of heterochronic parabiosis. Single-cell RNA-Seq and pseudotime analysis revealed age-related transcriptional heterogeneity in hippocampal microglia and identified intermediate states of microglial aging that also emerge following heterochronic parabiosis. We tested the functionality of intermediate stress response states via TGFβ1 and translational states using pharmacological approaches in vitro to reveal their modulation of the progression to an activated state. Furthermore, we utilized single-cell RNA-Seq in conjunction with in vivo adult microglia-specific Tgfb1 conditional genetic knockout mouse models to demonstrate that microglia advancement through intermediate aging states drives transcriptional inflammatory activation and hippocampal-dependent cognitive decline.