Selfee, self-supervised features extraction of animal behaviors
- Tsinghua University, China
- Chinese Academy of Sciences, China
Abstract
Fast and accurately characterizing animal behaviors is crucial for neuroscience research. Deep learning models are efficiently used in laboratories for behavior analysis. However, it has not been achieved to use an end-to-end unsupervised neural network to extract comprehensive and discriminative features directly from social behavior video frames for annotation and analysis purposes. Here, we report a self-supervised feature extraction (Selfee) convolutional neural network with multiple downstream applications to process video frames of animal behavior in an end-to-end way. Visualization and classification of the extracted features (Meta-representations) validate that Selfee processes animal behaviors in a way similar to human perception. We demonstrate that Meta-representations can be efficiently used to detect anomalous behaviors that are indiscernible to human observation and hint in-depth analysis. Furthermore, time-series analyses of Meta-representations reveal the temporal dynamics of animal behaviors. In conclusion, we present a self-supervised learning approach to extract comprehensive and discriminative features directly from raw video recordings of animal behaviors and demonstrate its potential usage for various downstream applications.
Data availability
Major data used in this study were uploaded to Dryad, including pretrained weights. Data could be accessed via:https://doi.org/10.5061/dryad.brv15dvb8.With the uploaded dataset and pretrained weights, our experiments could be replicated. However, due to its huge size and the limited internet service resources, we are currently not able to share our full training dataset. The full dataset is as large as 400GB, which is hard to upload to a public server and will be difficult for others users to download.For training dataset, it would be available from the corresponding author upon reasonable request (wei_zhang@mail.tsinghua.edu.cn), and then we can discuss how to transfer the dataset. No project proposal is needed as long as the dataset is not used for any commercial purpose.Our Python scripts could be accessed on GitHub: https://github.com/EBGU/SelfeeOther software used in our project include ImageJ(https://imagej.net/software/fiji/) and GraphPad Prism(https://www.graphpad.com/).All data used to plot graphs and charts in the manuscript can be fully accessed on Dryad (DOI 10.5061/dryad.brv15dvb8).
-
Data from: Selfee: Self-supervised features extraction of animal behaviorsDryad Digital Repository, doi:10.5061/dryad.brv15dvb8.
Article and author information
Author details
Funding
National Natural Science Foundation of China (32022029)
- Wei Zhang
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
Ethics
Animal experimentation: All mating experiments were approved by the Animal Care and Use Committee of the Institute of Neuroscience, CAS Center for Excellence in Brain Science and Intelligence Technology, Chinese Academy of Sciences, Shanghai, China (IACUC No. NA-016-2016)All studies and experimental protocols of CIS and OFT were approved by Institutional Animal Care and Use Committee (IACUC) at Tsinghua University (No. 19-ZY1). Experiments were performed using the principles outlined in the Guide for the Care and Use of Laboratory Animals of Tsinghua University.
Copyright
© 2022, Jia et al.
This article is distributed under the terms of the Creative Commons Attribution License permitting unrestricted use and redistribution provided that the original author and source are credited.
Metrics
-
- 3,821
- views
-
- 739
- downloads
-
- 14
- citations
Views, downloads and citations are aggregated across all versions of this paper published by eLife.
Download links
A two-part list of links to download the article, or parts of the article, in various formats.
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
Selfee, self-supervised features extraction of animal behaviors
eLife 11:e76218.
https://doi.org/10.7554/eLife.76218
Further reading
-
- Developmental Biology
- Neuroscience
Two major ligand-receptor signaling axes – endothelin Edn3 and its receptor Ednrb, and glial-derived neurotrophic factor (GDNF) and its receptor Ret – are required for migration of enteric nervous system (ENS) progenitors to the hindgut. Mutations in either component cause colonic aganglionosis, also called Hirschsprung disease. Here, we have used Wnt1Cre and Pax2Cre in mice to show that these driver lines label distinct ENS lineages during progenitor migration and in their terminal hindgut fates. Both Cre lines result in Hirschsprung disease when combined with conditional Ednrb or conditional Ret alleles. In vitro explant assays and analysis of lineage-labeled mutant embryos show that GDNF but not Edn3 is a migration cue for cells of both lineages. Instead, Edn3-Ednrb function is required in both for GDNF responsiveness albeit in different ways: by expanding the Ret+ population in the Pax2Cre lineage, and by supporting Ret function in Wnt1Cre-derived cells. Our results demonstrate that two distinct lineages of progenitors give rise to the ENS, and that these divergently utilize endothelin signaling to support migration to the hindgut.
-
- Neuroscience
Memory consolidation in Drosophila can be sleep-dependent or sleep-independent, depending on the availability of food. The anterior posterior (ap) alpha′/beta′ (α′/β′) neurons of the mushroom body (MB) are required for sleep-dependent memory consolidation in flies fed after training. These neurons are also involved in the increase of sleep after training, suggesting a coupling of sleep and memory. To better understand the mechanisms underlying sleep and memory consolidation initiation, we analyzed the transcriptome of ap α′/β′ neurons 1 hr after appetitive memory conditioning. A small number of genes, enriched in RNA processing functions, were differentially expressed in flies fed after training relative to trained and starved flies or untrained flies. Knockdown of each of these differentially expressed genes in the ap α′/β′ neurons revealed notable sleep phenotypes for Polr1F and Regnase-1, both of which decrease in expression after conditioning. Knockdown of Polr1F, a regulator of ribosome RNA transcription, in adult flies promotes sleep and increases pre-ribosome RNA expression as well as overall translation, supporting a function for Polr1F downregulation in sleep-dependent memory. Conversely, while constitutive knockdown of Regnase-1, an mRNA decay protein localized to the ribosome, reduces sleep, adult specific knockdown suggests that effects of Regnase-1 on sleep are developmental in nature. We further tested the role of each gene in memory consolidation. Knockdown of Polr1F does not affect memory, which may be expected from its downregulation during memory consolidation. Regnase-1 knockdown in ap α′/β′ neurons impairs all memory, including short-term, implicating Regnase-1 in memory, but leaving open the question of why it is downregulated during sleep-dependent memory. Overall, our findings demonstrate that the expression of RNA processing genes is modulated during sleep-dependent memory and, in the case of Polr1F, this modulation likely contributes to increased sleep.
