Traditional approaches to analyzing video data have involved researchers watching video playback and noting the times and locations of specific events of interest. These analyses are very time-consuming, require expert knowledge in the target species and experimental design, and are prone to user bias (Anderson and Perona, 2014). Video recordings are often made for many different animals and behavioral test sessions, but only reviewed for a subset of experiments. Complete sets of videos are rarely made accessible in published studies and the analysis methods are often vaguely described. There are variations in scoring criteria across researchers and labs, even over time by a single researcher. Collectively, these issues present major challenges for research reproducibility and the difficulty and cost of manual video analysis has led to the dominance of easy-to-use measures (lever pressing, beam breaks) in the neuroscience literature, and this has limited our understanding of brain-behavior relationships (Krakauer et al., 2017).

For example, ‘reward seeking’ has been a popular topic in recent years and is typically measured using beam breaks between response and reward ports located inside an operant arena (e.g., Cowen et al., 2012; Feierstein et al., 2006; Lardeux et al., 2009; van Duuren et al., 2009). By relying only on the discrete times when animals make a choice and receive a reward, it is not possible to describe how the animal moves during a choice or how it collects a reward. Animals may not move in the same way to a reward port when they expect a larger or smaller reward (e.g., Davidson et al., 1980). This could lead to, for example, a neural recording study labeling a cell as ‘reward encoding’ when it actually reflects differences in movement.

Commercial products (e.g., Ethovision by Noldus, Any-Maze by Stoelting) and open-source projects (e.g., JAABA: Kabra et al., 2013; SCORHE: Salem et al., 2015; OptiMouse: Ben-Shaul, 2017; ezTrack: Pennington et al., 2019) are available for semi-automated annotation and tracking of behaviors. These methods track animals based on differences between the animals and the background color or luminance. This can be challenging to do in naturalistic settings or for species or strains that do not have a uniform color (e.g., Long-Evans rats). These methods provide estimates of the overall position of an animal in its environment and can be used to measure the direction and velocity of its movements. These ‘center of mass’ tracking methods could be used to measure where an animal is and how fast it is moving. More sophisticated versions of these products may also detect the head and tail of common laboratory species such as rodents or zebrafish and draw inferences from the shape and location of the animal to classify a small subset of an animal’s behavioral repertoire. However, these simpler tracking methods cannot account for movements of discrete sets of body parts (e.g., head scanning in rodents, which is associated with a classic measure of reward-guided decisions called ‘vicarious trial-and-error’ behavior: see Redish, 2016, for review).

More advanced analyses could be used to quantify movements across many pixels simultaneously in video recordings. For example, Stringer et al., 2019, used dimensionality reduction methods to study the spontaneous coding of visual- and movement-related information in the mouse visual cortex in relation to facial movements. Musall et al., 2019, used video recordings of motion data from several parts of the face of mice as they performed a decision-making task and related the measures from the video recordings to cortical imaging data. While these analyses would go beyond what is possible to achieve with a simple tracking method, the multivariate methods developed by Stringer and Musall are not themselves capable of categorizing movements, measuring transitions between different types of movements, or quantifying the dynamics of movement sequences. For these measures, a different approach is needed.

Methods for capturing the pose of an animal (the location and configuration of its body) have emerged in recent years (e.g., DeepLabCut: Mathis et al., 2018a; SLEAP: Pereira et al., 2022). These methods can provide a description of an animal’s movement and posture during a behavioral task. They can be used to understand the dynamics of naturalistic movements and behaviors, as illustrated in Figure 1. Pose estimation methods provide information on the position and orientation of multiple parts of an animal, with recent methods being able to measure pose information for groups of animals (Chen et al., 2020; Lauer et al., 2021; Pereira et al., 2022; Walter and Couzin, 2021). Some recent methods now even allow for pose estimation to be run in real experimental time (Kane et al., 2020; Lopes et al., 2015; Pereira et al., 2022; Schweihoff et al., 2021).

Figure 1

Download asset Open asset

Methods for pose estimation emerged in computer vision research in the late 1970s (Marr et al., 1978; Nevatia and Binford, 1973). The methods became widely available for the analysis of pose in human behavior following improvements in computer vision (Behnke, 2003), deep learning (Szegedy et al., 2013), and computing using graphical processing units (GPUs) (Oh and Jung, 2004). However, these methods were often not robust or required a lot of training data, which were at the time not easily available for animal studies. As a result, a number of open-source tools emerged for pose estimation in animals (e.g., DeepLabCut: Mathis et al., 2018a, LEAP: Pereira et al., 2019; DeepPoseKit: Graving et al., 2019a). These tools are especially notable in that they were developed to address specific scientific questions by researchers and are not available from commercial sources. They are an outstanding example of the ‘open-source creative process’ (White et al., 2019).

One of these methods, DeepLabCut, has been shown to outperform the commercial software package EthoVision XT14 and a hardware-based measurement system from TSE Systems, based on IR beam breaks (Sturman et al., 2020). When tested across a set of common behavioral assays used in neuroscience (open field test, elevated plus maze, forced swim test), data from the pose estimation method was evaluated using a neural network classifier and performed as well as classifications by human experts, required data from fewer animals to detect differences due to experimental treatments, and in some cases (head dips in an elevated plus maze) detected effects of treatment (a drug) that was not detected by EthoVision.

In the case of reward seeking behavior, human annotation of videos could resolve the animal’s position and when and for how long specific behaviors occurred. These measurements could be made by annotating frames in the video recordings, using tools such as the VIA annotator (Dutta and Zisserman, 2019), and commercial (e.g., EthoVision) or open-source (e.g., ezTrack) methods for whole-animal tracking. These measurements would not be able to account for coordinated movements of multiple body parts or for the dynamics of transitions between different behaviors that together comprise reward seeking behavior. These measurements are easily made using methods for pose estimation. These methods learn to track multiple body parts (for a rodent, the tip of snout, the ears, the base of the tail) and the positions of these body parts can be compared for different kinds of trials (small or large reward) using standard statistical models or machine learning methods. These analyses, together, allow for movements to be categorized (e.g., direct and indirect approach toward a reward port) and for transitions between different types of movements to be quantified (e.g., from turning to walking). It would even be possible to detect unique states associated with deliberation (e.g., head scanning between available choice options). All these measures could then be compared as a function of an experimental manipulation (drug or lesion) or used to assist in the analysis of simultaneously collected electrophysiological or imaging data. None of these measures are possible using conventional methods for annotating video frames or tracking overall the overall position of the animal in a behavioral arena.

Pose estimation methods have been crucial for several recent publications on topics as diverse as tracking fluid consumption to understand the neural coding of reward prediction errors (Ottenheimer et al., 2020), accounting for the effects of wind on the behavior of Drosophila (Okubo et al., 2020), understanding the contributions of tactile afferents and nociceptors to the perception of touch in freely moving mice (Schorscher-Petcu et al., 2021), understanding interactions between tactile processing by the rodent whisker system and its ability to guide locomotion (Warren et al., 2021), and measuring the relationship between eye movements and neural activity in freely behaving rodents (Keshavarzi et al., 2022). While a number of studies are emerging that take advantage of methods for pose estimation, there is still not enough widespread adoption of the methods across the research community, perhaps in part due to the technical nature of collecting high-quality video recordings as well as setting up and using methods for pose estimation. These methods depend on access to computing systems with GPUs and the ability to set up and use the required computer software, which is usually available as computer code written in Python or MATLAB. A researcher who wants to get started with these approaches will therefore face a number of questions about how to set up video methods in a laboratory setting. New users may also need to learn some of the jargon associated with video analysis methods, and some of these terms are defined in Table 1. The primary goals of this document are twofold: to provide information for researchers interested in setting methods for video analysis in a research lab and to propose best practices for the use and development of video analysis methods.

In a typical setup for video recording, cameras are placed above, and in some cases to the side or below, the behavioral arena (Figure 1). The cameras send data to a computer and can be integrated with inputs from behavioral devices using custom-written programs using popular libraries such as OpenCV (Bradski, 2000), open-source data collection systems such as Bonsai (Lopes et al., 2015), or software included with many common commercial video capture boards (loopbio Motif). Video files can then be analyzed using a variety of open-source tools.

A common approach is to use methods for pose estimation, which track the position and orientation of the animal. This is done by denoting a set of ‘keypoints’ or “landmarks” (body parts) in terms of pixel locations on frames in the video recordings. Packages for pose estimation provide graphical user interfaces for defining keypoints and the keypoints are then analyzed with video analysis methods. In the example shown in Figure 1, keypoints are the colored dots on the tip of the snout, the ears, forelimbs and paw, midpoint of back, hindlimbs and paws, and base, middle, and end of tail. Once body parts have been defined, computer algorithms are used to track the skeleton formed by the points and to track the position and orientation of the skeleton over frames in the video file. Many open-source tools use machine learning methods for these intensive computational processes, which require GPUs to run in reasonable time. To run these analyses, many labs have either dedicated computers, institutional computing clusters, or cloud computing services such as Google Colab. The outputs of pose estimation can be analyzed to account for movement variability associated with different behaviors, to relate position and orientation to simultaneously collected brain activity (electrophysiology, optical imaging), or with algorithms that can describe and predict states and dynamical transitions of behaviors.

2D pose estimation and tracking

Pose tracking methods (Figure 2, part 1) enable researchers to extract positional information about the body parts of animals from video recordings. Tools for pose tracking (see Table 2) decompose the problem of pose tracking into sub-tasks outlined below. A note on nomenclature: pose estimation is the term typically reserved to mean single-animal keypoint localization within a single image; multi-animal pose estimation refers to; multi-animal pose estimation refers to keypoint localization and part grouping of multiple animals within a single image; and multi-animal pose tracking refers to combined keypoint localization, part grouping, and identification across video frames.

Figure 2

Download asset Open asset

Table 2

DeepLabCut	DeepLabCut (Mathis et al., 2018a; Mathis et al., 2018b) uses a popular architecture for deep learning (He et al., 2016), called ResNet. DeepLabCut models are pre-trained on a massive dataset for object recognition called ImageNet (Russakovsky et al., 2015). Through a process called transfer learning, the DeepLabCut model learns the position of keypoints using as few as 200 labeled frames. This makes the model very robust and flexible in terms of what body parts (or objects) users want to label as the model provides a strong backbone of image filters within their ResNet architecture. To detect the keypoint position, DeepLabCut replaces the classification layer of the ResNet with deconvolutional layers to produce spatial probability densities from which the model learns to assign high probabilities to regions with the user labeled keypoints. DeepLabCut can provide very accurate pose estimations but can require extensive time for training.
SLEAP	SLEAP (Pereira et al., 2022) is based on an earlier method called LEAP (Pereira et al., 2022), which performed pose estimation on single animals. SLEAP uses simpler CNN architectures with repeated convolutional and pooling layers. This makes the model more lightweight compared to DLC’s ResNet architecture and, hence, the model is faster to train with comparable accuracy. Similar to DeepLabCut, the model uses a stack of upsampling or deconvolutional layers to estimate confidence maps during training and inference. Unlike DLC, SLEAP does not solely rely on transfer learning from general-purpose network models (though this functionality is also provided for flexible experimentation). Instead, it uses customizable neural network architectures that can be tuned to the needs of the dataset. SLEAP can produce highly accurate pose estimates starting at about 100 labeled frames for training combined and is quick to train on a GPU (<1 hour).
DeepPoseKit	DeepPoseKit (Graving et al., 2019a; Graving et al., 2019b) uses a type of CNN architecture, called stacked DenseNet, an efficient variant of the stacked hourglass (Newell et al., 2016), and uses multiple down- and upsampling steps with densely connected hourglass networks to produce confidence maps on the input image. The model uses only about 5% of the amount of parameters used by DeepLabCut, providing speed improvements over DeepLabCut and LEAP.
B-KinD	B-KinD (Sun et al., 2021a; Sun et al., 2021b) discovers key points without human supervision. B-KinD has the potential to transform how pose estimation is done, as keypoint analysis is one of the most time-consuming aspects of doing pose estimation analysis. However, there are challenges for the approach when occlusions occur in the video recordings, e.g., recordings of animals tethered to brain recording systems.

Keypoint localization involves recovering the spatial coordinates of each distinct keypoint. This is normally done by estimating body part confidence maps, that is, image-based representations that encode the probability of the body part being located at each pixel. Recovering the coordinates of each body part is reduced to the task of finding the pixel with highest probability. A key consideration of this task is that the larger the image, the larger the confidence maps. Computer memory requirements can potentially exceed the capacity of most consumer-grade GPUs. This can be compensated by reducing the resolution of the confidence maps, though this comes at the cost of potentially reduced accuracy. Subpixel refinement methods are typically employed to compensate for this, but ultimately confidence map resolution is one of the most impactful choices for achieving reliable keypoint localization.

For single-animal videos, there will be at most one instance of each keypoint type present in the image, so keypoint localization is the only step strictly required. For multi-animal videos, however, there may be multiple instances of each keypoint type, for example, multiple ‘heads’. Part grouping refers to the task of determining the set of keypoint detections that belong to the same animal within an image. This is often approached in either a bottom-up or top-down fashion. In bottom-up models, all parts are detected, the association between them estimated (e.g., by using part affinity fields: Cao et al., 2017), and then grouped. In top-down models, the animals are detected, cropped out of the image, and then keypoints are located in the same fashion as in the single-animal case. These approaches have specific trade-offs. Analyses of bottom-up recordings tend to be more memory-intensive but also more robust to transient occlusions and work well with animals with relatively large bodies (e.g., rodents). By contrast, top-down recordings tend to be analyzed in less time since only subsets of the image are processed. Top-down views work best with smaller body types that have fewer complex occlusions (e.g., flies). A notable consideration is that all single-animal pose estimation models can be used in the multi-animal setting if the animals can be detected and cropped as a preprocessing step (Graving et al., 2019a; Pereira et al., 2019). While both methods will work on most types of data, drastic improvements in performance and accuracy can be obtained by selecting the appropriate one – most pose estimation tools allow users to select between each approach type.

Once animals are detected and their keypoints located within a frame, the remaining task in multi-animal pose tracking is identification: repeatedly detecting the same animal across frame sequences. This can be approached as a multi-object tracking (MOT) problem, where animals are matched across frames based on a model or assumption about motion; or a re-identification (ReID) problem, where distinctive appearance features are used to unambiguously identify an animal. Both MOT and ReID (and hybrids) are available as standalone functionality in open-source tools, as well as part of multi-animal pose tracking packages. While MOT-based approaches can function on videos of animals with nearly indistinguishable appearances, they are prone to the error propagation issue inherent in methods with temporal dependencies: switching an animal’s identity even once will mean it is wrong for all subsequent frames. This presents a potentially intractable problem for long-term continuous recordings which may be impossible to manually proofread. ReID-like methods circumvent this problem by detecting distinguishing visual features, though this may not be compatible with existing datasets or all experimental paradigms.

The single most significant experimental consideration that will affect the identification problem is whether animals can be visually distinguished. A common experimental manipulation aimed at ameliorating this issue is to introduce visual markers to aid in unique identification of animals. This includes techniques such as grouping animals with different fur colors, painting them with non-toxic dyes (Ohayon et al., 2013), or attaching barcode labels to a highly visible area of their body (Crall et al., 2015). Though an essential part of the pose tracking workflow, identification remains a challenging problem in computer vision and its difficulty should not be underestimated when designing studies involving large numbers of interacting animals. We refer interested readers to previous reviews on multi-animal tracking (Panadeiro et al., 2021; Pereira et al., 2020) for more comprehensive overviews of these topics.

Tools that are based on deep learning work by training deep neural networks (models) to reproduce human annotations of behavior. Methods that strictly depend on learning from human examples are referred to as fully supervised. In the case of animal pose tracking, these supervisory examples (labels) are provided in the form of images and the coordinates of the keypoints of each animal that can be found in them. Most pose tracking software tools fall within this designation and provide graphical interfaces to facilitate labeling. The usability of these interfaces is a crucial consideration as most of the time spent in setting up a pose tracking system will be devoted to manual labeling. The more examples and the greater their diversity, the better that pose tracking models will perform. Previous work has shown that hundreds to thousands of labeled examples may be required to achieve satisfactory results, with a single example taking as much as 2 min to manually label (Mathis et al., 2018a; Pereira et al., 2022; Pereira et al., 2019). To mitigate this, we strongly recommend adopting a human-in-the-loop labeling workflow. This is a practice in which the user trains a model with few labels, generates (potentially noisy) predictions, and imports those predictions into the labeling interface for manual refinement before retraining the model. This can drastically reduce the amount of time taken to generate thousands of labeled images necessary for reliable pose estimation models.

The rule of thumb is that ‘if you can see it, you can track it’, but this aphorism strongly depends on the examples provided to train the model. Important factors to consider in the labeling stage include labeling consistency and sample diversity. Labeling consistency involves minimizing the variability of keypoint placement within and across annotators which helps to ensure that models can learn generalizable rules for keypoint localization. This can be accomplished by formalizing a protocol for labeling, especially for ambiguous cases such as when an animal’s body part is occluded. For example, one convention may be to consistently place a ‘paw’ keypoint at the apex of the visible portion of the body rather than guessing where it may be located beneath an occluding object. Similarly, the principle of consistency should inform which body parts are selected as tracked keypoints. Body parts that are not easily located by the human eye will suffer from labeling inconsistency which may cause inferior overall performance as models struggle to find reliable solutions to detecting them. Sample diversity, on the other hand, refers to the notion that not all labeled examples have equal value when training neural networks. For example, labeling 1000 consecutive frames will ensure that the model is able to track data that looks similar to that segment of time, but will have limited capacity to generalize to data collected in a different session. As a best practice, labels should be sampled from the widest possible set of experimental conditions, time points, and imaging conditions that will be expected to be present in the final dataset.

Improving the capability of models to generalize to new data with fewer (or zero) labels is a currently active area of research. Techniques such as transfer learning and self-supervised learning aim to reduce the labeling burden by training models on related datasets or tasks. For example, B-KinD (Sun et al., 2021a) is able to discover semantically meaningful keypoints in behavioral videos using self-supervision without requiring human annotations. These approaches work by training models to solve similar problems and/or on similar data than those used for pose estimation, with the intuition that some of that knowledge can be reused and thereby will require fewer (or no) labeled examples before achieving the same performance as fully supervised equivalents. Future work in this domain is on track to produce reusable models for commonly encountered experimental species and conditions. We highly encourage practitioners to adopt open data and model sharing to facilitate these efforts where possible.

Having described how to set up and use video recording methods and analysis methods for pose estimation, we would like to close by discussing some best practices in the use and development of methods for video analysis, including recommendations for the open sharing of video data and analysis code.

We hope that our review of the current state of open-source tools for behavioral video analysis will be helpful to the community. We described how to set up video methods in a lab, provided an overview on currently available methods, and provided guidance for best practices in using and developing the methods. As newer tools emerge and more research groups become proficient at using available methods, there is a clear potential for the tools to help with advancing our understanding of the neural basis of behavior.

1. 1. Abdar M
   2. Pourpanah F
   3. Hussain S
   4. Rezazadegan D
   5. Liu L
   6. Ghavamzadeh M
   7. Fieguth P
   8. Cao X
   9. Khosravi A
   10. Acharya UR
   11. Makarenkov V
   12. Nahavandi S

   (2021) A review of uncertainty quantification in deep learning: techniques, applications and challenges

   Information Fusion 76:243–297.

   https://doi.org/10.1016/j.inffus.2021.05.008

   • Google Scholar
2. 1. Anderson DJ
   2. Perona P

   (2014) Toward a science of computational ethology

   Neuron 84:18–31.

   https://doi.org/10.1016/j.neuron.2014.09.005

   • PubMed
   • Google Scholar
3. 1. Bala PC
   2. Eisenreich BR
   3. Yoo SBM
   4. Hayden BY
   5. Park HS
   6. Zimmermann J

   (2020) Automated markerless pose estimation in freely moving macaques with openmonkeystudio

   Nature Communications 11:4560.

   https://doi.org/10.1038/s41467-020-18441-5

   • PubMed
   • Google Scholar
4. Preprint

   1. Balog M
   2. Gaunt AL
   3. Brockschmidt M
   4. Nowozin S
   5. Tarlow D

   (2017) DeepCoder: Learning to Write Programs

   arXiv.

   https://arxiv.org/abs/1611.01989

   • Google Scholar
5. Conference

   1. Batty E
   2. Whiteway M
   3. Saxena S
   4. Biderman D
   5. Abe T
   6. Musall S
   7. Gillis W
   8. Markowitz J
   9. Churchland A
   10. Cunningham JP
   11. Datta SR
   12. Linderman S
   13. Paninski L

   (2019)

   BehaveNet: nonlinear embedding and bayesian neural decoding of behavioral videos

   Advances in Neural Information Processing Systems.

   • Google Scholar
6. Book

   1. Behnke S

   (2003) Hierarchical neural networks for image interpretation

   Berlin, Heidelberg: Springer.

   https://doi.org/10.1007/b11963

   • Google Scholar
7. 1. Ben-Shaul Y

   (2017) OptiMouse: a comprehensive open source program for reliable detection and analysis of mouse body and nose positions

   BMC Biology 15:41.

   https://doi.org/10.1186/s12915-017-0377-3

   • PubMed
   • Google Scholar
8. 1. Berman GJ
   2. Choi DM
   3. Bialek W
   4. Shaevitz JW

   (2014) Mapping the stereotyped behaviour of freely moving fruit flies

   Journal of the Royal Society, Interface 11:20140672.

   https://doi.org/10.1098/rsif.2014.0672

   • PubMed
   • Google Scholar
9. 1. Berman GJ

   (2018) Measuring behavior across scales

   BMC Biology 16:23.

   https://doi.org/10.1186/s12915-018-0494-7

   • PubMed
   • Google Scholar
10. 1. Bradski G

    (2000)

    The OpenCV library

    Dr Dobb’s Journal of Software Tools 120:122–125.

    • Google Scholar
11. Conference

    1. Bregler C

    (1997) Learning and recognizing human dynamics in video sequences

    Presented at the Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition. pp. 568–574.

    https://doi.org/10.1109/CVPR.1997.609382

    • Google Scholar
12. 1. Breiman L

    (2001) Random forests

    Machine Learning 45:5–32.

    https://doi.org/10.1023/A:1010933404324

    • Google Scholar
13. Conference

    1. Cao Z
    2. Simon T
    3. Wei SE
    4. Sheikh Y

    (2017) Realtime Multi-person 2D Pose Estimation Using Part Affinity Fields

    2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR. pp. 7291–7299.

    https://doi.org/10.1109/CVPR.2017.143

    • Google Scholar
14. Preprint

    1. Chen T
    2. Guestrin C

    (2016) XGBoost: A Scalable Tree Boosting System

    arXiv.

    https://arxiv.org/abs/1603.02754

    • Google Scholar
15. Preprint

    1. Chen Z
    2. Zhang R
    3. Eva Zhang Y
    4. Zhou H
    5. Fang HS
    6. Rock RR
    7. Bal A
    8. Padilla-Coreano N
    9. Keyes L
    10. Tye KM
    11. Lu C

    (2020) AlphaTracker: A Multi-Animal Tracking and Behavioral Analysis Tool

    bioRxiv.

    https://doi.org/10.1101/2020.12.04.405159

    • Google Scholar
16. 1. Costa AC
    2. Ahamed T
    3. Stephens GJ

    (2019) Adaptive, locally linear models of complex dynamics

    PNAS 116:1501–1510.

    https://doi.org/10.1073/pnas.1813476116

    • PubMed
    • Google Scholar
17. 1. Cowen SL
    2. Davis GA
    3. Nitz DA

    (2012) Anterior cingulate neurons in the rat MAP anticipated effort and reward to their associated action sequences

    Journal of Neurophysiology 107:2393–2407.

    https://doi.org/10.1152/jn.01012.2011

    • PubMed
    • Google Scholar
18. 1. Crall JD
    2. Gravish N
    3. Mountcastle AM
    4. Combes SA

    (2015) BEEtag: a low-cost, image-based tracking system for the study of animal behavior and locomotion

    PLOS ONE 10:e0136487.

    https://doi.org/10.1371/journal.pone.0136487

    • PubMed
    • Google Scholar
19. 1. Davidson TL
    2. Capaldi ED
    3. Myers DE

    (1980) Effects of reward magnitude on running speed following a deprivation upshift

    Bulletin of the Psychonomic Society 15:150–152.

    https://doi.org/10.3758/BF03334493

    • Google Scholar
20. 1. Dunn TW
    2. Marshall JD
    3. Severson KS
    4. Aldarondo DE
    5. Hildebrand DGC
    6. Chettih SN
    7. Wang WL
    8. Gellis AJ
    9. Carlson DE
    10. Aronov D
    11. Freiwald WA
    12. Wang F
    13. Ölveczky BP

    (2021) Geometric deep learning enables 3D kinematic profiling across species and environments

    Nature Methods 18:564–573.

    https://doi.org/10.1038/s41592-021-01106-6

    • PubMed
    • Google Scholar
21. Conference

    1. Dutta A
    2. Zisserman A

    (2019) The VIA Annotation Software for Images, Audio and Video

    MM ’19. pp. 2276–2279.

    https://doi.org/10.1145/3343031.3350535

    • Google Scholar
22. 1. Feierstein CE
    2. Quirk MC
    3. Uchida N
    4. Sosulski DL
    5. Mainen ZF

    (2006) Representation of spatial goals in rat orbitofrontal cortex

    Neuron 51:495–507.

    https://doi.org/10.1016/j.neuron.2006.06.032

    • PubMed
    • Google Scholar
23. 1. Gebru T
    2. Morgenstern J
    3. Vecchione B
    4. Vaughan JW
    5. Wallach H
    6. Iii HD
    7. Crawford K

    (2021) Datasheets for datasets

    Communications of the ACM 64:86–92.

    https://doi.org/10.1145/3458723

    • Google Scholar
24. 1. Gosztolai A
    2. Günel S
    3. Lobato-Ríos V
    4. Pietro Abrate M
    5. Morales D
    6. Rhodin H
    7. Fua P
    8. Ramdya P

    (2021) LiftPose3D, a deep learning-based approach for transforming two-dimensional to three-dimensional poses in laboratory animals

    Nature Methods 18:975–981.

    https://doi.org/10.1038/s41592-021-01226-z

    • PubMed
    • Google Scholar
25. 1. Graving JM
    2. Chae D
    3. Naik H
    4. Li L
    5. Koger B
    6. Costelloe BR
    7. Couzin ID

    (2019a) DeepPoseKit, a software toolkit for fast and robust animal pose estimation using deep learning

    eLife 8:e47994.

    https://doi.org/10.7554/eLife.47994

    • PubMed
    • Google Scholar
26. Software

    1. Graving JM
    2. Chae D
    3. Naik H
    4. Li L
    5. Koger B
    6. Costelloe BR
    7. Couzin ID

    (2019b) DeepPoseKit, version v0.3.6

    Github.

    https://github.com/jgraving/DeepPoseKit
27. 1. Günel S
    2. Rhodin H
    3. Morales D
    4. Campagnolo J
    5. Ramdya P
    6. Fua P

    (2019) DeepFly3D, a deep learning-based approach for 3D limb and appendage tracking in tethered, adult Drosophila

    eLife 8:e48571.

    https://doi.org/10.7554/eLife.48571

    • PubMed
    • Google Scholar
28. 1. Hayden BY
    2. Park HS
    3. Zimmermann J

    (2022) Automated pose estimation in primates

    American Journal of Primatology 84:e23348.

    https://doi.org/10.1002/ajp.23348

    • PubMed
    • Google Scholar
29. Conference

    1. He K
    2. Zhang X
    3. Ren S
    4. Sun J

    (2016) Deep Residual Learning for Image Recognition

    2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR. pp. 770–778.

    https://doi.org/10.1109/CVPR.2016.90

    • Google Scholar
30. 1. Hsu AI
    2. Yttri EA

    (2021a) B-SOiD, an open-source unsupervised algorithm for identification and fast prediction of behaviors

    Nature Communications 12:5188.

    https://doi.org/10.1038/s41467-021-25420-x

    • PubMed
    • Google Scholar
31. Software

    1. Hsu AI
    2. Yttri EA

    (2021b) B-SOID, version 3.0

    Github.

    https://github.com/YttriLab/B-SOID
32. Conference

    1. Hutchinson B
    2. Smart A
    3. Hanna A
    4. Denton E
    5. Greer C
    6. Kjartansson O
    7. Barnes P
    8. Mitchell M

    (2021) Towards Accountability for Machine Learning Datasets

    FAccT ’21. pp. 560–575.

    https://doi.org/10.1145/3442188.3445918

    • Google Scholar
33. 1. Kabra M
    2. Robie AA
    3. Rivera-Alba M
    4. Branson S
    5. Branson K

    (2013) JAABA: interactive machine learning for automatic annotation of animal behavior

    Nature Methods 10:64–67.

    https://doi.org/10.1038/nmeth.2281

    • PubMed
    • Google Scholar
34. 1. Kane GA
    2. Lopes G
    3. Saunders JL
    4. Mathis A
    5. Mathis MW

    (2020) Real-Time, low-latency closed-loop feedback using markerless posture tracking

    eLife 9:e61909.

    https://doi.org/10.7554/eLife.61909

    • PubMed
    • Google Scholar
35. 1. Karashchuk P
    2. Rupp KL
    3. Dickinson ES
    4. Walling-Bell S
    5. Sanders E
    6. Azim E
    7. Brunton BW
    8. Tuthill JC

    (2021) Anipose: a toolkit for robust markerless 3D pose estimation

    Cell Reports 36:109730.

    https://doi.org/10.1016/j.celrep.2021.109730

    • PubMed
    • Google Scholar
36. 1. Keshavarzi S
    2. Bracey EF
    3. Faville RA
    4. Campagner D
    5. Tyson AL
    6. Lenzi SC
    7. Branco T
    8. Margrie TW

    (2022) Multisensory coding of angular head velocity in the retrosplenial cortex

    Neuron 110:532–543.

    https://doi.org/10.1016/j.neuron.2021.10.031

    • PubMed
    • Google Scholar
37. 1. Kingma DP
    2. Welling M

    (2019) An introduction to variational autoencoders

    Foundations and Trends in Machine Learning 12:307–392.

    https://doi.org/10.1561/2200000056

    • Google Scholar
38. 1. Krakauer JW
    2. Ghazanfar AA
    3. Gomez-Marin A
    4. MacIver MA
    5. Poeppel D

    (2017) Neuroscience needs behavior: correcting a reductionist bias

    Neuron 93:480–490.

    https://doi.org/10.1016/j.neuron.2016.12.041

    • PubMed
    • Google Scholar
39. 1. Lardeux S
    2. Pernaud R
    3. Paleressompoulle D
    4. Baunez C

    (2009) Beyond the reward pathway: coding reward magnitude and error in the rat subthalamic nucleus

    Journal of Neurophysiology 102:2526–2537.

    https://doi.org/10.1152/jn.91009.2008

    • PubMed
    • Google Scholar
40. Preprint

    1. Lauer J
    2. Zhou M
    3. Ye S
    4. Menegas W
    5. Nath T
    6. Rahman MM
    7. Di Santo V
    8. Soberanes D
    9. Feng G
    10. Murthy VN
    11. Lauder G
    12. Dulac C
    13. Mathis MW
    14. Mathis A

    (2021) Multi-Animal Pose Estimation and Tracking with DeepLabCut

    bioRxiv.

    https://doi.org/10.1101/2021.04.30.442096

    • Google Scholar
41. 1. Lopes G
    2. Bonacchi N
    3. Frazão J
    4. Neto JP
    5. Atallah BV
    6. Soares S
    7. Moreira L
    8. Matias S
    9. Itskov PM
    10. Correia PA
    11. Medina RE
    12. Calcaterra L
    13. Dreosti E
    14. Paton JJ
    15. Kampff AR

    (2015) Bonsai: an event-based framework for processing and controlling data streams

    Frontiers in Neuroinformatics 9:7.

    https://doi.org/10.3389/fninf.2015.00007

    • PubMed
    • Google Scholar
42. 1. Luxem K
    2. Mocellin P
    3. Fuhrmann F
    4. Kürsch J
    5. Miller SR
    6. Palop JJ
    7. Remy S
    8. Bauer P

    (2022a) Identifying behavioral structure from deep variational embeddings of animal motion

    Communications Biology 5:1267.

    https://doi.org/10.1038/s42003-022-04080-7

    • PubMed
    • Google Scholar
43. Software

    1. Luxem K
    2. Mocellin P
    3. Fuhrmann F
    4. Kürsch J
    5. Miller SR
    6. Palop JJ
    7. Remy S
    8. Bauer P

    (2022b) VAME, version 3.0

    Github.

    https://github.com/LINCellularNeuroscience/VAME
44. 1. Markowitz JE
    2. Gillis WF
    3. Beron CC
    4. Neufeld SQ
    5. Robertson K
    6. Bhagat ND
    7. Peterson RE
    8. Peterson E
    9. Hyun M
    10. Linderman SW
    11. Sabatini BL
    12. Datta SR

    (2018) The striatum organizes 3D behavior via moment-to-moment action selection

    Cell 174:44–58.

    https://doi.org/10.1016/j.cell.2018.04.019

    • PubMed
    • Google Scholar
45. 1. Marr D
    2. Nishihara HK
    3. Brenner S

    (1978) Representation and recognition of the spatial organization of three-dimensional shapes

    Proceedings of the Royal Society of London. Series B. Biological Sciences 200:269–294.

    https://doi.org/10.1098/rspb.1978.0020

    • Google Scholar
46. 1. Mathis A
    2. Mamidanna P
    3. Cury KM
    4. Abe T
    5. Murthy VN
    6. Mathis MW
    7. Bethge M

    (2018a) DeepLabCut: markerless pose estimation of user-defined body parts with deep learning

    Nature Neuroscience 21:1281–1289.

    https://doi.org/10.1038/s41593-018-0209-y

    • PubMed
    • Google Scholar
47. Software

    1. Mathis A
    2. Mamidanna P
    3. Cury KM
    4. Abe T
    5. Murthy VN
    6. Mathis MW
    7. Bethge M

    (2018b) DeepLabCut, version 3.0

    Github.

    https://github.com/DeepLabCut/DeepLabCut
48. 1. McInnes L
    2. Healy J
    3. Astels S

    (2017) Hdbscan: hierarchical density based clustering

    The Journal of Open Source Software 2:205.

    https://doi.org/10.21105/joss.00205

    • Google Scholar
49. Preprint

    1. McInnes L
    2. Healy J
    3. Melville J

    (2020) UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction

    arXiv.

    https://arxiv.org/abs/1802.03426

    • Google Scholar
50. 1. Musall S
    2. Kaufman MT
    3. Juavinett AL
    4. Gluf S
    5. Churchland AK

    (2019) Single-trial neural dynamics are dominated by richly varied movements

    Nature Neuroscience 22:1677–1686.

    https://doi.org/10.1038/s41593-019-0502-4

    • PubMed
    • Google Scholar
51. Conference

    1. Nevatia K
    2. Binford TO

    (1973)

    Structured descriptions of complex objects

    Proceedings of the 3rd international joint conference on artificial intelligence, IJCAI’73. pp. 641–647.

    • Google Scholar
52. Preprint

    1. Newell A
    2. Yang K
    3. Deng J

    (2016) Stacked Hourglass Networks for Human Pose Estimation

    arXiv.

    https://arxiv.org/abs/1603.06937

    • Google Scholar
53. Preprint

    1. Nilsson SR
    2. Goodwin NL
    3. Choong JJ
    4. Hwang S
    5. Wright HR
    6. Norville ZC
    7. Tong X
    8. Lin D
    9. Bentzley BS
    10. Eshel N
    11. McLaughlin RJ
    12. Golden SA

    (2020a) Simple Behavioral Analysis (SimBA) – An Open Source Toolkit for Computer Classification of Complex Social Behaviors in Experimental Animals

    bioRxiv.

    https://doi.org/10.1101/2020.04.19.049452

    • Google Scholar
54. Software

    1. Nilsson SR
    2. Goodwin NL
    3. Choong JJ
    4. Hwang S
    5. Wright HR
    6. Norville ZC
    7. Tong X
    8. Lin D
    9. Bentzley BS
    10. Eshel N
    11. McLaughlin RJ
    12. Golden SA

    (2020b) Simba, version v1.3

    Github.

    https://github.com/sgoldenlab/simba
55. 1. Oh KS
    2. Jung K

    (2004) Gpu implementation of neural networks

    Pattern Recognition 37:1311–1314.

    https://doi.org/10.1016/j.patcog.2004.01.013

    • Google Scholar
56. 1. Ohayon S
    2. Avni O
    3. Taylor AL
    4. Perona P
    5. Roian Egnor SE

    (2013) Automated multi-day tracking of marked mice for the analysis of social behaviour

    Journal of Neuroscience Methods 219:10–19.

    https://doi.org/10.1016/j.jneumeth.2013.05.013

    • PubMed
    • Google Scholar
57. 1. Okubo TS
    2. Patella P
    3. D’Alessandro I
    4. Wilson RI

    (2020) A neural network for wind-guided COMPASS navigation

    Neuron 107:924–940.

    https://doi.org/10.1016/j.neuron.2020.06.022

    • PubMed
    • Google Scholar
58. 1. Ottenheimer DJ
    2. Bari BA
    3. Sutlief E
    4. Fraser KM
    5. Kim TH
    6. Richard JM
    7. Cohen JY
    8. Janak PH

    (2020) A quantitative reward prediction error signal in the ventral pallidum

    Nature Neuroscience 23:1267–1276.

    https://doi.org/10.1038/s41593-020-0688-5

    • PubMed
    • Google Scholar
59. 1. Panadeiro V
    2. Rodriguez A
    3. Henry J
    4. Wlodkowic D
    5. Andersson M

    (2021) A review of 28 free animal-tracking software applications: Current features and limitations

    Lab Animal 50:246–254.

    https://doi.org/10.1038/s41684-021-00811-1

    • PubMed
    • Google Scholar
60. 1. Pennington ZT
    2. Dong Z
    3. Feng Y
    4. Vetere LM
    5. Page-Harley L
    6. Shuman T
    7. Cai DJ

    (2019) EzTrack: An open-source video analysis pipeline for the investigation of animal behavior

    Scientific Reports 9:19979.

    https://doi.org/10.1038/s41598-019-56408-9

    • PubMed
    • Google Scholar
61. 1. Pereira T.D
    2. Aldarondo DE
    3. Willmore L
    4. Kislin M
    5. Wang SSH
    6. Murthy M
    7. Shaevitz JW

    (2019) Fast animal pose estimation using deep neural networks

    Nature Methods 16:117–125.

    https://doi.org/10.1038/s41592-018-0234-5

    • PubMed
    • Google Scholar
62. 1. Pereira T.D
    2. Shaevitz JW
    3. Murthy M

    (2020) Quantifying behavior to understand the brain

    Nature Neuroscience 23:1537–1549.

    https://doi.org/10.1038/s41593-020-00734-z

    • PubMed
    • Google Scholar
63. 1. Pereira TD
    2. Tabris N
    3. Matsliah A
    4. Turner DM
    5. Li J
    6. Ravindranath S
    7. Papadoyannis ES
    8. Normand E
    9. Deutsch DS
    10. Wang ZY
    11. McKenzie-Smith GC
    12. Mitelut CC
    13. Castro MD
    14. D’Uva J
    15. Kislin M
    16. Sanes DH
    17. Kocher SD
    18. Wang SSH
    19. Falkner AL
    20. Shaevitz JW
    21. Murthy M

    (2022) SLEAP: A deep learning system for multi-animal pose tracking

    Nature Methods 19:486–495.

    https://doi.org/10.1038/s41592-022-01426-1

    • PubMed
    • Google Scholar
64. 1. Rameau F
    2. Park J
    3. Bailo O
    4. Kweon IS

    (2022) MC-calib: A generic and robust calibration toolbox for multi-camera systems

    Computer Vision and Image Understanding 217:103353.

    https://doi.org/10.1016/j.cviu.2021.103353

    • Google Scholar
65. 1. Redish AD

    (2016) Vicarious trial and error

    Nature Reviews. Neuroscience 17:147–159.

    https://doi.org/10.1038/nrn.2015.30

    • PubMed
    • Google Scholar
66. 1. Rudin C

    (2019) Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead

    Nature Machine Intelligence 1:206–215.

    https://doi.org/10.1038/s42256-019-0048-x

    • PubMed
    • Google Scholar
67. 1. Russakovsky O
    2. Deng J
    3. Su H
    4. Krause J
    5. Satheesh S
    6. Ma S
    7. Huang Z
    8. Karpathy A
    9. Khosla A
    10. Bernstein M
    11. Berg AC
    12. Fei-Fei L

    (2015) Image net large scale visual recognition challenge

    International Journal of Computer Vision 115:211–252.

    https://doi.org/10.1007/s11263-015-0816-y

    • Google Scholar
68. 1. Salem GH
    2. Dennis JU
    3. Krynitsky J
    4. Garmendia-Cedillos M
    5. Swaroop K
    6. Malley JD
    7. Pajevic S
    8. Abuhatzira L
    9. Bustin M
    10. Gillet JP
    11. Gottesman MM
    12. Mitchell JB
    13. Pohida TJ

    (2015) SCORHE: A novel and practical approach to video monitoring of laboratory mice housed in vivarium cage racks

    Behavior Research Methods 47:235–250.

    https://doi.org/10.3758/s13428-014-0451-5

    • PubMed
    • Google Scholar
69. Conference

    1. Schonberger JL
    2. Frahm JM

    (2016) Structure-from-motion revisited

    2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). pp. 4104–4113.

    https://doi.org/10.1109/CVPR.2016.445

    • Google Scholar
70. Book

    1. Schönberger JL
    2. Zheng E
    3. Frahm JM
    4. Pollefeys M

    (2016) Pixelwise view selection for unstructured multi-view stereo in

    In: Leibe B, Matas J, Sebe N, Welling M, editors. Computer vision – ECCV 2016, Lecture notes in computer science. Cham: Springer International Publishing. pp. 501–518.

    https://doi.org/10.1007/978-3-319-46487-9_31

    • Google Scholar
71. 1. Schorscher-Petcu A
    2. Takács F
    3. Browne LE

    (2021) Scanned optogenetic control of mammalian somatosensory input to map input-specific behavioral outputs

    eLife 10:e62026.

    https://doi.org/10.7554/eLife.62026

    • PubMed
    • Google Scholar
72. 1. Schweihoff JF
    2. Loshakov M
    3. Pavlova I
    4. Kück L
    5. Ewell LA
    6. Schwarz MK

    (2021) Deep lab stream enables closed-loop behavioral experiments using deep learning-based markerless, real-time posture detection

    Communications Biology 4:130.

    https://doi.org/10.1038/s42003-021-01654-9

    • PubMed
    • Google Scholar
73. Preprint

    1. Schweihoff JF
    2. Hsu AI
    3. Schwarz MK
    4. Yttri EA

    (2022) A-SOiD, an Active Learning Platform for Expert-Guided, Data Efficient Discovery of Behavior

    bioRxiv.

    https://doi.org/10.1101/2022.11.04.515138

    • Google Scholar
74. Book

    1. Sculley D
    2. Holt G
    3. Golovin D
    4. Davydov E
    5. Phillips T
    6. Ebner D
    7. Chaudhary V
    8. Young M
    9. Crespo JF
    10. Dennison D

    (2015)

    Hidden Technical Debt in Machine Learning Systems Advances in Neural Information Processing Systems

    Curran Associates, Inc.

    • Google Scholar
75. 1. Segalin C
    2. Williams J
    3. Karigo T
    4. Hui M
    5. Zelikowsky M
    6. Sun JJ
    7. Perona P
    8. Anderson DJ
    9. Kennedy A

    (2021a) The mouse action recognition system (mars) software pipeline for automated analysis of social behaviors in mice

    eLife 10:e63720.

    https://doi.org/10.7554/eLife.63720

    • PubMed
    • Google Scholar
76. Software

    1. Segalin C
    2. Williams J
    3. Karigo T
    4. Hui M
    5. Zelikowsky M
    6. Sun JJ
    7. Perona P
    8. Anderson DJ
    9. Kennedy A

    (2021b) MARS, version 1.8

    Github.

    https://github.com/neuroethology/MARS
77. Conference

    1. Shi C
    2. Schwartz S
    3. Levy S
    4. Achvat S
    5. Abboud M
    6. Ghanayim A
    7. Schiller J
    8. Mishne G

    (2021)

    Learning disentangled behavior embeddings

    Advances in neural information processing systems. pp. 22562–22573.

    • Google Scholar
78. Conference

    1. Stoyanovich J
    2. Howe B

    (2019)

    Nutritional labels for data and models

    A Quarterly Bulletin of the Computer Society of the IEEE Technical Committee on Data Engineering.

    • Google Scholar
79. 1. Stringer C
    2. Pachitariu M
    3. Steinmetz N
    4. Reddy CB
    5. Carandini M
    6. Harris KD

    (2019) Spontaneous behaviors drive multidimensional, brainwide activity

    Science 364:255.

    https://doi.org/10.1126/science.aav7893

    • PubMed
    • Google Scholar
80. 1. Sturman O
    2. von Ziegler L
    3. Schläppi C
    4. Akyol F
    5. Privitera M
    6. Slominski D
    7. Grimm C
    8. Thieren L
    9. Zerbi V
    10. Grewe B
    11. Bohacek J

    (2020) Deep learning-based behavioral analysis reaches human accuracy and is capable of outperforming commercial solutions

    Neuropsychopharmacology 45:1942–1952.

    https://doi.org/10.1038/s41386-020-0776-y

    • PubMed
    • Google Scholar
81. Preprint

    1. Sun JJ
    2. Ryou S
    3. Goldshmid RH
    4. Weissbourd B
    5. Dabiri JO
    6. Anderson DJ
    7. Kennedy A
    8. Yue Y
    9. Perona P

    (2021a) Self-Supervised Keypoint Discovery in Behavioral Videos

    arXiv.

    https://arxiv.org/abs/2112.05121

    • Google Scholar
82. Software

    1. Sun JJ
    2. Ryou S
    3. Goldshmid RH
    4. Weissbourd B
    5. Dabiri JO
    6. Anderson DJ
    7. Kennedy A
    8. Yue Y
    9. Perona P

    (2021b) BKinD, version 2.0

    Github.

    https://github.com/neuroethology/BKinD
83. Conference

    1. Sun JJ
    2. Kennedy A
    3. Zhan E
    4. Anderson DJ
    5. Yue Y
    6. Perona P

    (2021c) Task programming: Learning data efficient behavior representations

    IEEE Computer Society Conference on Computer Vision and Pattern Recognition. pp. 2875–2884.

    https://doi.org/10.1109/cvpr46437.2021.00290

    • Google Scholar
84. Software

    1. Sun JJ
    2. Kennedy A
    3. Zhan E
    4. Anderson DJ
    5. Yue Y
    6. Perona P

    (2021d) TREBA, version ceacb66

    Github.

    https://github.com/neuroethology/TREBA
85. Preprint

    1. Sun JJ
    2. Tjandrasuwita M
    3. Sehgal A
    4. Solar-Lezama A
    5. Chaudhuri S
    6. Yue Y
    7. Costilla-Reyes O

    (2022) Neurosymbolic programming for science

    arXiv.

    https://arxiv.org/abs/2210.05050

    • Google Scholar
86. Conference

    1. Szegedy C
    2. Toshev A
    3. Erhan D

    (2013)

    Deep neural networks for object detection

    Advances in neural information processing systems.

    • Google Scholar
87. 1. van Duuren E
    2. van der Plasse G
    3. Lankelma J
    4. Feenstra MGP
    5. Pennartz CMA

    (2009) Single-cell and population coding of expected reward probability in the orbitofrontal cortex of the rat

    The Journal of Neuroscience 29:8965–8976.

    https://doi.org/10.1523/JNEUROSCI.0005-09.2009

    • PubMed
    • Google Scholar
88. 1. Walter T
    2. Couzin ID

    (2021) Trex, a fast multi-animal tracking system with markerless identification, and 2D estimation of posture and visual fields

    eLife 10:e64000.

    https://doi.org/10.7554/eLife.64000

    • PubMed
    • Google Scholar
89. 1. Warren RA
    2. Zhang Q
    3. Hoffman JR
    4. Li EY
    5. Hong YK
    6. Bruno RM
    7. Sawtell NB

    (2021) A rapid whisker-based decision underlying skilled locomotion in mice

    eLife 10:e63596.

    https://doi.org/10.7554/eLife.63596

    • PubMed
    • Google Scholar
90. 1. White SR
    2. Amarante LM
    3. Kravitz AV
    4. Laubach M

    (2019) The future is open: Open-source tools for behavioral neuroscience research

    ENeuro 6:ENEURO.0223-19.2019.

    https://doi.org/10.1523/ENEURO.0223-19.2019

    • PubMed
    • Google Scholar
91. 1. Wilkinson MD
    2. Dumontier M
    3. Aalbersberg IJJ
    4. Appleton G
    5. Axton M
    6. Baak A
    7. Blomberg N
    8. Boiten JW
    9. da Silva Santos LB
    10. Bourne PE
    11. Bouwman J
    12. Brookes AJ
    13. Clark T
    14. Crosas M
    15. Dillo I
    16. Dumon O
    17. Edmunds S
    18. Evelo CT
    19. Finkers R
    20. Gonzalez-Beltran A
    21. Gray AJG
    22. Groth P
    23. Goble C
    24. Grethe JS
    25. Heringa J
    26. ’t Hoen PAC
    27. Hooft R
    28. Kuhn T
    29. Kok R
    30. Kok J
    31. Lusher SJ
    32. Martone ME
    33. Mons A
    34. Packer AL
    35. Persson B
    36. Rocca-Serra P
    37. Roos M
    38. van Schaik R
    39. Sansone SA
    40. Schultes E
    41. Sengstag T
    42. Slater T
    43. Strawn G
    44. Swertz MA
    45. Thompson M
    46. van der Lei J
    47. van Mulligen E
    48. Velterop J
    49. Waagmeester A
    50. Wittenburg P
    51. Wolstencroft K
    52. Zhao J
    53. Mons B

    (2016) The fair guiding principles for scientific data management and stewardship

    Scientific Data 3:160018.

    https://doi.org/10.1038/sdata.2016.18

    • PubMed
    • Google Scholar
92. 1. Wiltschko AB
    2. Tsukahara T
    3. Zeine A
    4. Anyoha R
    5. Gillis WF
    6. Markowitz JE
    7. Peterson RE
    8. Katon J
    9. Johnson MJ
    10. Datta SR

    (2020) Revealing the structure of pharmacobehavioral space through motion sequencing

    Nature Neuroscience 23:1433–1443.

    https://doi.org/10.1038/s41593-020-00706-3

    • PubMed
    • Google Scholar
93. Preprint

    1. Wu A
    2. Buchanan EK
    3. Whiteway MR
    4. Schartner M
    5. Meijer G
    6. Noel JP
    7. Rodriguez E
    8. Everett C
    9. Norovich A
    10. Schaffer E
    11. Mishra N
    12. Salzman CD
    13. Angelaki D
    14. Bendesky A
    15. Laboratory TIB
    16. Cunningham J
    17. Paninski L

    (2020) Deep Graph Pose: A Semi-Supervised Deep Graphical Model for Improved Animal Pose Tracking

    bioRxiv.

    https://doi.org/10.1101/2020.08.20.259705

    • Google Scholar
94. Preprint

    1. Zhan E
    2. Sun JJ
    3. Kennedy A
    4. Yue Y
    5. Chaudhuri S

    (2021) Unsupervised Learning of Neurosymbolic Encoders

    arXiv.

    https://arxiv.org/abs/2107.13132

    • Google Scholar

Author details

1. Kevin Luxem

   Cellular Neuroscience, Leibniz Institute for Neurobiology, Magdeburg, Germany

   Contribution

   Conceptualization, Writing - original draft, Writing - review and editing

   Contributed equally with

   Jennifer J Sun

   Competing interests

   No competing interests declared

2. Jennifer J Sun

   Department of Computing and Mathematical Sciences, California Institute of Technology, Pasadena, United States

   Contribution

   Conceptualization, Writing - original draft, Writing - review and editing

   Contributed equally with

   Kevin Luxem

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-0906-6589

3. Sean P Bradley

   Rodent Behavioral Core, National Institute of Mental Health, National Institutes of Health, Bethesda, United States

   Contribution

   Conceptualization, Writing - original draft, Writing - review and editing

   Competing interests

   No competing interests declared

4. Keerthi Krishnan

   Department of Biochemistry and Cellular & Molecular Biology, University of Tennessee, Knoxville, United States

   Contribution

   Conceptualization, Writing - original draft, Writing - review and editing

   Competing interests

   No competing interests declared

5. Eric Yttri

   Department of Biological Sciences, Carnegie Mellon University, Pittsburgh, United States

   Contribution

   Conceptualization, Writing - original draft, Writing - review and editing

   Competing interests

   No competing interests declared

6. Jan Zimmermann

   Department of Neuroscience, University of Minnesota, Minneapolis, United States

   Contribution

   Conceptualization, Writing - original draft, Writing - review and editing

   Competing interests

   No competing interests declared

7. Talmo D Pereira

   The Salk Institute of Biological Studies, La Jolla, United States

   Contribution

   Conceptualization, Writing - original draft, Writing - review and editing

   For correspondence

   talmo@princeton.edu

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0001-9075-8365

8. Mark Laubach

   Department of Neuroscience, American University, Washington D.C., United States

   Contribution

   Conceptualization, Funding acquisition, Writing - original draft, Project administration, Writing - review and editing

   For correspondence

   mark.laubach@gmail.com

   Competing interests

   No competing interests declared

   "This ORCID iD identifies the author of this article:" 0000-0002-2403-4497

• 12,312

  views

• 1,094

  downloads

• 37

  citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.