Processing pipeline and behavior representation. (a) Processing pipeline overview. We use a large set of video data of worm genetic strains and employ a contrastive learning approach to encode individual poses of the worm directly from the video frames. We next inspect these pose embeddings using their visualization in a 3D scatter plot. The trained pose embedding network is used to embed each video frame which is next an input to the sequence embedding network. Similarly to pose embeddings, we inspect the embedding space of worm behaviors using visualization techniques and motion features quantified with Tierpsy. (b) Visualization of the strain dataset behavior embedding space colored by the underlying genetic strain. (c) Visualization of the aging dataset behavior embedding space, illustrating the behavioral change with age in the direction of the arrow moving from young (blue) to old (red).

Data preprocessing and network architecture. 1) Data preprocessing pipeline: Artifacts are removed keeping only the worm as foreground object. We change the background to gray, crop the image to keep the worm centered and resize it to 128 × 128px. Finally, we rotate the worm to a vertical orientation. 2) A contrastive learning network is trained with images in random order to learn pose embeddings. 3) Using the ResNet-18 trained in (2) we embed sequences of 12 frames of moving C. elegans. Rotation information is concatenated with the encoded sequences and the last 5 frame embeddings are masked out. A Transformer-encoder learns behavior embeddings by imputing the masked sequence elements.

Visualization of the pose embedding space. (a) We reduced the embedding space to 3D using UMAP and colored it with the Tierpsy max_amplitude feature. Dark gray dots indicate poses for which this feature could not be quantified using Tierpsy. There is a gradient in coloring suggesting that similar poses occupy neighboring parts of the embedding space. Example images of poses are shown with an indication of their position in the embedding space. Strongly coiled and almost straight worms occupy opposite ends of the point cloud. (b-e) Pose embedding space colored according to their eigenworm 1 to 4 values.

Behavior embedding space of the strain dataset. (a) Embedding space colored by strain. Worm images above correspond to 1st, 6th and 12th frame of three example sequences. (b) Embeddings space colored by tail tip speed and (c) head speed. Gray dots in (b) and (c) indicate sequences for which these Tierpsy features are missing.

Behavior embedding space of the strain and aging datasets combined. (a) Embedding space colored by age. With age we refer to the day of adulthood of an individual C. elegans. Gray color indicates missing age data of worms from the strain dataset. (b) Behaviors of one individual linked over the course of its aging. Starting with blue at the last day of the L4 stage, progressing to red until the last recorded day (23) of adulthood.