MorphoNet 2.0: An innovative approach for qualitative assessment and segmentation curation of large-scale 3D time-lapse imaging datasets
- Laboratoire d'informatique, de robotique et de microélectronique de Montpellier, LIRMM, Université de Montpellier, CNRS, France
- Centre de Recherche de Biologie cellulaire de Montpellier, CRBM, Université de Montpellier, CNRS, France
- Montpellier Ressources Imagerie, Biocampus, Université de Montpellier, CNRS, INSERM, France
Figures
Figure 1
Illustration of the visualization of datasets of various complexity and nature in the MorphoNet standalone application.
(a–f) Visualization of a 64 cell stage Phallusia mammillata embryo with labeled cell nuclei and cell membranes. (a-c) Intensity images showing the nuclei (a), the membranes (b) or both (c). (d) Same as c with additional nuclei segmentation obtained with the Binarize plugin (see Materials and methods for the full description of curation plugins). (e) Same as b with additional membrane segmentation obtained with the Cellpose 3D plugin. (f) Same as c with a combination of several rendering possibilities of cell and nuclei segmentations. (g) Multi-colored shaders allow the simultaneous visualization of the expression patterns of multiple genes extracted from the ANISEED (Dardaillon et al., 2020) database and of tissue fate information. Ascidian embryo (Guignard et al., 2020) at stage 15 (mid neurula); cells with a single color are colored with larval tissue fate; multi-colored cells are colored with both larval tissue fate and the expression of selected genes. (h) Visualization of a 6 days post-fertilization Platynereis dumerilii embryo (Vergara et al., 2021) imaged by whole-body serial block face scanning electron microscopy followed by the automated whole-cell segmentation of 16,000 cells. (i-k) Visualization of a cell cycle 14 Drosophila melanogaster embryo imaged with SiMView microscopy and segmented with RACE (Stegmaier et al., 2016). (i) Projection on each segmented cell of the mean image intensity. (j) Projection on each segmented cell of the ratio between the length of the major and the minor axes of the ellipse that has the same normalized second central moments as the segmented cell. (k) Projection of the cell volume.
Figure 2
MorphoNet Standalone Schema.
From the local data (loaded from the green box), the MorphoNet Standalone application first computes the dual meshes for each of the segmented objects (in the module python in the yellow box). Then, using the 3D viewer (in the blue box), users identify detection, segmentation, or tracking issues using, if necessary, the cell lineage information, the raw images, and/or properties computed on the segmented objects. Errors are then corrected by choosing and executing the appropriate image processing plugin from the curation menu. Finally, new meshes are computed from the result of the plugin execution to update the visualization.
Figure 3 with 3 supplements
Unsupervised quality assessment and curation of the Tribolium castaneum embryo.
(a) The five steps of the curation pipeline. (b) View of the original intensity images (in green) of the first time step of the published data (Maška et al., 2023). Both Ground Truth channel (GT, red point) and Silver Truth (ST, white nucleus segmentation) are shown at the top. Blue segmentation corresponds to the match between ST and GT. Bottom, zoom at the GT region (ROI) without the intensity images. (c) Projection for the ST of the distance between the gravity center of the intensity inside the segmentation and the centroid of the segmentation. Color bar at the bottom of d. (d) Same as c for the curated pipeline. (e) Projection for the ST of the deviation of the intensity at the border of the segmentation. Color bar at the bottom of f. (f) Same as e for the curated pipeline. (g) Comparative histogram of the intensity_offset property distribution between the ST, the Step 1 and the Step 5 for the 181 curated nuclei (left) and the whole image (right). (h) Same as g for the distribution of the intensity_border_variation property. (i) Same as g for the distribution of the nuclei volume property.
Figure 3—figure supplement 1
Illustration of properties intensity_border_variation and intensity_offset for 2 nuclei.
2D slice of the intensity image of 2 distinct nuclei superimposed with their corresponding segmentation border (in yellow). The blue cross is the corresponding gravity center of the intensity images inside the segmentation. The blue cross corresponds to the geometrical center of the segmentation shape. The intensity offset is the Euclidean distance between the geometric center of the segmented object and the center of mass of the signal intensity. The intensity_border_variation is the standard deviation of the intensity images only at the border of the segmentation (in yellow). The left nucleus, representing a well-segmented example, exhibits low values for both properties (intensity_offset = 0.18 and intensity_border_variation = 1.84). The right nucleus, where the segmentation is misaligned, shows high values for both properties (intensity_offset = 2.06 and intensity_border_variation = 10.73).
Figure 3—figure supplement 2
Analysis of two automatically computed intensity properties.
(a) Published data, Silver Truth (ST) segmentation corresponding to the ROI of the Gold truth (GT). Color code: green: exact matching between a segmentation in the ST and the GT; red: identified over-segmentation in the ST close to the GT; blue: missing cell of the GT with no correspondence in the ST. (b) Projection of the property intensity_border_variation onto the nuclei segmentation selected in a. (c) Projection of the property intensity_offset onto the same nuclei segmentation. (d) Double projection of both property intensity_offset and intensity_border_variation. Colormap same as in b and c. (e) 2D plot of the values of both properties (intensity_offset in X-axis and intensity_border_variation in Y-axis) of the same selected nuclei segmentation. Color code is equivalent as in a. The blue line in each axis represents the last quartile of the distribution. (f) Double projection of both properties (same as in d) on the whole ST embryo. (g) Same as in f for the curated data obtained using the five steps pipeline (see Figure 3). (h) 2D plot of 40 ground truth cells and their corresponding segmentations in both the published and curated datasets, showing intensity_offset property on the X-axis and Intersection over Union (IoU) on the Y-axis. (i) Same as in h with the intensity_border_variation property on the X-axis. (j) Comparison of IoU values for 40 ground truth cells and their corresponding segmentations in the published and curated datasets. Points above the red line correspond to cells with improved IoU values in the curated dataset.
Figure 3—figure supplement 3
Distribution of three properties for the Tribolium reconstruction.
Comparative histogram of a specific property distribution between the silver truth (ST), the Step 1, and the Step 5 for the 181 curated nuclei (left) and the whole image (right). (a) Comparative histogram of the distribution of the intensity offset property. (b) Comparative histogram of the distribution of the intensity border deviation property. (c) Comparative histogram of the nuclei volume distribution.
Figure 4 with 1 supplement
Starfish whole-cell segmentation using Cellpose.
(a) Animal and Lateral view of the maximum intensity projection from the published dataset (Barone et al., 2024). (b) Segmented ground truth image. (c) Result of the segmentation using Cellpose Cyto2 model followed by the removal of small cells (<1000 voxels) with the Deli plugin (d) Result of a Cellpose segmentation with a model trained on P. miniata dataset followed by the Deli plugin. (e) New cells created (different between b and d) (f) Under segmentation and missing cells generated by d compared to b. (g) Vegetal view of c with smoothness representation. (h) Opposite view of d with smoothness representation. (i) Cell size distribution in the published segmentation (b), Cellpose cyto2 model (c), and Cellpose with P. miniata model (d). (j) Identical as i after application of the Deli plugin. Colors in b-f represent cell volume in µm3. Colors in g, h represent Smoothness.
Figure 4—figure supplement 1
Evaluation of Segmentation in Patiria miniata starfish embryo.
(a) Histogram of Intersection over Union (IoU) values comparing the published dataset (gray) with the Cellpose cyto2+Deli pipeline (orange) and the Cellpose P. miniata model + Deli pipeline (green).
Figure 5 with 1 supplement
Curation of a segmented shoot apical meristem of Arabidopsis thaliana.
Visualization of the time step n°19 for several types of curations using MorphoNet. (a) 3D view of an Arabidopsis thaliana shoot apical meristem (Willis et al., 2016). (b) Published 3D intensity images. (c) Published 3D segmentation of an Arabidopsis thaliana shoot apical meristem (Willis et al., 2016) obtained using MARS-ALT (Fernandez et al., 2010). (d) Comparative histogram based on the cell volume between the published segmentation (c), the result of the cyto2 prediction (e), and the final curated version (i) + Deli plugin for cells <1000 voxels. X and Y axes are in log scale. (e) Result of the Cellpose (Stringer et al., 2021) 3D MorphoNet plugin using the pretrained cyto2 model. (f) Result using the Cellpose 3D MorphoNet plugin using the model trained over the first 10 time steps with the Cellpose training plugin with the XY planes. (g) Result of the Cellpose 3D MorphoNet plugin using the model trained over the first 10 time steps with the Cellpose training plugin with each plane of the 3D images. (h) Selected masks larger than 300 µm3 in the published dataset (c). (i) Result of the Cellpose 3D MorphoNet on the selected masks (h) using the model trained over the ten first time steps with the Cellpose training plugin with each plane of the 3D images.
Figure 5—figure supplement 1
Comparison of the Cellpose models for the reconstruction shoot apical meristem of Arabidopsis thaliana.
(a) Result of the Cellpose prediction on the full image using the model cyto. Color bar on the right represents the cell volume in µm3. (b) Same as a with cyto2 Cellpose model. (c) Same as a with cyto3 Cellpose model. (d) Result of the Cellpose prediction using the model cyto3 with the option disconnect non-connected component activated. (e) Identical as d with the visualization for cells with a volume <10 µm3. (f) Result of the Cellpose prediction using the model cyto3 with the option disconnect non-connected component activated and the option to reassign objects under a volume <10 µm3. (g) Distribution of the segmented cell volume (µm3) for the time step 19. Left: comparison of the three default models of Cellpose (cyto,cyto2, and cyto3). Cellpose train 3D: the model was trained using XY, YZ, and XZ planes and the prediction is then applied on the full 3D image. Cellpose train 3D selected masks: the model was trained using XY, YZ, and XZ planes and the prediction is then applied only on the selected masks with volumes >300 µm3. (h) 3D view of the masks larger than 300 µm³ in the published dataset. Same orientation as in Figure 5a. (i) Same view of the corresponding masks after curation.
Figure 6 with 1 supplement
Curation of a Caenorhabditis elegans embryo dataset.
(a) Cell lineage viewer of all time steps colored by clonal cells of the published segmented dataset (Murray et al., 2008). (b) Comparative histogram based on the axis ratio between the published data and the curation. (c) 3D View of the intensity images at t=150. (d) 3D View of the published segmented dataset at t=150. Colors represent the ratio of the longest axis on the shortest axis of the shape. (e) Automatic selection (in gray) of the nuclei with an axis ratio >2.9. (f) Result of the Fusoc plugin applied on all selected nuclei. Colors represent the nuclei volume in µm3. (g) Same as f but only previously selected nuclei are shown. (h) Result of the Gaumi plugin applied independently on regions fused with four, three or two nuclei. Colors represent the axis ratio as in d.
Figure 6—figure supplement 1
Manual Validation of Curation for the Caenorhabditis elegans Embryo.
(a) 2D XZ view at Y=306 of the intensity image at time point t=150 using Napari. (b) Same view showing the published segmentation, where vertically elongated segmentation artifacts are visually apparent. (c) Same view after curation, where these artifacts have been corrected.
Figure 7 with 1 supplement
Curation of a Phallusia mammillata ascidian embryo named Astec-Pm9 in the publication (Guignard et al., 2020).
(a) Cell lineage of the cell b7.1 after the execution of the Disco and Deli plugins on the published data. Projection of the volume property, the colormap between b and c represents the cell volume in µm3. (b) Scatter view of the corresponding segmented embryo described in a. at t=28 colored by cell volume. Colormap identical as a. (c) Same view as b with the activation of the ‘highlight’ mode which focuses on the selected cell and shows other cells in transparent colors. (d) Several snapshots of the cell b8.21 at different time points with its associated cell lineage. Colormap represents the cell volume in µm3 which points to a missing division. (e) Cell lineage of the bilateral cells b7.11 and b7.11*. Color bar shows the lineage distance between the bilateral symmetrical cells. Black region represents snapshots with no matches between bilateral symmetrical cells. (f) Cell lineage of the bilateral cells a7.5 and b8.6. Color bar shows the compactness property. The property highlights that the delay of division between A7.5* and A7.5 is due to an under-segmented error of A7.5*. B8.6 and B8.6* have expected behavior. (g) Result of the Fuse plugin applied on an over-segmented cell. (h) Top line: Several snapshots of B7.7* cell under-segmented (between time point 31 and 33) from the original segmented embryo. Bottom: result of the Propa plugin applied backward from time t=34 where both cells are well separated. (i) Example of the result of the Wata plugin from a manual added seed (red dot) in the empty space. (j) Result of the Copy-Paste plugin from the selected cell (in gray) on the left side of the embryo to the right side (the new cell appears with a mesh shader in blue). Colormap represents cell volume in µm3. (k) Comparison of the lifetime of bilateral symmetrical cells. The X-axis shows the number of in-time points separating the division of bilateral cell pairs. The Y-axis corresponds to the number of cells (in log).
Figure 7—figure supplement 1
Cell lineage following curation of a Phallusia mammillata embryo.
(a) Cell lineage of the cell b7.1 after curation. Projection of the volume property as in Figure 7a. (b) Cell Lineage of the cell b8.21 after curation. Colormap represents the cell volume in µm3 as in Figure 7d. (c) Cell lineage of the bilateral cells b7.11 and b7.11* after curation. Color bar shows the lineage distance between the bilateral symmetrical cells as in Figure 7e. (d) Cell lineage of the bilateral cells 7.5 after curation. Color bar shows the compactness property as in Figure 7f.
Figure 8
Representation of the MorphoNet Plugins organized by the corresponding functionalities according to segmentation issues.
Color code represents the plugin family. Description of each plugin functionalities is accessible on the help web page: https://morphonet.org/help_curation#plugin_list.
Videos
Video 1
Tribolium castaneum.
The movie shows how to train a Cellpose model using a curated sub-part of a dataset, and how to fine-tune models with successive training on specific nuclei using image properties.
Video 2
Patiria miniata.
The movie shows how to create a dataset, how to use an already existing custom Cellpose model and how to easily correct usual Cellpose segmentation errors.
Video 3
Arabidopsis thaliana.
The movie shows how to train a Cellpose model on several time steps of a 3D+t dataset, and how to use it to predict under-segmentations on a specific part of a dataset.
Video 4
Caenorhabditis elegans.
The movie shows how to use the lineage and image properties to easily detect segmentation issues, and how to fix them in batches for a fast curation of dense datasets.
Video 5
Phallusia mammillata.
This movie shows how to identify and fix several issues on a segmented dataset, using the lineage viewer and a large array of plugins. It shows how to fix large curation errors using a couple of actions only.
Additional files
-
MDAR checklist
- https://cdn.elifesciences.org/articles/106227/elife-106227-mdarchecklist1-v1.docx
-
Supplementary file 1
Summary of the benchmarking datasets and device performance evaluation for the MorphoNet standalone application.
- https://cdn.elifesciences.org/articles/106227/elife-106227-supp1-v1.docx
-
Supplementary file 2
Overview of the MorphoNet documentation hub, organizing help resources by user profile and intended use cases.
- https://cdn.elifesciences.org/articles/106227/elife-106227-supp2-v1.docx
Download links
A two-part list of links to download the article, or parts of the article, in various formats.
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
MorphoNet 2.0: An innovative approach for qualitative assessment and segmentation curation of large-scale 3D time-lapse imaging datasets
eLife 14:RP106227.
https://doi.org/10.7554/eLife.106227.3
