CEM500K, a large-scale heterogeneous unlabeled cellular electron microscopy image dataset for deep learning
- Center for Molecular Microscopy, Center for Cancer Research, National Cancer Institute, National Institutes of Health, United States
- Cancer Research Technology Program, Frederick National Laboratory for Cancer Research, United States
Figures
Figure 1
Preparation of a deep learning appropriate 2D EM image dataset rich with relevant and unique features.
(a) Percent distribution of collated experiments grouped by imaging technique: TEM, transmission electron microscopy; SEM, scanning electron microscopy. (b) Distribution of imaging plane pixel …
-
Figure 1—source data 1
Details of imaging technique, organism, tissue type and imaging plane pixel spacing in collated imaging experiments.
- https://cdn.elifesciences.org/articles/65894/elife-65894-fig1-data1-v1.xlsx
Figure 2
CEM500K pre-training improves the transferability of learned features.
(a) Example images and colored label maps from each of the six publicly available benchmark datasets: clockwise from top left: Kasthuri++, UroCell, CREMI Synaptic Clefts, Guay, Perez, and Lucchi++. …
-
Figure 2—source data 1
IoU scores achieved with different datasets used for pre-training.
- https://cdn.elifesciences.org/articles/65894/elife-65894-fig2-data1-v1.xlsx
Figure 3
Features learned from CEM500K pre-training are more robust to image transformations and encode for semantically meaningful objects with greater selectivity.
(a) Mean firing rates calculated between feature vectors of images distorted by (i) rotation, (ii) Gaussian blur, (iii) Gaussian noise, (iv) brightness v. contrast, (vi) scale. Dashed black lines …
Figure 4
Models pre-trained on CEM500K yield superior segmentation quality and training speed on all segmentation benchmarks.
(a) Plot of percent difference in segmentation performance between pre-trained models and a randomly initialized model. (b) Example segmentations on the UroCell benchmark in 3D (top) and 2D …
-
Figure 4—source data 1
IoU scores for different pre-training protocols.
- https://cdn.elifesciences.org/articles/65894/elife-65894-fig4-data1-v1.xlsx
-
Figure 4—source data 2
IoU scores for different training iterations by pre-training protocol .
- https://cdn.elifesciences.org/articles/65894/elife-65894-fig4-data2-v1.xlsx
Appendix 1—figure 1
Deduplication and image filtering.
(a) Breakdown of fractions (top) and representative examples (bottom) of patches labeled ‘uninformative’ by a trained deep learning (DL) model based on defect (as determined by a human annotator). (b…
Appendix 1—figure 2
Randomly selected images from CEMraw, CEMdedup, and CEM500K.
Appendix 1—figure 3
Schematics of the MoCoV2 algorithm and UNet-ResNet50 model architecture.
(a) Shows a single step in the MoCoV2 algorithm. A batch of images is copied; images in each copy of the batch are independently and randomly transformed and then shuffled into a random order (the …
Appendix 1—figure 4
Randomly selected images from the Bloss et al., 2018 pre-training dataset.
Appendix 1—figure 5
Visual comparison of results on the UroCell benchmark.
The ground truth and Authors’ Best Results are taken from the original UroCell publication (Žerovnik Mekuč et al., 2020). The results from the CEM500K-moco pre-trained model have been colorized to …
Appendix 1—figure 6
Images from source electron microscopy (EM) volumes are unequally represented in the subsets of CEM.
The line at 45° shows the expected curve for perfect equality between all source volumes (i.e. each volume would contribute the same number of images to CEMraw, CEMdedup, or CEM500K). Gini …
Appendix 1—figure 7
Plot showing the percent of random crops from an image that will be 100% uninformative based on the percent of the image that is informative.
Tables
Table 1
Comparison of segmentation Intersection-over-Union (IoU) results for benchmark datasets from models randomly initialized and pre-trained with MoCoV2 on the Bloss dataset, and CEMraw, CEMdedup, and CEM500K.
* denotes benchmarks that exclusively contain electron microscopy (EM) images from mouse brain tissue. The best result for each benchmark is highlighted in bold and underlined.
| Benchmark | Random Init. (No Pre-training) | Bloss et al., 2018 | CEMraw | CEMdedup | CEM500K |
|---|---|---|---|---|---|
| All Mitochondria | 0.306 | 0.694 | 0.719 | 0.722 | 0.745 |
| CREMI Synaptic Clefts | 0.000 | 0.242 | 0.254 | 0.259 | 0.265 |
| Guay | 0.349 | 0.380 | 0.372 | 0.391 | 0.404 |
| *Kasthuri++ | 0.855 | 0.907 | 0.913 | 0.913 | 0.915 |
| *Lucchi++ | 0.788 | 0.899 | 0.880 | 0.890 | 0.894 |
| *Perez | 0.547 | 0.874 | 0.854 | 0.866 | 0.869 |
| UroCell | 0.208 | 0.638 | 0.652 | 0.699 | 0.729 |
| *Average Mouse Brain | 0.730 | 0.893 | 0.883 | 0.890 | 0.893 |
| Average Other | 0.216 | 0.489 | 0.499 | 0.518 | 0.536 |
Table 2
Comparison of segmentation IoU scores for different weight initialization methods versus the best results on each benchmark as reported in the publication presenting the segmentation task.
All IoU scores are the average of five independent runs. References listed after the benchmark names indicate the sources for Reported IoU scores.
| Benchmark | Training Iterations | Random Init. | IN-super | IN-moco | CEM500K-moco | Reported |
|---|---|---|---|---|---|---|
| All Mitochondria | 10000 | 0.587 | 0.653 | 0.653 | 0.770 | – |
| CREMI Synaptic Clefts | 5000 | 0.000 | 0.196 | 0.226 | 0.254 | – |
| Guay (Guay et al., 2020) | 1000 | 0.308 | 0.275 | 0.300 | 0.429 | 0.417 |
| Kasthuri++ (Casser et al., 2018) | 10000 | 0.905 | 0.908 | 0.911 | 0.915 | 0.845 |
| Lucchi++ (Casser et al., 2018) | 10000 | 0.894 | 0.865 | 0.892 | 0.895 | 0.888 |
| Perez (Perez et al., 2014) | 2500 | 0.672 | 0.886 | 0.883 | 0.901 | 0.821 |
| Lysosomes | – | 0.842 | 0.838 | 0.816 | 0.849 | 0.726 |
| Mitochondria | – | 0.130 | 0.860 | 0.866 | 0.884 | 0.780 |
| Nuclei | – | 0.984 | 0.987 | 0.986 | 0.988 | 0.942 |
| Nucleoli | – | 0.731 | 0.859 | 0.865 | 0.885 | 0.835 |
| UroCell | 2500 | 0.424 | 0.584 | 0.618 | 0.734 | – |
Additional files
-
Source data 1
Details of image datasets acquired from external sources.
- https://cdn.elifesciences.org/articles/65894/elife-65894-data1-v1.xlsx
-
Source data 2
Zipped folder containing .xl files for Figure 1, 2 and 4 source data.
- https://cdn.elifesciences.org/articles/65894/elife-65894-data2-v1.zip
-
Supplementary file 1
Details of benchmarks used in this paper.
- https://cdn.elifesciences.org/articles/65894/elife-65894-supp1-v1.docx
-
Supplementary file 2
IoU scores for pre-training with CEM500K after removing benchmark data from pre-training dataset .
- https://cdn.elifesciences.org/articles/65894/elife-65894-supp2-v1.docx
-
Supplementary file 3
IoU scores on Guay benchmark using different hyperparameter choices.
- https://cdn.elifesciences.org/articles/65894/elife-65894-supp3-v1.docx
-
Transparent reporting form
- https://cdn.elifesciences.org/articles/65894/elife-65894-transrepform-v1.docx
