Machine learning of dissection photographs and surface scanning for quantitative 3D neuropathology
- Martinos Center for Biomedical Imaging, MGH and Harvard Medical School, United States
- Centre for Medical Image Computing, University College London, United Kingdom
- Computer Science and Artificial Intelligence Laboratory, MIT, United States
- Biomedical Imaging Group, Universitat Politècnica de Catalunya, Spain
- BioRepository and Integrated Neuropathology (BRaIN) Laboratory and Precision Neuropathology Core, UW School of Medicine, United States
- Massachusetts Alzheimer Disease Research Center, MGH and Harvard Medical School, United States
- Neuroscience and Biomedical Engineering, Aalto University, Finland
- Department of Neurological Surgery, UW School of Medicine, United States
Figures
Figure 1
Examples of inputs and outputs from the MADRC dataset.
(a) Three-dimensional (3D) surface scan of left human hemisphere, acquired prior to dissection. (b) Routine dissection photography of coronal slabs, after pixel calibration, with digital rulers …
Figure 2
Qualitative comparison of SAMSEG vs Photo-SynthSeg: coronal (top) and sagittal (bottom) views of the reconstruction and automated segmentation of a sample whole brain from the UW-ADRC dataset.
Note that Photo-SynthSeg supports subdivision of the cortex with tools of the SynthSeg pipeline.
Figure 3
Dice scores of automated vs manual segmentations on select slices.
Box plots are shown for SAMSEG, Photo-SynthSeg, and two ablations: use of probabilistic atlas and targeted simulation with 4 mm slice spacing. Dice is computed in two-dimensional (2D), using manual …
Figure 4
Reconstruction error (in mm) in synthetically sliced HCP data.
The figure shows box plots for the mean reconstruction error as a function of spacing and thickness jitter. A jitter of means that the nth slice is randomly extracted from the interval (rather …
Figure 5
Steps of proposed processing pipeline.
(a) Dissection photograph with brain slices on black board with fiducials. (b) Scale-invariant feature transform (SIFT) features for fiducial detection. (c) Photograph from (a) corrected for pixel …
Figure 6
Intermediate steps in the generative process.
(a) Randomly sampled input label map from the training set. (b) Spatially augmented input label map; imperfect 3D reconstruction is simulated with a deformation jitter across the coronal plane. (c) …
Appendix 1—figure 1
Simulation and reconstruction of synthetic data.
Top row: skull stripped T1 scan and (randomly translated and rotated) binary mask of the cerebrum, in yellow. Second row: original T2 scan. Third row: randomly sliced and linearly deformed T2 …
Appendix 1—figure 2
Reconstruction with surface scan vs probabilistic atlas.
(a) Initialization, with contour of 3D surface scan superimposed. (b) Reconstruction with 3D surface scan. (c) Reconstruction with probabilistic atlas (overlaid as heat map with transparency); the …
Appendix 1—figure 3
Example of mid-coronal slice selected for manual segmentation and computation of Dice scores.
Compared with the FreeSurFer protocol, we merge the ventral diencephalon (which has almost no visible contrast in the photographs) with the cerebral white matter in our manual delineations. We also …
Videos
Video 1
Overview of the proposed method.
Tables
Table 1
Area under the receiver operating characteristic curve (AUROC) and p-value of a non-parametric Wilcoxon rank sum test comparing the volumes of brain regions for Alzheimer’s cases vs controls.
The volumes were corrected by age and sex using a general linear model. We note that the AUROC is bounded between 0 and 1 (0.5 is chance) and is the non-parametric equivalent of the effect size …
| Region | Wh matter | Cortex | Vent | Thal | Caud | Putamen | Pallidum | Hippoc | Amyg |
|---|---|---|---|---|---|---|---|---|---|
| AUROC | 0.45 | 0.52 | 0.73 | 0.48 | 0.65 | 0.64 | 0.77 | 0.75 | 0.77 |
| p-value | 0.666 | 0.418 | 0.016 | 0.596 | 0.086 | 0.092 | 0.005 | 0.009 | 0.007 |
Table 2
Correlations of volumes of brains regions estimated by SAMSEG and Photo-SynthSeg from the photographs against the ground truth values derived from the magnetic resonance imaging (MRI).
The p-values are for Steiger tests comparing the correlations achieved by the two methods (accounting for the common sample).
| Mask from MRI as reference | Probabilistic atlas as reference | |||||
|---|---|---|---|---|---|---|
| SAMSEG | Photo-SynthSeg | p-value | SAMSEG | Photo-SynthSeg | p-value | |
| White matter | 0.935 | 0.981 | 0.0011 | 0.886 | 0.935 | 0.0117 |
| Cortex | 0.930 | 0.979 | 0.0001 | 0.889 | 0.920 | 0.0366 |
| Ventricle | 0.968 | 0.988 | 0.0004 | 0.980 | 0.993 | 0.0006 |
| Thalamus | 0.812 | 0.824 | 0.4350 | 0.812 | 0.824 | 0.4252 |
| Caudate | 0.719 | 0.779 | 0.2525 | 0.733 | 0.792 | 0.2062 |
| Putamen | 0.904 | 0.779 | 0.9923 | 0.872 | 0.792 | 0.9598 |
| Pallidum | 0.727 | 0.694 | 0.6171 | 0.676 | 0.658 | 0.5698 |
| Hippocampus | 0.830 | 0.757 | 0.8873 | 0.764 | 0.776 | 0.4293 |
| Amygdala | 0.598 | 0.703 | 0.1663 | 0.576 | 0.763 | 0.0221 |
