A deep learning pipeline for mapping in situ network-level neurovascular coupling in multi-photon fluorescence microscopy
- Department of Medical Biophysics, University of Toronto, Canada
- Physical Sciences, Sunnybrook Research Institute, Canada
- Biological Sciences, Sunnybrook Research Institute, Canada
- Department of Laboratory Medicine and Pathobiology, University of Toronto, Canada
- Hurvitz Brain Sciences, Sunnybrook Research Institute, Canada
Figures
Figure 1
Photostimulation setup.
The excitation and stimulation light pass through a FV30-NDM690 dichroic mirror with two notch filters, at 458 nm and 552 nm, to excite TexasRed, EYFP, and ChR2 within the mouse. The emitted light passes through the objective, is reflected off the FV30-NDM690 dichroic mirror, and passes through a 650 nm barrier filter before reaching a 570 nm long pass filter (LPF) separating emitted light from EYFP and TexasRed, which respectively pass through 495–540 nm and 575–630 nm barrier filters to be collected via GaAsP detectors.
Figure 2
Computational analysis pipeline.
(A) The stacks of 2PFM slices were registered using ANTS rigid registration and aligned to the reference time point. (B) Images were upsampled using bicubic interpolation to an isotropic resolution of 0.99 x 0.99 × 0.99 μm. (C) An ensemble of UNETR deep learning models with dropout generated segmentation masks at each time point, producing probability maps. (D) The mean and standard deviation of the probability of each pixel being vasculature were computed and used to create binary vascular segmentation masks. (E) The union over the vascular segmentation masks for all time points was computed, and background pixel clusters within vessel masks were removed. (F) The vascular segmentation mask was thinned down to centerlines and rendered as a graph, where edges were vessel segments connecting branch points (nodes). This skeleton was overlaid on the vasculature channel from which the neuron channel was subtracted. (G) The plane orthogonal to the tangent to the vessel’s travel direction was computed every micrometer along the centerline. (H, I) 1D signal intensity profiles at each centerline vertex were computed in the orthogonal plane every 10°. (J) The boundary for each profile was placed at the minimum of the signal gradient for that signal intensity profile. (K) The raw intensity image with the detected boundary points, where outlier boundary points (in green) were defined as points over 2 standard deviations from the mean were excluded. (L) Visualization of the changes in vertex-wise radii on a sample vascular network.
Figure 3
Model performance metrics.
The Dice, precision, recall, mean surface distance, and HD95 distance for the vascular (A) and neuron (B) channels. Each model was evaluated on the same test dataset composed of nine images (250 x 507 × 507 μm each) from six mice. A Wilcoxon signed-rank test was used to compare the model’s performance on each performance metric for images from the test dataset. * p<0.05, ** p<0.005, and *** p<0.0005. p-values were not adjusted.
Figure 4
Visual model comparison.
(A) Raw images of the vascular channel with the neuron channel subtracted to facilitate vessel visualization. The first and last stacks in each row span from the cortical surface to 250 μm below the surface, while the middle stack spans from 250 μm below the surface to 500 μm below the surface. All images were from the test dataset, which was unseen during model training. (B) Ground truth segmentation masks for the vasculature were generated by a rater who utilized ilastik-assisted manual segmentation. (C) Ilastik predictions generated via a random forest model. (D) Binary segmentation masks generated by an ensemble of 3D UNet models. (E) Binary segmentation masks generated by an ensemble of 3D UNETR models.
Figure 5
Estimation of simulated radii changes.
(A) An image in the plane orthogonal to the local tangent to a capillary with the detected boundary (in blue) and with the estimated radius of 2.28 μm. On the right, this image was resized (upsampling, via bicubic interpolation, by 1.10 times) to simulate dilation. (B) The plot shows correspondence between the estimated radius following scaling and the simulated level of scaling. (C) An image in the plane orthogonal to the local tangent of a capillary with the detected boundary (in blue) and with the estimated radius of 3.65 μm. On the right, Gaussian noise with a sigma of 205.36 SU was added to the image. (D) The estimated % change in the vessel’s radius after the addition of varying levels of Gaussian noise, demonstrating the robustness of the radius estimated to noise.
Figure 6
Vascular graph examples.
(A) Baseline variability in vessel diameter estimated by the standard deviation of each vessel’s mean radius across baseline time frames. (B) Mean change in the vessel radius induced by optogenetic stimulation. (C) Mean change in the vertexwise radius, allowing the visualization of heterogeneity of radius changes within each vessel. (D) Distance from each vertex to the closest pyramidal neuron. Each row corresponds to the vascular graph of a different mouse.
Figure 7
Vertex-wise radii along vessel lengths of a sample artery, capillary, and venule at baseline vs. post-stimulation.
(A) MIP of an artery, vein, and capillary segments before (left) and after (right) optogenetic stimulation with 458 nm light at 1.1 mW/mm2. The artery and capillary dilated by 1.33±0.86 μm and 0.42±0.39 μm, respectively (for both p<1e-4, Mann-Whitney U test), whereas there was no significant change in the venular caliber upon photostimulation (p=0.22, Mann-Whitney U test). (B) Estimates of the vertex-wise radius obtained along each of the three vessels’ centrelines, before and after stimulation. (C) Vertex-wise radii changes in response to optogenetic stimulation. (D). The vertex-wise distance from the vascular surface to the closest YFP-expressing neuron.
Figure 8
Optogenetic activation-induced changes in vessel-wise microvascular radii.
Capillary responses included both dilatations, shown in (A), and constrictions, shown in (B), with changes in the magnitude of the capillary response with increased photostimulation power. * p<0.05, ** p<0.005, and *** p<0.0005. p-values were not adjusted. (C) Probability density function of constrictions and dilations for the 4.3 mW/mm2 photostimulation. (D) Changes to capillary radii are displayed in relation to the closest pyramidal neurons. The proportion of vessels constricting increased with the higher intensity of blue light stimulation, and constrictions tended to occur further away from pyramidal neurons than did dilations. (E) Mean cortical depth of responding capillaries showed a tendency for dilators to be closer to the surface and for constrictors to be deeper in the tissue.
Figure 9
Microvascular network coordination following optogenetic stimulation.
(A) Graph representation of a vascular network of 425 vascular segments from a single image stack. Vessel segments are depicted as nodes of the graph; vascular segments that are joined at junctions are connected by edges. Nodes are colored by the change in the mean vessel-wise radius following photostimulation with 458 nm light at 4.3 mW/mm2. (B) Assortativity of photostimulation-induced changes in mean capillary radius increased with increasing photostimulation power. (C) Photostimulation-induced changes in the efficiency of the capillary network. The capillary network efficiency changed by a median –0.16 PΩ–1 (IQR: –0.39–0.10 PΩ–1) in response to green light; –0.14 PΩ–1 (IQR: –0.55–0.27 PΩ–1) in response to lower intensity blue light; and 0.22 PΩ–1 (IQR = –0.43;1.47 PΩ–1) in response to higher intensity blue light. There was a significant increase (p=0.03) in the capillary network efficiency post 458 nm light at 4.3 mW/mm2, when compared to that following the control green illumination. The measurements came from 72 paired acquisitions of 32 image stacks acquired in 17 mice (9 M/8 F). * p<0.05, ** p<0.005, and *** p<0.0005. p-values were not adjusted.
Appendix 1—figure 1
Attrition.
Flow chart of animal numbers at each step of the experiment.
Appendix 1—figure 2
High temporal resolution (1.9–3.2 s per frame) time courses of microvascular radii.
Loess smoothed vascular radius estimates over time in significantly responding vessels as determined by an F-test comparing the variance in vascular radius before and after stimulation. Light blue corresponds to trials with 1.1 mW/mm2 photostimulation at 458 nm; dark blue, 4.3 mW/mm2 at 458 nm; green, 4.3 mW/mm2 photostimulation at 552 nm; and red, 2 mA, 3 Hz, 10 s on, 10 s off stimulation of the contralateral forepaw. Images were acquired for 5 min prior to stimulation and for 5 min following the end of the stimulation. Optogenetic stimuli were parametrized as described in Imaging. The forepaw stimulation started at the vertical blue line time and lasted for the duration of the scan. (A) Sample murine radii traces in vessels whose radius was significantly altered following photostimulations. This volume was acquired from a depth of 156–94 μm below the cortical surface with each volume acquisition lasting 2.98 s. (B) Another sample murine radii traces in vessels whose radius was significantly altered following photostimulations. This volume was acquired from a depth of 75–5 μm below the cortical surface with each volume acquisition lasting 3.17 s. (C, D) Traces of responding vessel radii from the third mouse. (C) This volume was acquired from a depth of 305–341 μm below the cortical surface with each volume acquisition lasting 1.93 s. (D) This volume was acquired from a depth of 276–235 μm below the cortical surface with each volume acquisition lasting 2.16 s.
Appendix 1—figure 3
Bead radius estimation sensitivity to noise.
(A) Sensitivity of vessel-wise radius estimate on the number of spokes used to estimate the radius. The radius estimate converges after 20 spokes have been used for estimation. Using 36 spokes initially, the vesselwise mean radius estimation was within 0.24±0.62% of the mean of radius estimates using 40–60 spokes. (B) The centerline was jittered in the perpendicular plane at each point along the line and then mean radius was estimated in 71 larger vessels (mean radius > 5 μm). The percent difference in the estimated radius at our selected vessel centerpoints vs. the jittered centerpoints is plotted. The percent difference in the mean radius estimation was 0.64 ± 3.44% (eg. 0.032±0.17 μm for a vessel with a radius of 5 μm) with 2.45±0.30 μm centerline jittering. (In contrast, photostimulation was estimated to elicit an average 25.4 ± 18.1% change in the magnitude of the radius of larger vessels, that is those with the baseline radius > 5 μm.).
Appendix 1—figure 4
Sample 2D slices of segmentation results.
(A) Raw slices of the vascular channel with the neuron channel subtracted to facilitate vessel visualization. The first slice was 29 μm below the cortical surface; the second slice 200 μm; and the third image 300 μm. Each slice is from a separate mouse. All images were taken from the test dataset, unseen during model training. (B) Ground truth segmentation masks for the vasculature were generated by a rater who utilized ilastik assisted manual segmentation. (C) Ilastik predictions generated via a random forest model. (D) Binary segmentation masks generated by an ensemble of 3D UNet models. (E) Binary segmentation masks generated by an ensemble of 3D UNETR models.
Appendix 1—figure 5
Vascular network characteristics.
(A) Probability density of the extracted mean radii of vascular segments. (B) Probability density of the lengths of extracted vessel segments between branch points or terminal ends. (C) Probability density of the mean vessel segment depths. (D) Probability density of the depths of vessel branch points. Terminal nodes were excluded from this probability density. 12,555 vessels and 6421 vascular junctions from 17 mice (9M/8F) were used to estimate these PDFs.
Appendix 1—figure 6
Large vessels' radius changes following optogenetic stimulation.
All vascular responses in large vessels (radius > 5 μm) separated into dilators (A) and constrictors (B) showing an increase in the magnitude of the vascular response as the photostimulation power increases. (C) Probability density function of diameter changes across all large vessels. (D) Radius changes to vascular radii in relation to the closest pyramidal neurons (within 80 μm). (E) Mean depth of the responding large vessels.
Appendix 1—figure 7
Registered images of the cortical microvasculature before and after optogenetic stimulation for five scan pairs over three different stimulation conditions.
The estimated radii changes along vessel segments are shown in the third row.
Appendix 1—figure 8
Maximum intensity projections of sample capillary constrictions at repeated time points following optogenetic stimulation.
Baseline (pre-stimulation) image is shown on the left and the post-stimulation image, on the right, with the estimated radius changes listed to the left.
Appendix 1—figure 9
THY1-ChR2 photostimulation-induced changes in vessel-wise microvascular radii compared to C57BL/6J.
(A) Vessel radius in responding vessels of the Thy1-ChR2 mice described in the manuscript vs. (B) Four wild-type C57BL/6J mice. Response to photostimulation was defined as a radius change above twice the standard deviation in the radius across baseline frames. 552 nm light was applied at 4.3 mW/mm2, while 458 nm light was applied at 1.1 mW/mm2 and 4.3 mW/mm2. In C57BL/6J mice (B), the radii distributions following either blue light photostimulation were not statistically distinguishable from that resulting from green photostimulation (‘response’ to control condition) using a Wilcoxon test.
Appendix 1—figure 10
Imaris segmentation examples.
(A) Raw slices of the vascular channel with the neuronal channel subtracted to facilitate vessel visualization. (B) Corresponding slices of the segmentation mask generated in Imaris 9.2.1 manually by a rater overlaid onto the raw data. (C) Corresponding segmentation masks generated by our UNETR segmentation model overlaid onto the raw data.
Appendix 1—figure 11
Results of VesselVio graph generation on our segmentation masks.
Graphs were generated on the images shown in column 4 of Appendix 1—figure 7. (A) VesselVio vascular graph extraction on a volume before blue light photostimulation produced 546 vessel segments. (B) VesselVio vascular graph extraction on the same imaging volume after blue light stimulation produced 642 vessel segments. (C) NOVAS3D generated graph of the vasculature with direct tracking of morphological changes.
Appendix 1—figure 12
Examples of the predictions of the deep learning model applied on out-of-distribution data from a different mouse strain (C57BL/6J), a different species (Fischer rat), and a different microscope (light sheet fluorescence microscope, Miltenyi UltraMicroscope Blaze).
Appendix 1—figure 13
Examples of 2D slices of the predictions of the NOVAS3D deep learning model applied on out-of-distribution data from a different mouse strain (C57BL/6), a different species (Fischer rat), and a different microscope (light sheet fluorescence microscope, Miltenyi UltraMicroscope Blaze).
Appendix 1—figure 14
Optogenetic activation-induced changes in vessel-wise microvascular radii with responders defined as vessels changing their radius by more than 10%.
Capillary responses included both dilatations, shown in (A), and constrictions, shown in (B) with potentiation of the capillary response with increased photostimulation power. * p<0.05, ** p<0.005, and *** p<0.0005. p-values were not adjusted. (C) Changes to capillary radii are displayed in relation to the closest labeled neuron. The proportion of vessels constricting increased with the higher intensity of blue light stimulation, and constrictions tended to occur further away from labeled neurons than did dilations. (D) Mean cortical depth of responding capillaries showed a tendency for dilators to be closer to the surface and for constrictors to be deeper in the tissue.
Tables
Table 1
Bead diameter estimates.
| Number of orthogonal planes | Number of spokes per plane | Mean diameter estimate (μm) |
|---|---|---|
| 1 | 3 | 7.54±0.68 |
| 2 | 4 | 7.44±0.51 |
| 4 | 12 | 7.34±0.38 |
| 10 | 36 | 7.34±0.32 |
Table 2
S1FL vascular network morphological properties.
| Metric | Mean ±SD | N=17 Mice (9 M/8 F) |
|---|---|---|
| Number of individual vessels per volume | 368±239 | 32 FOVs |
| Vessel density | 5705±3705 mm–3 | 32 FOVs |
| Number of vascular junctions per volume | 207±154 | 32 FOVs |
| Vascular junction density | 3215±2385 mm–3 | 32 FOVs |
| Number of terminal vessels per volume | 128±52 | 32 FOVs |
| Individual vessel length | 70.7±61.1 μm | 12555 vessel segments |
| Cumulative vessel length density | 0.40±0.22 m/mm3 | 32 FOVs |
| Baseline vessel radius | 2.19±1.66 μm Range: 0.66–15.88 μm | 12555 vessel segments |
| Baseline intra-vessel radius standard deviation | 0.53±0.47 μm | 12555 vessel segments |
| Baseline vascular volume density | 0.010±0.007 mm3/mm3 | 32 FOVs |
| Number of pyramidal neurons per volume | 313±202 neuronal somas | 32 FOVs |
| Pyramidal neuron density | 4872±3145 neuronal somas/mm3 | 32 FOVs |
Table 3
Details of responder (ΔR>2 * σRbaseline) vessels.
| Stimulation condition | Total number of vessel estimates | Average minimum distance to the closest neuron (μm) | Number of dilators | Minimum distance from dilators to the closest neuron (μm) | Average vessel depth of dilators (μm) | Diameter change (μm) | Number of constrictors | Minimum distance from constrictors to the closest neuron (μm) | Average vessel depth of constrictors (μm) | Diameter change (μm) |
|---|---|---|---|---|---|---|---|---|---|---|
| All vessels | Dilators | Constrictors | ||||||||
| Capillaries | ||||||||||
| 552 nm 4.3 | 5036 | 21.2±16.2 | 144 (2.9%) | 25.5±19.0 | 186±114 | 0.58±0.92 | 49 (1.0%) | 26.5±19.5 | 247±122 | –0.37±0.30 |
| 458 nm 1.1 | 10136 | 18.7±14.5 | 317 (3.1%) | 16.8±13.5 | 196±138 | 0.90±0.93 | 255 (2.5%) | 22.7±16.3 | 254±126 | –1.39±1.51 |
| 458 nm 4.3 | 12537 | 20.6±15.4 | 575 (4.6%) | 16.1±14.3 | 237±146 | 0.90±0.77 | 874 (7.0%) | 21.9±14.6 | 274±103 | –1.19±1.13 |
| Large vessels | ||||||||||
| 552 nm 4.3 | 225 | 43.1±19.5 | 1 (0.4%) | 75.4 | 82 | 13.98 | 0 (0%) | NA | NA | NA |
| 458 nm 1.1 | 545 | 38.4±19.5 | 1 (0.2%) | 26.1 | 402 | 1.97 | 1 (0.2%) | 19.0 | 179 | –3.65 |
| 458 nm 4.3 | 569 | 38.4±20.1 | 2 (0.35%) | 53.1±6.3 | 84±34 | 2.47±2.93 | 6 (1.1%) | 43.1±16.3 | 290±125 | –6.07±2.45 |
Additional files
-
Supplementary file 1
Physiological parameters of the mice during imaging.
Includes heart rate, breath rate, and oxygen saturation levels recorded via pulse oximetry.
- https://cdn.elifesciences.org/articles/95525/elife-95525-supp1-v1.docx
-
Supplementary file 2
Deep learning data augmentation parameters.
Detailed list of spatial and intensity transformations used for training the segmentation models.
- https://cdn.elifesciences.org/articles/95525/elife-95525-supp2-v1.docx
-
Supplementary file 3
Hyperparameter optimization grid search results.
Comparison of model performance across different loss functions, learning rates, and dropout settings.
- https://cdn.elifesciences.org/articles/95525/elife-95525-supp3-v1.docx
-
Supplementary file 4
Segmentation model performance metrics.
Quantitative evaluation of UNETR, U-Net, and ilastik models based on overlap and surface distance metrics.
- https://cdn.elifesciences.org/articles/95525/elife-95525-supp4-v1.docx
-
MDAR checklist
- https://cdn.elifesciences.org/articles/95525/elife-95525-mdarchecklist1-v1.pdf
Download links
A two-part list of links to download the article, or parts of the article, in various formats.
Downloads (link to download the article as PDF)
Open citations (links to open the citations from this article in various online reference manager services)
Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)
A deep learning pipeline for mapping in situ network-level neurovascular coupling in multi-photon fluorescence microscopy
eLife 13:RP95525.
https://doi.org/10.7554/eLife.95525.5
