A Deep Learning Pipeline for Mapping in situ Network-level Neurovascular Coupling in Multi-photon Fluorescence Microscopy

Reviewer #1 (Public Review):

Summary:

In this manuscript, the authors describe a new pipeline to measure changes in vasculature diameter upon opt-genetic stimulation of neurons.

The work is interesting and the topic is quite relevant to better understand the hemodynamic response on the graph/network level.

Strengths:

The manuscript provides a pipeline that allows for the detection of changes in the vessel diameter as well as simultaneously allowing for the location of the neurons driven by stimulation.

The resulting data could provide interesting insights into the graph-level mechanisms of regulating activity-dependent blood flow.

The interesting findings include that vessel radius changes depend on depth from the cortical surface and that dilations on average happen closer to the activated neurons.

Weaknesses:

The utility of a pipeline depends on the generalization properties.

While the proposed pipeline seems to work for the data the authors acquired, it is unclear if this pipeline will actually generalize to novel data sets possibly recorded by a different microscope (e.g. different brand), or different imagining conditions (e.g. illumination or different imagining artifacts) or even to different brain regions or animal species, etc.

The authors provide a 'black-box' approach that might work well for their particular data sets and image acquisition settings but it is left unclear how this pipeline is actually widely applicable to other conditions as such data is not provided.

In my experience, without well-defined image pre-processing steps and without training on a wide range of image conditions pipelines typically require significant retraining, which in turn requires generating sufficient amounts of training data, partly defying the purpose of the pipeline.

It is unclear from the manuscript, how well this pipeline will perform on novel data possibly recorded by a different lab or with a different microscope.

Analysis

Some of the chosen analysis results seem to not fully match the shown data, or the visualization of the data is hard to interpret in the current form. Additionally, some measures seem not fully adapted to the current situation (e.g. the efficiency measure does not consider possible sources or sinks). Thus, some additional analysis work might be required to account for this.

Reviewer #2 (Public Review):

Summary:

The authors develop a highly detailed pipeline to analyze hemodynamic signals from in vivo two-photon fluorescence microscopy. This includes motion correction, segmentation of the vascular network, diameter measurements across time, mapping neuronal position relative to the vascular network, and analyzing vascular network properties (interactions between different vascular segments). For the segmentation, the authors use a Convolution Neural Network to identify vessel (or neural) and background pixels and train it using ground truth images based on semi-automated mapping followed by human correction/annotation. Considerable processing was done on the segmented images to improve accuracy, extract vessel center lines, and compute frame-by-frame diameters. The model was tested with artificial diameter increases and Gaussian noise and proved robust to these manipulations.

Network-level properties include Assortativity - a measure of how similar a vessel's response is to nearby vessels - and Efficiency - the ease of flow through the network (essentially, the combined resistance of a path based on diameter and vessel length between two points).

Strengths:

This is a very powerful tool for cerebral vascular biologists as many of these tasks are labor intensive, prone to subjectivity, and often not performed due to the complexity of collecting and managing volumes of vascular signals. Modelling is not my specialty so I cannot speak too specifically, but the model appears to be well-designed and robust to perturbations. It has many clever features for processing the data.

The authors rightly point out that there is a real lack in the field of knowledge of vascular network activity at single-vessel resolution. Network anatomy has been studied, but hemodynamics are typically studied either with coarse resolution or in only one or a few vessels at a time. This pipeline has the potential to change that.

Weaknesses:

The authors apply their method to in vivo data. However, there are some weaknesses in the design that make it hard to accept many of the conclusions and even to see that the method could yield much useful data with this type of application. Primarily, the acquisition of a large volume of tissue is very slow. In order to obtain a network of vascular activity, large volumes are imaged with high resolution. However, the volumes are scanned once every 42 seconds following stimulation. Most vascular responses to neuronal activation have come and gone in 42 seconds so each vessel segment is only being sampled at a single time point in the vascular response. So all of the data on diameter changes are impossible to compare since some vessels are sampled during the initial phase of the vascular response, some during the decay, and many probably after it has already returned to baseline. The authors attempt to overcome this by alternating the direction of the scan (from surface to deep and vice versa). But this only provides two sample points along the vascular response curve and so the problem still remains.

A second problem is the use of optogenetic stimulation to activate the tissue. First, it has been shown that blue light itself can increase blood flow (Rungta et al 2017). The authors note the concern about temperature increases but that is not the same issue. The discussion mentions that non-transgenic mice were used to control for this with "data not shown". This is very important data given these earlier reports that have found such effects and so should be included. Secondly, there doesn't seem to be any monitoring of neural activity following the photo-stimulation. The authors repeatedly mention "activated" neurons and claim that vessel properties change based on distance from "activated" neurons. But I can't find anything to suggest that they know which neurons were active versus just labeled. Third, the stimulation laser is focused at a single depth plane. Since it is single-photon excitation, there is likely a large volume of activated neurons. But there is no way of knowing the spatial arrangement of neural activity and so again, including this as a factor in the analysis of vascular responses seems unjustified.

The study could also benefit from more clear illustration of the quality of the model's output. It is hard to tell from static images of 3-D volumes how accurate the vessel segmentation is. Perhaps some videos going through the volume with the masks overlaid would provide some clarity. Also, a comparison to commercial vessel segmentation programs would be useful in addition to benchmarking to the ground truth manual data.

Another useful metric for the model's success would be the reproducibility of the vessel responses. Seeing such a large number of vessels showing constrictions raises some flags and so showing that the model pulled out the same response from the same vessels across multiple repetitions would make such data easier to accept.

A number of findings are questionable, at least in part due to these design properties.

There are unrealistically large dilations and constrictions indicated. These are likely due to artifacts of the automated platform. Inspection of these results by eye would help understand what is going on.

In Figure 6, there doesn't seem to be much correlation between vessels with large baseline level changes and vessels with large stimulus-evoked changes. It would be expected that large arteries would have a lot of variability in both conditions and veins much less. There is also not much within-vessel consistency. For instance, the third row shows what looks like a surface vessel constricting to stimulation but a branch coming off of it dilating - this seems biologically unrealistic.

As mentioned, the large proportion of constricting capillaries is not something found in the literature. Do these happen at a certain time point following the stimulation? Did the same vessel segments show dilation at times and constriction at other times? In fact, the overall proportion of dilators and constrictors is not given. Are they spatially clustered? The assortativity result implies that there is some clustering, and the theory of blood stealing by active tissue from inactive tissue is cited. However, this theory would imply a region where virtually all vessels are dilating and another region away from the active tissue with constrictions. Was anything that dramatic seen?

As mentioned, the claims about distance to active neurons are not meaningful if there is no measure of which neurons were active and which weren't. But even still, the claim is overly strong as the average distance to the nearest neuron for dilators was ~17 microns and for constrictors it was ~22 microns - about a half a neuronal soma difference.

The distance to the nearest neuron likely will depend on depth as well - neurons are quite sparse superficially and very dense in layer 4. The capillary network varies much less (see Blinder et al 2016 Nature Neuroscience). So the distance of a neuron to the nearest capillary may not vary much with depth, but the distance from the capillary to the nearest neuron might vary quite a lot.
Why were nearly all vessels > 5um diameter not responding >2SD above baseline? Did they have highly variable baselines or small responses? Usually, bigger vessels respond strongly to local neural activity.

Author Response

We would like to thank the reviewers for providing constructive feedback on the manuscript. To address the weaknesses identified, we are performing additional experiments and generating additional data, to be added to the updated manuscript.

(1) The utility of a pipeline depends on the generalization properties.

While the proposed pipeline seems to work for the data the authors acquired, it is unclear if this pipeline will actually generalize to novel data sets possibly recorded by a different microscope (e.g. different brand), or different imagining conditions (e.g. illumination or different imagining artifacts) or even to different brain regions or animal species, etc.

The authors provide a 'black-box' approach that might work well for their particular data sets and image acquisition settings but it is left unclear how this pipeline is actually widely applicable to other conditions as such data is not provided.

In my experience, without well-defined image pre-processing steps and without training on a wide range of image conditions pipelines typically require significant retraining, which in turn requires generating sufficient amounts of training data, partly defying the purpose of the pipeline. It is unclear from the manuscript, how well this pipeline will perform on novel data possibly recorded by a different lab or with a different microscope.

To address generalizability, we are performing several validation experiments with data from different 1) channels, 2) species (rat), and 3) microscopes, to highlight the robustness of our deep learning (DL) segmentation model to out-of-distribution data with different characteristics and acquisition protocols. We first used our model to segment three images (507x507 x&y, 250-170 um z) from three C57BL/6 mice acquired on the same two-photon fluorescent microscope following the same imaging protocol. The vasculature was labelled with the Texas Red dextran, as in the current experiment. In place of the EYFP signal from pyramidal neurons (2nd channel), gaussian noise was generated with a mean and standard deviation identical to the acquired vascular channel. A second set of two images(507x507 x&y, 300-400 um z) from two Fischer rats with Alexa680-dextran label in the plasma; these rats were imaged on the same two-photon fluorescence microscope, but with galvano scanners (instead of resonant scanners). A second channel of random Gaussian noise was also added here. Finally, an image of vasculature from a ex-vivo cleared mouse brain (1665x1205x780 um) imaged on a light sheet fluorescence microscope (Miltenyi UltraMicroscope Blaze) was also segmented with our model. Lectin-DyLight 649 was used to label the vasculature in this cohort. The Dice Score, Precision, Recall, Hausdorff 95%, and Mean surface distance will be reported for all of these additional image segmentations, upon generation of ground truth images. Finally, examples of the generated segmentation masks are presented in Author response image 1 for visual comparison. Of final note, should the segmentation results on a new data set be unsatisfactory, the methods downstream from segmentation are still applicable and the model can be further fine-tuned on other out-of-distribution data.

Author response image 1.

Examples of the deep learning model output on out of distribution data from a different mouse strain, from a different species (Fischer rat), and on a different microscope using a different imaging modality.

(2) Some of the chosen analysis results seem to not fully match the shown data, or the visualization of the data is hard to interpret in the current form.

We are updating the visualizations to make them more accessible and we will ensure matching between tables and figures.

(3) Additionally, some measures seem not fully adapted to the current situation (e.g. the efficiency measure does not consider possible sources or sinks). Thus, some additional analysis work might be required to account for this.

Thank you for your comment. The efficiency metric was selected as it does not consider sources or sinks. We do agree that accounting for vessel subtypes in the analysis (thus classifying larger vessels as either supplying or draining) would be uniquely useful: notwithstanding, it is extremely laborious. We are therefore leveraging machine learning in a parallel project to afford vessel classification by subtype. The source/sink analysis is also confounded by the small field-of-view of in situ 2PFM. Future work will investigate network remodelling across the whole brain with ex-vivo light sheet fluorescence microscopy.

(4) The authors apply their method to in vivo data. However, there are some weaknesses in the design that make it hard to accept many of the conclusions and even to see that the method could yield much useful data with this type of application. Primarily, the acquisition of a large volume of tissue is very slow. In order to obtain a network of vascular activity, large volumes are imaged with high resolution. However, the volumes are scanned once every 42 seconds following stimulation. Most vascular responses to neuronal activation have come and gone in 42 seconds so each vessel segment is only being sampled at a single time point in the vascular response. So all of the data on diameter changes are impossible to compare since some vessels are sampled during the initial phase of the vascular response, some during the decay, and many probably after it has already returned to baseline. The authors attempt to overcome this by alternating the direction of the scan (from surface to deep and vice versa). But this only provides two sample points along the vascular response curve and so the problem still remains.

We thank the Reviewer for bringing up this important point.

Although vessels can show relatively rapid responses to perturbation, vascular responses to photostimulation of ChannelRhodopsin-2 in neighbouring neurons are typically long lasting: they do not come and go in 42 seconds. To demonstrate this point, we acquired higher temporal-resolution images of smaller volumes of tissue over 5 minutes preceding and following the 5-s photoactivation with the original parameters. Imaging protocol was different in that we utilized a piezoelectric motor, smaller field of view, and only 3x frame averaging, resulting in a temporal resolution of 1.57-2.63 seconds. This acquisition was repeated at 4 different cortical depths (325 um, 250 um, 150um, and 40 um) in a single mouse.The vascular radii were estimated using our presented pipeline. Raw data and LOESS fits are shown in Author response image 2 (below). Vessels shorter than 20 um in length were excluded from the analysis. A video of one of the acquisitions is shown along with the timecourses of select vessels’ caliber changes in Author response image 3. The vascular caliber changes following photostimulation persisted for several minutes, consistent with earlier observations by us and others1–4. These higher temporal-resolution scans of smaller tissue volumes will be repeated in two more mice; we will therein assess the repeatability of individual vessel responses to repeated stimulations.

Author response image 2.

A. The vascular radii of multiple vessels were imaged at 4 different cortical depths, each within a 507 x (75-150) x (30-45)um tissue volume. Baseline scanning lasted for 5 minutes, followed by 5 seconds of blue or green light stimulation at 4.3 mW/mm2, and culminating in 5 minutes of post-stimulation scanning. B. LOESS fits of the vessel radius estimates for each vessel segment identified.

Author response image 3.

Estimated vascular radius at each timepoint for select vessels from the imaging stack shown in the following video: https://flip.com/s/kB1eTwYzwMJE

(5) A second problem is the use of optogenetic stimulation to activate the tissue. First, it has been shown that blue light itself can increase blood flow (Rungta et al 2017). The authors note the concern about temperature increases but that is not the same issue. The discussion mentions that non-transgenic mice were used to control for this with "data not shown". This is very important data given these earlier reports that have found such effects and so should be included.

We will update the manuscript to incorporate the data on volumetric scanning in nontransgenic C57BL/6 mice undergoing blue light stimulation, with identical parameters as those used in Thy-ChR2 mice. As before, responders were identified as vessels that following blue light stimulation show a radius change greater than 2 standard deviations of their baseline radius standard deviation: their estimated radii changes are shown in Author response image 4 below. There were no statistical difference between radii distributions of any of the photostimulation conditions and pre-photostimulation baseline. A comparison of this with the transgenic THY1-ChR2-EYFP mice will be included in manuscript updates.

Author response image 4.

Radius change measurements for responding vessels from the Thy1-ChR2 mice described in the manuscript (top row) vs. 4 wild-type C57BL6/J mice (bottom row). Response to photostimulation was defined as a change above twice their baseline standard deviation. 458nm light was applied at 1.1 mW/mm^2 and 4.3 mW/mm^2; while 552 nm light was applied at 4.3 mW/mm^2. No statistically significant differences were observed between the radii distributions in any condition, Wilcoxon test, Bonferroni correction.

(6) Secondly, there doesn't seem to be any monitoring of neural activity following the photo-stimulation. The authors repeatedly mention "activated" neurons and claim that vessel properties change based on distance from "activated" neurons. But I can't find anything to suggest that they know which neurons were active versus just labeled. Third, the stimulation laser is focused at a single depth plane. Since it is single-photon excitation, there is likely a large volume of activated neurons. But there is no way of knowing the spatial arrangement of neural activity and so again, including this as a factor in the analysis of vascular responses seems unjustified.

Given the high fidelity of Channel-Rhodpsin2 activation with blue light, we assume that all labeled neurons within the volume of photostimulation are being activated. Depending on their respective connectivities, their postsynaptic neurons (whether or not they are labelled) are also activated. We indeed agree with the reviewer that the spatial distribution of neuronal activation is not well defined. We will revise the manuscript to update the terminology from activated to labeled neurons and stress in the Discussion that the motivation for assessing the distance to the closest labelled neuron as one of our metrics is purely to demonstrate the possibility of linking vascular response to activations in some of their neighbouring neurons and including morphological metrics in the computational pipeline. Of final note, the depth-dependence of the distance between labelled neurons and responding vessels can also readily be assessed using our computational pipeline.

(7) The study could also benefit from more clear illustration of the quality of the model's output. It is hard to tell from static images of 3-D volumes how accurate the vessel segmentation is. Perhaps some videos going through the volume with the masks overlaid would provide some clarity. Also, a comparison to commercial vessel segmentation programs would be useful in addition to benchmarking to the ground truth manual data.

We generated a video demonstrating the deep-learning model outputs and have made the video available here: https://flip.com/s/_XBs4yVxisNs Additional videos will be uploaded.

(8) Another useful metric for the model's success would be the reproducibility of the vessel responses. Seeing such a large number of vessels showing constrictions raises some flags and so showing that the model pulled out the same response from the same vessels across multiple repetitions would make such data easier to accept.

We have generated a figure demonstrating the repeatability of the vascular responses following photoactivation in a volume, and presented them next to the corresponding raw acquisitions for visual inspection. It is important to note that there is a significant biological variability in vessels’ responses to repeated stimulation, as described previously 2,5. Constrictions have been reported in the literature by our group and others 1,3,4,6,7, though their prevalence has not been systematically studied to date. Concerning the reproducibility of our analysis, we will demonstrate model reproducibility (as a metric of its success) in the updated manuscript.

Author response image 5.

Registered acquisitions of the vasculature before and after optogenetic stimulation for 5 scan pairs over 3 different stimulation conditions. The estimated radii along vessel segments are presented.

Author response image 6.

Sample capillaries constrictions from maximum intensity projections at repeated timepoints following optogenetic stimulation. Baseline (pre-stimulation) image is shown on the left and the post-stimulation image, on the right, with the estimated radius changes listed to the left.

(9) A number of findings are questionable, at least in part due to these design properties. There are unrealistically large dilations and constrictions indicated. These are likely due to artifacts of the automated platform. Inspection of these results by eye would help understand what is going on.

Some of the dilations were indeed large in magnitude. We present select examples of large dilations and constrictions ranging in magnitude from 2.08 to 10.80 um for visual inspection (for reference, average, across vessel and stimuli, magnitude of radius changes were 0.32 +/- 0.54 um). Diameter changes above 5 um were visually inspected.

Author response image 7.

Additional views of diameter changes in maximum intensity projections ranging in magnitude from 2.08 um to 10.80 um.

(10) In Figure 6, there doesn't seem to be much correlation between vessels with large baseline level changes and vessels with large stimulus-evoked changes. It would be expected that large arteries would have a lot of variability in both conditions and veins much less. There is also not much within-vessel consistency. For instance, the third row shows what looks like a surface vessel constricting to stimulation but a branch coming off of it dilating - this seems biologically unrealistic.

We now plot photostimulation-elicited vesselwise radius changes vs. their corresponding baseline radius standard deviations (Author response image 8 below). The Pearson correlation between the baseline standard deviation and the radius change was 0.08 (p<1e-5) for 552nm 4.3 mW/mm^2 stimulation, -0.08 (p<1e-5) for 458nm 1.1 mW/mm^2 stimulation, and -0.04 (p<1e-5) for 458nm 4.3 mW/mm^2 stimulation. For non-control (i.e. blue) photostimulation conditions, the change in the radius is thus negatively correlated to the vessel’s baseline radius standard deviation. The within-vessel consistency is explicitly evaluated in Figure 8 of the manuscript. As for the instance of a surface vessel constricting while a downstream vessel dilates, it is important to remember that the 2PFM FOV restricts us to imaging a very small portion of the cortical microvascular network (one (among many) daughter vessels showing changes in the opposite direction to the parent vessel is not violating the conservation of mass).

Author response image 8.

A plot of the vessel radius change elicited by photostimulation vs. baseline radius standard deviation.

(11) As mentioned, the large proportion of constricting capillaries is not something found in the literature. Do these happen at a certain time point following the stimulation? Did the same vessel segments show dilation at times and constriction at other times? In fact, the overall proportion of dilators and constrictors is not given. Are they spatially clustered? The assortativity result implies that there is some clustering, and the theory of blood stealing by active tissue from inactive tissue is cited. However, this theory would imply a region where virtually all vessels are dilating and another region away from the active tissue with constrictions. Was anything that dramatic seen?

The kinetics of the vascular responses are not accessible via the current imaging protocol and acquired data; however, this computational pipeline can readily be adapted to test hypotheses surrounding the temporal evolution of the vascular responses, as shown in Author response image 2 (with higher temporal-resolution data). Some vessels dilate at some time points and constrict at others as shown in Author response image 2. As listed in Table 2, 4.4% of all vessels constrict and 7.5% dilate for 452nm stimulation at 4.3 mW/mm^2. There was no obvious spatial clustering of dilators or constrictors: we expect such spatial patterns to more likely result from different modes of stimulation and/or in the presence of a pathology. The assortativity peaked at 0.4 (i.e. is quite far from 1 where each vessel’s response exactly matches that of its neighbour).

(12) Why were nearly all vessels > 5um diameter not responding >2SD above baseline? Did they have highly variable baselines or small responses? Usually, bigger vessels respond strongly to local neural activity.

In Author response image 9, we now present the stimulation-induced radius changes vs. baseline radius variability across vessels with a radius greater than 5 um. The Pearson correlation between the radius change and the baseline radius standard deviation was 0.04 (p=0.5) for 552nm 4.3 mW/mm^2 stimulation, -0.26 (p<1e-5) for 458nm 1.1 mW/mm^2 stimulation, and -0.24 (p<1e-5) for 458nm 4.3 mW/mm^2 stimulation. We will incorporate an additional analysis to address this issue by identifying responding vessels as those showing supra-threshold percent change in their radius (instead of SD).

Author response image 9.

A plot of the vessel radius change elicited by photostimulation vs. baseline radius standard deviation in vessels with a baseline radius greater than 5 um.

References

(1) Alarcon-Martinez L, Villafranca-Baughman D, Quintero H, et al. Interpericyte tunnelling nanotubes regulate neurovascular coupling. Nature. 2020;kir 2.1(7823):91-95. doi:10.1038/s41586-020-2589-x

(2) Mester JR, Bazzigaluppi P, Weisspapir I, et al. In vivo neurovascular response to focused photoactivation of Channelrhodopsin-2. NeuroImage. 2019;192:135-144. doi:10.1016/j.neuroimage.2019.01.036

(3) O’Herron PJ, Hartmann DA, Xie K, Kara P, Shih AY. 3D optogenetic control of arteriole diameter in vivo. Nelson MT, Calabrese RL, Nelson MT, Devor A, Rungta R, eds. eLife. 2022;11:e72802. doi:10.7554/eLife.72802

(4) Hartmann DA, Berthiaume AA, Grant RI, et al. Brain capillary pericytes exert a substantial but slow influence on blood flow. Nat Neurosci. Published online February 18, 2021:1-13. doi:10.1038/s41593-020-00793-2

(5) Mester JR, Bazzigaluppi P, Dorr A, et al. Attenuation of tonic inhibition prevents chronic neurovascular impairments in a Thy1-ChR2 mouse model of repeated, mild traumatic brain injury. Theranostics. 2021;11(16):7685-7699. doi:10.7150/thno.60190

(6) Mester JR, Rozak MW, Dorr A, Goubran M, Sled JG, Stefanovic B. Network response of brain microvasculature to neuronal stimulation. NeuroImage. 2024;287:120512. doi:10.1016/j.neuroimage.2024.120512

(7) Hall CN, Reynell C, Gesslein B, et al. Capillary pericytes regulate cerebral blood flow in health and disease. Nature. 2014;508(7494):55-60. doi:10.1038/nature13165