Nanoparticles are used extensively as biomedical imaging probes and potential therapeutic agents. As new particles are developed and tested in vivo, it is critical to characterize their biodistribution profiles. We demonstrate a new method that uses adaptive algorithms for the analysis of hyperspectral dark-field images to study the interactions between tissues and administered nanoparticles. This non-destructive technique quantitatively identifies particles in ex vivo tissue sections and enables detailed observations of accumulation patterns arising from organ-specific clearance mechanisms, particle size, and the molecular specificity of nanoparticle surface coatings. Unlike nanoparticle uptake studies with electron microscopy, this method is tractable for imaging large fields of view. Adaptive hyperspectral image analysis achieves excellent detection sensitivity and specificity and is capable of identifying single nanoparticles. Using this method, we collected the first data on the sub-organ distribution of several types of gold nanoparticles in mice and observed localization patterns in tumors.https://doi.org/10.7554/eLife.16352.001
Metallic elements like gold and silver can be made into particles that are one thousand times smaller than the width of a human hair. Researchers can create these “nanoparticles” in different sizes and shapes that exhibit unique properties. For example, gold can be made into rod-shaped particles that interact with infrared light. Other nanoparticles can be loaded with drug molecules and designed to bind to cancer cells. As a result, nanoparticles have been explored for use in a variety of biomedical imaging and therapy applications. However, we must fully understand how the nanoparticles bind to the cancer cells and how the body tolerates these nanoparticles before they can be used in humans.
Experiments that explore where nanoparticles accumulate in the body are typically called biodistribution studies. However, current techniques for studying biodistribution cannot simultaneously measure the uptake of particles into organs and reveal the fine structures inside the organs that interact with the particles.
SoRelle, Liba et al. aimed to address this problem by developing a new biodistribution technique called HSM-AD (short for hyperspectral microscopy with adaptive detection). This new technique combines a relatively recent method called hyperspectral dark-field microscopy, which can identify nanoparticles from their unique optical signatures, with versatile computer algorithms to detect nanoparticles.
HSM-AD is more sensitive than previously developed biodistribution techniques, and SoRelle, Liba et al. used it to produce highly detailed maps of nanoparticle uptake patterns in the organs of mice. These maps provide new insights into how cells and tissues in the body handle different nanoparticles. Moreover, HSM-AD was able to distinguish nanoparticles with unique shapes by their distinct optical signatures. Further experiments show that HSM-AD can reveal interactions between human tumor cells and nanoparticles specifically designed to target those cells.
HSM-AD will be a useful resource for researchers studying the effect of nanoparticles on the human body. Future studies will use this technique to explore which nanoparticles have the potential to be developed for medical uses.https://doi.org/10.7554/eLife.16352.002
Nanoparticles (NPs) can be fashioned in precise shapes and sizes from a wide variety of materials. This synthetic versatility makes NPs excellent tools for wide-ranging biomedical applications including in vivo imaging (Jokerst et al., 2012; Durr et al., 2007), drug delivery (Hauck et al., 2008), photothermal therapy (Huang et al., 2006; von Maltzahn et al., 2009), and gene transfection (Huang et al., 2009). In particular, metal and metal oxide NPs made from gold, silver, iron, and titanium are commonly used in biomedicine owing to their unique electromagnetic properties (Giustini et al., 2011; Husain et al., 2015; Lee and El-Sayed, 2006). Once administered to a living subject, these NPs may exhibit vastly different pharmacokinetics and uptake profiles that are contingent on NP shape, size, surface coating, and other factors (Owens III and Peppas, 2006; Chen et al., 2009; Zhang et al., 2009; He et al., 2010; Decuzzi et al., 2010). These differences manifest not only at the scale of whole organs but also at the cellular level (Giustini et al., 2011; Yang et al., 2014; Sadauskas et al., 2009). Ideally, biodistribution studies should address various scales – from whole animal to tissue and cellular interactions – in order to understand a given NP’s in vivo behavior.
Current studies commonly employ inductively-coupled plasma (ICP) techniques (Niidome et al., 2006) or electron microscopy (EM) (Giustini et al., 2011) to interrogate metallic NP biodistribution. However, each of these techniques has notable disadvantages. ICP can be coupled to mass spectrometry (MS) or atomic/optical emission spectrometry (AES/OES) to quantify the presence of a metallic species in tissues of interest with high sensitivity (~10 parts per billion); incidentally, detection of large metal NPs with ICP relies upon dissolving samples in strong acids. The need to dissolve NP-containing samples has severe drawbacks with respect to characterizing particle uptake including the complete loss of spatial insights such as NP distribution patterns within the given tissue. Moreover, the sample preparation itself can impede detection sensitivity, especially for small tissue samples and tissues with intrinsically low NP uptake, which must be diluted in acid. Conversely, EM studies provide exquisite high-resolution images of NP uptake by individual cells. Unfortunately, EM requires cumbersome sample preparation and acquires qualitative data over fields of view that are too small to be tractable for whole organ studies. Fluorescence (Zhang et al., 2009; He et al., 2010; Poon et al., 2015) and radioactivity (Kreyling et al., 2015; Collingridge et al., 2003) detection can also be used to assess NP biodistribution, however these techniques typically require the addition of a labeling moiety to the NP prior to in vivo use. Aside from the potential that labels may detach or even alter NP pharmacokinetic properties, whole-organ studies with these techniques can be impeded by poor spatial resolution.
Hyperspectral dark-field microscopy (HSM) is a technique that obtains scattered light spectra from a sample on a per-pixel basis (Roth et al., 2015). HSM is capable of identifying individual nanoparticles in pure solutions and cell culture by their intrinsic scattering spectra without the addition of a labeling molecule (Fairbairn, 2013; Fairbairn et al., 2013; Patskovsky et al., 2015). This approach may be ideal for detecting metallic nanoparticles with unique visible and near-infrared (NIR) spectral signatures. Unlike current methods that characterize NP biodistribution, HSM simultaneously achieves diffraction-limited spatial resolution and excellent detection sensitivity without destroying the sample. HSM has been used to study NP uptake in cell culture (Yang et al., 2014; Fairbairn, 2013; Fairbairn et al., 2013; Patskovsky et al., 2015) and the induction of toxic effects in tissue (Husain et al., 2015), but its use for characterizing NP biodistribution has not yet been demonstrated due to several outstanding constraints. The primary limitation that has prevented HSM from being used in evaluating the biodistribution of NPs in tissue is the inability to accurately distinguish NPs from the background of tissue scattering. To abate this limitation, we use a modified dark-field microscope that uses oblique sample illumination to enable 150-fold brightness enhancement and ~15-fold better signal to noise ratio (SNR) than standard dark-field optics (Badireddy et al., 2012; Zhang et al., 2015). Another challenge with HSM detection stems from the reality that individual NPs within a given sample do not exhibit the exact same spectrum. Furthermore, the NP uptake within tissues inevitably results in a combination of the NP spectrum with tissue scattering, which can be spectrally diverse. Current approaches such as spectral angle mapping (Roth et al., 2015; Kruse, 1993; De Carvalho and Meneses, 1999; Luc, 2005; Roth et al., 2015) (originally developed for non-biological applications) and manual delineation (Husain et al., 2015; Roth et al., 2015) cannot adapt to these conditions and may yield high false positive and false negative detection rates. It has been observed that no HSM method to date has demonstrated robust capabilities for quantifying false positive rates or other diagnostic measures (Roth et al., 2015). Thus, HSM methods must be customized to address spectral mixing and diffraction effects as well as detection sensitivity and specificity if they are to be successfully used for microscopic analyses of complex biological samples.
Here, we demonstrate Hyperspectral Microscopy with Adaptive Detection (HSM-AD), the first HSM method based on adaptive clustering, as a viable alternative to current techniques for assessing whole-organ biodistribution and cellular uptake of NPs. In this study, we collected tissues of interest from mice that were injected with large gold nanorods (LGNRs) (SoRelle et al., 2015), gold nanoshells (Nanoshells), and silica-coated gold nanospheres (GNS@SiO2), and we developed pre-processing and adaptive algorithms to identify NPs that accumulated in tissue sections based on their spectral signatures. The implementation of an adaptive classification algorithm for spectral classification extended HSM’s single NP detection capabilities to tissue samples with negligible false-positive detection. HSM-AD was sufficiently robust for detecting NPs in images of different organ tissues and images acquired using variable illumination conditions. This approach may be preferable to conventional biodistribution assays for studies that simultaneously require quantification of relative NP uptake in various clearance organs and wide-field high-resolution images with histological detail.
LGNRs (~100 × 30 nm) exhibiting a near infrared plasmonic peak (Figure 1a) were synthesized, biofunctionalized, and administered to healthy and tumor-bearing nude (Foxn1nu/nu) mice as previously reported (SoRelle et al., 2015; Liba et al., 2016). Mice were euthanized 24 hr post-injection, and various tissues were resected and fixed in 10% formalin. Fixed tissues were sectioned into 5 µm thick slices, mounted on glass slides, and stained with Hematoxylin and Eosin (H&E) as per standard histological preparation (Figure 1b). H&E-stained sections were imaged at 40x or 100x magnification in conventional dark-field and hyperspectral microscopy modes (CytoViva) (Figure 1—figure supplement 1). Conventional dark-field images (Figure 1c) were used to guide anatomical feature identification. All spectral data and quantitative comparisons presented in this report were derived from the analysis of hyperspectral images.
A hyperspectral camera with a detection range of 400–1000 nm was used to image scattered light from each sample in transmission mode. In the resulting images (Figure 1d), each pixel contains the spectral profile of the sample at the corresponding spatial position and can be used to detect LGNRs with near diffraction-limited resolution (1 μm). While standard dark-field images did not reveal notable differences between uninjected and injected samples, the hyperspectral images, in which three bands of the spectrum (800.0 nm, 700.6 nm, and 526.2 nm) were respectively color-coded as red, green, and blue, indicated that species with strong near infrared scattering (putative LGNRs, depicted in orange) were present in injected tissues but not observable in control tissues. We created HSM-AD, a method that combines pre-processing and adaptive classification algorithms to automatically detect and quantify LGNRs in hyperspectral images. The pre-processing stage is described in detail in the methods section (Figure 1—figure supplements 2,3). One of the first stages of processing includes vignette correction and a determination of whether each pixel in the image belongs to one of three categories—background, tissue, or potential LGNR—based on its average intensity across the measured spectrum. Only high-intensity pixels that belong to the potential LGNR group are classified by the adaptive algorithm (Figure 1—figure supplement 2). For the training of the adaptive algorithm, pre-processed images of tissue samples from LGNR injected mice were input into a standard k-means clustering algorithm (Bishop, 1995). Four initial clusters were identified using this scheme and were used to produce a spectral cluster library (Figure 1e). Owing to the unique spectral profile of LGNR scattering, the particle spectrum was automatically recognized as a separate cluster by the k-means algorithm. Next, a fifth cluster was manually added to the spectral cluster library to account for edge artifacts caused by chromatic aberrations that were frequently falsely detected as LGNRs (Figure 1—figure supplement 3). The five spectra in this library were used as cluster centers for automatic detection of LGNRs in tissue sections using a nearest centroid (or nearest-neighbor) classifier. In this scheme, pixels that exhibited a spectrum that was closest (in a Euclidean sense) to the LGNR cluster center were identified as containing LGNRs (denoted as LGNR+). The rest of the pixels were classified as not containing LGNRs (denoted as LGNR-). Initial validation of the algorithm shows that regions of the image that were detected as LGNR+ indeed exhibited the characteristic plasmonic peak at around 900 nm while pixels identified as LGNR- did not (Figure 1f). The mean and standard deviation spectra of pixels classified into each cluster for a representative image also indicate the high fidelity of the algorithm (Figure 1—figure supplement 4).
We characterized the sensitivity and specificity of HSM-AD by three methods. First, we measured the false positive rate in uninjected tissue samples to obtain a specificity of 99.7% (Figure 2—figure supplements 1–4). The false positives, which also have a spectral peak near 900 nm, usually appear near the edges of the tissue section. We attribute this red-shift of the spectrum to chromatic aberrations, ostensibly due to the spectral dependence of the diffraction diameter (Lipson and Lipson, 2010). Next, we measured the false negatives in an image of LGNRs in mounting media (CytoSeal 60, Electron Microscopy Sciences) on a glass slide and obtained a detection sensitivity of 99.4%. We attributed the false negatives to LGNRs with hybridized surface plasmon resonances (Funston et al., 2009), which resulted in spectral scattering that was different from the distinct plasmonic resonance of single LGNRs (Figure 2—figure supplements 5,6). Because all training and test samples were mounted using the same media, spectral shifts due to local refractive environments did not contribute to false detection (Figure 2—figure supplement 7). Independently we also calculated specificity and sensitivity by analyzing LGNR-injected tissue samples (see Methods). We obtained a sensitivity of 89.5% and a specificity of 98.5% using this approach. The high sensitivity of the automated algorithm is further evident from its ability to detect single LGNRs, both on a glass slide and in injected tissue samples (Figure 2—figure supplements 5,8).
We demonstrated HSM-AD as a potential biodistribution technique by analyzing various tissues resected from mice (Figure 2). For quantitative measurements of LGNR uptake, we analyzed kidney tissue from uninjected (Figure 2a, Figure 2—figure supplements 2a,3a,4a) and injected (Figure 2b, Figure 2—figure supplements 9a,10a,11a) mice. Our analysis found a relative LGNR signal of 4.8% ± 2.3% in injected mouse kidney tissue. By comparison, a relative LGNR signal of 0.08% ± 0.01% was measured from uninjected samples, indicative of the method’s high specificity (Figure 2c, Figure 2—figure supplement 1). Similar low false positive rates were measured in other organ tissues (Figure 2—figure supplements 2b–e,3b–e,4b–e). In addition to the kidney, HSM-AD was used to analyze LGNR uptake in liver, lung, muscle, and spleen sections to demonstrate an alternative to common biodistribution techniques. While a conventional biodistribution study of LGNRs has not yet been reported, HSM-AD analysis indicated that LGNRs exhibited a similar uptake profile (mostly in the liver and spleen) as commonly-used smaller gold nanorods (Zhang et al., 2009; Niidome et al., 2006). The greatest relative LGNR signal (38.5% ± 4.5%) was observed in the spleen. LGNRs were also concentrated in the liver (7.5% ± 1.5%). Particle uptake was minimal in lung tissue (0.5% ± 0.1%) and muscle tissue (0.8% ± 0.5%) sections (Figure 2d, Figure 2—figure supplements 9–11). Tissue sections without H&E staining were also analyzed and yielded results similar to those obtained for H&E stained sections (Figure 2—figure supplements 12–14).
HSM-AD imaging of histological sections (Figure 3a) enabled sensitive LGNR detection with sub-cellular resolution over large fields of view (Figure 3b–c), which afforded more detailed characterizations of NP uptake than those achieved by typical biodistribution methods. We used these advantages to investigate and quantify the sub-organ distribution of LGNRs. This analysis revealed well-defined patterns of LGNR uptake that appeared to be largely influenced by factors including particle size, innate immunological function, and waste-filtering anatomical structures.
The kidneys are responsible for filtering small, low molecular weight waste products from the bloodstream and diverting those products to the bladder for elimination (Rouiller, 2014). Waste-laden blood flows into capillary-dense structures called glomeruli within the kidney. Blood plasma containing small species including ions, biomolecules, cell fragments, and (in some cases) nanoparticles can extravasate from glomerular capillaries and traverse Bowman’s space before being collected into an extensive network of efferent tubes called Proximal Convoluted Tubules (PCT) and, further downstream, Distal Convoluted Tubules (DCT), which ultimately traffic waste from the kidney out to the bladder. Using our method, we observed that the vast majority of LGNRs within the kidney were concentrated in glomeruli and were virtually absent in the PCT and DCT, which are functionally downstream (Figure 3d–f). Glomerular uptake was 13-fold greater than uptake within the convoluted tubule network. These results can be explained by comparing the size of an individual LGNR with the narrow width of cellular junctions and the architecture of endothelial cells that form glomerular capillaries (Satchell and Braet, 2009). For reference, the sub-organ segmentation maps of all tissue images analyzed for quantification have been provided (Figure 4—figure supplement 1).
Along with the kidneys, the liver is instrumental in clearing waste from circulation. Partially because the size cutoff for hepatic filtration is notably larger than that of renal ultrafiltration (Longmire et al., 2008), we observed 1.6-fold greater LGNR uptake within the liver tissue compared to the kidney. Interestingly, the hepatic distribution of LGNRs also appeared to be non-uniform. Hepatocytes, which constitute the majority of liver tissue by mass, exhibited mild LGNR signal (2.9% ± 1.1%). By contrast, 15-fold more LGNR signal (43.5% ± 12.0%) in the liver was localized in a manner consistent with the shape, size, and number of Kupffer cells (Figure 4a, Figure 2—figure supplements 9b,10b,11b). We attribute this localization to the phagocytic function of Kupffer cells (Owens III and Peppas, 2006; Longmire et al., 2008). Thus, the variable localization of nanoparticles within the liver appears to be largely derived from the organ’s innate immunological functions. It is interesting to note that aggregation within Kupffer cells likely caused spectral hybridization of some LGNRs, an observation that is consistent with previous studies of cellular uptake of gold NPs (Chen, 2014). This spectral shifting, which was most prevalent at the centers of LGNR aggregates, caused a portion of LGNRs to remain undetected by HSM-AD. While these aggregates were undetected by algorithmic means, their manual identification as LGNRs was evident from the lack of similar morphological features in uninjected liver tissue.
The spleen comprises several unique cell types arranged into tissues with diverse biological functions including blood filtration, innate immunity, and lymphocyte activation (Mebius and Kraal, 2005). The spleen is largely composed of red pulp, white pulp, and the boundary between these two tissues, commonly referred to as the marginal zone. Consistent with each tissue’s biological function, we observed 1.7-fold greater relative LGNR pixel coverage in splenic red pulp than white pulp (Figure 4b, Figure 2—figure supplements 9e,10e,11e). A similar result has been previously reported for carbon-based nanomaterials (Chen et al., 2015). We did not definitively identify marginal zone tissue, but the radial distribution of LGNRs around white pulp follicles indicated that a significant portion of white pulp uptake may in fact be within the marginal zone (Figure 4—figure supplement 2).
Despite its dense network of alveolar capillaries, lung tissue exhibited minimal LGNR accumulation relative to the organs described above (Figure 4c, Figure 2—figure supplement 9c,10c,11c). This finding was consistent with existing biodistribution data for smaller particles, which can be explained by the lungs’ major functions of gas exchange to and from the blood rather than biomolecule or particle filtration and clearance.
While whole-organ analysis indicated the presence of LGNRs in muscle tissue, HSM-AD revealed that muscle tissue itself (which consists largely of myocytes and dense networks of extracellular collagen) was virtually devoid of LGNRs (Figure 4d, Figure 2—figure supplement 9d,10d,11d). Rather, the apparently high LGNR presence was traced to blood vessels found in between muscle fiber bundles. As with the accumulation patterns described for other organs within this study, this distinction would not have been possible through conventional biodistribution methods.
HSM-AD images acquired at higher objective magnification (100x) offered further insights into the cellular nature of LGNR uptake within the kidney and liver tissue (Figure 5). Within the kidney, LGNRs were observed mostly within or in close proximity to glomerular capillaries (Figure 5a,b). HSM-AD also revealed patterns of LGNR uptake within individual Kupffer cells resident in liver sinusoids (Figure 5c,d). LGNR signal was detected within the Kupffer cell cytoplasm, but not within the region of the cell nucleus. This pattern is consistent with the phagocytic function of Kupffer cells in clearing particulate matter from circulation. Interestingly, several bright regions within the Kupffer cell were not detected as LGNRs. We expect that these regions resulted from spectral hybridization of LGNRs, possibly due to aggregation induced by lysosomal acidification following particle phagocytosis.
We also injected mice with Nanoshells, which are morphologically distinct from LGNRs (Figure 6a). While Nanoshells and LGNRs both exhibit near-infrared plasmonic peaks, the Nanoshell spectrum is substantially broader than the LGNR spectrum (Figure 6b). A spectral cluster library was developed for H&E-stained Nanoshell+ tissues and was then used to quantify Nanoshell uptake as described for LGNRs (Figure 6c). While Nanoshells and LGNR displayed related uptake patterns, several differences including negligible Nanoshell uptake in kidney tissue and Nanoshell concentration within the splenic white pulp were observed (Figure 6d, Figure 6—figure supplements 1,2, Nanoshell+ pixels are shown in cyan).
HSM-AD was separately trained on a sample consisting of a mixture of pure Nanoshells and pure LGNRs. The spectral clusters identified during this training corresponded well to the spectra of each particle type and enabled high-specificity and high-sensitivity identification in samples of Nanoshells + LGNRs, Nanoshells-only, and LGNRs-only (Figure 6—figure supplement 3). These results demonstrate that HSM-AD can spectrally resolve plasmonic particles despite similarities in composition, although this capability was not tested in ex vivo tissues.
We used HSM-AD to characterize the tissue uptake of a third particle type, GNS@SiO2 (Figure 7, GNS@SiO2+ pixels are shown in green). Notably, GNS@SiO2 are distinct from LGNRs and Nanoshells in terms of shape, size, composition, particle surface, and plasmonic resonance (Figure 7—figure supplement 1a,b). Because GNS@SiO2 exhibit a visible regime plasmonic peak (~550 nm), HSM-AD analysis was performed on unstained tissue sections. First, a spectral cluster library was developed for GNS@SiO2 classification as described for LGNRs (Figure 7—figure supplement 1c). Control tissues classified with this library displayed negligible false positives (Figure 7a). GNS@SiO2 uptake in the liver and spleen was observed at 2 and 24 hr post-IV injection (Figure 7b,c). Interestingly, GNS@SiO2 uptake appeared to be even more localized to Kupffer cells than LGNR accumulation in the liver. Furthermore, GNS@SiO2 in the spleen are consistently found in the marginal zone, and presence within the red pulp and white pulp is minimal (Figure 7—figure supplement 2). Quantitative results from HSM-AD correlate well with those obtained using ICP-MS (Figure 7—figure supplements 3,4), although it should be noted that HSM-AD measurements are more relative rather than absolute with respect to the amount of gold present in each tissue. As for LGNR quantification, four FOVs per sample were analyzed (Figure 7—figure supplements 5,6).
One hallmark of tumor growth is angiogenesis, the stimulated development of new blood vessels to provide nutrients to rapidly dividing cancer cells. This newly-formed vasculature is composed of endothelial cells that express high levels of cell adhesion receptors including αVβ3 integrin (Avraamides et al., 2008). Thus, αVβ3 is commonly used as a target biomolecule for tumor imaging (Sipkins et al., 1998). Such studies have demonstrated that NPs targeted to αVβ3 exhibit greater accumulation in tumors in vivo than NPs coated with non-specific antibodies or small molecules. We hypothesized that the presence or absence of specific molecular targeting moieties would influence tissue-NP interactions beyond simply the degree of accumulation in target tissues. To test this, we used HSM-AD to observe the spatial patterns of targeted and non-targeted LGNR uptake within U87MG (human glioblastoma cells, αVβ3+) tumor xenografts. We observed 7.4-fold greater relative LGNR signal of anti-αVβ3 LGNRs than isotype LGNRs in tumor tissue (Figure 8a–d). However, the most striking differences were in the localization patterns of each LGNR type. Anti-αVβ3 LGNRs were present in high density around the edges of small blood vessels within the tumor while isotype LGNRs showed no such association (Figure 8c–f, Figure 8—figure supplement 1). The prevalence of anti-αVβ3 LGNRs around the edges of tumor capillaries is highly consistent with the expression pattern of αVβ3 in angiogenic vessels. Moreover, isotype LGNRs found outside of the vasculature were notably dispersed compared to extravascular anti-αVβ3 LGNRs, which often appeared in small clusters. While NPs are known to accumulate in tumors regardless of molecular specificity due to leaky vasculature, these results indicated that the enhanced extravascular accumulation of anti-αVβ3 LGNRs may have originated from specific binding of αVβ3 integrins present on the U87MG cells themselves.
The necessity of sample digestion with strong acids for ICP quantification effectively reduces an entire organ (a remarkably rich dataset by any measure) down to a single number representative of bulk NP accumulation. While the quantification offered by ICP is certainly valuable, it provides minimal insight into the patterns and mechanisms of NP uptake within individual cells or tissues. Unlike ICP methods, HSM-AD provides additional dimensions of anatomical detail at optical resolution to facilitate better understanding of the biology behind quantitative measurements of NP uptake.
The primary solution for dealing with the limitations of ICP has been to use EM, which provides excellent spatial resolution (at the nanometer scale) and particle sensitivity (down to individual nanoparticles). However, EM can only scan minimal fields of view—a typical transmission EM (TEM) image for studying NP uptake covers ~1 × 1 µm. For comparison, TEM scanning of the same area depicted in Figure 3c would require ~460,000 TEM images, which is infeasible for single tissue studies and virtually unrealistic for multiple-organ studies. The necessity of thin samples (~10 nm) for TEM imaging compared to samples analyzed using HSM-AD (~1 µm optical focus) would further multiply the number of TEM scans (>46 million) required for equivalent volumetric imaging. Other biodistribution techniques based on radioactivity (Kreyling et al., 2015; Collingridge et al., 2003), photoacoustic (Poon et al., 2015), and fluorescence (He et al., 2010) detection have been used previously as alternatives to ICP and TEM. By comparison, HSM-AD offers roughly 100-fold higher spatial resolution (~1 µm vs ~100 µm) than current fluorescence and photoacoustic biodistribution methods. Fluorescence-based methods may also suffer from high false positive detection arising from tissue autofluorescence, as has been observed for renal capsule tissue (Poon et al., 2015). While HSM-AD was excellently suited for exploring the sub-organ localization of NPs, it has been observed that radiolabeling approaches may be poorly-equipped for accurately determining particle distribution within organs (Kreyling et al., 2015). Moreover, single-particle detection sensitivity was not demonstrated by any of these alternatives to ICP and TEM.
Biodistribution methods based on imaging mass spectrometry were recently demonstrated to enable sub-organ quantification of carbon nanomaterials (Chen et al., 2015). This impressive approach can obtain images of full mouse tissue sections (cm in scale), but the limited spatial resolution (50 µm) precludes the study of NP uptake within individual cells. Because detection relies upon particle fragmentation and ionization, it is unclear whether imaging mass spectrometry can achieve single particle sensitivity—calculations based upon the reported data indicate that ~10 (von Maltzahn et al., 2009) particles per pixel are required for detection. However, the cited spatial resolution negates many of the potential advantages of single-particle sensitivity such as direct observation of NP uptake by cells through endocytosis or adhesion to the cell membrane. More generally, while mass spectrometry provides an approach to biodistribution studies of certain materials, its use for identifying gold NPs has been constrained to NPs smaller than 10 nm and typically requires the inclusion of 'mass barcode' molecules as capping agents (such as alkanethiols) on the NP surface (Zhu et al., 2008; Harkness et al., 2010). Incidentally, gold NPs have previously been demonstrated as assisting matrices to improve mass spectrometry detection of biomolecules (Su and Tseng, 2007; Huang and Chang, 2007). Notably, many metallic NPs are compositionally similar yet spectrally distinct (for example, gold nanospheres, nanorods, nanoshells, etc.), which may confound results in mass spectrometry-based analysis of samples containing more than one NP species. We have demonstrated that HSM-AD can successfully identify gold nanoparticles with different spectra, shape, size, and composition in tissues. Furthermore, Nanoshells and LGNRs were discernible from each other in particle mixtures. While not directly tested in this work, HSM-AD may thus be capable of distinguishing such NPs from each other in tissues, enabling biodistribution studies of multiplexed NPs. Thus, we expect that HSM-AD will extend the advantages of high sensitivity and resolution to the analysis of a variety of metallic NPs with unique spectral properties.
As reported in previous studies (Chen, 2014; Okamoto et al., 2000), we observed that the scattering spectrum from gold NPs is heavily influenced by the local refractive index (n). Spectral shifts of ~80 nm were evident between preparations of LGNRs in water (n = 1) relative to the same particles prepared in CytoSeal (n = 1.5) (Figure 2—figure supplement 7), and similar shifts were observed for Nanoshells and GNS@SiO2. Thus, in order to generate reliable spectral cluster libraries, it is critical to train the adaptive algorithm using images prepared in similar fashion to the samples being studied. The orientation of anisotropic NPs within a sample has also been shown to influence the observed spectrum (Biswas et al., 2012). Because LGNRs are anisotropic, variation in particle orientation may affect detection sensitivity in our case, but this effect appears to be negligible in light of empirically measured sensitivity values. Because it relies on spectral identification of NPs, HSM-AD cannot detect NPs that have shifted their spectrum markedly, such as concentrated NP aggregation within cells. Such spectral hybridization is most prominent in Kupffer cells within the liver, which likely results in an artificially low measure of LGNR uptake in that organ. However, the absence of these bright aggregates in hyperspectral images of uninjected control tissues confirms their identity as LGNRs. Because of plasmon hybridization, LGNR aggregates produce shifted spectra that can resemble the scattering from H&E-stained tissue, which can impede automated detection. Thus, future efforts to detect such aggregates should rely on the analysis of unstained tissues.
We tested alternate machine learning approaches including support vector machine (SVM) and logistic regression for nanoparticle detection. We found that unsupervised k-means outperformed these other methods when trained on images of tissues containing nanoparticles (Liba and Shaviv, 2014). Further advantages of unsupervised k-means include its ease of use and no need to pre-label samples for analysis. While HSM-AD based on k-means clustering provides a robust general platform for nanoparticle detection, it is conceivable that certain studies may benefit from learning methods tailored to address specific applications.
HSM-AD imaging simultaneously achieves excellent sensitivity and specificity for detecting NPs in tissues with sub-cellular resolution. In addition to improved diagnostic capabilities, the automated and adaptive features of HSM-AD enable standardized high-throughput analysis previously absent from biomedical HSM studies (Roth et al., 2015). Unlike Spectral Angle Mapping (the current gold standard for HSM image analysis), HSM-AD does not require the manual steps typically needed to create target spectral libraries, define particle intensity and size thresholds, filter false positives from libraries, and calibrate angular tolerance for accurate classification on an individual image basis (Figure 2—figure supplement 15) (Roth et al., 2015). Along with an ability to image millimeter-scale fields of view on reasonable timescales (<30 min) and simple sample preparation, these properties make HSM-AD a favorable alternative to existing methods for characterizing NP biodistribution. Beyond biodistribution, this work demonstrates that HSM-AD can be used for post-injection validation of NP localization in target tissues as a function of surface modifications. Because HSM-AD is non-destructive, samples can be further analyzed by a variety of conventional microscopy techniques including immunohistochemistry to provide additional molecular detail. Collectively, the results presented herein indicate that HSM-AD provides a new approach for studying interactions of cells and whole tissues with spectrally unique NPs commonly used in biomedical imaging and therapeutic studies.
LGNRs were synthesized using methods adapted from Ye et al (Ye et al., 2013). LGNRs were characterized using Transmission Electron Microscopy (TEM), visible/near-infrared spectrometry, and dark-field hyperspectral microscopy. As-synthesized LGNRs were prepared for biological use by removing excess CTAB from solution and coating the particles with poly(sodium 4-styrenesulfonate) (PSS, MW 70 kDa) as previously reported (SoRelle et al., 2015). PSS-coated LGNRs were then conjugated with IgG isotype antibody (clone eB149/10H5, eBioscience) for use in sub-organ biodistribution experiments. We also prepared LGNRs targeted to αVβ3 integrin (a cell-surface receptor that is overexpressed in angiogenic vasculature within tumors) by conjugating LGNRs with anti-αVβ3 antibody (clone 23C6, eBioscience).
Healthy female nude (Foxn1nu/nu) mice (6–8 weeks old, Charles River Labs) were anesthetized with 2% isoflurane by inhalation and intravenously injected with 250 µL of IgG isotype-coated LGNRs at optical density (OD) 470. In separate experiments, mice bearing U87MG tumors in the right ear pinna were injected with either IgG isotype-coated LGNRs or anti-αVβ3-coated LGNRs. Additional details of these experimental protocols can be found in the literature (SoRelle et al., 2015; Liba et al., 2016). In nanoshell experiments, nude mice were injected with 200 µL of OD 50 (2.5 mg/mL) nanoshells composed of 119 nm silica cores and 14 nm-thick gold shells with PEG coating (Nano Composix, San Diego, CA). In GNS@SiO2 experiments, healthy female Balb/C mice were anesthetized as described previously and injected intravenously with 150 µL of 0.8 nM GNS@SiO2 particles composed of 60 nm gold cores and 30 nm-thick SiO2 shells (Oxonica, Mountain View, CA). For all experiments, mice were euthanized 24 hr (or 2 hr, for GNS@SiO2) post-injection, and tissues including kidney, liver, lung, spleen, thigh muscle, and (when applicable) tumor were immediately resected and preserved in 10% formalin. Tissues were also resected from uninjected mice for control imaging experiments. These tissues were subsequently embedded in paraffin and sectioned into 5 µm thick samples. Sections were prepared with and without H&E stains and mounted on microscope slides using CytoSeal 60 (Electron Microscopy Sciences) as the mounting medium. All animal experiments were performed in compliance with IACUC guidelines and with the Stanford University Animal Studies Committee’s Guidelines for the Care and Use of Research Animals. Experimental protocols (APLAC #s 27499 and 29179) were approved by Stanford University’s Animal Studies Committee.
All tissue samples were imaged with a modified dark-field microscopy setup as shown in Figure 1—figure supplement 1. Light from a broadband halogen lamp was coupled via an optical fiber into a custom dark-field condenser (CytoViva, Auburn, AL), which produced a light cone for sample illumination. Light scattered from the sample was collected using either a 40x magnification dark-field air objective lens (Olympus UPlanFLN 40x, 0.75 NA) or a 100x magnification oil immersion objective lens (Olympus UPlanFLN 100x, 1.3 NA) and directed to one of two cameras depending on detection mode. Conventional dark-field and hyperspectral images were collected for all samples in this study. Conventional dark-field images were collected with a Dagexcel-M cooled camera (Dage-MTI, Michigan City, IN). Hyperspectral images were collected with a hyperspectral camera (iXon3, Andor, Belfast, UK). Each image has 509 × 512 pixels. With a 40x lens, the sampling resolution is 410 × 408 nm, which produces a 209 × 209 µm field of view. With a 100x lens, the sampling resolution is 163 × 160 nm. Only Figure 5 shows images acquired at 100x magnification. The spectrum from each pixel was acquired with 361 uniform samples at wavelengths ranging from 400 nm to 1000 nm. The acquired raw spectra of each pixel were lamp-normalized using the Cytoviva software package (ENVI 4.8) and exported after normalization.
Processing of the hyperspectral images was done with Matlab (Mathworks, Natick, MA). Hyperspectral images were created by color-coding the spectrum by integrating over three bands. The band centers were 800.0 nm, 700.6 nm and 526.2 nm for the red, green and blue channels, respectively. The integration was weighted by a Gaussian window with a width of 80 spectrum samples. Each channel was scaled separately for optimal viewing.
Automatic detection of NPs required preprocessing the spectra prior to training and classification. First, due to noise at lower wavelengths, the spectra were truncated to disregard values below a cutoff of 566 nm. As part of HSM-AD, we initially segmented the image into background, tissue, or potential NPs based on each pixel’s average intensity across its spectrum. This segmentation allowed a more accurate calculation of the biodistribution by measuring the number of pixels that correspond to tissue inside the field of view. The segmentation of potential NPs helped to avoid classifying low intensity edges that may be falsely detected as NPs due to chromatic aberrations. Before measuring the intensity of each pixel, we also applied a correction for vignetting. Vignetting is a common artifact in photography and microscopy in which image brightness is reduced at the periphery compared to the image center. We assumed a natural illumination fall-off that follows the 'cosine fourth' law, in which the light fall-off is proportional to the fourth power of the cosine of the angle at which the light impinges on the sensor. We measured the radial falloff of several images and found that it can be approximated by , in which , where is the calculated distance from the center of the image and was found by fitting to be 2 mm. In order to correct the vignetting, we divided the intensities of each field of view by . Next, we calculated the segmentation thresholds adaptively for each image (to account for different exposure times and variations in tissue scattering). The thresholds were obtained by analyzing the histogram of pixel intensities of each image (Figure 1—figure supplement 2). The histogram was calculated with 510 bins and then re-sampled (using interpolation) every 5 intensity units. Pixels with the lowest intensities were segmented as background. The threshold for segmenting the background was calculated as the first minimum of the histogram (minHist) multiplied by a user-defined parameter (which is slightly larger than 1) to allow fine tuning of the background threshold. Next, we assumed that pixels representing tissue without NPs have a relatively consistent intensity lower than that of NPs and therefore correspond to a peak in the histogram. The threshold for differentiating between pixels that correspond to tissue and those that can be potential NPs can be determined by the peak of the histogram (peakHist) multiplied by a pre-defined parameter (larger than 1) which allows tuning of the threshold. This intensity-based segmentation was confirmed to be effective by comparing results over 20 separate fields of view from all analyzed tissue types. Pixels segmented as potential NPs were preprocessed by smoothing their spectra using a Savitzky-Golay algorithm (Orfanidis, 1995) implemented by a Matlab’s built-in function. Next, we normalized the spectrum of each pixel by the maximal intensity across its spectrum. Training and classification were performed only on pixels which were segmented as potential NPs.
Training of the k-means algorithm (Bishop, 1995) was initially done with 3, 4, and 5 clusters on 6 images of injected tissue sections, from which ~500,000 pixels were binned into the potential LGNR group. The 4 clusters found by the k-means algorithm matched the expected spectra of the injected and stained tissue. One of the spectra automatically matched the spectrum of LGNRs, owing to their distinct spectrum compared to tissue and staining dyes, two of the spectra represent the Hematoxylin and Eosin stains, and the fourth is an intermediate cluster representing the sum of both stains (see Figure 1—figure supplement 4). Testing the algorithm on uninjected tissue samples showed false detections of LGNRs near the edges of the tissue. We attributed these false positives to spectral red-shifting caused by chromatic aberrations due to the larger point spread function of longer wavelengths compared to the smaller point spread function of shorter wavelengths. To minimize the number of these false positives, we manually added a cluster representing the spectrum of the chromatic aberrations by averaging the spectra of falsely detected pixels from uninjected samples. Indeed, this cluster showed a spectral peak near the resonance of the LGNRs, albeit much broader. The initial clusters found by k-means with the added cluster representing chromatic aberrations were used for classification of pixels as LGNR+ or LGNR- with a nearest centroid (or nearest neighbor) classifier, based on the Euclidean distance to the cluster centers. We then measured the sensitivity and specificity and also qualitatively assessed the results on injected tissue section using the algorithm with different numbers of clusters. We chose to use the cluster library which includes 5 clusters (4 obtained through k-means and the 5th added manually) because it produced the best results and clusters that were more consistent with the actual spectra present in the samples. Several other machine learning algorithms were also explored for this purpose, including support vector machine (SVM) and logistic regression, but k-means yielded better results (Liba and Shaviv, 2014). The classification results are presented as detection maps in which the average intensity at each pixel is displayed in grayscale and the pixels that are LGNR+ are shown in orange. Similar training and classification steps were performed for samples containing GNS@SiO2 (3 clusters used) and Nanoshells (5 clusters for images of ex vivo tissues, 2 clusters for Nanoshells + LGNRs).
We characterized the sensitivity and specificity of HSM-AD by three methods. First, we measured the false positive and true negative rates in uninjected tissue samples to obtain the specificity. Note that false positives and true negative rates can be measured reliably from tissue-only samples due to the absence of LGNRs. Next, we measured the false negatives and true positives in an image of pure LGNRs in mounting media on a glass slide and obtained the detection sensitivity. Note that false negatives and true positives can be measured reliably from LGNR-only samples due to the absence of tissue and scattering media other than the particles themselves. In order to calculate the sensitivity and specificity for detecting LGNRs in tissue samples, we created a 'ground truth' data set by randomly choosing > 200 pixels, half of which were detected by the algorithm to contain LGNRs. We then manually (and in a manner blind to the results from the algorithmic classification) determined whether pixels were LGNR+ or LGNR- based purely on observing their raw spectra and looking for the unique plasmonic peak of LGNRs. Low-intensity pixels that were considered as the background or tissue were not considered in this calculation. By comparing the ground truth to the results of the automated algorithm we obtained the number of true positives, true negatives, false positives and false negatives, and used them to calculate additional measures of sensitivity and specificity. Confidence intervals for these measurements, presented in Figure 2—figure supplement 1, were calculated using the 'log method' (Altman et al., 2000) with the aid of a statistical calculator (MedCalc Software, 2016).
The biodistribution in each field of view was measured as the relative LGNR signal in an image. This measurement takes into account the amount of tissue versus background in a frame and also the signal of the detected LGNRs. For this calculation, we refer to LGNR signal as the average intensity around the plasmonic peak of the LGNRs (833–988 nm). The relative LGNR signal is the sum of the LGNR signal over LGNR+ pixels divided by the number of tissue pixels (i.e., the number of all pixels minus the number of background pixels) and divided by the median LGNR signal over the LGNR+ pixels in the field of view. For whole-organ analysis, we measured the relative LGNR signal in four fields of view for each organ (taken from the same mouse) and calculated the mean and standard deviation for each organ. For the sub-organ calculation, the measurement of relative LGNR signal yielded results with substantial variability due to a small number of pixels and high variability of intensity within several regions of interest. Therefore, a simpler pixel ratio (termed pixel coverage) was calculated by dividing the number of LGNR+ pixels by the number of tissue pixels for each region of interest (ROI). ROI maps are presented for each field of view in Figure 4 as well as for the additional images used for quantification (Figure 4—figure supplement 1). The same calculations also apply for quantification of other particle types.
In order to determine whether HSM-AD is able to detect single LGNRs, we compared the theoretical point spread function of the microscope’s 40x lens with the shape of the increased intensity caused by an isolated LGNR+ pixel. The diffraction-limited point spread function of the microscope can be approximated by a Gaussian with a standard deviation of 0.25 μm (based on a numerical aperture of 0.75 and wavelength of 910 nm) (Lipson and Lipson, 2010), which may increase in the case of defocusing. The cross section of intensity in the wavelengths around the plasmonic peak of the LGNRs (833–988 nm) showed a close resemblance to the theoretical spot size (Figure 2—figure supplement 8). This analysis supports the capability to detect single LGNRs in tissue samples.
Statistics with ConfidenceBMJ Books.
Detection, characterization, and abundance of engineered nanoparticles in complex waters by hyperspectral imagery with enhanced Darkfield microscopyEnvironmental Science & Technology 46:10081–10088.https://doi.org/10.1021/es204140s
Neural Networks for Pattern RecognitionClarendon Press.
Orientation sensing with color using plasmonic gold nanorods and assembliesJournal of Physical Chemistry Letters 3:2568–2574.https://doi.org/10.1021/jz3009908
Mass spectrometry imaging reveals the sub-organ distribution of carbon nanomaterialsNature Nanotechnology 10:176–182.https://doi.org/10.1038/nnano.2014.282
Assessment of the In vivo toxicity of gold nanoparticlesNanoscale Research Letters 4:858–864.https://doi.org/10.1007/s11671-009-9334-6
Summaries of the 9th JPL Airborne EarthScience Workshop9, Spectral correlation mapper (SCM): An improvement on the spectral angle mapper (SAM), Summaries of the 9th JPL Airborne EarthScience Workshop, JPL Publication.
Size and shape effects in the biodistribution of intravascularly injected particlesJournal of Controlled Release 141:320–327.https://doi.org/10.1016/j.jconrel.2009.10.014
Hyperspectral darkfield microscopy of single hollow gold nanoparticles for biomedical applicationsPhysical Chemistry Chemical Physics 15:4163–4168.https://doi.org/10.1039/C2CP43162A
Enhancing the toxicity of cancer chemotherapeutics with gold nanorod hyperthermiaAdvanced Materials 20:3832–3838.https://doi.org/10.1002/adma.200800921
Cancer cell imaging and photothermal therapy in the near-infrared region by using gold nanorodsJournal of the American Chemical Society 128:2115–2120.https://doi.org/10.1021/ja057254a
In vivo integrity of polymer-coated gold nanoparticlesNature Nanotechnology 10:619–623.https://doi.org/10.1038/nnano.2015.111
The spectral image processing system (SIPS)—interactive visualization and analysis of imaging spectrometer dataRemote Sensing of Environment 44:145–163.https://doi.org/10.1016/0034-4257(93)90013-N
Gold and silver nanoparticles in sensing and imaging: sensitivity of plasmon response to size, shape, and metal compositionJournal of Physical Chemistry B 110:19220–19225.https://doi.org/10.1021/jp062536y
Automatic detection of nanoparticles in tissue sectionsCS229 Machine Learning.
Optical PhysicsCambridge: Cambridge University Press.
Optimized Spectral Angle Mapper classification of spatially heterogeneous dynamic dune vegetation, a case study along the Belgian coastline2, The 9th International Symposium on Physical Measurements and Signatures in Remote Sensing, 1.
Diagnostic test evaluationDiagnostic test evaluation.
PEG-modified gold nanorods with a stealth character for in vivo applicationsJournal of Controlled Release 114:343–347.https://doi.org/10.1016/j.jconrel.2006.06.017
Introdction to Signal ProcessingPrentice-Hall.
Opsonization, biodistribution, and pharmacokinetics of polymeric nanoparticlesInternational Journal of Pharmaceutics 307:93–102.https://doi.org/10.1016/j.ijpharm.2005.10.010
Identification of metal oxide nanoparticles in histological samples by enhanced darkfield microscopy and hyperspectral mappingJournal of Visualized Experiments 106:e53317.https://doi.org/10.3791/53317
Hyperspectral microscopy as an analytical tool for nanomaterialsWiley Interdisciplinary Reviews: Nanomedicine and Nanobiotechnology 7:565–579.https://doi.org/10.1002/wnan.1330
Morphology, Biochemistry, PhysiologyAcademic Press.
Microchip electrophoresis with enhanced dark-field illumination detection for fast separation of native single super-paramagnetic danoparticlesBulletin of the Korean Chemical Society 36:1172–1177.https://doi.org/10.1002/bkcs.10219
Multiplexed screening of cellular uptake of gold nanoparticles using laser desorption/ionization mass spectrometryJournal of the American Chemical Society 130:14139–14143.https://doi.org/10.1021/ja805392f
Gaudenz DanuserReviewing Editor; UT Southwestern Medical Center, United States
In the interests of transparency, eLife includes the editorial decision letter and accompanying author responses. A lightly edited version of the letter sent to the authors after peer review is shown, indicating the most substantive concerns; minor comments are not usually included.
[Editors’ note: this article was originally rejected after discussions between the reviewers, but the authors were invited to resubmit after an appeal against the decision.]
Thank you for submitting your work entitled "A hyperspectral method to assay the microphysiological fates of nanomaterials with single-particle sensitivity" for consideration by eLife. Your article has been reviewed by three peer reviewers, and the evaluation has been overseen by Gaudenz Danuser as the Reviewing Editor and Sean Morrison as the Senior Editor.
Our decision has been reached after consultation between the reviewers. Based on these discussions and the individual reviews below, we regret to inform you that your work will not be considered further for publication in eLife.
As you will see from the full reviews below, all reviewers praise the quality of the study and its relevance and timeliness. However, all three reviewers raised from different angles the same concern: How new and versatile is the technique. Reviewer #1 provides a number of papers that your approach should be carefully compared to – potentially with additional validation experiments. Similarly, Reviewer #3, an expert in machine learning, is not convinced that the approaches taken are all state-of-the-art. Some cross-validation with other methods would help easing this concern for a future reader of a publication. Reviewer #2 raises the most critical point: although the claim of generality is made, the data presented relies on a single type of nano-particles. In the discussion among the reviewers following the initial evaluation it became clear that manuscript could only be considered if data from multiple nano-particles were shown. We suppose that acquisition of such complementary data will exceed the typical time eLife grants for a revision. Therefore, we have decided at this point to reject the manuscript. That said, there is significant merit in the combination of approaches, leading to a significant result. Thus, with a more thorough comparison of your approaches to others and a demonstration of applicability to other nano-particles we would consider a new submission of this work and make our best effort to send the manuscript back to the same editor and reviewers.
In their work the authors propose a technique for quantifying nanoparticles distribution in histological samples. Hyperspectral imaging is based on the collection of images containing spectral information across a large (relatively speaking) range of the electromagnetic spectrum. Typical applications are found for military, geoscience, or environmentally studies. Here for each pixel within an image for example the signal across the visible or infrared spectrum is collected and hyperspectral data cubes are built and analyzed. Because different objects possess different optical properties, knowing a priori specific signatures allow classifying the information present within a scene.
In recent years this technique has found quite some success within the biomedical imaging field. Because metallic nanoparticles scatter light quite strongly at specific "resonant" wavelengths, the combination of darkfield microscopy and an hyperspectral approach has made it possible to successfully detect nanoparticles at high imaging speed and to separate them from the cellular background. Also the technique offers greater resolution when compared to other tools for studying biodistributions.
In the manuscript the authors report specifically on an imaging processing method to quantify nanoparticles distributions in tissue samples. The imaging setup is a standard one from CytoViva for "enhanced darkfield microscopy" equipped with a CCD camera for hyperspectral imaging. Previous work in the field is already present in my opinion. See for example a review paper from Brenner's group detailing recent works ("Hyperspectral microscopy as an analytical tool for nanomaterials"). Different groups have also presented several works demonstrating single-nanoparticle detection. See for example "Single-nanoparticle detection and spectroscopy in cells using a hyperspectral darkfield imaging technique" and others from Musken's group, and also another nice paper from Meunier group "Hyperspectral darkfield microscopy of PEGylated gold nanoparticles targeting CD44-expressing cancer cells" where 3D nanoparticles tracking is also demonstrated.
Because the authors concentrate on ex vivo tissue sections I found strange that very recent intriguing work from Brenner's group is not cited considering it is focusing basically on a very similar subject (i.e. "Identification of Metal Oxide Nanoparticles in Histological Samples by Enhanced Darkfield Microscopy and Hyperspectral Mapping" on JoVE 2015). It would have been nice to see a discussion and analysis of this work and in which respect the presented work differs from the cited one.
Having said that, I found the paper very interesting and very well written. There is a lot of work and data are very compelling. Also I think it could be of great interest. My only concern deals with the novelty of it (specifically see the paper mentioned above from JoVE).
Also, because the paper deals with the development of a machine learning algorithm I found myself a little bit in difficulty giving a judgement in this regard because I'm not a specialist of this particular field and I'm familiar with only the most common and basic approaches. From what I've seen a lot of sophisticated work has been done for hyperspectral classification in areas outside from the biological one (e.g. recently deep-learning based approaches for feature extraction for hyperspectral imaging have been implemented from different groups). Therefore, I honestly cannot judge in this regard and perhaps maybe someone with a specific imaging processing background in the field of hyperspectral classification (not necessarily limited to biomedical microscopy, but a more broad one) could perhaps provide more insights into it and comment on the novelty of the computational approach used here.
Overall, this is an interesting paper that seems to have been executed with significant care. The figures are of high quality and the text is well written.
The authors use hyperspectral imaging to detect large gold nanorods (100 x 30 nm) in ex vivo tissues with single particle detection capabilities and single micron resolution.
They analyze histologically stained tissue slices from various organs, and the appropriate controls. From a technical standpoint, everything looks fine. I was expecting this technique to not work for aggregated NPs, especially in liver Kupfer cells, and they report exactly that, the honesty of which I appreciate.
Unlike many papers that provide descriptive multivariate models for their data, in this one, they use a training dataset to build the model, and then they use the model on unknown samples. This shows that the model has a high predictive power. I would be curious to see how well this method could distinguish between different types of plasmonic particles. Also, the particles used are ~30x100 nm2.
In my opinion the biggest weakness of this paper is that the technique is only demonstrated for one single nanoparticle type, and especially one that is not commonly used, i.e. large gold nanorods that are 100 x 30 nm in size.
However, in the Discussion the authors specifically claim that their technique is applicable to many nanoparticle shapes "(for example, gold nanospheres, nanorods, nanocages, etc.)" […]. "ABIDE is capable of distinguishing such NPs from each other by spectral differences, enabling biodistribution studies of multiplexed NPs." This is not only a bold claim, which importantly is not supported by any data.
The technique is only interesting enough for a broad audience if multiple different nanoparticle shapes such as spheres, cages, regular sized nanorods etc., can be analyzed with this method.
And my concern is that the technique will run into issues with discriminating some of the other shapes from cellular components, as can already be predicted when considering the curves in "Figure 1—figure supplement 4". The large gold nanorods have a plasmonic peak in the near infrared, which may be much easier to discriminate by the machine-learning algorithm than for shapes such as e.g. spheres.
The authors need to show convincingly that this technique works for the other nanoparticle shapes as they claim.
The paper presents methods and results of combining mathematical modeling, data analysis, hyper spectral imaging, related to imaging of nano materials with applications in cancer, angiogenesis, and others. I find this area really interesting, and the potential impact large, although tissue and cell physiology are not my area of expertise. Though I find the ideas very stimulating, the main issue that I have is that the mathematical modeling/data analysis used here is not very clear, and may not be near top shelf work. Combined with certain choices of validation expanded below, my understanding is that the contributions to imaging methodology is not sufficiently novel, nor inspire a lot of confidence that results are as best as possible. See below for detailed comments.
Regarding modeling and data analysis, a few questions. The topic of discerning the contributions of different elements (in this case nano materials versus others such as Eosin, Hematoxylin, etc.) from spectral measurements is a well studied one. While I understand the intricacies of this imaging experiment are not the same as other more well studied spectral unmoving problems, I'm not convinced that the wealth of other methods for linear and nonlinear unmixing don't apply here. In the paper I did not find any discussion related to this. Classification methods of the type authors claim to have attempted (nearest neighbor, SVMs, etc.) could be applied on such unmixed data, and to me this would constitute a more standard way of doing things. As is, the methodology regarding data analysis coupled to imaging does not seem very novel, and it is not presented in a way that can be related to many other already proposed methods to other seemingly similar applications.
Regarding the data analysis for validation, a couple of things seemed unclear to me. For the detection of false positives and false negatives, were these evaluations performed separately? And if so, is the validation criterion computed by checking whether there is one single pixel indicating nonmaterial (in case of false positives) present? Similar comments for the reverse situation (false negatives). It would be necessary to know the specifics of these details better, and even better have the data (e.g. histograms and if pixel counts are used, how thresholds are utilized). Also, how is the user defined parameter (it seems a manually selected threshold is used to initially determine if a pixel is potentially LGNR) handled during validation? Is data used in the validation stage also used by the user when selecting this parameter? It is a little confusing when in the subsection “Data processing and automatic biodistribution detection” the authors comment "We then manually (and in a manner blind to algorithmic classification)…". How is this possible given that for a positive detection the pixel must be classified as potential LGNR first? Am I missing something, or is it possible that authors are mixing training and testing data?
A few more comments below:
The Results section reads more like a Methods section. If format is to be followed, I'd suggest only describing results in the Results section.
Results, second paragraph: K-means is most commonly referred as a clustering algorithm. Once clusters are identified, one can use this information to design a multitude of classification methods, but authors must specify which they used. Presumably the simplest would be the nearest neighbor cluster center. Are the cluster centers utilized as ground truth?
[Editors’ note: what now follows is the decision letter after the authors submitted for further consideration.]
Thank you for resubmitting your work entitled "A hyperspectral method to assay the microphysiological fates of nanomaterials with single-particle sensitivity" for further consideration at eLife. Overall, your revised article has been favorably evaluated by Sean Morrison (Senior editor), a Reviewing editor, and three reviewers.
The manuscript has been improved but there are some remaining issues that need to be addressed before acceptance. They mainly concern the tendency of 'over-selling' some of the approaches and accomplishments of the work. We especially encourage you to reconsider the use of an acronym for this technology. We are concerned about the use of acronyms as cheap eye-catchers. Moreover, a new expert in machine learning has been brought to the panel because previous Reviewer #3 was unavailable. Given the rapid pace of the machine learning field it seems inappropriate to call the application of a clustering method (and even a very classic one, which is taught in undergraduate computer science course) 'machine learning'. It is perfectly fine to use standard methods if they solve the task, but there is a discomfort among reviewers that machine learning is merely used as a catchy phrase in this case. This could fire back on your work. We thus encourage you to revise some of the language in your manuscript, including the Abstract, and to address the few other comments listed below.
I've found that the authors’ comments are very appropriate and to the point even though in some cases I'm not too familiar with some of the points discussed. My major concern is always related to my original comment. Previous similar work from Brenner using a basically identical setup hardware is already present in the literature. Because the scientific contribution of the submitted paper consists in proposing an analysis procedure based on a machine learning algorithm with the intent of extending the work of Brenner and co-workers and others, it is critical to determine the novelty of the proposed algorithm. Overall I found the data provided by the authors very compelling and particularly interesting.
The authors are now showing feasibility data from two other nanoparticle shapes, and have therefore satisfactorily addressed my previous main concern whether ABIDE may be versatile enough with regards to different nanoparticle shapes and sizes.
There are a few remaining issues.
General comment: The revised version does not include any tracked changes or other markings to indicate where changes were made, which made the review of the paper quite difficult. I am making this comment not because I want to review another version with track changes, but to make it clear that this limitation may have reduced my ability to catch all remaining or new issues.
1) In the PowerPoint slides the authors provided in their rebuttal (for the reviewers only), it says "Example of new in vivo data". I find this misleading, as I could not find any data in the entire paper that was acquired "in vivo". A reviewer who does not carefully examine the manuscript may be misled by the rebuttal summary slides that this is in fact all acquired in vivo and not catch this discrepancy between the summary and the actual paper. I am not expecting the authors to provide true in vivo data, but would like to clarify what the authors meant by that.
2) The title of the paper is overstated with regards to claiming "single particle sensitivity" and this needs to be changed or else would be misleading. Figure 2—figure supplement 8 is the only figure that shows any data that would support that, and only in the very large gold nanorods (which by some definitions would not represent a nanoparticle). "Likely" as is stated in the figure legend is probably an honest assessment by the authors, but not enough to make such a major claim, and there is no evidence for this to work in the other two nanoparticles that are now included. I suggest replacing "…with single particle sensitivity" with "… in tissue sections" or "… in histological slices".
I would be willing to accept the manuscript pending these clarifications/changes if the other reviewers agree that their areas of expertise were addressed sufficiently as well.
This revised manuscript describes an interesting and straightforward development of an approach for detecting nanoparticles in hyperspectral dark field images. It has been significantly improved based on comments in the initial reviews. The results presented demonstrate an impressive ability to detect and quantify these nanoparticles in tissue images. From an image processing/analysis/machine learning point of view, the approach is not novel or instructive (clustering spectra, especially with manual tuning, barely qualifies to be called machine learning, and does not seem to warrant a new acronym). Hence the significance of the manuscript must derive from the future importance of the method's application, something this reviewer is not qualified to judge.https://doi.org/10.7554/eLife.16352.050
- Elliott D SoRelle
- Orly Liba
- Cristina L Zavaleta
- Cristina L Zavaleta
- Adam de la Zerda
- Adam de la Zerda
- Adam de la Zerda
- Adam de la Zerda
- Adam de la Zerda
- Adam de la Zerda
- Adam de la Zerda
- Adam de la Zerda
- Adam de la Zerda
- Adam de la Zerda
- Adam de la Zerda
- Adam de la Zerda
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
This work was funded in part by grants from the Claire Giannini Fund, the United States Air Force (FA9550-15-1-0007), the National Institutes of Health (NIH DP50D012179), the National Science Foundation (NSF 1438340), the Damon Runyon Cancer Research Center (DFS# 06-13), the Susan G Komen Breast Cancer Foundation (SAC15-00003), the Mary Kay Foundation (017-14), the Donald E and Delia B Baxter Foundation, the Skippy Frank Foundation, a seed grant from the Center for Cancer Nanotechnology Excellence and Translation (CCNE-T U54CA151459), and a Stanford Bio-X Interdisciplinary Initiative Seed Grant. AdlZ is a Pew-Stewart Scholar for Cancer Research supported by The Pew Charitable Trusts and The Alexander and Margaret Stewart Trust. EDS wishes to acknowledge funding from the Stanford Biophysics Program training grant (T32 GM-08294). OL is grateful for a Stanford Bowes Bio-X Graduate Fellowship. CLZ is supported by the National Cancer Institute of the National Institutes of Health under award numbers K22 CA160834 and R21 CA184608. JLC acknowledges funding from the Victorian government of Australia in the form of a Victorian Postdoctoral Research Fellowship. We would like to thank Debasish Sen, Byron Cheatham, Jamie Uertz, and Michelle Rincon for technical assistance. We wish to thank Dor Shaviv for help with testing alternative detection methods.
Animal experimentation: All animal experiments in this study were performed in compliance with IACUC guidelines and with the Stanford University Animal Studies Committee's Guidelines for the Care and Use of Research Animals (APLAC Protocol #27499 and #29179).
- Gaudenz Danuser, Reviewing Editor, UT Southwestern Medical Center, United States
© 2016, SoRelle et al.
This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.