Introduction

Precise intraoperative cancer diagnosis is crucial for achieving optimal patient outcomes by enabling complete tumor removal. The standard method is the microscopic cellular examination of surgically excised specimens following various processing steps, including thin sectioning and hematoxylin and eosin (H&E) cell staining. However, this examination method is laborious and time-consuming. Furthermore, it has inherent artifacts that disturb accurate diagnosis, including tissue loss, limited two-dimensional (2D) information, and sampling error [1]. High-speed three-dimensional (3D) optical microscopy, which can visualize cellular structures without thin sectioning, holds promise for nondestructive 3D pathological examination as a complement of 2D pathology limitation [1–4]. Various high-speed 3D microscopy techniques have been developed [5–12], and open-top light sheet microscopy (OT-LSM) is a promising technique that enables high-throughput imaging with light sheet illumination and planar imaging on uneven tissue surfaces [13–16]. OT-LSM is a specialized form of LSM for large tissue specimens such as biopsy specimens with all the optical components placed below the sample holder. However, fluorescence 3D microscopy techniques including OT-LSM have shallow imaging depths in turbid tissue owing to light scattering and absorption, limiting the applicability of 3D histopathology. As a deep tissue imaging method, two-photon microscopy (TPM) has been used in both biological and optical biopsy studies [17–19]. TPM is based on nonlinear two-photon excitation of fluorophores and achieves high imaging depths down to a few hundred micrometers by using long excitation wavelengths, which reduce light scattering. Moreover, TPM provides additional intrinsic second harmonic generation (SHG) contrast for visualizing collagen fibers within the extracellular matrix (ECM). This feature proved advantageous for high-contrast imaging of cancer tissue and microenvironmental analysis [20–22]. However, TPM has low imaging speeds due to point scanning-based imaging. To address this limitation, two-photon LSM (TP-LSM) techniques were developed for high-speed imaging [23–27]. Although TP-LSM facilitated rapid 3D imaging of cancer cells and zebrafish, its applications were limited to small samples and biological studies due to geometric limitations. Conventional TP-LSM had a configuration of a horizontally oriented illumination objective and a vertically oriented imaging objective. This geometry imposed limitations on the sample size, rendering it unsuitable for the examination of centimeter-scale specimens. TP-LSM with open-top configuration is needed for 3D histological examination.

In this study, we present the development of an open-top TP-LSM (OT-TP-LSM) with a Bessel beam for high-throughput and high-depth 3D pathological examination of large tissue specimens. A nondiffractive Bessel beam was used and scanned in one-dimensional (1D) to generate an extended depth of field (EDOF) excitation light sheet with minimal side lobe effects [28, 29]. High imaging depths were achieved by using long excitation wavelengths and fluorescent nuclear probes emitting relatively long wavelengths. After development and characterization, the system was applied to high-throughput 3D imaging of various human cancer specimens, including skin, pancreas, and prostate in comparison with conventional H&E stained histological images for verification. Moreover, OT-TP-LSM images were converted to virtual H&E images using the deep learning-powered style transfer. We demonstrated that OT-TP-LSM images were comparable to histological images of H&E stained slides from several human cancer types, and OT-TP-LSM 3D imaging might have the potential for rapid and accurate nondestructive 3D pathology.

Results

Open-top two-photon light sheet microscopy (OT-TP-LSM) for 3D pathology

Schematics and characterization results of OT-TP-LSM are presented in Figure 1. OT-TP-LSM is an orthogonally arranged dual objective LSM with a liquid prism interface (Figure 1A). All optical components of OT-TP-LSM were positioned underneath a sample holder to accommodate large size specimens without physical interference with the illumination and imaging optics. Two 20x air objective lenses with a moderate numerical aperture (NA) of 0.42 were used to deliver a thin excitation light sheet and to collect emission light efficiently. The custom liquid prism was used to transmit excitation light into the specimen and to collect emission light out with minimal aberration by refractive index (RI) matching with different mediums where the sample was immersed (Figure 1B). The liquid prism was filled with diluted glycerol in distilled water, whose RI was matched to that of a quartz (RI: 1.45) window in a sample holder. A Ti-Sapphire laser beam was expanded and converted to a Bessel beam using an axicon lens. The Bessel beam was reflected on a galvanometer scanner and relayed to the sample through the liquid prism by a combination of a tube lens and the illumination objective lens. The scanner scanned the Bessel beam to form an EDOF excitation light sheet. Emission light generated by the sample was collected by the imaging objective lens through the liquid prism, split into two channels, one for fluorescence and the other for SHG, using a dichroic mirror, and imaged simultaneously by two scientific complementary metal-oxide–semiconductor (sCMOS) cameras with individual gain control.

Figure 1.

The excitation light sheet was designed to have a DOF of 180 μm with a thickness of 0.9 μm (Figure 1 - figure supplement 1). The imaging field of view was 600 μm × 170 μm with 1,850 × 512 pixels and was limited by the DOF of the Bessel beam. Emission light scattering in the specimen further limited the imaging depth. Selective planar images along the oblique excitation light sheet were acquired sequentially with a stepwise X-axis translation of the specimen by the motorized stage (Figure 1C). A graphical user interface based on LabVIEW was used to generate trigger signals for both the cameras and the motorized stage. OT-TP-LSM was characterized by imaging fluorescent microspheres (0.5 μm in diameter) embedded in agarose gel, whose RI was matched to that of the solution in the liquid prism (Figure 1D). The image throughput was 0.24 mm²/s at an acquisition rate of 400 fps and step size of 1 μm. The image resolution was measured to be 0.9 μm in all dimensions. The lateral resolution in the Y-Z’ plane was approximately 128% of the theoretical value due to spherical aberration induced by the RI mismatch between air and the immersion medium, but this was sufficient for detecting cellular features.

The imaging depth of OT-TP-LSM was evaluated using freshly excised human skin specimens. Proflavine was topically instilled onto the specimen for nuclear labeling before imaging. A representative Y-Z’ plane image is shown in Figure 1E. The imaging depth in the skin was approximately 90 μm in the 45°-tilted image plane, and the actual depth was 64 μm from the surface. To compare the imaging depth, the same skin specimen was imaged using conventional one-photon excitation-based LSM (1PLSM). The imaging depth was approximately 28 μm in the 45°-tilted image plane and 20 μm from the surface, which was approximately three times less than that of OT-TP-LSM. While the imaging depth of OT-TP-LSM was sufficient for visualizing 3D cell structures in the skin, it was relatively low compared to that of conventional point-scanning based TPM due to emission photon scattering.

3D OT-TP-LSM imaging of human skin cancers

After development and characterization, the performance of OT-TP-LSM was assessed in human skin cancer specimens. Representative OT-TP-LSM images of two basal cell carcinoma (BCC) specimens are presented along with H&E stained histology images in Figure 2. The H&E images were obtained from the same specimens at slightly different depths. The OT-TP-LSM images visualized both cell structures and ECM in the cross-sectioned skin specimens, where proflavine-labeled cell nuclei and SHG-emitting collagen were depicted in green and blue, respectively. A mosaic image of the first BCC specimen visualized the epidermis as a thin layer on the top and the underlying dermis, most of which was occupied by BCC nests (Figure 2A). The epidermis and dermis were easily identifiable with SHG contrasts. Two regions of interest (ROI) were selected in the epidermis and BCC, and magnified images of these ROIs are presented to show the detailed cell structures. Magnified image in the epidermis (ROI 1) showed layered keratinocytes (Figure 2B). Keratinocytes in the basal layer were relatively large and individually resolved, while those in the upper layers were unresolved and appeared as a band. It could be attributed to the upper layers being comprised of flat cells with relatively small cytoplasm, resulting in little space between nuclei. Additionally, strong autofluorescence signal in the stratum corneum might prevent visualization of the cells in the superficial layer. Magnified images of BCC (ROI 2) showed many irregularly shaped tumor nests composed of densely packed monomorphic tumor cells of the same size (Figure 2C).

Figure 2.

The mosaic image of the second BCC specimen mainly showed two different cell structures in the dermis (Figure 2D). The structures on the right side were BCCs, while those on the left side had intact ECM composition and differed from BCCs. The corresponding H&E stained histology image confirmed these as BCCs and normal gland structures in the dermis, respectively. Two ROIs were selected in the nontumorous skin region, and magnified images at two different depths are presented along with their corresponding H&E stained images. Magnified images of ROI 3 visualized clusters of small round tubular structures (Figure 2E). The image at depth 2 showed a ductal structure composed of single-layered cuboidal epithelial cells, which is typical of eccrine glands. Magnified images of ROI 4 at two different depths showed pear-shaped clusters consisting of cells with clear and bulging cytoplasm, which is typical of sebaceous glands (Figure 2F). Using 3D visualization, normal glandular structures in the dermis were distinguished from BCC tumor nests (Video 1). Both eccrine and sebaceous glands could appear similar to BCC nests in 2D images at certain depths. Hence, nondestructive 3D visualization of cell structures would be important for distinguishing them, serving as a complement to the traditional 2D H&E images.

3D OT-TP-LSM imaging of human pancreatic cancers

We then used OT-TP-LSM to visualize human pancreas specimens including those with pancreatic cancer. Instead of proflavine, propidium iodide (PI) was used as a nuclear labeling agent to increase the imaging depth (Figure 3 - figure supplement 1). The proflavine-labeled pancreas specimens had a relatively shallow imaging depth (up to 30 µm approximately) owing to the dense desmoplastic stromal cell composition. PI labeling allowed for deeper imaging up to approximately 55 µm from the surface due to the longer peak emission wavelength of 640 nm compared to proflavine. PI cytoplasmic labeling also occurred in addition to nuclear labeling of both acinar and neoplastic cells, probably owing to PI labeling of both DNA and RNA (Figure 3 - figure supplement 2) [30, 31]. Cell nuclei were identified with relatively strong fluorescence in the membrane and weaker fluorescence in the core. Representative OT-TP-LSM images and corresponding H&E stained slide images of normal pancreas and premalignant lesion specimens, and a pancreatic cancer (also known as pancreatic ductal adenocarcinoma, PDAC) specimen are shown in Figures 3 and 4, respectively. The mosaic image of the normal pancreas visualized a dense distribution of acini composed of exocrine acinar cells in good correlation with the corresponding H&E image (Figure 3A). Two ROIs were selected, and magnified images are presented to show detailed cytologic features of the normal pancreas. Magnified images of ROI 1 visualized the cellular structure of acini and islets of Langerhans expressing different fluorescence intensities (Figure 3B). Acini were composed of lobules of acinar cells, and the boundaries of the acini were identified with relatively low PI fluorescence. The islets of Langerhans expressed relatively low PI fluorescence, probably due to the high composition of crystallized insulin in beta cells [32]. Magnified images of ROI 2 visualized a duct in the normal pancreas (Figure 3C). The duct was composed of single-layered cuboidal epithelial cells with round nuclei and was surrounded by dense fibrous tissue expressing SHG. The mosaic image of the pancreatic premalignant lesion exhibited different and irregular structures, including dilated ducts with intraductal proliferation of papillary structures (Figure 3D). Two different types of precursor lesions, intraductal papillary mucinous neoplasm (IPMN) and pancreatic intraepithelial neoplasia (PanIN), were observed [33, 34], and tumor cells were tall columnar containing intracytoplasmic mucin [35]. A corresponding H&E histology image confirmed the precursor lesions. Magnified images of PanIN (ROI 3) at two different depths showed proliferation of papillary epithelial cells with intracytoplasmic mucin, and 3D morphology of the PanIN (Figure 3E). OT-TP-LSM images were very similar to the corresponding H&E images, indicating OT-TP-LSM provided comparable histopathological information to that of H&E stained slides without thin sectioning.

Figure 3.

Figure 4.

A mosaic image of the PDAC arising from the IPMN specimen showed various abnormal ductal structures including cancer glands, IPMN, and surrounding pancreatitis, which were confirmed by the corresponding H&E stained slide image (Figure 4A). More desmoplastic fibrous stroma and a few abnormal cancer glands were observed compared to normal pancreas tissue [36, 37]. Two ROIs were selected in the PDAC region and the boundary region with nonneoplastic pancreatitis, and magnified images are presented to show detailed cellular structures. Magnified images of ROI 1 (PDAC) at two different depths showed irregularly shaped glands with sharp angles and 3D structural complexity including unstable bridging structure inside (Figure 4B). An irregular and distorted architecture amidst desmoplastic stroma is one of the important diagnostic factors for PDAC [35]. The cancer glands exhibited disorganized cancer cell arrangement with nuclear membrane distortion. Magnified images of ROI 2 showed both nonneoplastic ducts and cancer glands in different cell arrangements (Figure 4C). The nonneoplastic ducts showed single-layered epithelium with small, evenly distributed cells expressing relatively high nuclear fluorescence. Cancer glands, on the other hand, had disorganized and multilayered structure with large nuclei. OT-TP-LSM visualized the 3D invasiveness of cancer glands within tissues nondestructively, which could not be identified from limited 2D information.

3D OT-TP-LSM imaging of human prostatic cancers

OT-TP-LSM was used to visualize human prostatectomy specimens, and two representative OT-TP-LSM images of benign and adenocarcinoma specimens are presented in Figure 5. The mosaic image of the histologically confirmed benign prostate showed benign glands dispersed throughout the specimen with relatively weak SHG composition between the glands (Figure 5A). An ROI was selected on benign glands, and magnified images are presented to show cytologic features alongside the corresponding H&E stained slide image (Figure 5B). Multiple cell layers, consisting of secretory cells and basal cells, lined the central lumen in the ducts of benign glands. Although images at two different depths are presented, the cell distribution was similar because of the relatively small depth difference compared to the gland size. The mosaic image of the prostate adenocarcinoma in Figure 5C showed both the benign and cancer regions, whose boundary was marked with a white solid line, and they were confirmed by the corresponding H&E stained image. The cancer region was full of crowded small gland structures, whereas the benign region consisted of relatively large glands surrounded by fibrous stroma. Magnified images of ROI 2 in the cancer region showed small glands composed of monolayered cell walls (Figure 5D). The cancer glands had single-layered malignant secretory cells only due to the loss of basal cell layers. The corresponding H&E histology was diagnosed as Gleason score 3+3 prostatic adenocarcinoma. OT-TP-LSM provided histological 3D information equivalent to that of the H&E stained image without the need for sectioning.

Figure 5.

Deep learning-based style transfer for virtual H&E

OT-TP-LSM images visualized histological features with high similarity to those in H&E images, but they may be unfamiliar to pathologists and clinicians accustomed to traditional H&E stained slide histology. PI-labeled OT-TP-LSM images were converted to virtual H&E images using Cycle-consistent generative adversarial network (CycleGAN) [38]. Representative virtual H&E images of pancreatic premalignant (PanIN) and PDAC are presented along with the corresponding real H&E stained images for comparison in Figure 6. Mosaic virtual H&E images of both PanIN and PDAC showed similar patterns to the corresponding real H&E stained histology images (Figure 6A, C). Although the normal region in the lower left corner of the virtual PanIN image was not transformed properly possibly due to insufficient image data of normal acini for training (Figure 6A), the magnified virtual H&E images clearly displayed distinct cell distributions of PanIN and PDAC by labeling cell cytoplasm and nuclei in pink and purple, respectively (Figure 6B, D). PanINs were visualized as slightly dilated, irregularly clustered ducts with preserved structural integrity (Figure 6B), whereas PDAC had disorganized cell arrangement with haphazard patterns, infiltrating into the surrounding tissue with desmoplastic stroma (Figure 6D). Histological features found in the virtual H&E images were diagnostic factors for the discrimination of PDAC and PanIN. Virtual H&E staining of OT-TP-LSM via CycleGAN could provide comparable cellular information to conventional H&E stained histology.

Figure 6.

Discussion

OT-TP-LSM was developed for the rapid and precise nondestructive 3D pathological examination of excised tissue specimens during both biopsy and surgery, as a compliment to traditional 2D H&E pathology by visualizing 3D cell structures. OT-TP-LSM generated a two-photon excitation light sheet using one-dimensional Bessel beam scanning, and planar images were acquired by collecting emission light from the excitation light sheet with sCMOS cameras at 400 fps maximum. Proflavine and PI were used for nuclear labeling and cell structure visualization, and intrinsic SHG contrast was used for ECM visualization. The performance of OT-TP-LSM was characterized in comparison with histopathologic images of H&E stained slides in various human cancer specimens including skin, pancreas, and prostate. 3D OT-TP-LSM images were well matched with the corresponding 2D H&E histological images in all the human tissue specimens evaluated. The relatively high imaging depths of OT-TP-LSM enabled the nondestructive visualization of detailed 3D cell structures with high contrast and without distortion and allowed a distinction between cancer and normal cell structures as well as the detection of cancer invasiveness within tissues. These have been challenging with 2D histological images. The high-quality images of OT-TP-LSM might be partially attributed to the use of the nondiffractive Bessel beam for the generation of an EDOF excitation light sheet [39].

OT-TP-LSM could visualize 3D cell structures in the skin down to 60 μm deep from the surface by using long wavelength excitation light and proflavine labeling. Although the imaging depth was not high compared to the one of conventional TPM, it was sufficient to visualize 3D cell structures in the skin without sectioning artifacts. The imaging depth varied depending on the tissue type, and the imaging depths of both pancreas and prostate were relatively low in comparison to that of the skin. This low imaging depth could be attributed to the dense cell composition and higher RI than the current immersion medium. To increase the imaging depth, PI with a relatively long emission wavelength was used. Additionally, an immersion medium with 1.5 RI could improve the imaging depth [40], and a quick optical clearing method could also be used to enhance the imaging depth [41].

We trained and applied CycleGAN to PI-labeled OT-TP-LSM images of pancreas specimens to generate virtual H&E images, and the cytoplasm and cell nuclei, which exhibited intensity differences in greyscale, were transformed to H&E-like pseudo colors. Virtual H&E images in pancreatic cancer provided histological features equivalent to the conventional H&E. Although CycleGAN showed the potential for virtual H&E staining, further development is needed to have robust performance and universal application. Therefore, CycleGAN will be trained by using more OT-TP-LSM images from various tissue types such as skin and prostate.

The current OT-TP-LSM has several limitations. One issue is spherical aberration arising from the RI mismatch between the liquid prism and air by using air objective lenses in both illumination and imaging arms. Spherical aberration degraded the image resolution, particularly in the imaging arm. An immersion objective lens with a high NA would be desirable to improve both image resolution and emission light collection efficiency. The excitation light sheet was produced by swiftly scanning a single Bessel beam, yielding a shorter pixel dwell time compared to that of a traditional single-photon 2D light sheet. Employing multiple Bessel beams for excitation light sheet generation would enhance pixel dwell time and image contrast [42, 43]. The relatively short excitation wavelength of 770 nm was used for all the imaging to efficiently collect intrinsic SHG light [20], but the imaging depth was limited. Longer excitation wavelengths with long emission wavelength probes would increase the imaging depth beyond those presented here. Although OT-TP-LSM enabled high-speed 3D imaging, the post-processing time of the OT-TP-LSM image datasets was relatively long due to the large data size, sequential processing of dual channel images, and manual stitching. The long post-processing time needs to be resolved for intraoperative applications. To speed up processing, these processing steps can be performed using field-programmable gate array (FPGA)-based data acquisition with graphics processing unit (GPU)-based computing. The processing time can be further reduced by coding the algorithm in a C++-based environment. Furthermore, ImageJ-based software such as the Bigstitcher plugin can be used for automatic 3D image processing [44].

In conclusion, OT-TP-LSM was developed for high-throughput and high-depth tissue examination. OT-TP-LSM was tested with various human cancers including skin, pancreas, and prostate by visualizing 3D cell structures. Intrinsic SHG contrast was used for visualizing ECM and detecting fibrosis. Normal glands and BCC nests were clearly distinguished, and both pancreatic and prostate cancers were clearly identified using 3D visualization of cell structures with OT-TP-LSM. Furthermore, virtual H&E staining of OT-TP-LSM using CycleGAN provided histopathologic information comparable to that from H&E stained slides. Therefore, OT-TP-LSM could be useful for rapid and precise nondestructive 3D pathology.

Materials and methods

OT-TP-LSM setup

The schematics of OT-TP-LSM are shown in Figure 1A. As the light source, OT-TP-LSM used a titanium–sapphire pulse laser (Chameleon II, Coherent) with 140-fs pulse width and 80-MHz pulse repetition rate. The excitation wavelength was set at 770 nm for imaging human tissue specimens. In the illumination arm, the laser beam was expanded from 1.4 mm to 1.86 mm and then converted to a Bessel beam by using an axicon lens (131-1270 + ARB825, Eksma). The Bessel beam was relayed onto the specimen through two lens pairs of L1 (AC254-045-B, Thorlabs) and L2 (AC254-080-B, Thorlabs), and L3 (AC254-150-B, Thorlabs) and a 20× air illumination objective lens with 0.42 NA (MY20X-804, Mitutoyo). The Bessel beam diameter at the back aperture of the objective lens was approximately 6.8 mm to prevent beam clipping, and the effective illumination NA was 0.34. The excitation light sheet was generated by Y-axis scanning of the Bessel beam with a galvanometric scanner (GVS102, Thorlabs), which was positioned between L1 and L2. The liquid prism transmitted the excitation light normally through a coverslip (HSU-0101122, Marienfeld Superior), RI-matching solution, and a 0.25 mm thick round quartz window (26043, Ted Pella) at the bottom of the sample holder, and then to the sample (Fig. 1B). Diluted 85% glycerol (Samchun) in distilled water was used for RI-matching with the quartz window (RI: 1.45). The sample holder was connected to an x-y motorized translation stage (XY MS-2000, Applied Scientific Instrumentation) for sample translation during imaging. To remove air gaps between the quartz window of the sample holder and irregular-shaped tissue specimen, the specimen was partially immersed in the RI-matching solution and light pressure was applied on the top of the specimen. In the imaging arm, emission light was collected by the imaging objective lens (MY20X-804, Mitutoyo) through the liquid prism, spectrally separated into two channels by a dichroic mirror (T425lpxr, Chroma), and collected at two sCMOS cameras (pco-edge 4.2, PCO) after passing through a tube lens (AC254-200-A, Thorlabs), respectively. SHG light was collected in the short pass channel with an emission filter (ET376/30x, Chroma), and fluorescence was collected in the long pass channel with an emission filter (ET680SP, Chroma). Both fluorescence and SHG images were simultaneously acquired with individual gain control.

Human skin cancer specimen collection and imaging protocol

Fresh skin cancer specimens were collected under the Institutional Review Board (IRB) of Yonsei University, Severance Hospital, approved protocol (approval # 4-2021-0128). The study was conducted according to the Declaration of Helsinki Principles. OT-TP-LSM imaging was conducted within 12 h post-excision. Patient information was deidentified. Before imaging, fresh skin specimens were cut cross-sectionally to approximately 4 mm thickness and topically instilled with 400 µg/mL proflavine solution (Sigma) for 5 min, and then rinsed in phosphate-buffered saline. OT-TP-LSM imaging was conducted in 3D on the sectioned surface. After imaging, the skin specimens were fixed and processed for histopathological evaluation with H&E stained slides. Microscopic images were obtained by using a wide-field microscope (Leica Z16 APO, Leica Microsystems).

Human pancreatic and prostate cancer specimen collection and imaging protocol

Formalin-fixed, paraffin-embedded (FFPE) tissue blocks of pancreatic and prostate cancers were collected under approval (approval #, 2021-0393) from the IRB of Asan Medical Center, University of Ulsan College of Medicine. 4 μm thickness sections were made for H&E stained slides, and the remaining blocks were used for OT-TP-LSM imaging. The FFPE tissue blocks were deparaffinized with xylene (99%, Daejung) solution overnight, 1:1 xylene-100% methanol (Optima grade, Fisher scientific) mixture solution for 1 h with shaking, followed by sequentially decreasing the methanol concentration in the xylene and methanol mixture to 90%, 80%, and 50% for 1 h each. After deparaffinization, specimens were stained with 80 µg/mL PI solution (Sigma) for 5 min, and then OT-TP-LSM imaging was conducted on the sectioned surface.

Image acquisition and post-processing

Raw image datasets from dual sCMOS cameras were acquired and processed on a workstation with 128lJGb RAM and a 2lJTB SSD drive. The imaging time and data size per 1cm² area with 400 fps was 7 min and 318 GB (= (7 × 60) s × 400 fps × (1850 × 512 × 2) byte) for each channel, respectively. The raw image strip was sheared at 45° with respect to the sample surface, and a custom image processing algorithm was used to transform the image data in the XYZ coordinate. The processing for en-face image was conducted in MATLAB and took ∼1.7lJslJGb⁻¹ after loading the image dataset at ∼6.8lJslJGb⁻¹ in the current laboratory setting. Mosaic images were generated by joining the image strips manually.

Virtual H&E staining of OT-TP-LSM via deep learning network

CycleGAN is an unsupervised deep learning network without the need for well-aligned image pairs [38] and was used for style transfer of OT-TP-LSM images to virtual H&E images [45–47]. CycleGAN model consisted of two generators (G: real OT-TP-LSM → fake H&E and F: real H&E → fake OT-TP-LSM), and two corresponding discriminators (D_OT-TP-LSM and D_H&E) to identify each fake image (Figure 6 - figure supplement 1). Cycle-consistent loss was combined with adversarial loss to prevent the well-known problem of mode collapse, where all input images mapped to the same output images [38]. For CycleGAN training, PI-labeled OT-TP-LSM and real H&E images were collected from pancreas specimens, including each of normal, premalignant lesions, and cancer. Collected images were cropped into 512 × 512 pixels patch images, and augmented by applying rotation (90°, 180°, 270°) and flip (vertical, horizontal). The total training dataset was 14,322 and 22,644 patches for OT-TP-LSM and H&E, respectively. Adam solver was used as an optimizer [48]. Hyper-parameters were set with a batch size of 1 and an initial learning rate of 0.0002 and 100 epochs. Once the training was completed, generator G was used to transform OT-TP-LSM patches into virtual H&E ones. The CycleGAN training and testing were performed using a Nvidia GeForce RTX 3090 with 24lJGB RAM. The network was implemented using Python version 3.8.0 on a desktop computer with a Core i7-12700K CPU@3.61lJGHz and 64lJGB RAM, running Anaconda (version 22.9.0). The inference time for converting OT-TP-LSM patch image into virtual H&E patch image was measured as 160 ms.

Acknowledgements

This work was supported in part by the National Research Foundation (NRF) grant funded by the Korea government (NRF-2020R1A2C3009309); and the Korea Medical Device Development Fund (KMDF) grant funded by the Korean government (Project Number: 9991006749, KMDF_PR_20200901_0076).

Conflict of interest statements

No conflicts of interest were declared.

Supplementary figure legends

Figure 1 - figure supplement 1.

Figure 3 - figure supplement 1.

Figure 3 - figure supplement 2.

Figure 6 - figure supplement 1.

Video legend

Video 1. Three-dimensional (3D) visualization of basal cell carcinoma (BCC) structures and normal cell structures.