Deep-learning-based three-dimensional label-free tracking and analysis of immunological synapses of CAR-T cells

Abstract
Introduction
Results
Discussion
Materials and methods
Data availability
References
Article and author information
Metrics

Abstract

The immunological synapse (IS) is a cell-cell junction between a T cell and a professional antigen-presenting cell. Since the IS formation is a critical step for the initiation of an antigen-specific immune response, various live-cell imaging techniques, most of which rely on fluorescence microscopy, have been used to study the dynamics of IS. However, the inherent limitations associated with the fluorescence-based imaging, such as photo-bleaching and photo-toxicity, prevent the long-term assessment of dynamic changes of IS with high frequency. Here, we propose and experimentally validate a label-free, volumetric, and automated assessment method for IS dynamics using a combinational approach of optical diffraction tomography and deep learning-based segmentation. The proposed method enables an automatic and quantitative spatiotemporal analysis of IS kinetics of morphological and biochemical parameters associated with IS dynamics, providing a new option for immunological research.

Introduction

Understanding the immune response at the cellular scale requires knowledge regarding interactions between immune cells and their microenvironment. For T lymphocyte, which is one of the major immune cell types involved in adaptive immune responses, it communicates with antigen-presenting target cells via the formation of a nanoscale cell-cell junction, which is called the immunological synapse (IS). Specifically, the engagement of T cell receptor (TCR) by peptide-loaded MHC complex (pMHC) presented on the target cells leads to the formation of IS that coordinates the downstream signaling events required for the initial activation of T cells (Bromley et al., 2001; Basu and Huse, 2017). Previous studies have shown that the TCR-mediated IS comprises segregated concentric rings of supramolecular activating clusters, which are crucial for stabilization of the IS as well as the secretion of lytic granules (Basu and Huse, 2017; Monks et al., 1998).

Alternatively, recent studies have highlighted the IS structures formed by chimeric antigen receptor (CAR), a synthetic fusion protein comprised of an extracellular targeting and hinge domain, a transmembrane domain, and intracellular signaling domains (Lee and Kim, 2019; Mukherjee et al., 2017). The extracellular targeting domain of CAR is typically adopted single-chain variable fragment (scFv) of monoclonal antibodies, allowing the CAR-T cells to recognize various types of surface antigens independent of its MHC restriction. In particular, CD19-specific CAR-T cells have demonstrated remarkable anti-cancer efficacy in patients with B-cell malignancies (van der Stegen et al., 2015). Although CAR/antigen and TCR/pMHC complexes have different IS structures (Davenport et al., 2018), the distinct dynamics and mechanochemical properties of the IS driven by these complexes remain understudied.

A variety of imaging techniques have been used to reveal the hierarchical details of the IS structures and their relevant functions. For instance, electron microscopy and single-molecule localization microscopy have resolved the spatial distributions of subcellular IS compartments beyond optical diffraction limits (Choudhuri et al., 2014; Hu et al., 2016). However, assessing dynamical changes in IS formation requires rapid and continuous imaging of immune cells. Fluorescence microscopy is useful in this regard (Balagopalan et al., 2011). Such fluorescence-based techniques have the advantage of chemical specificity. However, they are limited due to photo-bleaching and photo-toxicity, which necessitates the use of complementary label-free, rapid three-dimensional (3D) microscopy methods to assess long-term dynamic changes in IS morphologies (Skylaki et al., 2016). Recently developed lattice light-sheet microscopy enables long-term high-speed volumetric imaging (Chen et al., 2014), and thus can be potentially used for the study of the dynamics of CAR-T and the structures of IS. However, fluorescence labeling process requires complicated and time-consuming sample preparation, especially when multiplex fluorescence imaging is needed.

The development of label-free IS imaging has been limited to phase-contrast and differential interference contrast microscopy. The aim is to develop quantitative phase imaging (QPI) as a quantitative label-free imaging method to studying IS (Park et al., 2018a). Optical diffraction tomography (ODT) is a promising 3D QPI technique for imaging the 3D refractive index (RI) distribution of cells at a sub-micrometer spatial resolution (Kim et al., 2016a). Unlike nonlinear scanning microscopy that requires a long acquisition time due to weak signal intensities (Zumbusch et al., 1999; Freudiger et al., 2008; Squier et al., 1998; Zipfel et al., 2003), ODT enables fast 3D imaging via holographic recording. Also, because the reconstructed RI profile correlates with total cellular protein densities, ODT enables quantitative, photobleaching-free analyses of cell dynamics.

ODT has been actively used to study single-cell morphology (Yoon et al., 2017; Kim et al., 2018; Park et al., 2008; Yang et al., 2017). However, it has not yet been used to study cell–cell interactions that include immune responses. One of the primary reasons is the lack of an accurate 3D segmentation framework to distinguish interacting cell-to-cell interfaces, which is also a problem with other microscopy methods (Uchida, 2013). Manual marking is the most primitive segmentation strategy. It is effective but is too laborious and difficult for time-resolved volumetric segmentation. To overcome this barrier, automatic segmentation has been developed based on basic algorithms that include intensity thresholding, filtering, morphological operations, region accumulation, and deformable models (Dimopoulos et al., 2014). However, these methods often result in poor segmentation, particularly for adjoining cell segmentations, which occur in immune responses. To accurately and precisely segment immunologically interacting cells in an automated manner, a novel computational framework is needed.

Here, we present DeepIS, a computational framework for the systematic, label-free analysis of 3D IS dynamics of immune cells in ODT. Our framework is based on deep convolutional neural network (DCNN) that distinguishes adjoining immune cells, target cells, and IS surfaces from the obtained RI tomogram. The proposed framework enables the general, high-throughput, and automated segmentation of more than 1000 immune-target cell pairs. To validate the method, we applied this method to study the dynamics of CAR/antigen-mediated IS formed between CD19-specific CAR-engineered T cells (CART19) and CD19-positive K562 cancer cells (K562-CD19). The combined use of high-speed imaging capability of ODT enabled 3D high-speed CAR IS tracking in which a tomogram was measured every 3 to 8 s for a prolonged period of time (300 s to 10 min depending on the cell type). Exploiting the linear proportion between RI and protein density (Barer et al., 1953), we also demonstrate quantitative analyses of CAR IS kinetics using the morphological and biochemical properties. The results suggest that DeepIS offers a new analytical approach to immunological research.

Results

3D time-lapse RI measurement of the CART19 and K562-CD19 cell conjugates using optical diffraction tomography

In order to perform ODT experiments in our study, we employed an experimental setup which is based on off-axis holography equipped with a high-speed illumination scanner using a digital micro-mirror device (DMD; DLP6500FLQ DLP 0.65 1080 p Type A DMD, Texas Instrument) (Figure 1). The setup enables the high-speed acquisition of a single tomogram within 500 milliseconds (Figure 1a; Shin et al., 2015; Lee et al., 2017). A 1 × 2 single-mode FC/APC fiber coupler was utilized to split a coherent, monochromatic laser (λ = 532 nm) into a sample and reference arms. The DMD was then placed onto the sample plane of the sample arm to control the illumination angle of the first-order diffracted beam striking the sample. To scan the illuminations at large tilt angles, a 4-f array consisting of a tube lens (Lens 1, f = 250 mm) and a condenser objective (UPLASAPO 60XW, Olympus Inc, Japan) magnified the illumination angle. The light scattered by live cells in a live-cell chamber (TomoChamber, Tomocube Inc, Republic of Korea) was then transmitted through the other 4 f array formed by an objective lens (UPLASAPO 60XW, Olympus Inc, Japan) and a tube lens (Lens 2, f = 175 mm). The sample beam was combined with the reference beam by a beam splitter and filtered by a linear polarizer. The resultant off-axis hologram was then recorded by a CMOS camera (FL3-U3-13Y3M-C, FLIR Systems, Inc, USA) synchronized with the DMD to record 49 holograms of the sample illuminated at different angles. Using a phase-retrieval algorithm, the amplitude and phase images of the 1:1 conjugate between a CART19 and K562-CD19 cell were retrieved from the measured holograms (Figure 1b). Based on the Fourier diffraction theorem with Rytov approximation (Wolf, 1969; Devaney, 1981), the 3D RI tomogram of the sample was reconstructed from the retrieved amplitude and phase images (Figure 1c). To fill the uncollected side scattering signals due to the limited numerical apertures of objective lenses, a regularization algorithm based on the non-negative constraint was used (Lim et al., 2015). The maximum theoretical resolutions of the ODT system were respectively 125 nm laterally and 471 nm axially, according to the Lauer criterion defined as the Nyquist sampling period (Lauer, 2002; Park et al., 2018b). Note that the empirical spatial resolution of ODT has been known to be between the Nyquist sampling period and the Abbe criterion (Simon et al., 2017), so the Nyquist sampling period was used to set the lowest bound of the resolution. Finally, we approximated the protein densities from the reconstructed RI values using a RI increment per protein concentration, α = dRI/dc = 0.185 mL/g²⁵. We assumed this RI increment to be constant over cells for two reasons: (1) as in Barer et al., 1953, most protein molecules have narrow ranges of RI incremental values from 0.179 to 0.195 mL/g, (2) lymphocytes contain lipid-rich environment localized mostly on a 4-nm-thick membrane site, whose size is beyond optical resolution and thus the relatively voluminous adjacent cytoplasm would contribute the reconstructed RI tomogram rather than sub-resolved membrane layers. The average illumination intensity was 11.46 mW/cm², at which the samples did not suffer from phototoxicity or signal losses during long-term assessment (Figure 1—video 1).

Figure 1 with 1 supplement see all

Download asset Open asset

Data acquisition in optical diffraction tomography (ODT).

(a) The experimental setup for ODT is based on a digital micro-mirror device (DMD) for high-speed illumination scanning. (b) Forty-nine holograms of 1:1 conjugate between a CART19 and a K562-CD19 cell were recorded at various illumination angles, and their amplitude and phase delay distributions were retrieved. (c) A reconstructed refractive index (RI) map.

DeepIS establishment for automated assessment of CART19 IS dynamics

The IS tracking analysis is preceded by segmentation, which involves dividing volumetric sections for background, cell domains, and IS in the ODT RI map. However, iteration is required for parameter tuning of the manual segmentation method to obtain a single well-segmented label, which is prohibitive to obtain a dynamic dataset. Therefore, we established DeepIS framework based on the DCNN supervised learning method to enable general, high-throughput, and automated segmentation for 3D RI tomograms (Figure 2). The framework was developed in the following order: (i) Dataset preparation, (ii) Training stage, and (iii) Inference stage. These are detailed next.

Figure 2 with 4 supplements see all

Download asset Open asset

Dataset preparation

The preliminary step of supervised learning of DCNN is to prepare an annotated dataset (Figure 2a). To annotate the 3D masks of the CART19 and K562-CD19 cells, we applied a combination of image processing and the watershed algorithm to a raw RI tomogram according to the following steps (Figure 2—figure supplement 1, also see the Codes). First, we annotated the cell masks from a raw RI tomogram using manual selections of four hyper-parameters: (i) initial seed locations of each cell to obtain a 3D distance-transform map, (ii) RI threshold for defining cell boundaries, (iii) voxel dilation sizes for merging over-segmented grains into one discrete region, and (iv) standard deviation of the Gaussian smoothing mask. The processed data were then multiplied to the 3D distance-transform map of the cell regions and segmented by the watershed algorithm. Finally, after iterative adjustment of the parameters from over 2000 data points, three experts in cellular biology quantitatively validated consistent annotation performances via simultaneous truth and performance level estimation (Figure 2—figure supplement 2), and heuristically curated the 236 pairs of well-annotated 3D tomograms with consensus (Warfield et al., 2004). The curated data uniformly reflected the various stages of the IS dynamics, which ensured providing information about the immunological response in both the early and late stages.

Training stage

The segmentation tasks were challenged by the lack of distinct boundaries between CART19/K562-CD19 conjugates in RI distributions, diverse morphology of cells, and the demand for precise segmentation at high resolution. As a consequence, although typical segmentation tasks of DCNN were assigned to infer voxel-wise label classification, various types of failure occurred, such as fragmented labels and unnatural IS.

To overcome these limitations and improve the segmentation accuracy and robustness, we designed DCNN to predict the distance map (y_pred), which was adapted from a previous study (Wang, 2018; Figure 2b, also see Materials and methods and Figure 2—figure supplement 3). We first conducted pre-processing of the 3D annotated masks of CART19 and K562-CD19 cells using the Euclidean distance transform to obtain a true distance map (y_true). A primary difference from the prior study (Wang, 2018) is that the CART19 and K562-CD19 cells were distinguished by the signs of the distance maps (i.e. positive/negative for CART19/K562-CD19 cells). Presently, we set the background to zero.

With the pre-processed data, the DCNN was optimized using the Adam optimizer (Initial learning rate = 0.001, decay rate for the first moment estimate β₁ = 0.5, decay rate for the second moment estimate β₂ = 0.99, learning decay rate = 0.5 per 50 epochs) during the training stage to predict the signed 3D distance map of CART19 and K562-CD19 cells. The boundary-weighted L1 function was used as a loss function. To prevent overfitting that might arise from a relatively small number of the training data compared with a large number of parameters, early stopping and data augmentation were used. For early stopping, the obtained annotated dataset was into two disjoint subsets. In one subset, 198 tomogram data pairs were used for the optimization of model parameters. In the other subset, the remaining 36 tomogram data pairs were used for internal validation. Training of the model parameters was stopped if the performance of the model on the validation set had not improved for five consecutive epochs. For data augmentation, a larger annotated dataset was simulated using random rotations, horizontal reflection, cropping, and elastic transformation to make the resulting model more robust to irrelevant sources of variability. The network was trained on four graphics processing units (GPUs; GEFORCE GTX 1080 Ti) for 400 epochs, which took approximately 6 hr. Selection of a model for inference among trained models was based on performance on the internal validation set.

Inference stage

In the inference stage, the trained network was used to regress the distance maps of the CART19 and K562-CD19 cells from unlabeled RI tomograms (Figure 2c). In the post-processing stage, the output distance map was post-processed to yield cell domains masks of each CART19 and K562-CD19 cell through simple thresholding using a value of 54.5 nm, which is approximately half of a voxel pitch. The IS of CART19/K562-CD19 conjugate was defined by dilating the CART19 and K562-CD19 masks by a comparable length to the axial resolution of ODT (two voxels, 437 nm) and finding an overlapping region. The cell surface and the cell interior were obtained using binary image erosion by one voxel (218 nm) and defined as the boundary of the 3D cell mask and the remaining interior mask respectively (Figure 2—figure supplement 4).

Evaluation of DeepIS segmentation performance

Because the label-free segmentation approach aims to distinguish the boundaries between two attached cells at a sub-micrometer spatial resolution, it was essential to validate whether DeepIS provides sufficient segmentation accuracy for our purpose. This was first addressed by comparing the segmentation performances between DeepIS and manual segmentation (Figure 3, also see Figure 3—video 1 and Datasets). In the training dataset, this comparison showed that the model could define IS boundaries. Notably, DeepIS generally displayed better segmentation performance than the manual segmentation in the untrained dataset, without notable segmentation problems such as fragmentations and discontinuous boundaries. This observation indicated that DeepIS was well trained and could be exploited to predict the IS boundaries.

Figure 3 with 1 supplement see all

Download asset Open asset

Representative segmentation results.

(Top row) Masks annotated by manual parameter selections. (Bottom row) Segmentation results using DeepIS. Blue shade indicates the curated data in the training stage. Red shade indicates the data which were poorly segmented by parameter-based annotation. White arrows illustrate poorly segmented regions. Lower right graph: rendering colormaps determined by the ranges of RI and RI gradient norm.

We then examined whether DeepIS could be applied for high-throughput analysis of IS morphology. Wide-area segmentation of CART19 and K562-CD19 cells was carried out over a lateral field-of-view exceeding 1 mm² (Figure 4a). When cells were located based on RI contrast, DeepIS allowed rapid, automated, and on-site semantic segmentation (Figure 4b). Interestingly, DeepIS successfully labeled adjoining CART19/K562-CD19 cell conjugates as well as individual CART19 and K562-CD19 cells (Figure 4c), which validated the high-throughput, general segmentation performance of DeepIS.

Figure 4

Download asset Open asset

High-throughput semantic segmentation using DeepIS.

(a) RI tomogram over 0.98 × 1.05 × 0.04 mm³ obtained by stitching method. (b) Segmentation using DeepIS. (c) Representative CART19, K562-CD19, and CART19/K562-CD19 cell conjugates are magnified.

The segmentation performance of DeepIS was further evaluated by quantifying the segmentation accuracy using manually delineated 3D labels obtained from correlative fluorescence microscopy (Figure 5). To prepare 3D manual labels, we mixed T cells and K562 cells expressing CAR-G4S-mCherry and hCD19-G4S-Zsgreen fusion proteins, respectively, and imaged fixed cell conjugates using correlative RI and fluorescence microscopy to delineate manual labels (Figure 5a, Materials and methods). We found that RI and fluorescence images overlapped well with the plasma-membrane fluorescence images, confirming that correlative RI and CAR/CD19 fluorescence images could sufficiently resolve the 3D cell topologies (Figure 5—figure supplement 1). Furthermore, multi-color 3D confocal microscopy showed with higher spatial resolution that CAR and CD19 were spread throughout the plasma membranes of CART19 and K562-CD19 cells respectively and coalesced into the IS, validating that CAR/CD19 fluorescence images were sufficient for defining the IS (Figure 5—figure supplement 2). We then compared the manually drawn labels with the segmentation masks obtained from DeepIS (Figure 5b). When 3D Pearson correlation coefficients for volumetric masks were quantified, we obtained the mean ± standard deviation (SD) values of 87.9 ± 7.7% and 92.0 ± 4.3% for CART19 and K562-CD19 cells, respectively, implying a greater than 80% good overlap between the manual labels and automatically segmented labels using DeepIS. In addition, the mean ± SD value for boundary displacement error was 2.82 ± 1.74 voxels, which corresponded to a sub-micrometer displacement error of 615.5 ± 379.8 nm. Considering that 3D segmentation is more challenging than 2D segmentation, these results indicated significantly small boundary errors (Wang, 2018; Lalaoui and Mohamadi, 2013).

Figure 5 with 2 supplements see all

Download asset Open asset

Quantitative analysis of segmentation performance using correlative fluorescence microscopy and ODT.

(a) Representative 3d rendering images, the xy- and the xz- cross-sections of RI (first column), fluorescence (second column), and their overlapped images (third column). CART19 and K562-CD19 labels were manually delineated based on the correlative images (fourth column) and compared with the labels obtained from DeepIS (fifth column). (b) Quantifications of DeepIS segmentation performance (n = 17). Pearson correlation coefficient was measured for CART19 (87.9 ± 7.7%) and K562-CD19 (92.0 ± 4.3%). Boundary displacement error was measured for IS (2.82 ± 1.74 voxels; 615.5 ± 379.8 nm). Each boxplot indicates the median, upper, and lower quartiles of each population.

For a conclusive validation of the segmentation performance, we lastly compared the IS defined by DeepIS with the IS imaged by high-resolution fluorescence microscopy methods. For this purpose, we integrated the DeepIS framework with multicolor 3D structured illumination microscopy (3D-SIM; see Materials and methods) (Mobahi et al., 2011; Gustafsson et al., 2008; Demmerle et al., 2017; Kim et al., 2017). The integrated setup enabled us to image the protein compositions at CAR IS at sub-200-nm lateral and nearly sub-400-nm axial spatial resolution defined as full width at half-maximum (Figure 6—figure supplement 1).

Using 3D-SIM, we first assessed the 3D location of the synaptic cleft (Figure 6). We used the negative stain method by adding the FITC-labeled dextran solution with two different hydrodynamic diameters (4 and 54 nm) into the chemically fixed conjugates of K562-CD19 cells and T cells expressing CAR-G4S-mCherry (see Materials and methods). The resultant 3D-SIM images showed that the synaptic cleft was visible only when we used the smaller dextran (4 nm), suggesting that, similar with the IS of NK cells, CAR IS excluded dextran molecules above a size threshold (Cartwright et al., 2014; Figure 6a, also see Figure 6—figure supplement 2). We then compared the imaged synaptic clefts, the CAR fluorescence, and the IS drawn by DeepIS for validation (Figure 6b). The 3D overlaps of the three images validated the 3D segmentation accuracy of our proposed method (Figure 6c). We laterally plotted the signals across the IS to quantitatively assess the segmentation accuracy (Figure 6d). The displacements of the IS from the peak intensities of the CAR and the synaptic cleft were within 200 and 600 nm, respectively. The result suggested that the IS drawn by DeepIS reflected the IS boundary closer to CAR-T cells, whose accuracy was comparable to the resolution limit of ODT. Moreover, consistent with the previous study using confocal fluorescence microscopy (Cartwright et al., 2014), the lateral full-width half-maximum of the synaptic cleft was measured to be 894 nm, which may imply the presence of a spatial gap larger than tens of nanometers across the IS (McCann et al., 2003).

Figure 6 with 2 supplements see all

Download asset Open asset

High-resolution validation of 3D IS using a negative stain method.

(a) The XY- and XZ-slice images of fluorescein-labeled dextran. White dashed lines: slice regions. (b) Corresponding XY- and XZ- images of SIM and ODT. (c) Comparison of dextran fluorescence, CAR fluorescence, and the IS defined by DeepIS. (d) Mean lateral line profiles for lines drawn across the IS.

Additionally, we validated whether the proposed method could be used to detect organelles at CAR IS. We specifically investigated the distributions of lytic granules, CAR and CD19 proteins, by imaging T cells expressing CD19-specific CAR-G4S-mCherry stained with lysotracker dyes and K562 cells expressing hCD19-G4S-Zsgreen. To verify successful staining of lytic granules before 3D-SIM, we employed widefield deconvolution fluorescence microscopy and confirmed the rapid dynamics of lytic granules coalescing into the IS (Figure 7—figure supplement 1, also see Figure 7—video 1). We also confirmed the RI detection sensitivity of the lytic granules using a histogram analysis, which showed that, compared to cell bodies, the mean RI value was lower for CAR IS and higher for lytic granules (Figure 7—figure supplement 2). We then chemically fixed stable CART19/K562-CD19 conjugates and imaged them using 3D-SIM and ODT for high-resolution multi-protein composition analysis (see Materials and methods).

As a result, our DeepIS framework demarcated the interface regions in close proximity to the overlapping areas of CAR and CD19 proteins (Figure 7a). Importantly, the demarcated region by DeepIS was consistent with the 3D-SIM images, which provided clearer 3D features of the IS between CART19 and K562-CD19 cells than widefield fluorescence microscopy. Along the IS demarcated by DeepIS, we quantitatively analyzed the en face images of CAR, CD19 and lytic granules imaged by 3D-SIM (Figure 7b). In agreement with the previous report (Davenport et al., 2018), the protein compositions of CAR exhibited asymmetric and granular distributions along the CAR IS. We analyzed the correlations between the total protein concentration distribution and the imaged proteins at CAR IS. The correlative line and surface profiles of the protein signals indicated the highest correlations of CAR with lytic granules, as well as colocalizations of CD19 proteins with the CAR clusters (Figure 7c). Interestingly, the total surficial protein densities approximated by ODT exhibited both correlated and uncorrelated clustered regions with the dense multi-protein clusters. Since ODT quantitatively estimates the total protein concentration, the uncorrelated signals are highly likely to imply the presence of clusters of other dominant proteins such as F-actin, Lck, and supramolecular attack particles (Xiong et al., 2018; Bálint et al., 2020).

Figure 7 with 3 supplements see all

Download asset Open asset

High-resolution analysis of 3D IS compositions using 3D-SIM and DeepIS.

(a) XY- and XZ-slice images of representative CART19/K562-CD19 conjugates. White-dashed lines: slice regions. White bold lines in SIM + ODT: 3D IS areas defined by DeepIS. (b) *En face* projected images of the defined IS areas in a. (c) Surface (left each) and line (right each) profiles along the dashed lines of normalized fluorescence intensities and surficial protein densities. Arrows indicate representative colocalized signals.

Taken together, these results suggest that our DeepIS method based on ODT can be used to define IS area with high accuracy, and also provides collective information about the distribution of total proteins within the IS which may not be easily measured by using conventional high-resolution fluorescence microscopy.

Quantitative kinetic analysis of CART19 IS formation using DeepIS

The successfully established DeepIS was implemented in the detailed kinetic analysis of the IS formation between CART19 and K562-CD19 cells using morphological and biochemical parameters (Figure 8). Specifically, we analyzed 27 sets of IS dynamic datasets measured over 300 s to 10 min at time intervals of 3 to 8 s to determine the kinetics of synapse area, membrane protein density, and intracellular protein density.

Figure 8 with 4 supplements see all

Download asset Open asset

Quantification of initial IS formation kinetics of CART19 cells.

(a) Representative snapshots of a video of CART19 cells responding to K562 (top row) and K562-CD19 (bottom row) cells. Purple: K562, Blue: CART19. 0 s: the initial contact point of the effector cell and target cells. White arrow: the blebbing point. (b) Temporal changes in the synapse areas of CART19 cells responding to K562 cells (n = 5) and K562-CD19 cells (n = 22). Black line: fitting curve with A(t)=*A_maxt*/(*t + τ*_1/2), where *A_max* = 106.16 μm² and τ_1/2 = 39.63 seconds, Pearson correlation coefficient (ρ)=0.93. (c) Maximum *en face* projection of membrane protein density of IS (top row) and non-IS T surface (bottom row) for CART19/K562-CD19 conjugate. The direction of the en face projection and the projection plane: the green arrow and a dashed line in (a). (d) Temporal changes in the mean membrane protein density of CART19 immunological synapses (circles) and non-IS T surfaces (crosses) responding to K562-CD19 cells. (e) Maximum z-axis projection of intracellular protein density distributions of the CART19/K562-CD19 conjugate. The white contours: the boundaries of the CART19 mask. (f) Temporal changes in the displacements of the center-of-masses of CART19 and K562-CD19 cells from their initial locations. Black lines: Δd(t) = Δd_max t/(*t + τ*_1/2), where Δ*d_max* = 7.48 and 1.74 μm, τ_1/2 = 37.84 and 14.56 s, ρ = 0.91 and 0.58 for CART19 and K562-CD19 respectively. (g) The average changes for 10 s in the early (0 s) and late (100 s) stages are marked by colored arrows, and statistically compared by two-tail paired Wilcoxon tests for CART19 and K562-CD19 cells, respectively. CART19: Δd/Δ*t_10s =* 818.5 ± 788.5 nm/s, and 26.6 ± 223.6 nm/s in the early and late stages. K562-CD19: Δd/Δ*t_10s =* 194.3 ± 246.5 nm/s, and 63.7 ± 125.6 nm/s in the early and late stages. *** indicates p<0.001. Each shaded error bar indicates the mean and standard deviation of line plots. Each boxplot indicates the median, upper, and lower quartiles of each population. The central points and shades in Figures 7b, d and f indicate the mean and standard deviations respectively. The numbers of analyzed cells per each experiment are listed in Table 1.

Table 1

Numbers of analyzed cells per each experiment.

Data	Experiment	1	2	3	4	5	6	7	8	9	10	Total
# of data	Statistics (Figure 5)	7	10	-	-	-	-	-	-	-	-	17
	dynamics (Figure 7)	9	3	3	1	3	3	1	1	2	1	27
	4-1BB (Figure 8)	20	19	25	-	-	-	-	-	-	-	64
	CD28 (Figure 8)	1	11	10	9	23	-	-	-	-	-	54
	Statistics (Figure 5—figure supplement 1)	8	22	-	-	-	-	-	-	-	-	30
	10 kDa (Figure 6—figure supplement 2)	3	7	-	-	-	-	-	-	-	-	10
	2000 kDa (Figure 6—figure supplement 2)	6	3	-	-	-	-	-	-	-	-	9
	4-1BB (Figure 8—figure supplement 1)	1	1	2	1	-	-	-	-	-	-	5
	CD28 (Figure 8—figure supplement 1)	2	1	-	-	-	-	-	-	-	-	3

First, we examined the temporal changes of synapse areas depending on the expression of the target antigen, CD19, on K562 cells (Figure 8a). As expected, CART19 cells could not form a stable synapse with K562 cells (CD19-negative) in five independent experimental trials (Figure 8a, also see Figure 8—video 1). By contrast, CART19 cells formed a stable IS with K562-CD19 cells (CD19-positive), and induced apoptotic blebbing on the target cells about 9 min after the initial contact (Figure 8—video 2). For the statistical analysis of the initial IS area changes, DeepIS was applied to both dynamic datasets and successfully segmented CART19 cells, target cells, and IS boundaries. The individual temporal plots of IS showed that each data exhibited smooth IS increase and indicated no undulation problems due to ill segmentations in most cases (Figure 8—figure supplement 1). The temporal graphs of the mean synapse area showed that, whereas the IS for K562 cells was not stably formed for 300 s, the IS for K562-CD19 expanded to the half of the maximum synapse area (A_max = 106.16 μm²) within 40 s and reached a steady-state within only 3 min (Figure 8b).

Next, we also assessed whether the membrane protein amount differs between the IS and non-IS areas of CART19 cells by comparing the 2D maximum projected snapshots of membrane protein densities in each area (Figure 8c). Within the IS surface, a dramatic increase in membrane protein density, as well as the synapse area, was observed. By contrast, the non-IS T surfaces of CART19 cells maintained the constant, lower membrane protein densities. In line with this observation, quantification of temporal changes in the mean membrane protein densities revealed that up to 27 ± 4 fg/μm² at 300 s have been accumulated in the IS surface, which was higher than average protein density (19 ± 1 fg/μm²) in the non-IS surface (Figure 8d).

We further explored the cell mechanics of CART19 and K562-CD19 cells during their initial IS formation. As in Figure 8e, the time-lapse snapshots of maximum 2D projection of intracellular protein densities visualized the dynamic action of CART19 cells, which incorporated the polarization of the intracellular protein contents. Furthermore, CART19 cells exerted mechanical forces during the dynamic IS formation, which led to the subsequent translational displacement of the K562-CD19 cells (Figure 8—video 3). To quantify the cell translocation dynamics, we traced the temporal changes in the displacement of the center-of-mass (the average displacement weighted by the intracellular protein density) with respect to the initial location for each cell (Figure 8f). As observed, CART19 cells exhibited more dynamic motions (Δd_max = 7.48 μm) than K562-CD19 cells (Δd_max = 1.74 μm) during the IS formation. For both cells, the mechanical translocations were more dramatic in the earlier stage of the IS formation, as confirmed statistically by comparing the average change in the cell translocations in the early- and late-stage (Figure 8g).

Overall, the results presented here generally recapitulate previously observed phenomena in the synapse studies based on conventional microscopy techniques. The rapid increase in the membrane protein density in the synapse area likely reflects the influx of large amounts of IS protein components, including CARs, actins, and other adhesion molecules (Xiong et al., 2018). In addition, perhaps the intracellular protein density changes in CART19 cells, which indicates polarization of the intracellular organelles in CART19 cells until stabilization, can be explained by the centrosome polarization of cytotoxic T lymphocytes (Ritter et al., 2015). Lastly, the force exerted by the CART19 cells on target cells during the IS formation suggests that CART19 cells also integrate mechanical potentiation during target cell killing similar to TCR-based cytotoxic T cells (Basu et al., 2016).

Statistical analysis of IS parameters depending on the co-stimulatory domains of CAR

Kymriah and Yescarta are the only two CD19-targeting CAR-T cell therapies that have been approved by US-FDA to date. Since one of the major differences between the two 2^nd generation CAR-T cells lies in the sequence of the costimulatory signaling domain used (4-1BB for Kymriah and CD28 for Yescarta), we next thought to apply our method to compare the IS characteristics of the CAR-T cells with different costimulatory signaling domains. Primary human T cells transduced with the lentiviral vectors encoding CD19-28z or CD19-BBz CAR showed comparable transduction efficiency as well as similar surface expression levels of CARs as determined by flow cytometry (Figure 9—figure supplement 1).

Exploiting our established method, we studied the effect of CD28 or 4-1BB co-stimulatory domains on CAR IS characteristics (see Materials and methods). We first observed the early IS dynamics between CD28 and 4-1BB based CAR-T cells in the presence of K562-CD19 target cells, and found the stable IS formation within five minutes without statistically significant kinetic differences (Figure 9—video 1, also see Figure 9—figure supplement 2). We then compared the statistics of other IS parameters between the CD19-28z and CD19-BBz CAR-T cells in a steady state. Specifically, we incubated each type of CAR-T cells with K562-CD19 cells for 15 min to allow sufficient time for stable IS formation, and fixed them with 4% paraformaldehyde solution. When we analyzed the images for conjugates (Figure 9a), we found no significant difference in IS areas between the two CAR-T cell types (Figure 9b). However, statistical analysis indicated significantly higher IS protein densities and total IS protein amounts for CD19-BBz CAR-T cells, approximately by 10% compared with CD19-28z CAR-T cells (Figure 9c–d). Collectively, our results indicate that the quantitative analysis of IS parameters using DeepIS, in conjunction with other analytical methods such as fluorescence-based microscopy and quantitative mass-spectrometry (Salter et al., 2018), may help to elucidate the mechanistic details underlying the functional differences observed for the CAR-T cells with different signaling domains (Lee and Kim, 2019; van der Stegen et al., 2015; Guedan et al., 2019).

Figure 9 with 3 supplements see all

Download asset Open asset

Statistical analyses of synapse morphologies depending on the co-stimulatory domains.

(a) Representative images for maximum z-axis projection of intracellular protein density distributions (top row) and *en face* projection of surface protein density distributions of IS (bottom row). White boundaries indicate the annotated membranes of CART19 cells using DeepIS. (**b–d**) Scatterplots of (b) synapse areas, (c) mean surface protein density, and (d) total IS protein amount. Each boxplot indicates the median, upper, and lower quartiles of each population. The vertical lines indicate population ranges. Perpendicular shades indicate normalized population density distributions. Two-tail unpaired Wilcoxon tests were performed. mean ± SD data are presented. The numbers of analyzed cells per each experiment are listed in Table 1.

Discussion

The collective results demonstrate that DeepIS in combination with ODT and deep neural network enables the label-free, time-lapse 3D IS tracking in an automated manner. The successful segmentation performance of DeepIS could not be possible without careful validations in the initial development stage, including data curation, model training, and qualitative and quantitative tests for general segmentation capabilities. This platform was applied to define the IS parameters related to their morphological and biochemical traits, and to quantify the IS dynamics of CART19 cells. We anticipate that the proposed method can be generalized to study a broad range of IS that are mediated by different types of immune receptors such as TCR and B-cell receptor (BCR) as well as invariant receptors expressed on innate natural killer (NK) cells. In particular, the model will be a powerful way to test whether the IS morphology is affected by chemical treatment and genetic mutations (Tamzalit et al., 2019).

Previously, Xiong W et al. have evaluated the quality of the CAR IS formed by CD28 co-stimulatory domain including 2nd CAR (CD19-28z CAR) or CD28 + 4-1BB costimulatory domain including 3rd CAR (CD19-28BBz CAR) through confocal microscopic analysis using glass-supported planar lipid bilayers (Xiong et al., 2018). Although they showed that the 3rd generation CAR incorporates more IS-related proteins (F-actin, phospho-CD3z) than 2nd generation CAR, there were no direct comparative experiments on the properties of 2nd CAR IS depending on the types of co-stimulatory domain. Our present work fills the missing gaps, suggesting that CAR IS can also vary among the groups of 2nd generation CAR. In particular, we demonstrated that the surface protein density and total protein amount differ between CD28 and 4-1BB based CAR IS, which recalls the importance of quantitative biochemical analysis of CAR IS.

The present work is, to the best of our knowledge, the first application of ODT and deep learning to understand IS dynamics. Refinements are anticipated in an immediate follow-up study. For example, more rapid immune dynamics can be investigated at higher spatiotemporal resolution in a more sophisticated experimental setup, which may reveal dynamic diffusions of subcellular organelles (Kim et al., 2016b) during the immune responses. Also, more efforts in data curation and network design may allow robust time-lapse IS tracking. In order to do so, segmentation based on active contour methods, and new network models based on recurrent neural networks (Mikolov et al., 2010), Bayesian neural network (Snoek, 2015), and pyramid pooling (Bian, 2018) may be incorporated. Although we have demonstrated the label-free imaging and analyzing IS, one of the limitations of the current work is that the error of synaptic cleft determination is on the order of 500 nm. To overcome this, the negative staining with the smallest dextran or correlative electron micrographs may be used to provide a ground truth for improving the precision of synaptic cleft determination.

Our study focused on the platform capable of quantitatively tracking 3D IS dynamics in a completely label-free manner. The RI contrast used in the present study alone does not provide information about individual proteins that are transported during the IS formation of CART19 cells. To provide such biochemically relevant information, we expect that correlative imaging with fluorescence microscopy will circumvent the inherent lack of chemical specificities (Demmerle et al., 2017; Kim et al., 2017). It also remains unclear how much force was associated during the IS formation, and whether this force correlates with the cytotoxic intensity of CART19 cells. To test the hypothesis, the simultaneous use of microscopic force measurement techniques and ODT will be helpful. For instance, a combined technique of ODT and holographic optical tweezers (Kim et al., 2015; Kim and Park, 2017) or traction force microscopy can supplement our DeepIS framework for understanding the mechanical potentiation during the IS kinetics. We expect that these future studies may be exploited for label-free predictions of dynamic molecular transports occurring in the IS formation, and provide complementary information required to elucidate the IS formation mechanisms.

Materials and methods

Key resources table

Reagent type (species) or resource	Designation	Source or reference	Identifiers	Additional information
Cell line (Homo sapiens)	K562	Other	RRID:CVCL_0004	Obtained from Dr. Travis S. Young at California Institute for Biomedical research. The original stock was purchased from ATCC
Cell line (Homo sapiens)	K562-CD19	Other		Obtained from Dr. Travis S. Young at California Institute for Biomedical research. CD19-negative K562 cells were transduced wtih lentiviral vector encoding human CD19.
Cell line (Homo sapiens)	Lenti-X 293T	Takara	Cat. #: 632180	Lentivirus production
Recombinant DNA reagent	pLV-ΔLNGFR-P2A-CD19-BBz CAR	This paper		Signal peptide: aa 1–28 hLNGFR (Uniprot TNR16_Human) Extracellular domain: aa 29–250 hLNGFR (Uniprot TNR16_Human) Transmembrane domain: aa 251–272 hLNGFR (Uniprot TNR16_Human) Cytosolic sequence: aa 273–275 (Uniprot TNR16_Human) ΔLNGFR was fused with P2A sequence (GGAAGCGGAGCTACTAACTTCAGCCTGCTGAAGCAGGCTGGCGACGTGGAGGAGAACCCTGGACCT) and followed by CAR sequence Signal peptide: aa 2–21 hCD8 (Uniprot CD8A_Human) Extracellular domain: anti-CD19 scFv (Clone: FMC 63); VL –G4S linker VH Hinge/Transmembrane domain aa 138–206 hCD8 (Uniprot CD8A_Human) Co-stimulatory domain sequence: aa 214–255 (Uniprot TNR9_Human) Signaling domain: aa 52–164 (Uniprot CD3Z_Human)
Recombinant DNA reagent	pLV- ΔLNGFR -P2A-CD19-28z CAR	This paper		Signal peptide: aa 1–28 (Uniprot TNR16_Human) Extracellular domain: aa 29–250 (Uniprot TNR16_Human) Transmembrane domain: aa 251–272 (Uniprot TNR16_Human) Cytosolic sequence: aa 273–275 (Uniprot TNR16_Human) ΔLNGFR was fused with P2A sequence (GGAAGCGGAGCTACTAACTTCAGCCTGCTGAAGCAGGCTGGCGACGTGGAGGAGAACCCTGGACCT) and followed by CAR sequence Signal peptide: aa 2–21 hCD8 (Uniprot CD8A_Human) Extracellular domain: anti-CD19 scFv (Clone: FMC 63); VL-G4S linker-VH Hinge/Transmembrane domain: aa 138–206 (Uniprot CD8A_Human) Co-stimulatory domain sequence: aa 180–220 (Uniprot CD28_Human) Signaling domain: aa 52–164 (Uniprot CD3Z_Human)
Recombinant DNA reagent	pLV-CD19-BBz CAR-G4S-mcherry	This paper		Signal peptide: aa 1–21 (Uniprot CD8A_Human) Extracellular domain: anti-CD19 scFv (Clone: FMC 63) VL –G4S linker VH Hinge/Transmembrane domain: aa 138–206 (Uniprot CD8A_Human) Co-stimulatory domain sequence: aa 214–255 (Uniprot TNR9_Human) Signaling domain: aa 52–164 (Uniprot CD3Z_Human) CAR and mcherry was fused with G4S linker mCherry: aa 2–236 (Uniprot X5DSL3_ANAMA)
Recombinant DNA reagent	pLV-hCD19-G4S-Zsgreen	This paper		Signal peptide: aa 1–19 (Uniprot CD19_Human) Extracellular domain: aa 20–291 (Uniprot CD19_Human) Cytoplasmic domain: aa314-556 (Uniprot CD19_Human) CD19 and Zsgreen was fused with G4S linker Zsgreen: aa 2–231 (Uniprot GFPL1_ZOASP)
Recombinant DNA reagent	pMDLg/pRRE	other	RRID:Addgene12251	3^rd Lentivirus packaging vector
Recombinant DNA reagent	pRSV-Rev	other	RRID:Addgene12253	3^rd Lentivirus packaging vector
Recombinant DNA reagent	pMD2.G	other	RRID:Addgene12259	Lentivirus envelop vector
Recombinant DNA reagent	pMACS LNGFR	Miltenyi Biotec	Cat. #:130-091-890
Peptide,recombinant protein	rhIL-2	BMI KOREA		300IU/mL
Peptide,recombinant protein	rhCD19-Fc	ACRO Biosystems	Cat. #: CD9-H5259	FACS
Peptide,recombinant protein	AF647-conjugated streptavidin	Biolegend	Cat. #: 405237	FACS
Antibody	InVivoMab anti-human CD3	Bio X cell	Cat. #: BE0001-2-5MG	OKT3 clone 4 μg/mL coated
Antibody	InVivoMab anti-human CD28	Bio X cell	Cat. #: BE0291-5MG	CD28.2 clone 2 μg/mL soluble
Antibody	hLNGFR-APC antibody	Miltenyi Biotec	Cat. #: 130-113-418	FACS
Antibody	hCD19-APC antibody	Biolegend	Cat. #: 302212	FACS
Other	DPBS	Welgene	Cat. #: LB001-02
Other	Fetal Bovine Serum (FBS)	Gibco	Cat. #: 26140–079
Other	DMEM	Gibco	Cat. #: 11965–118
Other	RPMI	Gibco	Cat. #: 21870–092
Other	HBSS with Ca²⁺ and Mg²⁺	Gibco	Cat #: 14-025-092
Other	HEPES	Gibco	Cat. #: 1530080
Other	2-Mercaptoethanol	Sigma	Cat. #: M6250-100mL
Other	Lipofectamine 2000	ThermoFisher	Cat. #: 11668019
Other	Glutamax	Gibco	Cat. #: 35050–061
Other	Protamine Sulfate	Sigma	Cat. #: P3369
Other	MEM Non-essential amino acid	Gibco	Cat. #: 111400500
Other	Sodium pyruvate	ThermoFisher	Cat. #: 11360070
Other	penicillin/streptomycin	Gibco	Cat. #: 15140–122
Other	Tomodish	Tomocube
Other	Paraformaldehyde Solution, 4% in PBS	ThermoFisher	Cat. #: AAJ19943K2	Fixation
Other	CellBrite Fix 640 Membrane Dye	Biotium	Cat. #: 30089	Plasma membrane staining
Other	VECTASHIELD Hardset w/DAPI	VECTOR Laboratory	Cat. #: H-1400	Mounting Medium
Other	LysoTracker Deep Red	ThermoFisher	Cat #: L12492	Dye for labeling and tracking lytic granules
Other	TetraSpeck Microspheres,0.1 µm, fluorescent blue/green/orange/dark red	ThermoFisher	Cat #: T7279	Fiducial markers
Other	FITC-dextran 10 kDa	TdB Labs	Cat #: 20682	Fluorescein-labeled dextran, 10 kDa
Other	FITC-dextran 2000 kDa	TdB Labs	Cat #: 20584	Fluorescein-labeled dextran, 2000 kDa
Commercial assay or kit	CD271 Microbeads kits, Human	Miltenyi Biotec	Cat. #: 130-099-023	Isolation of CAR⁺ T cells
Commercial assay or kit	SepMate PBMC Isolation tube	STEMCELL	86460	PBMC Isolation
Software, algorithm	Flowjo	Flowjo	Version 10
Software, algorithm	Prism	GraphPad	Version 7
Software, algorithm	ImageJ	NIH

Primer name	Source or reference	Identifiers	Additional information (5`>3`)
LNGFR F	This paper	PCR primers	GGGGATCCCCCCATCAGTCCGCAAAG
LNGFR R-P2A	This paper	PCR primers	AGGTCCAGGGTTCTCCTCCACGTCGCCAGCCTGCTTCAGCAGGCTGAAGTTAGTAGCTCCGCTTCCCCACCTCTTGAAGGCTATGTAGG
P2A-CD19-BBz F	This paper	PCR primers	GGAAGCGGAGCTACTAACTTCAGCCTGCTGAAGCAGGCTGGCGACGTGGAGGAGAACCCTGGACCTGCCTTACCAGTGACCGCCTTGCTC
CD19 BBz R	This paper	PCR primers	TGTCGACTTAGCGAGGGGGCAGGGCCTGC
CD28/CD3 F	This paper	PCR primers	GTTATCACCCTTTACTGCAGGAGTAAGAGGAGCAGGCTC
CD28/CD3 R	This paper	PCR primers	GTCGACTTAGCGAGGGGGCAGGG
CD8Hinge/TM F	This paper	PCR primers	CAGTCACCGTCTCCTCAAC
CD8Hinge/TM R	This paper	PCR primers	GAGCCTGCTCCTCTTACTCCTGCAGTAAAGGGTGATAACCAG
mCherry F	This paper	PCR primers	GTGAGCAAGGGCGAGGAGGAT
mCherry-Sal1 R	This paper	PCR primers	TTGTCGACCTACTTGTACAGCTCGTCCATGCCGCCGG
G4S-mCherry F	This paper	PCR primers	GCTCCGGTGGTGGTGGTTCTGTGAGCAAGGGCGAGGAGGATAAC
CD19 BBz CAR F	This paper	PCR primers	CCGGGGATCCATGGCCTTACCAGTGACCG
CD19 BBz CAR G4S R	This paper	PCR primers	AGAACCACCACCACCGGAGCCGCCGCCGCCAGAACCACCACCACCGCGAGGGGGCAGGGCCTGCATGTGA
hCD19 F	This paper	PCR primers	GTTGGATCCATGCCACCTCCTCGCCTCCT
hCD19-G4S R	This paper	PCR primers	AGAACCACCACCACCGGAGCCGCCGCCGCCAGAACCACCACCACCCCTGGTGCTCCAGGTGCCCAT
G4S-Zsgreen F	This paper	PCR primers	GGTGGTGGTGGTTCTGGCGGCGGCGGCTCCGGTGGTGGTGGTTCTGCCACAACCATGGCCCAGTCCAAGC
Zsgreen R	This paper	PCR primers	GATTACGCGTATTGCTAGCTCAGGGCAAGGCGGAG

Share this article

Cite this article

Data acquisition in optical diffraction tomography (ODT).

Data flowchart in DeepIS framework.

Representative segmentation results.

High-throughput semantic segmentation using DeepIS.

Quantitative analysis of segmentation performance using correlative fluorescence microscopy and ODT.

High-resolution validation of 3D IS using a negative stain method.

High-resolution analysis of 3D IS compositions using 3D-SIM and DeepIS.

Quantification of initial IS formation kinetics of CART19 cells.

Numbers of analyzed cells per each experiment.

Statistical analyses of synapse morphologies depending on the co-stimulatory domains.

Author details

Moosung Lee

Contribution

Contributed equally with

Competing interests

Young-Ho Lee

Contribution

Contributed equally with

Competing interests

Jinyeop Song

Contribution

Contributed equally with

Competing interests

Geon Kim

Contribution

Competing interests

YoungJu Jo

Contribution

Competing interests

HyunSeok Min

Contribution

Competing interests

Chan Hyuk Kim

Contribution

For correspondence

Competing interests

YongKeun Park

Contribution

For correspondence

Competing interests

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

Categories and tags

Further reading