The HRMAn pipeline (Figure 1), is designed to work with all file types acquired on any HCI platform or fluorescence microscope. Plate maps including experimental layouts, sample groups and replicates can be loaded, enabling HRMAn to automatically cluster results and perform error analysis. Once fed into the HRMAn pipeline, images are automatically pre-processed and clustered based on user-defined parameters (i.e. imaging specifications) and corrected for illumination. The subsequent image analysis proceeds in two stages: in stage 1, HRMAn segments images into pathogen and host cell features for single cell analysis. It then classifies these features using a decision tree learning algorithm previously trained on an annotated dataset. In stage 2, HRMAn analyzes host cell features associated with the pathogens using a CNN HRMAlexNet (derived from AlexNet architecture) trained to distinguish complex phenotypic patterns of host-protein recruitment (Krizhevsky et al., 2012). Robust classification is achieved by passing segmented regions of interest through multiple non-linear convolutional filters to identify characteristic phenotypic details.

Figure 1

Download asset Open asset

Finally, data are output as a single spreadsheet file providing the researcher with ≥15 quantitative descriptions of a pathogen and its interaction with host factors at population and single cell levels (Figure 1; Readouts). Importantly, by separating the analysis, HRMAn offers researchers the flexibility to perform fast, simple quantitative analysis of infection parameters using stage 1, without analyzing host protein recruitment.

To train for detection and analysis of host–pathogen interactions, HRMAn was provided an annotated dataset of host cells infected with an eGFP-expressing version of the parasite Toxoplasma gondii (Tg) and stained for various host cell features (Figure 2A) (Seibenhener et al., 2004; Clough et al., 2016). For stage one pathogen detection and enumeration training a simplistic ML strategy – decision tree performed remarkably well. Over 35,000 Tg-vacuoles were analyzed by decision tree, gradient boosted tree, and random forest machine learning classification algorithms and cross-validated (Figure 2B). As each performed equally, a simple decision tree with Minimum Description Length (MDL) pruning, to limit overfitting, was employed for speed and accuracy of pathogen detection (>99.5% for Tg). Using these parameters, in addition to the readouts from stage 1 (see Figure 1), HRMAn detected and quantified Tg-containing vacuoles harbouring 1, 2, 4 or >4 fluorescent Tg (Figure 2C).

Figure 2 with 1 supplement see all

Download asset Open asset

For stage 2, host protein recruitment, the CNN was trained for ubiquitin and p62 recruitment using segmented Tg vacuoles defined in Stage 1. Robust classification of host protein recruitment was achieved by passing these regions of interest through multiple non-linear filters to identify and differentiate between no recruitment, recruitment, and analysis artefacts (Figure 2D). Training over 80 epochs with negative log likelihood as a loss function, the deep CNN achieved 92.1% classification accuracy confirmed by expert-based cross-validation. Precision for ‘no recruitment’, ‘recruitment’, and ‘artefacts’ classes was 0.92, 0.92 and 0.71, while recall was 0.94, 0.89 and 1 respectively, hence achieving the accuracy of a human operator and far exceeding human capacity (Figure 2E).

To assure that uninvaded Tg parasites do not skew the data, stringent synchronization of infection by centrifugation and washing procedures were employed. In a pilot experiment (Figure 2—figure supplement 1), staining with the Tg vacuole marker GRA2 (Figure 2—figure supplement 1A–B) revealed that more than 98% of all parasites captured in the images have successfully invaded and established a PV, irrespective of the Tg strain used for infection (Figure 2—figure supplement 1B). Using a multiplicity of infection (MOI) of 3 for experiments resulted in up to 90% type I and 80% type II Tg infected host cells (Figure 2—figure supplement 1C). In line with this, we often observed that a single host cell can contain more than one PV.

To demonstrate the ability of HRMAn and to expand how researchers define and classify host–pathogen interactions, the impact of IFNγ on Tg replication and ubiquitin/p62 recruitment to Tg vacuoles was analyzed (Figure 3).

Figure 3 with 7 supplements see all

Download asset Open asset

Previous reports indicate that HeLa cells restrict the growth of Tg through ubiquitination of parasitophorous vacuoles and subsequent non-canonical, p62-dependent autophagy (Selleck et al., 2015; Clough et al., 2016). HeLa cells infected with eGFP Tg ±IFNγ were fixed 6 hr post-infection (hpi) and stained with Hoechst (nuclei) and antibodies directed against ubiquitin and p62. A total of 1,350 4-colour images were acquired on an automated microscope and loaded into HRMAn for analysis.

HRMAn automatically detected and analyzed more than 15,000 HeLa cells resulting in 15 quantitative outputs of host–pathogen interaction (Figure 3). Population level readouts from stage one indicated that IFNγ treatment did not impact the percentage of infected cells but decreased the number of vacuoles within host cells as well as the number of parasites per cell (Figure 3A). As eGFP fluorescence is lost when parasites are killed, a reduction in the ratio between vacuoles and host cells serves as an indirect measurement for parasite killing. At the single cell level, HRMAn found that IFNγ treatment resulted in a significant reduction of vacuoles per cell and a minor reduction in mean vacuole size, without impacting vacuole position (Figure 3B). Concomitant with this reduction in vacuole size, both the percentage of replicating parasites, and the number of parasites per vacuole were significantly reduced by IFNγ treatment (Figure 3C). Thus, IFNγ−mediated control of Tg in HeLa cells involves both parasite-killing and restriction of Tg replication. Importantly, HRMAn offers a wide range of readouts in stage 1 analysis allowing for more detailed information on the dynamics of infection and clearance than typically seen with manual counting. To allow the user to decide which readouts are best suited to answer their specific research question some redundancy has been purposely built in (e.g. mean vacuole size vs. % Replicating). For example, here we focused on parasites per vacuole and the proportion of infected cells, as opposed to the number of individual vacuoles per host cell.

In stage 2, analysis of the >25,000 vacuoles identified in stage 1, showed that the number of host cells with ubiquitin/p62-positive vacuoles and the percentage of ubiquitin/p62-positive vacuoles per host cell increased with IFNγ (Figure 3D). Distribution analysis indicated that in untreated cells, only 5.92% of vacuoles were decorated with ubiquitin, p62, or both. This number rose to 27.61% in IFNγ-treated cells, the majority of which (20.92%) were double-positive for ubiquitin/p62 (Figure 3D). By quantifying the radial fluorescence intensity distribution of these host factors, HRMAn revealed that ubiquitin was more closely associated with Tg vacuoles than p62 and that recruitment of both was increased by IFNγ treatment (Figure 3E). This is in agreement with the notion that p62 binds a ubiquitinated vacuole substrate through its UBA domain (Seibenhener et al., 2004; Clough et al., 2016). Finally, by analyzing vacuoles that recruit ubiquitin/p62, HRMAn indicated that restriction of Tg replication occurs in vacuoles decorated with these host defense proteins (Figure 3F). Collectively, this data indicates that in HeLa cells, IFNγ drives both parasite killing as well as recruitment of ubiquitin/p62 to Tg vacuoles, which acts to restrict parasite replication (Figure 3). The results demonstrate the capacity of HRMAn to provide a quantitative, multi-parametric readout of host–pathogen interaction at population and single-cell levels.

As a high-throughput, high-content analysis program, HRMAn removes experimental size constraints imposed by manual quantification. To illustrate this, HRMAn was used to systematically analyze the impact of IFNγ treatment on type I and type II Toxoplasma strains in five human cell lines: HeLa (cervical carcinoma epithelial), PMA-differentiated THP-1 (macrophage-like), A549 (lung carcinoma epithelia), HFF (primary fibroblasts), and HUVEC (primary endothelial cells) (Figure 3—figure supplements 1–7).

First, stage 1 HRMAn was used to ascertain the impact of varying concentrations of IFNγ (50–500 IU/ml) on Tg infection, killing, and replication. (Figure 3—figure supplement 1). For each host cell line (Figure 3—figure supplement 1A), a dose-dependent reduction in Tg infection was seen (Figure 3—figure supplement 1B). Assessment of the vacuole:cell ratio and mean vacuole size indicated that THP-1s, HFFs, and HUVECs limit infection by IFNγ-dependent Tg killing, while HeLas and A549s do so by restricting replication (Figure 3—figure supplement 1C–D). Quantification of the number of parasites per vacuole indicated that HeLas and A549s acutely restrict type I and type II Tg replication at all concentrations of IFNγ (Figure 3—figure supplement 2B–C), while THP-1s, HFFs, and HUVECs are far more limited in this capacity (Figure 3—figure supplement 2A,D–E).

Next, HRMAn was employed on all 5 cell lines infected with either type I and type II Tg ±100 IU/ml IFNγ and immuno-stained for ubiquitin and p62. Figure 3—figure supplement 3–7 display the 15 quantitative readouts compiled by HRMAn of 9000 fields of view (~90 GB) and >175,000 vacuoles identified in stage 1. Taking advantage of the large-scale capabilities of HRMAn, we found that all host cell types can mediate IFNγ-dependent type I and II Tg killing (Figure 3—figure supplement 3B–C), and growth restriction (Figure 3—figure supplement 4A–B) to similar levels. Tg vacuoles show strain-dependent (A549, HUVEC), and strain-independent (HFFs) IFNγ-stimulated movement towards the nucleus (Figure 3—figure supplement 4C). HRMAn revealed that type II Tg grew slower than type I Tg in each host cell line and that their growth decreased more upon treatment with IFNγ (Figure 3—figure supplement 5A–B). Consistent with this, stage 2 HRMAn showed that all cell types could recruit ubiquitin and/or p62 equally well (Figure 3—figure supplement 6A), while a greater percentage of type II vacuoles per cell were decorated in response to IFNγ-priming (Figure 3—figure supplement 6B). The exception to this were THP-1 cells, which did not mount a strain-specific response (Figure 3—figure supplement 6B). Distribution analysis further indicated that THP-1s display a higher intrinsic capacity to decorate Tg vacuoles than other cell lines, even in the absence of IFNγ (Figure 3—figure supplement 6C). While no cell-type dependent differences in ubiquitin or p62 coat distance were observed (Figure 3—figure supplement 7A), THP-1s not only decorate vacuoles with more ubiquitin upon IFNγ stimulation, they also appear to recruit p62 in an IFNγ-independent fashion (Figure 3—figure supplement 7B). Decorated vacuoles in all host cell types displayed a greater ability to restrict the growth of type II versus type I Tg upon IFNγ treatment (Figure 3—figure supplement 7C–D). These results highlight the ability of HRMAn to provide high-throughput and quantitative single-cell analysis of host–pathogen interactions at a scale not achievable by automated bulk or manual quantification.

To demonstrate its flexibility, HRMAn was trained to recognize the bacterium Salmonella enterica Typhimurium (STm) - a pathogen 16x smaller than Tg (0.5 μm vs. 8 μm) - and then set to analyze the impact of IFNγ on bacterial killing, replication, and ubiquitin recruitment. Stage 1 outputs showed that similar to Tg (Figure 3), IFNγ treatment in HeLa cells reduced the ratio of STm vacuoles/cell and the bacterial load, without impacting the percent of infected host cells (Figure 4A). At the single cell level, HRMAn found a significant reduction in the number of STm vacuoles/cell, consistent with a reduction in vacuole size, percent of replicating bacteria, and reduced numbers of STm/vacuole (Figure 4B–C). These results demonstrate that HeLa cells can control infection with STm through IFNγ−dependent bacterial killing and growth restriction.

Figure 4

Download asset Open asset

For stage 2, we used the Tg recruitment model as input to retrain HRMAn for quantification of ubiquitin recruitment to STm (Figure 4D). This allowed us to achieve 69.9% classification accuracy, confirmed by expert-based cross-validation, in just 40 epochs using 10-fold less non-augmented image data (Figure 4D). It’s known that HeLa cells restrict STm growth by maintaining vacuole integrity; the small percentage of bacteria that escape vacuoles are decorated with ubiquitin and subsequently cleared by autophagy (Noad et al., 2017; van Wijk et al., 2017). Interestingly, stage 2 HRMAn showed that the percent of host cells which recruit ubiquitin to STm doubles upon IFNγ treatment, while the percent of decorated vacuoles/cell increases only slightly (Figure 4E). As seen with Tg (Figure 3E, Figure 3—figure supplement 7A), IFNγ does not impact the distance of the ubiquitin coat to STm but increases its thickness (Figure 4F). This indicates that more ubiquitin is recruited to cytosolic STm in the presence of IFNγ and growth of decorated bacteria was restricted (Figure 4G). Consequently, although IFNγ treatment increases the number of host cells that recruit ubiquitin to STm and the intensity of that recruitment, at the single-cell level HeLa cells appear to have reached their capacity for detection and autophagy-mediated clearance of cytosolic/ubiquitinated STm independent of IFNγ treatment (Figure 4E–G).

To illustrate the versatility of HRMAn and the advantage of the modular architecture combined with the accessible user interface that comes with the KNIME Analytics platform, we performed experiments to stress HRMAn’s applicability and adaptability to study pathogen-driven parameters of infection. Using transgenic Tg lines expressing different parasite virulence factors, we were able to reproduce and expand upon published data (Virreira Winter et al., 2011). We confirmed that expression of ROP16 from type I Tg or lack of GRA15 in otherwise isogenic type II parasites (PruA7) reduces the recruitment of murine guanylate binding proteins (Gbps) to the vacuoles. Similarly, expression of type I ROP18 in type III parasites (CEP) also reduced recruitment of murine Gbps 1, 2 and 5 compared to isogenic type III parasites (Figure 5A). This analysis shows that HRMAn can be used to study effects of pathogen effector proteins on an established host phenotype.

Figure 5

Download asset Open asset

Next, we asked whether HRMAn can accurately measure parasite effector proteins targeted elsewhere within infected host cells. Tg is known to secret multiple effector proteins upon invasion and during replication. Examples include the parasite proteins GRA16, GRA24 and TgIST, that are secreted beyond the boundaries of its PV and subsequently translocated to the host cell nucleus (Bougdour et al., 2013; Braun et al., 2013; Gay et al., 2016). Using HRMAn, we were able to visualize accumulation of these three Tg effector proteins (tagged with an HA-tag) in the nucleus of the host cell. HRMAn, further indicated that the levels of nuclear accumulation correlated to the number of parasites contained within the infected cell (Figure 5B). Thus, HRMAn can be employed to analyze both host and parasite parameters during infection in an unbiased, accurate and high content manner.

All open-source KNIME workflows used in this publication can be found at: https://github.com/HRMAn-Org/HRMAn and on the homepage https://hrman.org/ under GPLv3 open-source software license to allow for rapid and open dissemination and free availability for the research community. The models and their respective weights obtained through training will be deposited on GitHub and the homepage as well.

All images used to train the neural networks and other machine learning algorithms in this study are available upon request.

For simple infection analysis (stage 1), 96-well plates (see Microscopy sample generation) were imaged on an ArrayScan VtI Live High Content Imaging Platform (Thermo Scientific) using 20x magnification and depending on the experiment, 15–20 fields of view per well.

For recruitment analysis to Toxoplasma gondii (Tg) vacuoles, glass-bottom 96-well plates were imaged on an ArrayScan VtI Live High Content Imaging Platform but using 40x magnification and depending on the experiment, 50 fields of view per well. In both cases, following image acquisition, the images were exported from HCS Studio Cell Analysis Software as single channel 16-bit tiff files before they were fed into the HRMAn analysis pipeline.

For recruitment analysis to Salmonella enterica Typhimurium (STm) vacuoles, images of coverslips were acquired on a Ti-E Nikon microscope equipped with an LED-illumination and an Orca-Flash4 camera using a 60x magnification. 75 fields of view per coverslip were acquired using multi-position acquisition. Images were exported as single channel 16-bit tiff files with Nikon NIS Elements software before they were fed into the HRMAn analysis pipeline.

Generally, HRMAn can work with any common image file format, but the use of uncompressed, lossless formats like tiff (or png) is recommended. Furthermore, HRMAn can work with images acquired on any type of fluorescence microscope and is truly platform independent.

Following image acquisition, the images were loaded into the HRMAn pipeline. Images can be in any common file format, preferably as single-channel tiff files. The used image reader loads images from all file formats supported by SCIFIO (more information can be found here: http://scif.io). If the images were not acquired on a high-content imaging platform, they can be renamed with HRMAn to mimic the file names and the plate format. This is needed to cluster the output data and perform error calculation. Furthermore, the OME-XML-metadata is loaded and information on the image is extracted (e.g. image size, type and origin). While the images are loaded into KNIME, the user is asked to provide some basic information on the image acquisition and the type of analysis to be performed. This includes the used magnification, type of analysis, channel number and order and providing a plate map to cluster the data.

HRMAn then pre-processes the images, metadata and provided information and lets the user inspect the input images arranged into a grid and sorted by the field of view. Next the input images undergo illumination correction by dividing the background as a mean image of all acquired images in a channel-wise fashion. Following this step, the individual channels are segmented to detect the Nuclei, the pathogens and the cells:

1. Nuclei are detected using Otsu’s method thresholding (Otsu, 1979), a watershed and connected component analysis. Fields of view containing insufficient numbers of nuclei (i.e. empty fields) are excluded from the following analysis.

2. The pathogens (or vacuoles) are detected after image normalization and filtering through thresholding using Otsu’s method. Incomplete labels are corrected by filling holes and pathogen vacuoles are separated through water-shedding. Labels are created with a connected component analysis.

3. Cell labels are created using Huang thresholding (Huang et al., 2004) and a Voronoi segmentation using the nuclei labels as starting points. Optionally the images can be enhanced using Contrast Limited Adaptive histogram equalization (CLAHE) to improve the segmentation accuracy. All cell labels touching the border of an image are excluded from the analysis.

Furthermore, the created labels are filtered based on their size and the user-defined parameters such as magnification and detector size. The filter values for STm and Tg were empirically determined using thousands of images from different experiments. Based on which pathogen type is chosen by the user, HRMAn will adjust the filters automatically. Using labelling arithmetic, pathogen labels that are not contained within a cell label are removed from the dataset, as they represent extracellular pathogens. The created labels can then be inspected by the user through an interactive label viewer displaying the original image next to the labels.

Using the created labels, the infection readouts for stage 1 are created: First, cell numbers N_Cells and vacuole numbers N_Vacuoles are determined by counting the numbers of respective labels in all acquired fields per well (=replicate). Using these values, the vacuole to cell ratio is calculated:

Next, the dependencies between the cell labels and vacuole labels are used to calculate the proportion of infected cells and the infection levels of the cells. The label dependencies determine which vacuole labels V₁, V₂, …, V_i are contained by which cell label C₁, C₂, …, C_j. If a cell label C contains at least one vacuole label V, the cell is considered as infected cell C_inf. This is used to calculate the proportion of infected cells:

$\%Infectedcells=\frac{N_{Cinf}}{N_{Cells}}$, with N_Cinf as the number of infected cells C_inf

To determine the more precise distribution describing how many vacuoles are contained by which proportion of cells (=Infection levels) the cells C are split into subgroups according to the number of vacuoles they contain (no vacuoles or uninfected = C₀, 1 vacuole = C₁, …, 5 or more vacuoles per cell = C_≥5) and then the proportion is calculated:

0 vacuoles per cell (uninfected): $uninfected=\frac{N_{C0}}{N_{Cells}}$

1 vacuole per cell: $1vac/cell=\frac{N_{C1}}{N_{Cells}}$

…

5 or more vacuoles per cell: ${\displaystyle >5vac/cell=\frac{N_{C\ge 5}}{N_{Cells}}}$

Also based on the dependencies, the mean euclidean distance $d$ between the centroid of the vacuole labels (with coordinates X_V and Y_V) within a cell and its nucleus’ centroid (with coordinates X_N and Y_N) is determined as the position of the vacuole:

Using the vacuole labels, of vacuoles contained within cells, and working on the original images the properties of the vacuoles are measured as mean values for each well. These include mean vacuole size, shape descriptors (Circularity, Perimeter, Convexity, Extent, Diameter) and fluorescence properties (Minimum, Mean and Maximum Fluorescence).

Using the above determined values as attributes, a decision-tree machine learning algorithm determines good classifiers and employs them to classify each vacuole label V_i based on how many individual pathogens P_Vi it contains. This step requires providing an annotated dataset.

Based on this classification the vacuoles can be divided into individual groups for the number of pathogens they contain (1/ vac = V_1Vac, 2/ vac = V_2Vac, 4/ vac = V_4Vac and four or more/vac = V_≥4Vac) and the number of vacuoles in each group is counted (e.g. number of vacuoles that contain just one pathogen = N_V1Vac). To calculate the proportion of replicating pathogens, the number of vacuoles that contain at least two pathogens is divided by the total number of vacuoles:

Similarly, the individual proportions of the vacuole groups are calculated to illustrate pathogen replication distribution:

1 pathogen per vacuole: $1/vac=\frac{N_{V1Vac}}{N_{Vacuoles}}$

2 pathogens per vacuole: $2/vac=\frac{N_{V2Vac}}{N_{Vacuoles}}$

…

4 or more pathogens per vacuole: ${\displaystyle >4/vac=\frac{N_{V\ge 4Vac}}{N_{Vacuoles}}}$

Combining the information on the number of vacuoles and the number of pathogens P_Vi each individual vacuole V_i contains, the total number of pathogens N_Pathogens is calculated:

This can be used to determine the Pathogen load by normalization to the cell number:

This concludes stage 1 infection analysis performed by HRMAn. In the end of the analysis, the values calculated for each well or replicate is combined with the values for the other wells belonging to the same sample group based on the user-provided plate map and error calculation is performed. If the user decides to perform only stage 1 infection analysis the HRMAn image analysis pipeline will stop here, if host protein recruitment analysis is to be performed the data will be fed into the second stage for which the implemented deep Convolutional Neural Network (CNN) has to be created and trained first.

To classify pathogen recruitment, we employed a deep Convolutional Neural Network (CNN) HRMAlexNet (Figure 1) inspired by published AlexNet (Krizhevsky et al., 2012). Our architecture consisted of a total of 5 convolutional layers, where the first two were immediately followed by local response normalization layers and max pooling layers. The last three convolutional layers were followed by one max pooling layer connected to a fully connected layer. All these layers used rectified linear unit (ReLU) as activation function (Nair and Hinton, 2010). To ensure our neural network can be implemented by other researchers with no coding, it was based on the open source DeepLearning4J library implementation in KNIME Analytics Platform. Changes to the original AlexNet architecture were partially introduced by the KNIME AG team and partially by us in the process of optimizing the architecture to the fluorescence microscopy data. Here further improvements in architectures may be made through recently introduced KNIME-Keras integration (Chollet, 2015).

While having the same number of convolutional layers as AlexNet, as introduced by the KNIME team, their dimensions are different. HRMAlexNet has two fully connected layers instead of three, which pushes the fitting burden more to the convolutional layers (Figure 1).

We have changed the input layer dimensions to fit the multichannel fluorescence microscopy data, since our input two channels, rather than the standard RGB dimension of three. Hence, unlike the original AlexNet, HRMAlexNet was suited to take 100 by 100 by two pixels images as input and designed to run on a single graphic processing unit (GPU).

Furthermore, we have modified the SoftMax (Bishop, 2006) output layer of the architecture to fit our dataset and classification problem. While optimizing the network’s hyperparameters, we found that learning rate updating algorithm proposed by default in the KNIME architecture, as well as in Krizhevsky et al. - Stochastic Gradient Descent (SGD) with Nesterov momentum - was failing to converge on our fluorescence microscopy data (Krizhevsky et al., 2012). Therefore, we have substituted this algorithm for the more advanced learning rate update algorithm ADAM (Kingma and Ba, 2014) with both mean and Variance decay parameters, which proved to converge well on various data and initial learning rate ranges.

The choice of DeepLearning4J as a deep learning library allowed us to use 16-bit microscopy images directly, preventing information loss upon conversion of scientific imaging data. Our deep learning hardware was based on a single Nvidia 1080 Ti GPU set up in Intel Core i7 4790K system equipped with 32 Gb of RAM and a SSD.

Our neural networks were trained using SGD based backpropagation (using ADAM updater with ADAM Mean Decay of 0.9 and ADAM Variance Decay of 0.999 (Kingma and Ba, 2014)) on the augmented original data over at least 80 epochs. To fully utilize the GPU capacity, training was performed in mini-batches of 200. We employed the Xavier algorithm for the weight initialization strategy and negative log likelihood (NLL) as our loss function (Kingma and Ba, 2014). We used learning rates between 0.001 and 0.01 adjusted accordingly to ensure the optimal loss curve decay during training. Together with the weights initialization strategy and the updater choice these were the main hyperparameters optimized in multiple iterations to ensure good training process.

To visualize the attention of the trained HRMAlexNet, we have created a class activation map (CAM) based on a test image from ‘recruitment’ class (Figure 2). Since HRMAlexNet architecture does not have a Global Average Pooling layer to visualize the CAM we have used the occlusion technique (Zeiler and Fergus, 2013).

Vacuole images used for creation of our deep learning model were segmented from large field of view micrographs obtained from high-content imaging. To ensure the dimension of the images are uniform, we padded all vacuole with zero-value padding to a uniform 100 by 100 pixels size. Next, we manually labelled the segmented vacuoles into recruited, non-recruited and artefactual (in case of erroneous segmentation of the vacuole). This labelled dataset was then split into the training and test datasets.

To ensure our neural network has sufficiently diverse learning data, upon splitting the original labelled dataset into training and test subsets we performed data augmentation using a custom developed macro for ImageJ. During the augmentation processing, the original labelled dataset was concatenated with a modified version of it. The modifications included various rotations, image reflecting, image translation within the field of view. As microscopy data are typically rotation-, translation- or reflection-invariant, such modification allowed us to create a better dataset aiming at a more generalized model.

Model validation was performed using the non-augmented test fraction of the labelled dataset previously unseen by the model. For this, we used the trained model as first input and passed the labelled test data through the classifier in the second input. The classification accuracy was assessed by accuracy score, numbers of true positive, false positive, true negative and false negative, as well as Cohen’s kappa values. A direct summary of the accuracy was visualized in a confusion matrix illustrating a mismatch between original label (Ground Truth) and the class assigned by the classifier (Figure 2).

For recruitment analysis, the vacuole labels created in stage 1 of the analysis are dilated over 20 iterations to create non-overlapping regions of interest (ROIs) around them. Simultaneously, the fluorescence images of the pathogen and the respective channel with fluorescence signal of the host protein are merged into a dual channel image. The created ROIs are used to crop the dual channel images, which creates images of the pathogen and its surrounding stained host protein. The images are clipped to 100 by 100 pixel and fed into a feedforward predictor (classification) which uses the provided and trained deep convolution neural network (CNN) to classify the pathogens vacuoles based on their coating.

Once the vacuoles are separated into two groups, they are analyzed with the above described methods of stage one infection analysis but additionally comparing recruited versus non-recruited vacuoles. Thus, in addition to the overall infection parameters from stage one the user is provided with the same parameters but further layered for the cellular response.

In the case of co-recruitment analysis, two images of each vacuole are created with both containing the pathogen signal, but each containing a different second channel, representing the different stainings. After classification with the CNN, the vacuoles can be compared for recruitment or co-recruitment and all the above described parameters are calculated for them individually. Using the previously determined label dependencies of vacuoles V_i and cells C_i and the classification of the vacuoles V_i by the CNN, HRMAn can furthermore calculate the proportion of cells that do respond to infection by decorating at least one vacuole and the proportion of vacuoles decorated per cell, if a single cell contains more than one pathogen vacuole.

Furthermore, working only on the decorated vacuoles, we used a custom-made Fiji code to create a pixel-wise radial intensity profile starting from the pathogen centroid. The distance of the maximum fluorescence intensity is then used to define the distance of the coat from the pathogen centre. Moreover, the mean fluorescence intensity of the coat is determined and can be used as a readout for the amount of protein recruited to each pathogen vacuole.

Finally, all mean values and errors for the replicate conditions, as defined by the user’s plate map, are calculated and written into a single spreadsheet file. Before this, the user can also define a scaling factor between pixel and actual metric values which will adjust the output values from pixel (px) to μm or to μm² respectively.

THP-1 (TIB202, ATCC; RRID:CVCL_0006) were maintained in RPMI with GlutaMAX (Gibco) supplemented with 10% FBS (Sigma), at 37 ˚C in 5% CO₂. THP-1s were differentiated with 50 ng/mL phorbol 12-myristate 13-acetate (PMA, P1585, Sigma) for 3 days and then rested for 2 days by replacing the differentiation medium with complete medium without PMA. Cells were not used beyond passage 20. Human Umbilical Vein Endothelial cells, HUVECs, (C12203, Promocell), were maintained in M199 medium (Gibco) supplemented with 30 μg/mL endothelial cell growth supplement (ECGS, 02–102, Upstate), 10 units/mL heparin (H-3149, Sigma) and 20% FBS (Sigma). Cells were grown on plates, pre-coated with 1% (w/v) porcine gelatin (G1890, Sigma) and cultured at 37 ˚C in 5% CO₂. HUVEC were not used beyond passage 6. HeLa (ECACC, Sigma; RRID:CVCL_0030), A549 (CCL-185, ATCC; RRID:CVCL_0023), mouse embryonic fibroblasts (MEF) and human foreskin fibroblasts, HFFs (SCRC-1041, ATCC; RRID:CVCL_3285), were cultured in DMEM with GlutaMAX (Gibco) supplemented with 10% FBS (Sigma), at 37 ˚C in 5% CO₂. HeLa and A549 cells were not used beyond passage 25 and HFFs were not used beyond passage 15. All cell cultures were performed without addition of antibiotics and the cells were regularly tested for mycoplasma contamination by immunofluorescence, PCR and agar test and found to be mycoplasma-negative.

All five cell lines used in this publication were stimulated for 16 hr in complete medium at 37 ˚C with addition of 100 IU/mL human IFNγ (285-IF, R&D Systems) prior to infection, if not indicated otherwise.

Tg expressing luciferase/eGFP (type I RH, type II Prugniaud (Pru) and type III CEP), type II PruA7 (Kim et al., 2007), type II PruA7 + ROP16_I, type II PruA7 ΔGRA15 and type II CEP + ROP18_I (all from Virreira Winter et al., 2011) were maintained in vitro by serial passage on monolayers of HFF cells, cultured in DMEM with GlutaMAX (Gibco) supplemented with 10% FBS (Sigma), at 37 ˚C in 5% CO₂. Type II Tg expressing TgIST-HA-Flag (Pru ΔKU80 + TgIST HF), type I Tg expressing GRA16-HA (RH ΔKU80 + GRA16 HA) and type I Tg expressing GRA24-HA-Flag (Pru ΔKU80 + GRA24 HF) were a gift from Mohamed-Ali Hakimi (Bougdour et al., 2013; Braun et al., 2013; Gay et al., 2016).

Parasites were always passaged the day before infection onto new HFFs to obtain parasites with a high viability for infection. Tg were prepared from freshly 25G syringe-lysed HFF cultures in 10% FBS by centrifugation at 50 x g for 3 min and transferring the cleared supernatant into a new tube and subsequent centrifugation at 500 x g for 7 min and re-suspension of the pelleted parasites into fresh complete medium. Then, the parasites were added to the experimental cells at a MOI of 3 for both type I and type II strains. The cell cultures with added Tg were then centrifuged at 500 x g for 5 min to synchronize the infection. Two hours post-infection, the cultures were thoroughly washed two times with warm PBS (806552, Sigma) to remove any uninvaded parasites and fresh complete medium was added prior to culturing at 37 ˚C, 5% CO₂ for the required time.

Salmonella enterica Typhimurium 12023 strain containing the plasmid pFVP25.1, carrying gfpmut3A under the control of the rpsM constitutive promoter (Valdivia and Falkow, 1996) were grown in Luria Bertani (LB) medium supplemented with 50 µg/ml ampicillin (11593027, Gibco). Prior to infection, bacteria were grown to induce SPI-1 T3SS expression: cultures of STm were grown at 37°C in LB, diluted 1:50 into fresh LB containing 300 mM NaCl (746398, Sigma) the next morning and incubated shaking at 200 rpm until OD600 = 0.9–1.0 was reached. Bacteria were washed in medium without FBS before use. Cells were infected at a MOI of 50 and infections were synchronized by centrifuging bacteria onto the cells at 750 x g for 10 min. 15 min post infection, the cells were thoroughly washed three times with warm PBS to remove extracellular bacteria and medium containing 100 ug/mL Gentamicin (15750060, Gibco) was added for 30 min. Then, Gentamicin concentration was reduced to 10 μg/mL and cells were incubated further at 37°C, 5% CO₂ for the appropriate amount of time.

Antibodies used in this study were rabbit pAb anti-p62 (#PM045, MBL; RRID:AB_1279301), mouse mAb anti-GRA2 (A1298, Biovision) mouse mAb anti-ubiquitin FK2 (PW8810, Enzo Lifesciences; RRID:AB_10541840), mouse mAb anti-SAG1 (home-made) and rat mAb anti-HA (11867423001, Sigma). Secondary antibodies used were Alexa Fluor 647-conjugated goat anti-rabbit (A-21245, Invitrogen; RRID:AB_141775), anti-rat (A-21247, Invitrogen; RRID:AB_141778) or anti-mouse (A-21236, Invitrogen; RRID:AB_141725), Alexa Fluor 488-conjugated goat anti-mouse (A-11001, Invitrogen; RRID:AB_2534069) and Alexa Fluor 568-conjugated goat anti-mouse (A-11004, Invitrogen; RRID:AB_141371).

Data was plotted using Prism 7.0e (GraphPad Inc.) and presented with error bars as standard error of the mean (SEM). Significance of results was determined by non-parametric one-way ANOVA or unpaired t-test as indicated in the figure legends. Correction for multiple comparisons was performed by controlling the False Discovery Rate with Two-Stage step-up method after Benjamini, Krieger and Yekutieli.

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

We thank all members of the Frickel lab for productive discussion. We thank Mohamed-Ali Hakimi for providing transgenic Toxoplasma lines. We thank Valentin Bazarevsky and Stefan Helfrich for discussion and critical reading of the manuscript. This work was supported by the Francis Crick Institute, which receives its core funding from Cancer Research UK (FC001076), the UK Medical Research Council (FC001076), and the Wellcome Trust (FC001076). EMF was supported by a Wellcome Trust Career Development Fellowship (091664/B/10/Z). DF was supported by a Boehringer Ingelheim Fonds PhD fellowship. AY, JM were supported by core funding to the MRC Laboratory for Molecular Cell Biology at University College London (JM), the European Research Council (649101-UbiProPox), the UK Medical Research Council (MC_UU12018/7).

© 2019, Fisch et al.

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.