Shallow classification using manually defined features.

(A) Image overview of 1321N1 (top) and SH-SY5Y (bottom) cells after CP staining acquired at higher magnification (Plan Apo VC 60xA WI DIC N2, NA = 1,20). Scale bar is 30µm. The channel, wavelength and dye combination used is listed in the table below the figure. This color code and channel numbering is used consistently across all figures. (B) UMAP dimensionality reduction using handcrafted features. Each dot represents a single cell. The color reflects either cell type condition (left) or replicate (right). This shows UMAP clustering is a result of cell type differences and not variability between replicates. (C) Feature importance deducted from the UMAP (feature maps). Each dot represents a single cell. Three exemplary feature maps are highlighted alongside the quantification per cell type. These feature representations help understanding the morphological features that underly the cluster separation in UMAP. (D) Random Forest classification performance on the manually defined feature dataframe with and without exclusion of redundant features. Average confusion matrix (with redundant features) and Mean Decrease in Impurity (reflecting how often this feature is used in decision tree splits across all random forest trees). All features used in the UMAP are used for RF building. Each dot in the violinplot represents the F-score of one classifier (model initialization, N = 30). Classifiers were trained 10x with 3 different random seeds.

Convolutional neural network classification of monoculture conditions.

(A) Schematic of image pre-processing for CNN classification. (B) CNN accuracy and confusion matrix on cell type classification in monocultures. Each dot in the violinplot represents the F-score of one classifier (model initializations, N = 9). (C) Shallow vs. deep learning accuracy for a varying number of input instances (ROIs). Each dot in the violinplot represents the F-score of one classifier (model initialization, N). The ribbon represents the standard deviation on the classifiers. (D) The impact of experimental variability in the training dataset on model performance for shallow vs. deep learning. Classifiers were trained on either 1 (single rep.) or multiple (mixed reps.) replicates (where each replicate consists of N new biological experiment). Mann-Whitney-U-test, p-values resp. 0.7304 and 0.000197. Each violinplot represents the F-score of one classifiers (model initializations). (E) The performance of deep learning classifiers trained in panel D on single replicates (low variability) or mixed replicates (high variability) on unseen images from either the training replicate (cross-validation) or an independent replicate (independent testing) (where each replicate consists of a new biological experiment). Kruskal-Wallis test on single training condition, p-value of 0,026 with post-hoc Tukey test. Mann-Whitney-U-test on mixed training, p-value of 7,47e-6. Each violinplot represents the F-score of one classifier (model initialization, N = 3). (F) Images of example inputs given to the CNN. The composite image contains an overlay of all CP channels (left). The GradCAM image overlays the GradCAM heatmap on top of the composite image, highlighting the most important regions according to the CNN (right). One example is given per cell type.

Required input quality for cell type classification.

(A) Performance of the CNN when only a selection of image channels is given as training input. Each boxplot represents the F-score of one classifier (model initialization, N = 3). Channel numbering is in accordance with fig. 1A. (1 = DAPI; 2 = FITC; 3 = Cy3; 4 = Cy5) (B) Simulation of the effect of increased pixel size (reduced spatial resolution) on the classification performance. Each point is the average of 3 CNN model initializations (N), with bars indicating the standard deviation between models. Red line indicates the average F-score of the original crops. (C) Simulation of the effect of added gaussian noise (reduced signal-to-noise ratio) on the classification performance. Each point is the average of 3 CNN model initializations, with bars indicating the standard deviation between models. Red line indicates the average F-score of the original images. Statistics were performed using a Kruskal-Wallis test with post-hoc Tukey test. (D) CNN performance on 1321N1, RPE1 and ARPE cells. Dots represent different model initializations (N = 3).

Required regional input for accurate CNN training in cultures of high density.

(A) Selected images and insets for increasing culture density with the density categories used for CNN training. (B) Results of 3 CNN models trained using different regional input (the full cell, only nucleus or nucleocentric area) and evaluated on data subsets with increasing density. Each dot represents the F-score of one classifier (model initialization, N = 3) tested on a density subset. Classifiers were trained with the same random seed. Ribbon represents the standard deviation. (C) CNN performance (F-score) of CNN models using different regional inputs (full cell, only nucleus or nucleocentric area). Each boxplot represents 3 model initializations for 3 different random seeds (N = 9). (D) Images of example inputs for both the nuclear and nucleocentric region. The composite image contains an overlay of all CP channels (top). The GradCAM image overlays the GradCAM heatmap on top of the composite image, highlighting the most important regions according to the CNN (bottom). One example is given per cell type. (E) Systematic in- and decrease (default of 18µm used in previous panels) of the patch size surrounding the nuclear centroid used to determine the nucleocentric area. Each dot represents the results of one classifier (model initialization, N = 3). Ribbon represents the standard deviation. The analysis was performed using a mixed culture dataset of 1321N1 and SH-SY5Y cells (Fig. 5).

Cell type prediction in mixed cultures by post-hoc ground truth identification.

(A) Schematic overview of virtual vs. physically mixed cultures and the subsequent CNN models. For virtual mixing, individual cell types arise from distinct cultures, the cell type label was assigned based on the culture condition. For physical mixing, the true phenotype of each crop was determined by intensity thresholding after post-hoc staining. Three model types are defined: Mono2Mono (culture-based label), Co2Co (cell-based label) and Mono2Co (trained with culture-based label and tested on cell-based label). (B) Determining ground-truth phenotype by intensity thresholding on IF (top) or pre-label (bottom). The threshold was determined on the intensity of monocultures. Only the pre-labelled ground-truth resulted in good cell type separation by thresholding for 1321N1 vs. SH-SY5Y cells. (C) Mono2Mono (culture-based ground truth) vs. Co2Co (cell-based ground truth) models for cell type classification. Analysis performed with full cell segmentation. Mann-Whitney-U-test p-value 0,0027. Monoculture-trained models were tested on mixed cultures. Pretrained models were trained on independent biological replicates. These could be finetuned by additional training on monoculture images from the same replicate as the coculture. This was shown to reduce the variation between model iterations (Median performance: Mann-Whitney-U-test, p-value 0,0015; Coefficient of variation: Mann-Whitney-U-test, p-value 3,48e-4). Each dot in the violinplots represents the F-score of one classifier (model initialization, N = 9). Classifiers were trained 3x with 3 different random seeds.

iPSC-derived differentiation staging using morphology profiling.

(A) Schematic overview of guided vs. spontaneous neural differentiation. DIV = days in vitro. Selected timepoints for analysis of the spontaneously differentiation culture were 13, 30, 60 and 90 days from the start of differentiation of iPSCs. (B) Representative images of morphological staining (color code as defined in fig. 1A) and post-hoc IF of primed and differentiated iPSC-derived cultures (guided differentiation). Ground-truth determination is performed using TUBB3 (for mature neurons) and Ki67 (mitotic progenitors). (C) Fraction of neurons vs. NPC cells in the primed vs. differentiated condition as determined by IF staining. Upon guided differentiation, the fraction of neurons increased. (D) left: CNN performance when classifying neurons (Ki67-/TUBB3+) vs. NPC (Ki67+/TUBB3-) cells using either a condition-based or cell-type based ground truth. Each dot in the violinplots represents the F-score of one classifier (model initialization). Classifiers were trained with different random seeds. Mann-Whitney-U-test, p-value 4,04e-4. Right: comparison of CNN vs. RF performance. Mann-Whitney-U-test, p-value 2,78e-4. (E) Fractional abundance of predicted cell phenotypes (NPC vs. neurons) in primed vs. differentiated culture conditions using the cell-based CNN. (F) Unsupervised and supervised UMAP of the cell-based CNN feature embeddings. Plot color coded by cell type. Points represent individual cells. (G) Representative images of spontaneously differentiating neural cultures. Color code as defined in fig. 1A. (H) Prediction of differentiation status using the cell-based CNN model trained on guided differentiated culture.

iPSC cell type identification using morphology profiling.

(A) Representative images of iPSC-derived neurons, astrocytes and microglia in monoculture with morphological staining. Color code as defined in fig. 1A. Prediction accuracy of a CNN trained to classify monocultures of iPSC-derived astrocytes, microglia and neurons with confusion matrix (average of all models). Each dot in the boxplot represents the F-score of one classifier (model initialization, N = 9). Classifiers were trained 3x with 3 different random seeds. (B) Representative images of monocultures of iPSC-derived neurons and microglia treated with LPS or control. Color code as defined in fig. 1A. Prediction accuracy and confusion matrix (average of all models) are given. Each dot in the violinplot represents the F-score of one classifier (model initialization, N = 3). (C) Representative images of a mixed culture of iPSC-derived microglia and neurons. Ground-truth identification was performed using IF. Each dot in the violinplot represents the F- score of one classifier (model initialization). Classifiers were trained 3 different random seeds. Results of the CNN are compared to shallow learning (RF). The same analysis was performed for mixed cultures of neurons and microglia with LPS treatment or control. A layered approach was used where first the neurons were separated from the microglia before classifying treated vs. non-treated microglia. Each dot in the violinplot represents the F-score of one classifier (model initialization, N = 9). Classifiers were trained 3x with 3 different random seeds.

Specifications of morphological staining composition.

Used antibodies.

Specifications of the used laser lines, excitation and emission filters.

Python packages used for image and data analysis alongside the software version.

Definition of handcrafted features (according to the scikit-image documentation). All of these features were extracted for each fluorescent channel (1-4) and region (nucleus, cytoplasm and whole-cell) in the cell painting images.

(A) PCA dimensionality reduction on handcrafted features extracted from monocultures of 1321N1 and SH-SY5Y cells. Each point represents an individual cell. (B) UMAP of feature embeddings of the CNN trained to classify 1321N1 and SH-SY5Y monocultures. Each point represents an individual cell. (C) Examples of misclassified ROIs. (D) Left: Representative images of 1321N1 cells with increasing density alongside their cell and nuclear mask produced using resp. Cellpose and Stardist. Images are numbered from 1-5 with increasing density. Upper right: The number of ROIs detected in comparison to the ground truth (manual segmentation). A ROI was considered undetected when the intersection over union (IoU) was below 0,15. Each bar refers to the image number on the left. The IoU quantifies the overlap between ground truth (manually segmented ROI) and the ROI detected by the segmentation algorithm. It is defined as the area of the overlapping region over the total area. IoU for increasing cell density for cell and nuclear masks is given in the bottom right. Each point represents an individual ROI. Each bar refers to the image number on the left. (E) Examples of segmentation mistakes made by the Stardist segmentation algorithm for nuclear segmentation for different culture densities. (F) Nuclear area in function of density. (G) Definition of cell regions given as training input for nuclear and nucleocentric model training.

Influence of the nuclear/cellular size on CNN prediction and association with the input background (space artificially set to zero outside of the segmentation mask).

(A) Quantification of average nuclear/cellular size per cell type. (B) GradCAM images for 10 random seeds for crops and CNN models trained with background either set to zero or ‘random speckle’. (C) CNN prediction results for models trained on crops with background either set to zero or ‘random speckle’. Each dot in the violinplots represents the F-score of one classifier (model initialization, N = 3). (D) Feature map (see UMAP in figure 1 B and C) of 1321N1 and SH-SY5Y cells showing the contribution of nuclear/cellular area to the cell type cluster separation. Each point represents an individual cell.

GradCAM maps per region and density.

Fluorescence quenching over time using LiBH4.

(A) Images before and after quenching for all 4 fluorescence channels. (B) Time curve of normalized image-level fluorescence intensity during incubation with 1mg/ml LiBH4.

Methods.

(A) Pipeline used for morphological profiling in mixed cultures. (B) Overview of the steps within the image analysis pipeline. (C) Evaluation of accuracy, true negative and true positive rate during CNN training (1321N1 vs. SH-SY5Y in monocultures) across all 50 epochs.