Software Overview:

an end-to-end GUI pipeline, from left to right, for studying interactions between cell populations, here illustrated with target and effector populations. After loading an experiment project that mimics a multiwell plate structure, the user can apply preprocessing steps to the 2D time-lapse microscopy images before segmentation. Target and effector cells are then segmented, tracked, and measured independently. Events are detected from the resulting time series, and the co-culture images are distilled into tables of single-cell measurements. The neighbourhood module links cells in spatial proximity, and the cell-pair signal analysis framework facilitates the investigation of interactions between cell pairs. Eye and brush icons indicate steps where visual control and corrections are possible, with an appropriate viewer.

Comparative table of software functionalities with a selection of available solutions.

A ✓ is attributed if the task can be carried out without coding. The use of an integrated solution or plugin is indicated in parentheses. ∗Cell-cycle stages only. †Static per-frame neighbour measurement only; no dynamic pair tracking or time series analysis.

Functional response of immune cells in a spreading assay.

A) Schematic (top) and snapshot (bottom) of a spreading assay imaged by time-lapse RICM. Primary NK cells sediment on an HER2-coated surface in the presence of bispecific anti-CD16 × HER2 antibodies. Cells touch the surface in a desynchronised manner and may spread after a stochastic hovering duration. The contours of the cells are represented in orange for cells classified as spread on the surface and blue for hovering cells (corresponding colours are also used in the schematic). B) A traditional segmentation pipeline is used to generate a first instance segmentation of the cells imaged in RICM. Cell masks can be manually curated before training a new Cellpose [37] model, either from scratch or by fine-tuning an existing generalist model. Barplot comparing segmentation scores obtained for three different methods that can be called within the software. With the improved results yielded by the new model, the user can proceed to tracking and measurements. C) Intensity time series for a cell performing a contact and spreading sequence (see text for details). D) Single-cell intensity time series colour-coded for three classes: 1) the cell spreads during the movie (’event’, orange), 2) the cell is not observed to spread during the movie (’no-event’, green) and 3) the cell is already spread at the beginning of the movie (’left-censored’, purple). E) Benchmarking of cell class automatic determination reported by a confusion matrix, and comparison of spreading times determined automatically. F) Individual cell time series (tonal, morphological, etc – grey traces) are synchronised with respect to a characteristic time to measure the average population response at the event time (coloured lines).G) Hovering survival curves for different bsAb concentrations: fraction of cells that have not yet spread as a function of time after surface contact. The shaded area represents the 95% confidence interval of the survival estimate. H) Spreading rate (dA/dt at tspread) for different bsAb concentrations, displayed as boxplots. The error bars correspond to the 95% confidence interval. Statistical significance was assessed using the Kolmogorov-Smirnov test, with P > 0.05: ns, P < 0.05: *, P < 0.01: **, P < 0.001: ***, P < 0.0001: **** .

High-throughput cytotoxic response of cancer cells in co-culture with immune cells.

A. Schematic side-view of target/NK cells co-culture assay for bispecific ADCC (top) and representative multimodal composite images, obtained at two different time points, with target nuclei labelled in blue, dying cells in red and NK cells in green (bottom). Corresponding colours are also used in the schematics. B. Decomposition of partly overlapping fluorescence channels and benchmark of segmentation DL models. Brightfield and fluorescence images at three different time points. Hoechst channel (target nucleus) is taken as input to the existing StarDist versatile fluo or Cellpose nuclei models (available directly in Celldetective). Two new models trained in Celldetective were benchmarked: a transfer of the StarDist versatile fluo model on our MCF-7 nuclei dataset, with Hoechst as input, and a StarDist multimodal model trained from scratch using 4 channels as its input. C. Time series of nuclear fluorescence intensities and nucleus apparent area for a set of non-dying target cells (left) and of dying cells (right); the reference time t0 is the death time for the dying cells. Individual traces in black, average in colour. The coloured shaded area the standard deviation. D. Benchmark of three methods for event classification and regression (Threshold method on PI, DL classifier on PI, DL classifier on PI and nuclear area). Top row: confusion matrices with 3 classes (fraction of predictions displayed). Bottom row: correlation plots for the precision on death time determination. E. Survival curves of target cells without NK cells or in the presence of NK cells for different bsAb concentrations. The shaded area represents the 95% confidence interval of the survival estimate. F. Schematics for neighbour counting and histogram of target neighbour counts (time-averaged per cell). The dashed line at 3.5 neighbours represents the median of the distribution, used to split the population into low- and high-density subgroups. G. Survival curves for two subpopulations of targets at 100 pM bsAb, split as a function of the local target cell density (below or above 3.5 neighbours). The shaded area represents the 95% confidence interval of the survival estimate. The non-integer threshold of 3.5 arises because the neighbour count is time-averaged over the observation period for each cell.

Single time point analysis of effector-target interactions.

In the absence of effector tracking, fluorescent bsAbs were used to study the distribution of Ab on effectors and targets. A. bsAb intensity measured on effector cells on targets with high or low antigen expression. bsAb CE4-21 binds to both HER2 and CD16, while CE4-X control antibody binds only to the HER2 antigen on target cells. B. Representative snapshot of fluorescence channel for each condition of A. C. Empirical cumulative distribution functions (ECDFs) for the simulated (grey, sum of CE4-21/no contact + CE4-X/contact) and observed (orange, CE4-21/contact) distributions. The dotted black line represents the Kolmogorov-Smirnov statistic, i.e. the maximum separation between the two ECDFs.

Effector-target dynamic interactions.

A. Fraction of LAMP1-positive effector cells for different bsAb conditions, decomposed by contact with the targets (over all time points). Bars indicate the mean fraction across three replicate positions; error bars denote the standard deviation. B. Average effector velocity for different bsAb conditions, decomposed by target contact (up to two estimates per effector). C. Examples of killer-victim identification with one (left) or two (right) killers. One target cell (centre) is connected to effector neighbours via coloured segments. Dynamic effector-target interaction monitoring includes PI signal of target cell, relative distance and relative velocity between target and effector, as well as LAMP1 signal in effector neighbours. D, E. Characteristic parameters for killer/victim and non-killer/victim pairs. Effector cells are manually annotated as potential killer or not. Each pair of points represents one victim target, with the average over all neighbours of the parameter decomposed by killer class. D. LAMP1 intensity. E. Relative effector-target velocity.

Main GUI windows:

A) Experiment project selection window. B) GUI to generate a new experiment, where the user provides the metadata. C) Main interface after loading a project. The process block for the effector population is unraveled, showing the 4 main steps detailed in Fig. 1.

Processing modules to extract single cells:

Processing modules are shown on the left, with brief illustrations of the main options, while the output and visualisation modules are represented on the right side, following Celldetective’s graphical structure. A typical pipeline features successively (from top to bottom): A) an instance segmentation (either with traditional thresholding method or a deep learning model, stored in a model zoo, using StarDist [36] or Cellpose [38]). The output data are masks, which can be visualised and annotated using napari [57], for correction or model retraining. B) Bayesian tracking associated with feature extraction (btrack [42]), the output data being trajectories, which can also be visualised with napari. C) Single-cell measurements of intensity, morphology, texture (based on cell position or on mask), the output being tables of trajectories enriched with features.

Background correction methods.

Screenshots of the background correction tabs in the software and illustration of the associated pipeline. Coloured bounding boxes match graphical parameters to their role in the pipeline. The thick black arrows indicate the starting point of the pipelines. A) The steps for a fit-based correction involve selecting a channel of interest, setting a threshold on the standard-deviation filtered image to exclude the cells (done graphically with a viewer). Then a model (either plane or paraboloid) is fit on the background pixels for each image. The fitted background is either subtracted or divided, with or without clipping. B) With the model-free approach, a single background is reconstructed per well, following the same threshold approach but leveraging the multi-positional information to construct a median background. Instead of applying the background directly, it is amplified to minimise the difference with the current image’s background. As with A), the background is then either subtracted or divided, with or without clipping.

Designing a traditional segmentation pipeline.

Screenshots of the traditional segmentation pipeline configuration interface applied to a normalised RICM image of spreading primary NK cells. Top: User-selected mathematical operations (subtraction, absolute value) and preprocessing filters (Gaussian blur) are applied to the image, with results displayed in real-time on the right. A binary mask, set using an interactive thresholding slider, is overlaid on the transformed image in purple. Spot detection is then performed on the Euclidean Distance Transform of the binary mask, using parameters for footprint size and minimum object distance; detected spots are shown as red scatter points. Bottom: After applying the watershed algorithm, the instance segmentation of cells is overlaid on the original normalised RICM image. The user can set filters on object features to exclude false positives; rejected cells are marked in red in both the feature scatter plot and the image.

Segmentation correction and annotation.

To view segmentation results, the user can click the “eye” button in the segmentation module, which opens images and labels in napari. Shown is a screenshot of a segmentation outcome for a normalised RICM image of spreading primary NK cells, with labels overlaid on the image. The user can correct the masks directly on the image, for example, to separate cells indicated by white arrows. New labels can be assigned and saved, and the full set of corrected masks can either be saved in place (top-right plugin) or exported as a training sample for a segmentation model (bottom-right plugin).

Overview of segmentation strategies.

The principle is illustrated for a mixture of two cell populations, effector and targets. Celldetective provides several entry points (black arrows) to perform segmentation, with the intent of specifically segmenting a cell population (left: effectors, right: targets). The generalist DL models are listed in Tab. S1. Specific DL models are listed in Tab. S2 for targets and Tab. S3 for effectors. The traditional pipeline refers to a thresholding method accessible through the GUI (see Fig. S4) and which can serve as a starting point to prepare a dataset for training a new DL model. The masks output from each segmentation technique can be visualised and manually corrected in napari. Exporting those corrections into a dataset of paired image/masks can be used either to fit a generalist model (transfer learning) or train a new model from scratch.

Event detection model architectures.

The event detection models consist of two CNN-based models called back to back during inference to 1) classify single-cell time series and 2) detect the event times from the same time series. The models share a similar architecture of a first 1D convolution, followed by ResBlocks encoding the time series into either three neurons with a softmax activation for the classification problem (three classes) or one neuron with a linear activation for the regression problem. As illustrated here, only the time series that were classified as “event” are sent to the regressor model.

Neighbour counting methods:

A) All neighbouring cells in contact or within the isotropic neighbourhood are linked to the reference cell (inclusive method). B) Each neighbouring cell is linked exclusively to the closest reference cell (exclusive method). C) Neighbouring cells in contact or within the isotropic neighbourhood are linked to the reference cell, with weights inversely proportional to their number of neighbours.

Effect size at each time point:

Cliff’s Delta between the simulated sum of two distributions (CE4-21/off-contact/HER2+ + CE4-X/in-contact/HER2+) and CE4-21/in-contact/HER2+ computed at each timepoint, independently.

Representative LAMP1 antibody staining.

Fluorescence image of the co-culture assay showing NK effector cells stained with anti-LAMP1 APC antibody. Top: Control without bsAb. Bottom: in the presence of 1nM bsAb CE4-21. Left: MCF7 WT targets. Right: MCF7-HER2+ targets.

Generalist deep learning segmentation models.

This table lists the different generalist models (Cellpose or StarDist) which can be called natively in Celldetective. The sample images are cropped to (200 × 200) px and rescaled homogeneously to fit in the table.

MCF7 nuclei segmentation models in the presence of primary NK cells.

Each model was trained on the same dataset of ADCC images, picking only the relevant channels.

Primary NK segmentation models.

The models have been trained on a dataset of annotated primary NKs in ADCC images (primary NKs w MCF7).

Event detection models.

We trained the following 1D DL models to classify and regress events of interest. The mean event response, centred at the event time, is shown for each channel in the pattern column.