Celldetective, an AI-enhanced image analysis tool for unraveling dynamic cell interactions

eLife Assessment

CellDetective is an important software package for segmentation, tracking, and analysis of time-lapse microscopy datasets, specifically designed to be accessible to researchers without coding expertise. The authors provide convincing evidence of its capabilities through comprehensive validations and well-executed comparisons across immunological assays, and the latest version adds support for defining and visualizing multiple cell subsets. The current implementation remains limited to 2D widefield imaging, though the authors provide a sound rationale for this scope, and one interface issue (a fixed main-window size on some systems) still affects usability. Overall, this work will be of significant interest to the bioimaging community, especially those in immunology and cell biology, and has applicability extending well beyond immune profiling.

https://doi.org/10.7554/eLife.105302.3.sa0

Significance of the findings:

Important: Findings that have theoretical or practical implications beyond a single subfield

Landmark
Fundamental
Important
Valuable
Useful

Strength of evidence:

Convincing: Appropriate and validated methodology in line with current state-of-the-art

Exceptional
Compelling
Convincing
Solid
Incomplete
Inadequate

During the peer-review process the editor and reviewers write an eLife Assessment that summarises the significance of the findings reported in the article (on a scale ranging from landmark to useful) and the strength of the evidence (on a scale ranging from exceptional to inadequate). Learn more about eLife Assessments

Abstract
Introduction
Results
Discussion
Materials and methods
Data availability
References
Article and author information
Metrics

Abstract

Analysis of multimodal and multidimensional data capturing dynamic interactions between diverse cell populations is a current challenge in bioimaging, especially in the context of immunology and immunotherapy research. Here, we introduce Celldetective, an open-source Python-based software tool designed for high-performance end-to-end analysis of image-based in vitro immune and immunotherapy assays. Celldetective is purpose-built for multicondition, 2D multi-channel time-lapse microscopy of mixed cell populations. Although it is optimised for the needs of immunology assays, it is nevertheless broadly applicable to any biological system involving interacting cell populations. The software seamlessly integrates AI-based segmentation, tracking, and automated single-cell event detection, all within an intuitive graphical interface that supports interactive visualisation, annotation, and training options. We showcase its capabilities with original datasets of single immune effector cell interactions with an activating surface mediated by bispecific antibodies and pairwise interactions in antibody-dependent cell cytotoxicity events.

Introduction

An overarching goal of post-genomic biomedical research is to decipher cell states and transitions, as well as cell interactions, in functional dynamic assays (Rafelski and Theriot, 2024; Armingol et al., 2024). This requires high-throughput and high-resolution experimental methods supported by advanced and powerful analyses (Kramer et al., 2022; Bagheri et al., 2022). The field of immunology presents specific challenges, related to complex cell states characterised by transient and heterogeneous receptor expression, highly motile behaviour, and immune function relying on dynamic cell-cell interactions (Malissen and Bongrand, 2015; Belardi et al., 2020). These features are largely out of reach in ensemble biochemical measurements (Kiesgen et al., 2021), as well as low time-resolution single-cell multi-omic data (Ginhoux et al., 2022).

Microscopy-based in vitro assays can bridge temporal and spatial scales, helping to decipher lymphocyte activation from molecular to cellular scales (Limozin and Puech, 2019; Bongrand, 2023; Sengupta et al., 2024). Combining high content and high spatial resolution, imaging flow cytometry was applied to reconstruct disease progression in single immune cell populations (Eulenberg et al., 2017) or to functionally characterise therapeutic antibodies against liquid tumours (Shetab Boushehri et al., 2023). The analysis is, however, limited to single cells or cell doublets. Multi-channel fluorescence microscopy was successfully applied to image-based profiling of human T lymphocytes and natural killer (NK) cells (German et al., 2021) or therapeutic targets in drug discovery (Chandrasekaran et al., 2021), but it provided limited dynamic information on the system. In some cases, nanowell grids were used to organise cell contacts and facilitate image analysis (Olofsson et al., 2014; Lu et al., 2019) at the expense of imposing less physiological conditions.

High-content multi-channel spatiotemporal imaging of living cells provides rich, dynamic, and multiscale information, from molecular to cell population scale (Alieva et al., 2023). In single-cell image sequences, time-dependent signals contain functional information, traditionally collected or treated manually in immunological settings (Vanherberghen et al., 2013). Recent methods performed profiling on time-dependent data for cellular phenotyping, but without measuring single events in time (Wiggins et al., 2023). Functional dynamic information like calcium imaging can also be extracted for immune cell classification (Salles et al., 2013; This et al., 2024) or to interpret the brain’s neuronal activity (Sità et al., 2022). Most recent works on cell dynamics, especially in the context of cancer immunotherapy, involve specialised microscopy techniques and complex analysis protocols (Lee et al., 2020; Ichise et al., 2022; Liu et al., 2024), sometimes using proprietary software (Dekkers et al., 2023; Roudot et al., 2023).

In recent years, machine learning and deep learning (DL) have revolutionised the field of image analysis (Eulenberg et al., 2017; Berg et al., 2019; Moen et al., 2019; Arzt et al., 2022; Shetab Boushehri et al., 2023; Jan et al., 2024; Shroff et al., 2024). A critical step in the context of cell images is segmentation, for which DL tools are highly successful (Vicar et al., 2019; Falk et al., 2019; Weigert et al., 2020; Stringer et al., 2021; Pachitariu and Stringer, 2022; Greenwald et al., 2022). However, they often rely on pre-trained models for which the original dataset is not always accessible or can be too remote from users’ data (Edlund et al., 2021), limiting adaptability. For dynamic data, cell tracking is also crucial and can be performed by available tools (Wen et al., 2021; Ulicna et al., 2021; Ershov et al., 2022). Recent methods that integrate profiling with time-dependent information significantly enhance cell tracking performance (Cuny et al., 2022; Soelistyo et al., 2023; Ollion et al., 2024). However, current software tools associating the two tasks are mostly applied to the analysis of cell lineage (Lugagne et al., 2020; Ulicna et al., 2021; Padovani et al., 2022; Schwartz et al., 2023). Most popular-free bioimage analysis software tools such as Fiji (Schindelin et al., 2012), Icy (de Chaumont et al., 2012), or Cellprofiler (McQuin et al., 2018) were not originally designed for DL methods (Gómez-de-Mariscal et al., 2021), relying on extensions and plugins to bring in these new methods, or were not conceived for time-dependent data (Stirling et al., 2021).

Finally, end-to-end analysis tools accessible to experimentalists, adapted for both images and time series, are currently lacking, preventing dataset curation and DL-based event detection. With the exception of MIA (Körber, 2023), existing partial solutions require coding skills (Windhager et al., 2023) or proficiency in integrating/mastering disparate tools (von Chamier et al., 2021).

We introduce Celldetective, an open-source software tool designed to tackle the challenge of analysing image-based in vitro immune and immunotherapy assays. It addresses the problem of cell interactions by extracting and interpreting single-cell behaviour for user-defined cell populations in a 2D co-culture, with pairwise descriptions linking any two populations. The software adapts to varying imaging conditions and cell types, incorporating dataset curation and DL model training features directly within the graphical user interface (GUI). Celldetective organises and automates complicated and time-consuming processes while providing a user-friendly, no-code GUI, making it accessible to experimental scientists without programming expertise.

We tested the software on two applications in immunology and immunotherapy: (i) measuring immune cell behaviour in contact with an antigen-presenting surface using a label-free microscopy technique, and (ii) assessing dynamic cell-cell interactions in a multiwell co-culture of immune and target cells, an established cytotoxic assay, using multicolour fluorescence microscopy.

Results

Software analysis pipeline

Celldetective is implemented as a Python package with a PyQt5-based GUI, available through the PyPi package repository and regularly updated. The source code can be accessed on GitHub (https://github.com/celldetective/celldetective; Torro et al., 2026), while all models, datasets, and demos are hosted in a public Zenodo repository (https://doi.org/10.5281/zenodo.17068651). Extensive documentation is available online (https://celldetective.readthedocs.io). The software is designed to run locally on a single computer equipped with either a graphics processing unit (GPU) or multiple processors. Image processing is conducted frame by frame with garbage collection to significantly reduce memory requirements, allowing it to operate on most laboratory computers. Celldetective is compatible with multi-channel time-lapse microscopy image stack files in TIF/OME.TIF format, which are typically generated by acquisition software such as Micro-Manager (Edelstein et al., 2014).

As illustrated in Figure 1, the software input consists of an experimental project that organises raw data into a two-level folder structure emulating a multiwell plate, commonly used in in vitro immune assays (startup window in Figure 1—figure supplement 1A). At the top level, folders represent wells (or biological conditions), while at the second level, folders represent positions (a field of view associated with the biological condition). Experiment metadata—including channel names and order, spatiotemporal calibration, and biological condition labels—are stored in a configuration file created during project setup (Figure 1—figure supplement 1B). This structure organises the experimental data by biological conditions and fields of view, facilitating the processing of large data volumes.

Figure 1 with 3 supplements see all

Download asset Open asset

Software overview: an end-to-end graphical user interface (GUI) pipeline, from left to right, for studying interactions between cell populations, here illustrated with target and effector populations.

After loading an experiment project that mimics a multiwell plate structure, the user can apply preprocessing steps to the 2D time-lapse microscopy images before segmentation. Target and effector cells are then segmented, tracked, and measured independently. Events are detected from the resulting time series, and the co-culture images are distilled into tables of single-cell measurements. The neighbourhood module links cells in spatial proximity, and the cell-pair signal analysis framework facilitates the investigation of interactions between cell pairs. Eye and brush icons indicate steps where visual control and corrections are possible, with an appropriate viewer.

Upon loading a project, a modular interface organises all possible user actions (Figure 1—figure supplement 1C, with single-cell extraction modules illustrated in Figure 1—figure supplement 2). Image preprocessing enables users to transform images prior to segmentation (Figure 1—figure supplement 3). Independent processing blocks drive the single-cell extraction for each user-defined population. Population-specific segmentation, optional tracking, and measurements reduce the image stacks into tables, where each row represents a single cell at a specific time point. With tracking information available, single-cell measurements are structured and interpreted as time series to detect events. Alternatively, users can classify cells at each time point from their measurements. A viewer is integrated at each step for quality control, error correction, or data annotation. Notably, both segmentation masks and cell tracks can be interactively corrected within the same napari environment, enabling a seamless curation workflow that is directly linked back to the analysis pipeline. The final module addresses cell interactions within or across populations, enabling users to describe and annotate cell pairs. Throughout the process, users can plot and compare single-cell or cell pair measurement distributions via a table exploration module designed for manipulating tracks. Additional analysis modules allow users to perform survival studies and represent synchronised time series over cell populations.

Rather than reinventing existing efficient solutions, we incorporated into Celldetective established open-source DL segmentation methods StarDist (Weigert et al., 2020) and Cellpose (Stringer et al., 2021) (see model list in Supplementary file 1, table S1), alongside the Bayesian tracker bTrack (Ulicna et al., 2021) and the trackpy linking algorithm (Allan et al., 2025). The software provides intuitive graphical options for tuning input and control parameters of these algorithms. The diverse software functionalities are listed and compared to existing software in Table 1. Notably, Celldetective stands out by offering training and transfer options in segmentation, enabling detection of a population of interest within a mixed population, as well as time series annotation and survival analysis, not currently integrated in any other software. These features will be illustrated in the two biological applications described below.

Table 1

Comparative table of software functionalities with a selection of available solutions.

A ✓ is attributed if the task can be carried out without coding. The use of an integrated solution or plugin is indicated in parentheses. *Cell-cycle stages only. †Static per-frame neighbour measurement only; no dynamic pair tracking or time series analysis.

Software feature	ImageJ/Fiji	CellProfiler	CellACDC	MIA	Celldetective
Ref.	Schindelin et al., 2012	McQuin et al., 2018	Padovani et al., 2022	Körber, 2023	This work
Traditional segmentation	✓	✓	✓	✗	✓
DL segmentation	✓	✓	✓	✓	✓
Segmentation correction	✓ (Labkit)	✗	✓	✗	✓ (napari)
Training annotation	✓ (Labkit)	✓ (GIMP)	✓	✓	✓ (napari)
Training	✗	✗	✗	✓	✓
Transfer	✗	✗	✗	✓	✓
Tracking	✓ (TrackMate)	✓	✓	✓	✓ (bTrack, Trackpy)
Track visualisation	✓ (TrackMate)	✗	✓	✗	✓ (napari)
Track correction	✓ (TrackMate)	✗	✓	✗	✓ (napari)
Classification from measurements	✗	✓	✗	✗	✓
Time series annotations	✗	✗	✓^*	✗	✓
Interaction analysis	✗	✓^†	✗	✗	✓
Experiment manager	✗	✗	✓	✗	✓
Multicondition data exploration	✗	✓ (CP Analyst)	✗	✗	✓
Survival analysis	✗	✗	✗	✗	✓

High-throughput and time-resolved analysis of cell adhesion and spreading responses

Bispecific antibodies (bsAbs) act as molecular bridges to force the contact between immune effector cells and cancer cells. Despite the therapeutic interest of these constructs to improve cancer cell killing, their mode of action is poorly understood. In this first application, we study how a bsAb controls the interaction between NK cells and a surrogate cancer cell surface (Turini et al., 2014; González et al., 2019). The bsAb bridges the NK cell’s activating receptor CD16 and the human epidermal growth factor receptor 2 (HER2) coated on the surface. Cell-surface contact and spreading is monitored using reflection interference contrast microscopy (RICM), a label-free technique sensitive to submicrometre distances between the cell membrane and the surface (Limozin and Sengupta, 2009; González et al., 2019; Robert et al., 2021; Sengupta et al., 2024). The assay consists in depositing freshly isolated human primary NK cells on the HER2 surface, in the presence of known concentrations of bsAb. RICM imaging results in variations of grey levels under the cell in a grey background and provides a high spatiotemporal signal related to cell-surface contact without labelling (Figure 2A).

Figure 2 with 3 supplements see all

Download asset Open asset

Functional response of immune cells in a spreading assay.

(A) Schematic (top) and snapshot (bottom) of a spreading assay imaged by time-lapse reflection interference contrast microscopy (RICM). Primary natural killer (NK) cells sediment on a human epidermal growth factor receptor 2 (HER2)-coated surface in the presence of bispecific anti-CD16 × HER2 antibodies. Cells touch the surface in a desynchronised manner and may spread after a stochastic hovering duration. The contours of the cells are represented in orange for cells classified as spread on the surface and blue for hovering cells (corresponding colours are also used in the schematic). (B) A traditional segmentation pipeline is used to generate a first instance segmentation of the cells imaged in RICM. Cell masks can be manually curated before training a new Cellpose (Stringer et al., 2021) model, either from scratch or by fine-tuning an existing generalist model. Barplot comparing segmentation scores obtained for three different methods that can be called within the software. With the improved results yielded by the new model, the user can proceed to tracking and measurements. (C) Intensity time series for a cell performing a contact and spreading sequence (see text for details). (D) Single-cell intensity time series colour-coded for three classes: (1) the cell spreads during the movie (’event’, orange), (2) the cell is not observed to spread during the movie (’no-event’, green), and (3) the cell is already spread at the beginning of the movie (’left-censored’, purple). (E) Benchmarking of cell class automatic determination reported by a confusion matrix and comparison of spreading times determined automatically. (F) Individual cell time series (tonal, morphological, etc.—grey traces) are synchronised with respect to a characteristic time to measure the average population response at the event time (coloured lines). (G) Hovering survival curves for different bispecific antibody (bsAb) concentrations: fraction of cells that have not yet spread as a function of time after surface contact. The shaded area represents the 95% confidence interval of the survival estimate. (H) Spreading rate (dA/dt at $t_{s p r e a d}$ ) for different bsAb concentrations, displayed as boxplots. The error bars correspond to the 95% confidence interval. Statistical significance was assessed using the Kolmogorov-Smirnov test, with p>0.05: ns, p<0.05: *, p<0.01: **, p<0.001: ***, p<0.0001: ****.

Cell segmentation in RICM images is challenging because the cells can be locally brighter or darker than the background, causing direct thresholding techniques to fail (Limozin and Sengupta, 2009). To address this, previous approaches have adopted special techniques, such as background correction and applying a variance filter before thresholding (Sengupta et al., 2006; Dillard et al., 2014; Sengupta et al., 2024), which, however, was often inadequate and required expert intervention. Our goal here was to train a DL model to segment lymphocytes spreading or hovering on the surface, without the need for expert corrections. The complete strategy—background subtraction (Figure 1—figure supplement 3), generating an initial set of labels with the traditional segmentation pipeline (Figure 2—figure supplement 1), correcting errors in the viewer to build a training dataset (Figure 2—figure supplement 2), and training a DL model from this dataset—is illustrated in Figure 2A and B. Moving away from this application, Celldetective provides a whole ecosystem to perform segmentation and adapt DL methods (see Figure 2—figure supplement 3). Here, we trained a new Cellpose model on the RICM dataset (both the dataset and model are available on Zenodo). We benchmarked this model against the traditional segmentation pipeline and the cyto3 generalist Cellpose model (Pachitariu and Stringer, 2022) using a test set, quantifying lymphocyte detection accuracy and segmentation quality separately (see Materials and methods). Hovering cells, which appeared as bright blobs with interference patterns and had an ill-defined area in RICM, were excluded from the segmentation quality score. Our model achieved a higher detection accuracy (0.86), compared to the traditional pipeline (0.62) or the generalist model (0.5). The traditional pipeline tended to produce many false-positive detections, while the generalist model struggled to simultaneously detect spread and hovering cells. Segmentation quality, however, was higher for the pipeline (0.93) than for our model (0.89) or the generalist model (0.88). Since most spread cell labels in the test set required little manual correction (except for contiguous cells), we were not surprised to observe the highest segmentation quality with the pipeline method. The generalist model segmented spread cells well when it could detect them. Overall, the new Cellpose model achieved the best average performance across both tasks (0.88), making it the most effective approach for segmenting cells in this assay.

Next, tracking of cells—which move while hovering and are quasi-stationary while spreading—was performed using a versatile motion model (see Materials and methods), without feature inputs to prevent sharp morpho-tonal transitions from disrupting tracking. We enabled track extrapolation from the beginning of the movies, allowing intensity measurements to start before the cells reached the surface (Figure 2C and D). With track post-processing, we could measure intensities prior to cell arrival, with a position-centred average intensity measurement, in a user-defined disk (position-based), which compared well with the average over the cell mask when available (mask-based). Notably, the position-based intensity showed a spike during the spreading phase, which is characteristic of the cell nucleus approaching the surface and that cannot be detected in the mask-based intensity.

We defined two characteristic event times (Figure 2C): contact time ( $t_{c o n t a c t}$ )—the moment when the cell first appeared in RICM, with an average intensity greater than 1; and the onset of spreading ( $t_{s p r e a d}$ ) when the average intensity dropped below 1. The hovering time ( $t_{s p r e a d} - t_{c o n t a c t}$ ), for spreading cells, is the time required to make a spreading decision. $t_{c o n t a c t}$ is automatically detected as the time of first segmentation. For $t_{s p r e a d}$ , spreading events were identified through condition-based classification (mask-based RICM intensity < 1) and an irreversible event hypothesis (see Materials and methods). We benchmarked classification performance and hovering duration estimates against a manually annotated test set (Figure 2D). The set was annotated with a dedicated viewer (see Video 1). A confusion matrix indicates perfect agreement between automatic classification and ground truth, while a correlation plot comparing estimated hovering durations to the ground truth shows excellent agreement, with an MAE of 0.35 min (Figure 2E).

Video 1

Download asset

posterframe for video — Interactive and dynamic multi-event representation.

The reflection interference contrast microscopy (RICM) channel is loaded in greyscale into the dynamic event annotation viewer, looping at a user-controlled frame rate. First, the user selects the ‘firstdetection’ class, corresponding to the initial contact event of cells with the surface, and interacts with individual cells (represented by centroids marked on the image) to verify and adjust the detected $t_{c o n t a c t}$ times. The outer circle of each cell’s marker represents the event class: red for observed events, blue for no observed event. The inner cross indicates the current status within the class: blue for ‘event not yet happened’, and red for ‘event happened’. By clicking directly on a cell in the image, users can view the RICM intensity time series for that cell, displayed on the left side. Next, the user switches to the ‘spread’ class to review and adjust spread event detection. The user can correct event classifications and timings, seamlessly switching between event types.

After automatically extracting the event times, we examined average population response based on single-cell time series, by synchronising each single-cell measurement to a specific event time to calculate the mean response across the population (Figure 2F). Next, we tested whether varying the concentration of bsAb on the surface impacts spreading. First, we estimated the ‘hovering survival’ by plotting the fraction of cells hovering at a given time (Figure 2G). We observe a higher fraction of spreading cells and shorter spreading decision times with increasing bsAb concentrations. The acceleration of the spreading decision time was particularly apparent in the 10–1000 pM bsAb concentration range, saturating beyond 100 pM. Next, using the table exploration feature in Celldetective, we differentiated the area with respect to time for each cell track, extracting the spreading rate at the onset of spreading ( $t_{s p r e a d}$ ), which increases in the 14–1400 pM range (Figure 2H). Overall, determining event times for individual cells, measured at high throughput, highlights Celldetective capabilities in evaluating cell-surface interactions.

Cytotoxic response in a co-culture with overlapping channels

In the second application, we further tested the capacity of the same bsAbs to induce cell-cell contacts and trigger antibody-dependent cell cytotoxicity (ADCC). For this purpose, we used a high-density co-culture of target tumour cells (MCF-7) and effector cells (primary NK). The concentration of anti-CD16 × HER2 bsAb, as well as tumour cell HER2 expression, are varied. The lysis of target cells is monitored by propidium iodide (PI). The assay also includes a fluorescent label of the live target cell nucleus, as well as the effector cell cytoplasm, in order to locate both cell types over time. A multicolour composite representation of a single position is shown at endpoints in Figure 3A, illustrating how the number of dead (red) nuclei increases over time.

Figure 3 with 2 supplements see all

Download asset Open asset

High-throughput cytotoxic response of cancer cells in co-culture with immune cells.

(A) Schematic side view of target/natural killer (NK) cells co-culture assay for bispecific antibody-dependent cell cytotoxicity (ADCC) (top) and representative multimodal composite images, obtained at two different time points, with target nuclei labelled in blue, dying cells in red, and NK cells in green (bottom). Corresponding colours are also used in the schematics. (B) Decomposition of partly overlapping fluorescence channels and benchmark of segmentation deep learning (DL) models. Brightfield and fluorescence images at three different time points. Hoechst channel (target nucleus) is taken as input to the existing StarDist *versatile fluo* or Cellpose *nuclei* models (available directly in Celldetective). Two new models trained in Celldetective were benchmarked: a transfer of the StarDist *versatile fluo* model on our MCF-7 nuclei dataset, with Hoechst as input, and a StarDist multimodal model trained from scratch using four channels as its input. (C) Time series of nuclear fluorescence intensities and nucleus apparent area for a set of non-dying target cells (left) and of dying cells (right); the reference time $t_{0}$ is the death time for the dying cells. Individual traces in black, average in colour. The coloured shaded area represents the standard deviation. (D) Benchmark of three methods for event classification and regression (threshold method on propidium iodide [PI], DL classifier on PI, DL classifier on PI and nuclear area). Top row: confusion matrices with three classes (fraction of predictions displayed). Bottom row: correlation plots for the precision on death time determination. (E) Survival curves of target cells without NK cells or in the presence of NK cells for different bispecific antibody (bsAb) concentrations. The shaded area represents the 95% confidence interval of the survival estimate. (F) Schematics for neighbour counting and histogram of target neighbour counts (time-averaged per cell). The dashed line at 3.5 neighbours represents the median of the distribution, used to split the population into low- and high-density subgroups. (G) Survival curves for two subpopulations of targets at 100 pM bsAb, split as a function of the local target cell density (below or above 3.5 neighbours). The shaded area represents the 95% confidence interval of the survival estimate. The non-integer threshold of 3.5 arises because the neighbour count is time-averaged over the observation period for each cell.

First, we studied the lysis behaviour of target cells over time, which entailed achieving a single-cell description for the target cells. Directly using generalist nuclear segmentation models (StarDist versatile fluorescence and Cellpose nuclei models) on the Hoechst channel, we could only achieve a poor accuracy, limited by occlusion of the underlying target cells by effector cells. While it was possible to segment all cells and classify them either simultaneously (Pang et al., 2025) or in two steps (Juppet et al., 2021), label contamination from overlapping populations remained an issue. We produced expert annotations of the MCF-7 nuclei using all available multi-channel and time information. Since the MCF-7 nuclei are larger than primary NK cell nuclei, we expected to be able to train an efficient model detecting the MCF-7 nuclei from the Hoechst channel only, failing only on blurry images, where the size separation was no longer obvious (Figure 3B). We used Celldetective to retrain the StarDist versatile fluorescence model on our dataset using the Hoechst channel as its input and considerably improved the performance, from 0.55 to 0.82. Last, we trained a completely multi-channel model on all four available channels to detect the MCF-7 nuclei and achieve a slightly higher performance (0.83). All of the models benchmarked are accessible directly in Celldetective (see Supplementary file 1, table S1, S2). Target cell tracking is detailed in Materials and methods.

An expert annotated the death events, defined as a sharp increase in PI signal at $t_{d e a t h}$ . Single-cell time series were then grouped by event class and synchronised to $t_{d e a t h}$ (Figure 3C). The average PI intensity for cells undergoing death shows a distinctive step-like transition with low lateral variance, validating the annotation accuracy (Figure 3C, top-right). The PI signal in live cells, by contrast, rises slightly over time but remains well below the levels observed in dead cells (Figure 3C, top-left). Notably, the apparent nuclear area of dying cells contracts sharply at $t_{d e a t h}$ (Figure 3C, bottom-right), while the nuclear area of non-dying cells remains constant (Figure 3C, bottom-left).

Event detection was evaluated for three methods on this dataset: (1) a condition-based classification using the irreversible event hypothesis on the PI intensity time series, (2) a DL model trained to detect death events from PI alone, and (3) a DL model trained to detect events using both the PI intensity and nuclear area time series. Detailed information on the training procedures for these models is provided in the Materials and methods section, and the DL architecture on Figure 3—figure supplement 1. Figure 3D presents the confusion matrices for each method, showing agreement between predicted and ground-truth event classes, along with a correlation plot evaluating the accuracy of death time extraction. Method 1 incorrectly predicted about 15% of the events as death events when none had occurred. This method also had the highest mean absolute error (MAE) for death time estimates (7.1 min). Method 2 performed better, with only 3% of its predicted death events corresponding to an absence of events. The multivariate model (Method 3), incorporating both PI and nuclear area data, achieved the highest precision in death event detection (96% of predicted death events were true events). However, this model showed a reduced accuracy in identifying cells as already dead, with 20% of these cells (2% of the total number of cells) observed to die later. This remains a minor drawback here, since the fraction of initially dead cells is low. Celldetective’s event annotation viewer allows the user to efficiently review and manually reclassify these cells, recovering them for the survival study. As before, quality check and manual corrections were performed with the event annotation viewer, using a multi-channel composite of the images (Video 2).

Video 2

Download asset

We next investigated whether the probability of tumour cell lysis depended on bsAb concentration. Using $t = 0$ min as the synchronisation time, we excluded cells already dead at the start (always less than 5%) and disregarded cells born from mitosis (during tracking post-processing). We represented the MCF-7 survival curve using $t_{d e a t h}$ as the event time (Figure 3E), which shows an increase in the fraction of dead MCF-7 cells (endpoint) occurring at a faster rate with rising bsAb concentrations.

Our single-cell approach also permits us to link cell survival with local context. To do this, we calculated a target-target neighbourhood within a 32 μm radius (chosen to encompass approximately two cell diameters—MCF-7 nuclear diameter ≈15 μm), thus capturing the nearest layer of neighbours while excluding more distant cells (isotropic method, see Materials and methods and Figure 3—figure supplement 2) to count the local density experienced by each target cell until death (Figure 3F). We then applied a filter in Celldetective to compare survival outcomes for cells in low-density versus high-density environments (Figure 3G), observing a significant survival advantage for targets in high-density regions. This effect was present at all tested bsAb concentrations, though less pronounced at lower concentrations where the overall killing rate was reduced.

From a single time point to a dynamic readout of effector-target interactions

After describing the ADCC assay from the targets’ perspective, we shifted our focus to the effectors. The aim here was to characterise how effector cells interact with their target neighbours: first, we focus on the analysis of the distribution of fluorescently labelled bsAbs at the cell surfaces at the population level using instantaneous measurements; second, we study the relative motion between targets and effectors in relation to degranulation markers using cell tracking and pair analysis. Effector cells were segmented using a custom multi-channel Cellpose model that combines brightfield, Hoechst, and CFSE, a cytoplasmic fluorescence label (see Supplementary file 1, table S3). We chose Cellpose for effectors because NK cells have irregular, non-convex shapes, for which the Cellpose algorithm is better suited, while StarDist, which assumes star-convex objects, was more appropriate for the roughly elliptical MCF-7 nuclei segmented in the previous section. Unlike targets, effectors exhibit high motility and can form dense clusters in the assay, limiting reliable tracking.

When effector tracking was unfeasible (e.g. at an effector-to-target ratio of 5:1 and frame interval greater than 2 min), we could still perform a multicondition analysis of the cells at each time point. We aimed to measure the signal of antibodies tagged with a single ATTO 647 dye. Specifically, we compared the CE4-21 bsAb with the CE4-X control antibody, which only binds to the HER2 antigen on targets, across two target cell lines with distinct antigen expression levels (WT and HER2+) (Gonzàlez Gutierrez et al., 2025). Figure 4 presents the mean bsAb fluorescence intensity under effector cell masks at 1 hr, broken down by target cell type, antibody, and contact state (in or off contact with targets, see Materials and methods and Video 3). For WT targets (lowest HER2 expression), the cells exhibited a signal in the CE4-21 bsAb condition, regardless of the contact state (Figure 4A). In contrast, no signal was measured with the CE4-X antibody condition, supporting that the bsAb-condition signal came from the effector cells (visual assessment in Figure 4B). With the HER2+ targets and CE4-X antibody, the signal measured off-contact was negligible, confirming that the antibody did not bind to the effectors (Figure 4A and B). The signal measured in contact did not originate from the effector cells but from the target cells below. With the CE4-21 bsAb, on the other hand, the signal measured in contact was a sum of contributions from both the target and effector cells.

Figure 4 with 1 supplement see all

Download asset Open asset

Single time point analysis of effector-target interactions.

In the absence of effector tracking, fluorescent bispecific antibodies (bsAbs) were used to study the distribution of Ab on effectors and targets. (A) bsAb intensity measured on effector cells on targets with high or low antigen expression. bsAb CE4-21 binds to both human epidermal growth factor receptor 2 (HER2) and CD16, while CE4-X control antibody binds only to the HER2 antigen on target cells. (B) Representative snapshot of fluorescence channel for each condition of A. (C) Empirical cumulative distribution functions (ECDFs) for the simulated (grey, sum of CE4-21/no contact+CE4-X/contact) and observed (orange, CE4-21/contact) distributions. The dotted black line represents the Kolmogorov-Smirnov statistic, i.e., the maximum separation between the two ECDFs. Primary NK segmentation models.

Video 3

Download asset

Based on the measured signals, we sought to determine whether the added effector signal from the CE4-21/off-contact condition (without target contribution) and the effector signal from the CE4-X/in-contact condition (without effector contribution) could approximate the CE4-21/in-contact signal. Using the distributions generated by the software, we simulated the distribution of this summed signal with a Monte Carlo approach, and compared it with the CE4-21/in-contact condition. Figure 4C reveals that the simulated signal distribution was significantly lower than that of the CE4-21/in-contact condition, with a small effect size (Cliff’s Delta > 0.33). This excess signal suggests that, upon direct effector-target contact, the bsAb is enriched at the cell-cell interface through simultaneous engagement of CD16 on the NK cell and HER2 on the target, providing direct image-based evidence of bsAb bridging at the immunological synapse. By calculating this effect size at each time point, we further confirmed the robustness of this finding, as the effect consistently maintained the same direction, with a medium effect size observed more often than not (Figure 4—figure supplement 1).

If effector tracking is achievable (e.g. at an effector-to-target ratio of 1:1 and frame interval lower than 1 min), we can explore the measurement time series. To monitor the lytic activity of effector cells dynamically, we added a degranulation marker (fluorescent anti-LAMP1), exploiting Celldetective’s multivariate time series capabilities. An example of such labelling is shown in Figure 5—figure supplement 1. In this experiment, we compared two bsAbs, CE4-21 and CE4-28, formed of the same CE4 nanobody against HER2 and two different nanobodies against CD16 on the effector side.

First, we investigated whether LAMP1 expression was influenced by biological conditions, mirroring our approach with fluorescent antibodies. For each target cell type and antibody condition, we reported the fraction of LAMP1-positive effectors, decomposed by contact state, across all time points (Figure 5A). The results indicated that, except for the no-antibody controls, the fraction of LAMP1-positive effector cells was consistently higher in-contact with target cells.

Figure 5 with 1 supplement see all

Download asset Open asset

Effector-target dynamic interactions.

(A) Fraction of LAMP1-positive effector cells for different bispecific antibody (bsAb) conditions, decomposed by contact with the targets (over all time points). Bars indicate the mean fraction across three replicate positions; error bars denote the standard deviation. (B) Average effector velocity for different bsAb conditions, decomposed by target contact (up to two estimates per effector). (C) Examples of killer-victim identification with one (left) or two (right) killers. One target cell (centre) is connected to effector neighbours via coloured segments. Dynamic effector-target interaction monitoring includes propidium iodide (PI) signal of target cell, relative distance and relative velocity between target and effector, as well as LAMP1 signal in effector neighbours. (**D, E**) Characteristic parameters for killer/victim and non-killer/victim pairs. Effector cells are manually annotated as potential killer or not. Each pair of points represents one victim target, with the average over all neighbours of the parameter decomposed by killer class. (D) LAMP1 intensity. (E) Relative effector-target velocity.

From this time-integrated approach, we shifted our focus to the dynamics of the effector cells, specifically investigating whether their motility was influenced by target cell proximity. Given that velocity estimators can be noisy, we computed a time-averaged velocity estimate for each cell track, decomposed by contact state. The resulting velocities for all conditions are presented in Figure 5B. In each condition, the contact velocities were consistently lower than off-contact, regardless of antibody presence or type. This difference was always significant, with a small effect size for WT targets (Cliff’s Delta > 0.147) and a large effect size for HER2+ targets (Cliff’s Delta > 0.474). The lowest contact velocities were observed in the presence of antibodies, showing a significant drop from the target-specific control, with medium effect sizes for the WT bsAbs and HER2+ CE4-28, and a large effect size for HER2+ CE4-21. However, we did not detect any difference between the two antibodies, separately for each cell type. We note that off-contact velocities appear slightly higher on HER2+ targets across all conditions; this is likely due to differences in target cell monolayer coverage and density between wells, which can influence effector motility independently of the antibody condition.

After computing effector-target neighbourhoods, we measured features of cell pairs, including relative distance and angle, as well as their time derivatives. Figure 5C illustrates time series data for pairs, showing the PI intensity of a target cell that died at 5 min alongside the relative distance, velocity, and LAMP1 signal of neighbouring effectors. Notably, only the selected neighbour (highlighted in red) exhibits a strong LAMP1 signal, which rises around $t_{d e a t h}$ , consistent with degranulation preceding or coinciding with target membrane permeabilisation, and remains elevated until the end of the observation.

Using the pair annotation feature, we aimed to identify ‘killer-victim’ pairs. An expert annotated each neighbourhood containing a dead target cell, identifying the most probable effector cell killer to establish these pairs. This annotation was done via a cell pair viewer, similar to the viewer designed for single cells, which leveraged colour cues for pre-determined target cell death events and immobilised effector cells (Video 4). The expert viewed only the brightfield movie, blinded to the LAMP1 signal, and identified effectors with prolonged, restricted movement in contact with the target cell. Effectors that arrived post-mortem were excluded, and in ambiguous cases, multiple pairs could be labelled as ‘killer-victim’ within a neighbourhood. Due to the complexity of this task, precise event times were not annotated. Figure 5D and E displays average relative velocity and LAMP1 expression for each neighbourhood, with values grouped by killer-victim status and shown in paired boxplots. Results indicate significantly higher LAMP1 expression and lower relative velocity in killer-victim pairs, with both metrics showing a large effect size (Cliff’s Delta > 0.474).

Video 4

Download asset

Discussion

Here, we developed a new software tool to study multi-channel time-lapse microscopy data at scales ranging from single cells to cell populations. It integrates original methods for event detection and cell-pair analysis with established methods for cell segmentation and tracking (Weigert et al., 2020; Stringer et al., 2021; Ulicna et al., 2021). Although the pipeline steps are primarily sequential (see Figure 1, Figure 1—figure supplement 1) and could be performed independently with different tools and scripts, we demonstrate how integrating them within a unified, interactive software environment enhances data exploration and facilitates the analysis of dynamic cell interactions in large datasets.

Celldetective demonstrated its capacity to manage datasets of hundreds of gigabytes, generating high-quality plots that include hundreds of single-cell traces and data points. As a standalone application, it empowers biology experts to analyse single-cell data, annotate labels and events, and train models to automate tasks without requiring coding skills—similar to recent advances for fixed images (Körber, 2023). Celldetective integrates the open-source annotation tool napari for label correction and introduces unique tools for event annotation and pair classification. The software combines traditional threshold-based image and signal analysis methods with DL-based approaches, which can be used complementarily for enhanced results.

We showed how recording multivariate time series may reveal multiple phenomena occurring during the same event. For example, cell death, monitored through an increase in PI fluorescence, was observed alongside apparent nuclear shrinkage (see Figure 3C). We interpret this as cell de-adhesion upon death, releasing the pushing pressure on the nucleus and allowing its axial shape relaxation (Versaevel et al., 2012). In immunotherapy assays involving human primary cells, we prioritised a fluorescence-based labelling strategy that avoids genetic modification. Due to the exchange of fluorescent dyes between cell populations and cell lysis during the assay, along with significant size disparities between effector and target cells, generalist DL segmentation models perform poorly. To address this, we trained custom models within the software, using StarDist for target nuclei segmentation (convex object) and Cellpose for effector cells (any shape). Performance improved slightly with the simultaneous use of multiple input channels (see Figure 3B). On closer inspection, the improvement was more noticeable on difficult data (blurry, noisy). This approach also benefited event detection: training a model on both PI and nuclear size time series enhanced the event classification performance on the two most important classes for survival studies (see Figure 3D). Our spatially resolved immunotherapy assay documents the effect of heterogeneity in target cell density. Indeed, we observed how the high density of target cells reduces their ADCC killing rate (see Figure 3G). This survival advantage in dense regions could reflect a direct protective effect of close target neighbours (for instance through physical shielding or competition for effector engagement) or alternatively a sampling effect, where effectors in high-density areas are diluted across more targets, reducing the per-target encounter rate. Disentangling these mechanisms would require further experiments with controlled cell densities, which is beyond the scope of this software-focused study.

Immune cells leverage transient and weak contacts to make rapid decisions (Bongrand, 2023). We demonstrate the power of high-throughput imaging and analysis to explore such mechanisms. In a cell-surface assay, we measured by interference microscopy the decision time for NK cells to spread on an antigen surface in the presence of bsAb. Celldetective provides a higher statistical power than in earlier works (Robert et al., 2021). Thus, we observed differences in NK response to bsAb concentration in the minute range, which could not be resolved in earlier studies on similar systems such as TCR-pMHC (Brodovitch et al., 2015), where the smaller cell numbers and lower throughput limited the statistical power. Celldetective’s high-throughput capabilities could in principle help resolve such subtle kinetic differences in T cell assays, provided the experimental conditions (e.g. imaging modality and cell density) permit reliable segmentation and tracking. Here, the detection of sharp cell morphology changes during adhesion and spreading could be further enhanced by combining RICM with brightfield imaging, since each imaging modality exhibits a contrast specific to each cell state (González et al., 2019).

Proximity between effector and target provides spatial information inaccessible to traditional flow cytometry, revealing clear differences in LAMP1 signal and relative velocity in neighbourhoods of targets. We introduce a classification scheme, which allows quantification of a subpopulation based on arbitrary thresholds on any measurement. Additionally, instantaneous classification can be complemented by a time series analysis, when cell tracks are available. To extend this analysis to effector-target dynamic interactions, we addressed the question of killer:victim pair identification. The strategy consisted of collecting relative measurements like velocity and manually annotating the potential killer for each target from the images. Our data are consistent with one or more killers. The most prominent parameters for killer identification are relative velocity and LAMP1 signal. Since cell pairs were implemented like single cells, Celldetective allows users to train an event detection model from pair, target, and effector time series to automate killer:victim pair identification. As a future direction, further work is required to automatically analyse the dynamic signalling and provide more robust prediction. Celldetective offers in principle the complete environment for such an advanced analysis, as demonstrated by the automatic detection of death times.

For its applicability, Celldetective does not require coding skills. It was tested by graduate students from our labs with diverse backgrounds in biophysics, cell biology, and immunology. Its release is accompanied by worked examples, tutorials, and comprehensive documentation structured around the Diátaxis framework (tutorials, how-to guides, explanations, and reference; see https://diataxis.fr/).

The experimental systems analysed here are largely two-dimensional, but at the cellular scale, exhibit a three-dimensional aspect. Celldetective is specifically optimised for high-throughput, high-temporal-resolution imaging of quasi-2D systems. By prioritising temporal sampling over Z-axis depth, the software enables the capture of rapid biological dynamics that are often the focal point of interaction studies. Future software expansions could incorporate 2.5D or 3D sequence analysis, facilitated by the use of 3D-compatible backends (StarDist, Cellpose, bTrack). Currently, to our knowledge, only proprietary solutions are offered and exploited in immunotherapy assays (Dekkers et al., 2023). Notably, cell-interaction studies benefit from higher time resolution, which may limit compatibility with extensive 3D analysis.

The methodologies we demonstrate here should adapt well to commercial high-throughput automated microscopes for long-term cell co-culture studies (Edlund et al., 2021), for which no efficient analysis solutions are available, to our knowledge. A distinctive aspect of our approach is the use of compact, efficient models that are user-trainable on a single GPU setup. This contrasts with approaches relying on large-scale databases (Edlund et al., 2021) and extensive DL models (Marks et al., 2025). Moreover, the software’s integrated visualisation and annotation features encourage meticulous review of segmentation and tracking results, enhancing confidence in the analysis. Convolutional models used by the software, such as U-Net and ResNet, are prone to forgetting previously learned tasks upon transfer (Ma et al., 2024). To address this, Celldetective allows the addition of the original dataset during fine-tuning from a previously trained model. Future extensions could include foundation models like Segment Anything (Archit et al., 2025) for broader adaptability.

Conclusion

In conclusion, Celldetective is a user-friendly software tool designed for non-specialists, providing advanced cell-scale analysis for large cell populations. Combining high time-resolution with robust statistical capabilities, we show its potential to uncover new fundamental parameters characterising dynamic interactions between immune and tumour cells. We demonstrated the efficiency of Celldetective through two applications in the context of immunotherapy by isolating time-dependent cell-scale parameters and pair interactions in a high cell-density environment. We believe that such an approach holds great potential for enhancing the design and discovery of drugs in a more powerful and rational manner, while its general design suggests potential for broader application to other biological contexts where dynamic cell-cell interactions are of interest.

Materials and methods

NB: A detailed protocol related to some of the present experiments can be found in Díaz-Bello et al., 2025.

Share this article

Cite this article

Software overview: an end-to-end graphical user interface (GUI) pipeline, from left to right, for studying interactions between cell populations, here illustrated with target and effector populations.

Comparative table of software functionalities with a selection of available solutions.

Functional response of immune cells in a spreading assay.

Interactive and dynamic multi-event representation.

High-throughput cytotoxic response of cancer cells in co-culture with immune cells.

Interactive and dynamic event representation.

Single time point analysis of effector-target interactions.

Interactive single-cell classification representation.

Effector-target dynamic interactions.

Cell interaction viewer.

Author details

Rémy Torro

Contribution

For correspondence

Competing interests

Beatriz Díaz-Bello

Contribution

Competing interests

Dalia El Arawi

Contribution

Competing interests

Ksenija Dervanova

Contribution

Competing interests

Lorna Ammer

Contribution

Competing interests

Florian Dupuy

Contribution

Competing interests

Patrick Chames

Contribution

Competing interests

Kheya Sengupta

Contribution

Competing interests

Laurent Limozin

Contribution

For correspondence

Competing interests

Citations by DOI

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

Categories and tags

Research organism