Misic, a general deep learning-based method for the high-throughput cell segmentation of complex bacterial communities

  1. Swapnesh Panigrahi
  2. Dorothée Murat
  3. Antoine Le Gall
  4. Eugénie Martineau
  5. Kelly Goldlust
  6. Jean-Bernard Fiche
  7. Sara Rombouts
  8. Marcelo Nöllmann
  9. Leon Espinosa
  10. Tâm Mignot  Is a corresponding author
  1. CNRS-Aix-Marseille University, Laboratoire de Chimie Bactérienne, Institut de Microbiologie de la Méditerranée and Turing Center for Living Systems, France
  2. Centre de Biochimie Structurale, CNRS UMR 5048, INSERM U1054, Université de Montpellie, France

Abstract

Studies of bacterial communities, biofilms and microbiomes, are multiplying due to their impact on health and ecology. Live imaging of microbial communities requires new tools for the robust identification of bacterial cells in dense and often inter-species populations, sometimes over very large scales. Here, we developed MiSiC, a general deep-learning-based 2D segmentation method that automatically segments single bacteria in complex images of interacting bacterial communities with very little parameter adjustment, independent of the microscopy settings and imaging modality. Using a bacterial predator-prey interaction model, we demonstrate that MiSiC enables the analysis of interspecies interactions, resolving processes at subcellular scales and discriminating between species in millimeter size datasets. The simple implementation of MiSiC and the relatively low need in computing power make its use broadly accessible to fields interested in bacterial interactions and cell biology.

Introduction

Bacterial biofilms and microbiomes are now under intense study due to their importance in health and environmental issues. Within these spatially structured communities, analysis of cell-cell interactions requires powerful descriptive tools to link molecular mechanisms in single cells to cellular processes at community scales. Microscopy-based imaging methods combining multiple imaging modalities (e.g. bright-field, phase-contrast microscopy, fluorescence microscopy) directly record morphological, spatio-temporal, and intracellular molecular data in a single experiment. However, extraction of quantitative high-resolution information at high-throughput requires accurate computational tools to detect bacterial cells and correctly assign analyzed properties to these cells. While this task might seem trivial to the naked eye, it is a significant computational challenge to detect bacterial cells in a 2D image with high accuracy in dense microcolonies where the cells are in tight contact and under various imaging modalities.

Semantic segmentation of an image assigns pixels to the object that it belongs to, for example bacterial cells or background (Jeckel and Drescher, 2021). Traditionally in an image, pixel intensities that exceed a given threshold are assigned to detected objects while pixels with intensities lower than the threshold are assigned to background, producing so-called segmented masks. Detecting bacterial cells among the objects can be obtained in a number of ways using morphometric procedures. This is a fast moving field and an extensive list of available methods is described by Jeckel and Drescher, 2021. In 2D, broadly used example methods such as MicrobeJ (Ducret et al., 2016) and Oufti (Paintdakhi et al., 2016) use characteristic morphometric parameters (length, area, circularity, feature detection (ie septa) etc...) to filter non-cell objects from the segmented image and fit the remaining objects to embedded cell models, allowing cell pole detection, septa detection, protein localization etc… While these methods are highly performant to study isolated bacterial cells, they are ill-suited to perform segmentation of dense bacterial communities, mostly because adjacent cells are not easily resolved by intensity-thresholding when bacteria are in tight contact due to lower contrast at the interior of the colony (the so-called shade-off artifact). At best, when performed with Oufti, segmentation of single bacteria within micro-colonies requires extensive manual-tuning of multiple parameters limiting its robustness for high-throughput automatic data extraction (Paintdakhi et al., 2016; Stylianidou et al., 2016).

Machine-learning-based techniques are powerful alternatives to overcome the above limitations of traditional segmentation approaches. For example, one of them, Supersegger, combines intensity-based thresholding with supervised cell boundary recognition on phase contrast images, successfully resolving individuals cells at low contrast at the colony interior (Stylianidou et al., 2016). This method is, however, limited to phase contrast images and requires a number of filter applications (ie thresholding, contrast filter and watershed) to appropriately segment these images. Such tuning which may need to be adjusted for each field of view (in addition to image intensity normalization) renders Supersegger difficult to use for the segmentation of large colonies of various bacterial species captured under various imaging modalities. Deep-learning algorithms called Convolutional Neural-Networks (CNNs) have recently shown great promise for image classification and in particular semantic segmentation with reasonable computational power (Van Valen et al., 2016). Van Valen and collaborators (Van Valen et al., 2016) provided a general proof of concept that CNNs (DeepCell) could be used to segment both eukaryotic and bacterial cells in dense contexts with limited training datasets. However, while the study provides important tips toward the successful training of a CNN for a specialized cell segmentation task, it does not provide a trained CNN for the general segmentation of dense bacterial communities. Nevertheless, the approach provided promising perspectives to segment E. coli microcolonies on agar and confirming this, a CNN-based tool (DeLTA, Lugagne et al., 2020) was recently developed to detect and track E. coli cells immobilized in microfluidic devices (the so-called mother machine, Lugagne et al., 2020). Inspired by these methods, we decided to develop MiSiC (Microbial Segmentation in dense Colonies), a general CNN-based tool to segment bacteria in single or multispecies 2D bacterial communities at very high throughput. MiSiC is based on U-net, a CNN encoder-decoder architecture that has previously been applied for detection and counting of eukaryotic cells (Falk et al., 2019), that relies on shape rather than intensity information and thus performs semantic segmentation of microbial colonies under any microscope modality. Thus, contrarily to most other softwares, MiSiC is insensitive to specific imaging conditions which often require tailored training data sets. Operating MiSiC requires minimal parameter tuning and can be run with standard computational power, in a Napari Graphic User Interface (GUI, a complete user handbook for installation and use is provided) under Python (https://napari.org/). Both semantic segmentation and instance segmentation (in which each cell is defined as a distinct object [Jeckel and Drescher, 2021]) masks can thus be easily obtained with minimal computational expertise, making MiSiC broadly applicable to the field of bacterial cell biology.

Results

The MiSiC workflow

We sought to develop a prediction workflow that converts an input image taken under phase contrast, fluorescence, or bright field into a binary mask for cell bodies. However, when microscopy is performed under different modalities, pixel intensity variations between imaging conditions make it difficult to perform semantic segmentation with a single procedure. Thus, to minimize the impact of image intensity fluctuations that inevitably arise from varying imaging sources, the input images were transformed into intermediate image representations obtained from the shape and curvature (the Hessian or second-order differentiation) of the imaged objects. This strategy is possible because in rod-shaped bacteria, the characteristic dome-shaped curvature of the poles is remarkably conserved across division cycles (Campos et al., 2014). The curvature changes in the intensity field of an image are thus represented in a so-called Shape Index map (SI) derived from the eigenvalues of the Hessian of the image (Koenderink and van Doorn, 1992) (see Materials and methods section). Therefore, all microscopy images can be transformed into SI images with pixel intensity values varying as a function of object curvature and ranging from –1 to +1, a –1 value representing a negative dome-shape and +1 representing a positive dome-shape (Koenderink and van Doorn, 1992, Figure 1A). SI images of bacterial cells acquired under various experimental conditions were used to train a U-Net.

Figure 1 with 1 supplement see all
MiSiC: A U-netbased bacteria segmentation tool.

(a) Examples of Shape index maps (SI) calculated from Phase Contrast, Bright Field and Fluorescence images of the same field of Myxococcus xanthus cells. (b) A set of annotated bright-field images of Escherichia coli and Myxococcus xanthus along with synthetic labeled data with additive Gaussian noise was used to generate a training dataset of input images, X, consisting of Shape Index Map of intensity images (at three scales) and segmented images, Y, consisting of contours (Y1) and cell body (Y2). A CNN with U-net architecture was trained to segment the Shape Index Maps into cell body and contour of bacterial cells. Prediction using MiSiC requires that the mean width of the bacteria in the input image is close to 10 pixels, which is easily obtained by rescaling the input image based on the average width of the bacteria under consideration. Gaussian noise may be added to the input image to reduce false positives (Materials and methods).

U-net type architectures allow fast learning from a relatively small body of labeled data because the embedded skip-connections allow the convolutional kernels between both encoder and decoder ends to be shared (Falk et al., 2019). Nevertheless, the labeled data must be representative of the broadly varying experimental conditions to produce reliable outputs: in our case, different bacterial species recorded under varying imaging modalities. A schematic of the training strategy is shown in Figure 1 and detailed in the Materials and methods section. Specifically, the U-Net was trained to segment cells by learning shapes of individual bacteria and patterns emerging from the tight contact between cells. Accordingly, we curated a hand-segmented dataset of 350 bright-field images (n = 34807 cells) of two rod-shaped bacterial species, Escherichia coli and Myxococcus xanthus. This annotated data was however insufficient for the robust segmentation of bacterial cells, thus it was further enriched with synthetic representations of rod-shaped bacteria of varying length but fixed 10 pixels width corresponding to ≈ 0.6 µm. Overall, the ground truth data used for training had two classes: one corresponding to bacteria cell bodies and the other corresponding to the contour of the detected cell (Figure 1B). This makes post-processing possible when there is not enough edge information for the proper separation of tightly connected bacterial cells, using algorithms like watershed, conditional random fields, or snake segmentation (Yang and Cao, 2013, see below).

Prior to segmentation, two parameters must be adjusted to generate a SI image from an input image. Because the width of bacterial cells was set to 10 pixels in the training data set, the input image must be scaled similarly so that the width of bacterial cells is also contained in 10 pixels. However, this scaling often smoothens the original image, which in turns smoothens the corresponding SI Image (Figure 1—figure supplement 1). This is potentially problematic because the U-Net distinguishes smooth curvatures with well-defined boundaries and noise reduction can lead to increased false positive segmentation in the scaled images. Such false positive detection is more frequent in images where the number of bacteria is sparse. This problem can be solved by adding synthetic noise to the scaled images. In total, the MiSiC workflow takes raw input images of any imaging modality, scales them and adds noise to generate SI that are then segmented with the above described U-Net (Figure 1B). For the users, scaling and potential addition of noise can be easily adjusted in the Napari GUI, which is explained in detail in a dedicated handbook.

MiSiC can segment bacteria under varying microscope modalities

We first tested whether MiSiC can efficiently segment images of bacteria of distinct shapes (E. coli, smaller and thicker cells and M. xanthus, longer and thinner cells) in dense colonies, captured by phase contrast, fluorescence or brightfield. To quantify the accuracy of segmentation and compare the relative performance of MiSiC for each modality, we compared the obtained MiSiC masks with hand-annotated masks (considered ground truth masks) of the same images and measured the Jaccard index (JI) as a function of the Intersection-over-Union (IoU) threshold for each modality (see Materials and methods, Figure 2a and b, Figure 1—figure supplement 1 a,b). In all cases, high JI values ( ≥ 0.8) were observed for IoU thresholds 0–0.6, indicating that MiSiC can robustly segment all modalities. In fact, MiSiC performance was comparable to the observer’s eye for each modality (or even slightly better) because similar JI scores were obtained when the same ground-truth data (generated by the same observer consistently throughout the study) was compared to data annotated by another independent observer (Figure 1—figure supplement 1c). As would be expected, the quality of the segmentation was nevertheless variable (albeit slightly) as a function of the imaging modality, the best results being obtained for fluorescence (if the fluorescence is homogenous between cells as seen in E. coli, less so for Myxococcus cells where the fluorescence levels were more variable), followed by phase contrast and bright field.

Figure 2 with 1 supplement see all
MiSiC predictions under various imaging modalities.

(a) MiSiC masks and corresponding annotated masks of fluorescence, phase contrast and bright field images of a dense E. coli microcolony. (b) Jaccard index as a function of IoU threshold for each modality determined by comparing the MiSiC masks to the ground truth (see Materials and methods). The obtained Jaccard score curves are the average of analyses conducted over three biological replicates and n = 763, 811, 799 total cells for Fluorescence, Phase Contrast and Bright Field, respectively (bands are the maximum range, the solid line is the median). The fluorescence images were pre-processed using a Gaussian of Laplacian filter to improve MiSiC prediction (describe in Materials and methods).

MiSiC can segment colonies of various species and is tolerant to shape deviations

We next tested if MiSiC can segment bacterial species other than those that it was trained on (E. coli and M. xanthus), other rod shapes (Pseudomonas aeruginosa), curved shapes (Caulobacter crescentus) and filamentous shapes (Bacillus subtilis) (Figure 3a and b). For each analysis, we derived JI scores based on the comparison between MiSiC masks and hand-annotated data as described above. MiSiC predicted accurate masks for each bacterial species with JI scores ≥ 0.8 up to 0.5 IoU thresholds for all cases. Nevertheless, and as expected the segmentation accuracy was lower for curved bacteria than rod shaped bacteria (compare P. aeruginosa and C. crescentus). For filamentous bacteria, the filaments were remarkably well resolved but that was not always the case for cell separations (septa) within the filaments, likely because of insufficient edge information at cell septa in the raw image. Note that this problem may be resolved by post-processing, for example by applying a watershed algorithm to the MiSiC mask (Figure 2—figure supplement 1), which effectively resolves additional undetected septa. Although MiSiC can thus detect a number of bacterial species, a current limitation lies in its ability to segment round or oval cells. In fact, such shapes were initially omitted from the original training dataset to omit the spurious detection of round non-cell objects that are frequently observed in background images.

Figure 3 with 1 supplement see all
MiSiC predictions in various bacterial species and shapes.

(a) MiSiC masks and corresponding annotated masks of phase contrast images of another Pseudomonas aeruginosa (rod-shape), Caulobacter crescentus (crescent shape) and Bacillus subtilis (filamentous shape). (b) Jaccard index as a function of IoU threshold for each species determined by comparing the MiSiC masks to the ground truth (see Materials and methods). The obtained Jaccard score curves are the average of analyses conducted over three biological replicates and n = 1149,101, 216 total cells for P. aeruginosa, B. subtilis and C. crescentus, respectively (bands are the maximum range, solid line the median). Note that the B. subtilis filaments are well predicted but edge information is missing for optimal detection of the cell separations.

Figure 3—source data 1

IoU over threshold -Bacillus subtilis.

https://cdn.elifesciences.org/articles/65151/elife-65151-fig3-data1-v2.csv
Figure 3—source data 2

IoU over threshold -Caulobacter crescentus.

https://cdn.elifesciences.org/articles/65151/elife-65151-fig3-data2-v2.csv
Figure 3—source data 3

IoU over threshold – Pseudomonas aeruginosa.

Thresholds are the same in Figure 2.

https://cdn.elifesciences.org/articles/65151/elife-65151-fig3-data3-v2.csv

MiSiC as a tool to study dynamic bacterial communities

Encouraged by these results, we tested whether MiSiC could be further used to study bacterial multicellular organization and inter-species interactions accurately at very large scales. As a model system we used Myxococcus xanthus, a delta proteobacterium living in soil, that predates collectively in a process whereby thousands of cells move together to invade and kill prey colonies (Figure 4, Pérez et al., 2016). In the laboratory, spotting a Myxococcus colony next to a prey colony (here E. coli) results in invasion and complete digestion of prey cells in 48 H (Figure 4b). To capture predator-prey interactions at single cell resolution, we set up a predation assay where Myxococcus and E. coli interact on a 1 cm2 agar surface directly on a microscope slide (Figure 4—figure supplement 1a). Under these conditions, the entire invasion process occurs over a single prey cell layer allowing identification of single predator and prey cells at any given stage. This area is nevertheless quite large, and to record it with cell-level resolution, we implemented a multi-modal imaging technique termed ’Bacto-Hubble’ (in reference to the Hubble telescope and its use for the reconstruction of large scale images of the galaxies) that scans the entire bacterial community with a 100 X microscope objective and reconstructs a single image by near neighbor end-joining of multiple tiles of 80 nm/pixel resolution images (Figure 4—figure supplement 1b). Application of this method requires addressing practical considerations that are detailed in the methods section. Bacto-Hubble images (phase contrast and multi-channel fluorescence) thus capture cellular processes in a native community environment. We next tested whether MiSiC addresses the computational challenges posed by the analysis of such complex (dense population and mixed species) and large size data sets.

Figure 4 with 1 supplement see all
MiSiC can be applied to the study of cellular processes at the mesoscale.

(a) The Myxococcus xanthus predation cycle. Myxococcus cells use motility (yellow colony) to form so-called rippling waves to invade and consume prey colonies (gray). When encountering starvation (outside the prey) or after the prey is consumed, the Myxcococcus cells form aggregate that matures into fruiting bodies (black dots) where the cells differentiate into spores. These spores remain dormant until another prey is encountered which provokes the cycle to resume (see Pérez et al., 2016 for details about the cycle).(b) The Myxococcus xanthus cycle can be observed directly on a Petri dish. Shown are 24 H and 48 H time points. At 48 H, spore-filled fruiting bodies are observed forming in the nutrient depleted area but not in the former prey area where the Myxococcus cells are actively growing. This stage corresponds to the rippling stage shown in (a). Scale bar = 0.5 mm. (c–e) MiSiC can segment dense bacterial swarms.(c) An M. xanthus swarm expressing SgmX-GFP, observed at colony edges and captured under phase contrast, fluorescence and corresponding magnified images.(d) MiSiC prediction mask obtained on the phase contrast image shown in b (e) Demograph representation of the segmented cells in the MiSiC mask (d) and corresponding localization of the SgmX-GFP fluorescent clusters. The horizontal axis represents the number of cells ordered by cell length. The vertical axis represents cell coordinates in µm and aligned such that polar fluorescence clusters have positive values (with respect to the position of the cell middle set to 0). The color of the clusters reflects the number of cells for each given cell length (bins = 0.05 µm, maximum = 20 cells for [3.9–3.95] µm). n = 1672 detected cells after filtering the MiSiC mask. (f–j) Mapping of M. xanthus cell division in the M. xanthus-E. coli community. (f–g) Bacto-Hubble image of a predatory field containing FtsZ-NG labeled Myxococcus xanthus cells and unlabeled Escherichia coli prey cells. The composite image results from the assembly of 15 × 15 Tile images. The dotted circle marks the limits of the original prey colony. The white line (mean profile) indicates the axis used for the analysis shown in (i). (f), a single image tile showing a representative density of fluorescent cells. (h) Detection of dividing cells. FtsZ-NG fluorescent clusters are detected at midcell. The FtsZ clusters can be detected as fluorescence intensity maxima. Shown is a projection of the position of fluorescence intensity on a mean cell contour for a subset of n = 3490 cells (representing 14 % of the total detected cells with a cluster), revealing that as expected, the clusters form at mid-cell. The blue square marks the cell shown as an example in (h). (i) Counting dividing cells. The example shows segmentation of the field shown in (h). The position of Z-ring foci detected as fluorescent maxima was linked to all fluorescent cells segmented in the MiSiC mask. (j) Myxococcus cells divide in the prey colony. The spatial density of dividing cells, the total fluorescent cells (Materials and methods) and the proportion of dividing cells (density of dividing cells/density of total cells, Materials and methods) were determined all across the prey area shown in (e) (dotted circle). The mean ratio and standard deviation are plotted along a spatial axis (distance along profile) corresponding to areas outside and inside (dotted rectangle) of the prey area (mean profile, white segment in (e)).

First, we tested the capacity of MiSiC to segment closely-packed swarms of Myxococcus xanthus cells captured in a single image tile. To test the fidelity of segmentations in these conditions, we imaged a swarm composed of cells expressing SgmX-sfGFP, a motility protein that localizes to the cell pole (Mercier et al., 2020; Potapova et al., 2020) in both fluorescence and phase contrast modalities (Figure 4c). Phase contrast images were used to obtain a MiSiC segmentation mask (Figure 4d). Subsequently, the mask was processed under MicrobeJ (Ducret et al., 2016) to remove objects that do not correspond to cells (Materials and methods, less than 1.4%, n = 1695) and calculate the localization pattern of SgmX-GFP foci with respect to the long axis of each segmented cell (Figure 3e). As expected, SgmX-GFP loci localized to a cell pole, consistent with most Myxococcus cells in swarms being properly segmented by MiSiC.

Second, to show that MiSiC can be used to quantitatively study cellular processes in entire Bacto-Hubble images, we mapped a Myxococcus cellular process directly during prey invasion. Cell division is expected to occur mostly in prey-areas in absence of any other source of nutrients. Like all rod-shaped bacteria, dividing Myxococcus cells assemble a polymeric FtsZ bacterial tubulin ring to initiate cell division (Schumacher et al., 2017). When it is fused to fluorescent proteins, the FtsZ ring is observed as a dot at mid-cell (Treuner-Lange et al., 2013), which can be used as a proxy to determine which cells enter division. Thus, we first engineered M. xanthus cells expressing FtsZ fused to Neon-Green (NG, Materials and methods) and mixed Myxococcus FtsZ-NG+ (5%) with non-labeled cells (95%) in the presence of an Escherichia coli prey cell colony. A fluorescence Bacto-Hubble image spanning ~5 mm2 (representing 225 tiles of 500 × 500 pixels images) of the community during the invasion phase (Figure 4f) was then captured and segmented tile-by-tile using MiSiC (Figure 4f–g). Due to the fact that MiSiC uses SI, it is remarkably insensitive to noise and intensity variations between images (Figure 4—figure supplement 1c,d) and thus it easily allows segmentation of multi-tile images where signal intensity variations between tiles is to a large extent unavoidable. Cells with mid-cell FtsZ-NG fluorescence clusters were clearly observed suggesting that cell division is ongoing (Figure 4h). Dividing cells were therefore counted across the entire image (Figure 4i, Materials and methods) to determine where they localize spatially within the community. Figure 4j, shows that cell division is markedly increased in the prey area, demonstrating directly that Myxococcus grows during prey invasion. Thus, MiSiC is appropriate for the automated detection of cellular processes (detected at subcellular resolution) at community scales.

Third, we explored how MiSiC could be further adapted to segment and classify multiple bacterial species intermingling and interacting in space; here Myxococcus cell groups invading the tightly-knitted E. coli prey colony. To segment each bacterial species directly from unlabeled phase contrast-images, new training datasets were produced and used to retrain the U-NET. These labeled datasets were obtained by imaging GFP-labeled Myxococcus and mCherry-labeled E. coli (see Materials and methods). Images were captured for each channel (GFP, mCherry) and segmented separately with MiSiC to obtain masks for each species (Figure 5a). These masks were used to retrain the U-NET, the predictive value of which was tested by deriving classification masks directly from a phase-contrast image of a mixed Myxococcus (GFP) - E. coli (mCherry) population (Figure 5). Comparison of the MiSiC classification to hand annotated GFP (Myxococcus) and mCherry (E. coli) images over selected fields where each species interact (n = 4, ie Figure 5b) respectively yielded JI scores of 0.95 ± 0.036 (n = 200 cells) and 0.89 ± 0.047 (n = 545 cells), suggesting that the classification is highly accurate despite the tight interactions between Myxococcus and E. coli cells. To further show that species classification is reliable in a large dataset, we next classified Myxococcus from E. coli cells directly in a phase contrast Bacto-Hubble image (840 tiles, 2.1.109 pixels). We could thus discriminate cells belonging to each species in the predation area (Figure 5c) for a total detection of ~402,000 Myxococcus and ~630,000 E. coli cells in the entire image. Given the large size of the dataset, it would be impossible to test the accuracy of the classification procedure exhaustively by comparing it to the ground truth, instead, we tested whether the distribution of shape descriptors, such as the extent (E = area/bounding box area), solidity (S = area/convex area) and minor axis length, matched the distribution of these descriptors obtained from images of each single bacterial species (Figure 5d). The observed distributions were indeed consistent with an efficient separation of species in the mixed community, suggesting that the classification is also robust in large images. Inevitably, infrequent ambiguous classifications arise in the areas where the cells interact tightly due to low contrast in these areas. These instances generally appear as bi-color objects because the prediction is not homogeneous inside such objects. Given that Myxococcus cells and E. coli cells have clear morphometric differences, these uncertain cells can be easily eliminated by filtering with discriminating parameters (i.e. each of them or combined, Figure 5d).

Semantic segmentation of M. xanthus and E. coli from single phase contrast images.

(a) Semantic classification network. A U-Net was trained to discriminate M. xanthus cells from E. coli cells. The training dataset consisted of GFP+ M. xanthus cells mixed with mCherry+ E. coli cells, which were imaged in distinct fluorescent channels (GFP and mCherry) and segmented using MiSiC to produce ground truth data for each species. The network uses unlabeled phase contrast images as input (X) and produces one output for each labeled species (Y, Mx or Ec). (b) Semantic classification of intermixed Myxococcus and E. coli cells. Shown is a mixed population of GFP+ Myxococcus and mCherry+ E. coli cells. The classification (Myxococcus, Mx) and E. coli (Ec) were obtained directly from the phase contrast image. Corresponding fluorescence images of GFP+ and mCherry+ are shown for comparison and were used to estimate the accuracy of the classification. (c–d) Direct semantic classification of M. xanthus interacting with E. coli in a Bacto-Hubble image. (c) Bacto-Hubble image of M. xanthus cells invading an E. coli colony after 24 hours. The composite image corresponds to 20 × 42 image tiles captured by phase contrast and segmented tile-by-tile to produce the resulting classification. Inset: zoom of an area where M. xanthus cells interact tightly with E. coli cells within the E. coli colony. Phase Contrast and corresponding predictions (M. xanthus in orange and E. coli in blue) are shown. (d) Morphological analyses of the classified cell population and comparison with the ground truth data. Morphological parameters (Extent, Solidity and minor axis length) were determined for the cells predicted in the M. xanthus (Mx mixed) and E. coli masks (E. coli mixed) in the context of a mixed colony and compared to the same parameters obtained from MiSiC segmented from images of pure cultures (Mx/E. coli pure).

Discussion

In this article, we have presented MiSiC, a deep-learning based bacteria segmentation tool capable of segmenting bacteria in dense colonies imaged through different imaging modalities. The main novelty of our method is the use of a shape index map (SI) as a preprocessing step before network training and segmentation. The SI depends on the Hessian of the image, thus preserving the shape of bacterial masks rather than the raw intensity values, which vary as a function of microscopy modalities. In general, the use of SIs rather than image intensity is a promising lead for any deep learning approach to cell segmentation that relies on shape, which could also solve modality issues for eukaryotic cell and cellular organelle segmentation. Another important asset of MiSiC is the use of synthetic data to enrich training data sets, greatly facilitating ground truth annotations. Combined with the use of SIs, we show that this strategy only requires two adjustment parameters (scaling and noise addition) and it makes segmentation agnostic to imaging modality, and adapted to different bacterial species with different morphologies, provided that they do not deviate too largely from rod shapes. We have currently omitted detection of oval or round cells, to avoid false positive detection of round objects, but in theory, MiSiC could be adapted to such application by simple retraining with an adapted model using both real images and synthetic data. Currently, such retraining requires computational expertise but it is imaginable that a future version of MiSiC would include a GUI to generate training data depending on the needs of the community.

MiSiC is appropriate for the automated analysis of complex images, such as fluorescence (Bacto-Hubble) images tiles. In our hands, FtsZ-NG-expressing cells could not be properly segmented across tiles with intensity-based methods. In fact, due to the large size of the sample and probable issues with its overall flatness, fluorescence intensities vary across tiles, which required parameter adjustment for each tile, making the task overly complex. These problems are solved in MiSiC because image tiles conversion into SIs cancels fluorescence intensity fluctuations and MiSiC is quite robust to noise (Figure 4—figure supplement 1c,d). Tile-by-tile segmentation in MiSiC can thus also allow extraction of high-resolution information from large size data sets while allowing prediction of large images with no exceptional memory requirements Typically, a large data set containing up to 3,000 tiles of 500 × 500 pixels (7.5.109 pixels) may be segmented in ≈ 50 minutes on a GPU-equipped (Quadro P5000) computer (RAM 64 Gb, Intel Xeon 2.10 GHz). In addition, the Napari environment provides a simple GUI, allowing the generation of MiSiC masks from any image by any user without the need of computational expertise.

MiSiC is therefore broadly applicable to diverse applications in bacterial cell biology and generates prediction masks that can be used with other available softwares. For example, Supersegger which corrects improper segmentation based on cell contours may likely be used to correct aberrant segmentations in a MiSiC mask, combining the advantage of SI-based segmentation and contour correction. Along similar lines, non-cell or improperly separated cell objects an appear in the MiSiC masks and while some can be removed by the introduction of noise, an easy way to do it is to apply a post-processing filter using morphometric parameters to remove objects that are not bacteria. This can be easily done using Fiji, MicrobeJ or Oufti. MicrobeJ also contains a GUI and associated toolbox to cure improper cell contours manually. MicrobeJ and Oufti are especially useful from downstream analyses of the masks because they both allow cell tracking as well as determining protein localization at sub-diffraction resolution.

Finally, we show that MiSiC can be further implemented for the semantic classification of bacterial cell types directly from phase contrast images. While the network developed herein works specifically for M. xanthus and E. coli discrimination, it provides a proof of concept that the approach can be extended to segment and classify any number of bacterial species provided that a dataset (i.e. fluorescence labeling) is available to train a U-Net with ground truth generated in MiSiC. Given that these tasks must be tailored to specific applications, we did not implement them in this package but in principle, adapting MiSiC for such applications could be developed by any laboratory without deep expertise of CNNs. This is an exciting prospect at a time where tremendous efforts are injected to reconstruct micro communities in synthetic contexts for mechanistic studies (Wolfe et al., 2014) . Although MiSiC is designed for the analysis of bacteria that develop in 2D, recent toolboxes for the study of micro communities in 3D are also emerging (Hartmann et al., 2021; Zhang et al., 2020); we therefore foresee that quantitative microscopy approaches will profoundly impact microbiome research of health and environmental significance.

Materials and methods

Bacterial strains and predation assays

Request a detailed protocol

The complete list of the strains used for the study is compiled in Table S1 Supplementary file 1. For predation assays, cells of Myxococcus xanthus (DZ2, Table S1, Supplementary file 1) were grown overnight in 100 mL flasks in 10–20 mL of CYE (Bustamante et al., 2004) media without antibiotics at 32°C with shaking. In parallel a colony of Escherichia coli (MG1655, Table S1) was grown in 5 mL LB medium in a glass tube at 37°C with shaking. The next day, OD600 nm were measured and cultures of both strains were washed twice at 5000 rpm in CF (Bustamante et al., 2004) minimal media to discard CYE and LB traces. After the washes, the density of the cultures was brought to 5 OD units in CF media. Pads of CF agar 0.5% were poured in precast frames (in situ GeneFrame, 65 µL, ABGene, AB-0577) that were mounted on glass slides and briefly dried. One µL of both Myxococcus xanthus and prey cell suspensions were spotted as close as possible to one another on the pad making sure that they would not merge. Glass slides were kept in a sealed humid dish for 6, 24, 48, or 72 hr at 32°C. Thirty min before observation, the agar pad around the colonies was cut out and discarded and the pad was sealed with a cover slip and observed by microscopy.

Bacillus subtilis strains were grown in LB medium at 37°C until they reached an OD600 nm of 0.6, were transferred into wells and covered in a low melting LB-based agarose suspension (2%) before observation. Pseudomonas aeruginosa was grown in LB medium and cells were observed at an OD600 nm of 0.5. Caulobacter crescentus was cultured in PYE on the benchtop without shaking for 3–4 days. Cells were imaged from disrupted fragments of a pellicle biofilm and were transferred from the air-liquid interface to glass slides for imaging (Marks et al., 2010). Anabaena nostoc was grown in BG11 medium at 30°C with illumination (40 µE m-2s-1). Finally, Desulfovibrio vulgaris cells were grown until mid-exponential phase (OD600 nm of approximately 0.4–0.5) in LS4D medium supplemented with 1 g/L of yeast extract (LS4D-YE) at 33 °C in an anaerobic chamber (COY Laboratory Products) filled with a 10% H2-90% N2 mixed-gas atmosphere. Cultures (200 µL) were centrifuged, and the pellet was resuspended in 100 µl of 10 mM Tris-HCl (pH 7.6), 8 mM MgSO4 and 1 mM KH2PO4 buffer (TPM buffer). The cells were placed between a coverslip and an agar pad containing 2% of agarose.

Molecular biology and strain construction

Request a detailed protocol

To follow cell cycle progression in single cells of Myxococcus xanthus, a merodiploid strain of DZ2 expressing both native FtsZ and the fusion protein FtsZ-neonGreen (FtsZ-NG, Table S3 Supplementary file 3) was built (DM14, Table S1, Supplementary file 1). To do so, the coding sequence of DZ2 ftsZ gene (MXAN_5597) along with its predicted promoter sequence was amplified by PCR with primers oDM1 and oDM2 (Table S2, Supplementary file 2) and cloned in the non-replicative plasmid pKA32 (Treuner-Lange et al., 2013) allowing for its site-specific integration at the DZ2 attmx8 phage attachment site on the M. xanthus chromosome. The coding sequence of the neonGreen protein was amplified from a plasmid (Shaner et al., 2013) using primers oDM16 and oDM17 (Table S2 Supplementary file 2) allowing the in frame addition of neonGreen at the C-terminus of the FtsZ protein. When grown in CYE rich medium, DM14 did not present any significant defect in growth rate or cell shape. DM14 cell size is not significantly different from that of the isogenic wild type DZ2 strain. DM14 cells were spotted on thin CF agar pads to follow FtsZ localization in axenic cultures and allowed us to confirm that cell cycle progression was accompanied with the relocalization of Ftsz-nG from being diffuse in the cytoplasm to forming a discrete fluorescent focus at mid-cell before cell septation as previously described (Treuner-Lange et al., 2013).

To generate Dataset 2 (see below), strains of E. coli (EC500, Table S1 Supplementary file 1) and M. xanthus (DM31, Table S1 Supplementary file 1) expressing soluble versions of mCherry and sfGFP fluorophores respectively were used. To generate DM31, a plasmid allowing for the constitutive expression of sfGFP was built (pDM14 Table S3 Supplementary file 3). To obtain a high and constitutive expression of sfGFP in M. xanthus, we sought for the closest homolog of the constitutively expressed E. coli EF-TU (Translation elongation factor) in M. xanthus genome which is MXAN_3068. The 1000 bp region upstream of MXAN_3068 (p3068) was amplified by PCR using oDM53 and oDM54 and cloned upstream the coding sequence of sfGFP (amplified using primers oDM61 and oDM62) in a pSWU19 plasmid. The transcriptional fusion was then integrated on DZ2 chromosome at the attmx8 site through transformation. DM31 cells display a constitutive bright diffuse fluorescent signal in our growth conditions.

Microscopy and image acquisition

Request a detailed protocol

All microscopy images were acquired with an inverted optical microscope (Nikon TiE) and a 100 x NA = 1.45 Phase Contrast objective. Camera used was Orca-EM CCD 1000 × 1000 (Hamamatsu) camera mostly set with binning 2 × 2. Acquisition software was Nikon NIS-Elements with specific module JOBS. Fluorescence acquisition used a diode-based excitation device (Spectra-X Lumencore).

The Bacto-Hubble image is a composite image of rasters of the entire area and requires that the scanning speed must be sufficiently fast to avoid image shifts due to ongoing cell dynamics. To minimize the shift in focus from tile to tile we used Nikon perfect focus system (PFS) equipped with servo-control of the focus with an infrared LED. This is especially challenging because continuous focus alignment of the microscope slows down the acquisition times dramatically. To obtain a satisfactory compromise allowing both fast scanning and correct focusing we: (i) reduced the number of dynamic elements on the microscope set up: we replaced shutters by Light Emitting Diodes (LEDs, Spectra-X for fluorescence source and a white diode for transmitted light) which could be switched with a high-frequency rate (100 kHz). In addition, a double band dichroic mirror for the fluorescent cube was used to avoid switching the filters’ turret for each snapshot. (ii) used an EM-CCD camera set to a 2 × 2 binning mode to reduce the size of images (500 × 500 pixels at 0.16 µm/px) and acquisition time, and (iii) sped up the vertical movements by means of a piezoelectric stage. In its largest scanning mode, Bacto-Hubble thus captures 80 × 40 raster images covering a total surface of 20 mm2 (containing up to 0.8 billion pixels, an acquisition-time up to 4 hr), enabling a continuous magnification display from eye visible structures to single-cells. Individual tiles for Bacto-Hubble images were acquired with the scan large field capabilities of NIS-Elements software. A key point for Bacto-Hubble large images was the quality of the sample slide mounting. The samples were placed on a glass slide with a thin double-sided sticky frame (in situ GeneFrame, 65 µL, ABGene, AB-0577). An agar pad was poured inside the frame and a microliter of cells was placed on it. The chamber was closed with a glass cover slide. This assembly allows very good flatness and rigidity.

U-NET as base architecture

Request a detailed protocol

We implement a U-net inspired in encoder-decoder architecture with skip-connections and use it as a base network for segmentation tasks. This architecture is now widely used for segmentation tasks and has many advantages that are discussed in previous articles (Falk et al., 2019; Ronneberger et al., 2015). The original U-net architecture (Falk et al., 2019) was modified to include relu activation for all layers except for the output layer where sigmoid activation was used. The general U-net was implemented in Python programming language using tensorflow (https://www.tensorflow.org/tutorials/images/segmentation), where the number of input channels (say, n) and output classes (say, m ) could be varied as required by different models. The number of encoder layers were fixed to four with filter lengths [32,64,128,256] for the encoder side. The loss function was also modified from the original implementation. A combination binary cross entropy and the Jaccard, 1912 index was used as the loss function with Adams optimizer (learning rate = 0.001) for the minimization.

For brevity, we denote the network as a mapping between input X and output y as,, where X is a set of images with n channels and y is the output with m classes. Thus given a training data set of Xtrain with sizes (N × S × S× n) and multiclass images ytrain (of size N × S × S× m), the generalized implementation of U-NET learns to predict the segmented image from unknown images X.

Training datasets

Two training datasets were used for this study. Dataset one to segment bacterial cells (MiSiC) and Dataset two to classify Myxococcus and E. coli cells in mixed cultures.

Dataset 1

Request a detailed protocol

Combination of hand-annotated data with synthetic data provided the most accurate segmentation after training. When the network was trained with synthetic data only, segmentation was less performant presumably due to the fact that the synthetic data was generated by randomly throwing cell-shaped objects onto an image, which does not capture the intricate patterns created by cell-cell interactions observed in real images of dense cell populations.

This training dataset consists of three parts:

  1. A hand annotated dataset corresponding to: 263(training) +87(test) cases of Bright-field images of Escherichia coli and Myxococcus xanthus with segmented masks. This data set corresponds to images sizing from 157 × 157 to 217 × 217 pixels and containing a variable number of cells (up to 400 cells per image) for a total number of cells of 34,807 manually drawn cells.

  2. 200 null cases that contain background images without bacteria taken from bright-field, fluorescence and phase contrast data.

  3. A synthetic data set consisting of 600 cases was generated, 30% containing 16 cells per image (256 × 256 pixels) for sparse densities and 70% containing a maximum of 327 cells for high densities per image. Furthermore, 200 null cases, generated by inclusion of random gaussian noise and circular objects, were also included in the dataset. The synthetic data was generated with a simple model for rod-shaped bacteria with a width ranging from 8 to 10 pixels. An overlap threshold of 2% was used to obtain a dense cell population. The binary mask created was then smoothed with a Gaussian filter and Gaussian noise was added to emulate noise in real images.

The ground truth in this training dataset (denoted as [X′,y ]) has two classes: one with the mask of bacteria and the other with the contour of the detected cell. The test set consists of 87 cases of labeled bright-field images unseen by the trained network. The accuracy of the network is calculated over this test set.

Dataset 2

Request a detailed protocol

This dataset was used to train the classification network to discriminate Myxococcus xanthus and Escherichia coli cells directly from phase contrast images. To construct the data set, we obtained images from predatory interactions of M. xanthus and E. coli, where M. xanthus is tagged with green fluorescence (GFP, DM31, Table S1, Supplementary file 1) and Escherichia coli with red fluorescence (mCherry, EC500 -Shaner et al. 2004- Table S1). The fluorescence images were then processed with a gamma adjustment and segmented using MiSiC to produce clean masks and contours of two classes, namely, Myxococcus xanthus and Escherichia coli. Thus, y in dataset two contains two channels corresponding to a mask of each class. A total of 4,000 such pseudo-annotated images of size (256 × 256) were used for dataset 2. These images were obtained after random cropping of microscope images (1000 × 1000) and the number of cells varied from image to image.

MiSiC, shape-index map-based segmentation

Request a detailed protocol

σThe shape index (SI) map of an image, x, calculated over a scale , is defined as

SI(x,σ)=2πtan1(k2+k1k2k1),

where k1, k2 (with k1> k2) are the eigenvalues of the Hessian of the image,x, calculated over a scale σ(Koenderink and van Doorn, 1992). The SI map remains within the range [−1, 1] and preserves the MiSiC shape information while being independent of the intensity values of the original image. Using the Dataset 1, we pre-process the input images X′ to generate a train set: Xtrain of size 1260 × 256 × 256 × 3. Each channel in Xtrain is the shape-index map calculated at three different scales [1,1.5,2] to obtain shape index information of the cells at various scales. An instance of the U-net is trained over this data set to produce a network able to map data represented by Xtrain → ytrain. The network learns to reject the noise in the shape-index map and produces masks and boundaries of the cell like structures in the shape-index map. The trained network was tested over 87 cases of labelled bright-field images, that were previously unseen by the network leading to a segmentation accuracy of 0.76 computed with Jaccard, 1912 coefficient.

Preprocessing

Request a detailed protocol

Preprocessing the input image to enhance the edge contrast and homogenising intensities helps in obtaining a good segmentation via MiSiC. Some of the preprocessing that gave good results are gamma correction for homogenising, unsharp masking for sharpening the image and sometimes a gaussian of laplace of the image that removes intensity variations in the entire image and keeps edge-like features.

Parameters: scale and noise variance

Request a detailed protocol

The dataset one used to train the MiSiC contains cells with a width in the range of 8–10 pixels.Thus, to obtain a satisfactory segmentation, the input image must be scaled so that the average bacteria width is around 10 pixels. However, the scaling often modifies the original image leading to a smoother shape index map. Since, MiSiC has basically learnt to distinguish between smooth curvatures with well-defined boundaries from noisy background. A smooth image without inherent noise leads to a lot of false positive segmentations. Therefore, counterintuitively, synthetic noise must be added to the scaled or original image for a proper segmentation. It must be kept in mind that the noise variance should not reduce the contrast of the edges in the original image while it should be enough to discard spurious detections. Gaussian noise of a constant variance may be added to the entire image or alternatively, the variance could be a function of the edges in the input image.

Myxococcus xanthus and Escherichia coli classification

Request a detailed protocol

A U-NET was trained on dataset two to segment a single channel phase-contrast image into an image containing semantic classification of each species. The probability map for each label is color coded (blue = E. coli. coli, orange = M. xanthus) such that each pixel has a probability value to be part of a given class. In rare instances, bi-color objects are obtained because in these cases the prediction is not homogeneous inside the predicted objects. For subsequent analysis, these objects were filtered for the morphometric analysis shown in Figure 5c. A trained model and a python script illustrating how it was used is available at: https://github.com/pswapnesh/MyxoColi.

Image analysis and statistics

Request a detailed protocol

The validation of the automated MiSiC results shown in Figures 2 and 3 and Figure 2—figure supplement 1, was performed by comparing MiSiC masks to hand-annotated masks obtained by a single observer, herein referred to as ground-truth. This ground truth was also compared to data annotated by a second observer to test variability between observers. For comparison, curves of Jaccard Index vs. IoU threshold were constructed based on Jeckel and Drescher, 2021. Such curves allow a precise estimate of the accuracy and quality of shape detection and prediction.

For this, the IoU=XYXY (X = Ground Truth object; Y = Predicted object ) was determined by overlaying each predicted MiSiC mask with its corresponding ground truth mask and the number of True Positives (TP), False Positives (FP) and False Negatives (FN) that remain above varying IoU thresholds, applied from values ranging from 0.00 to 1.00 in 0.01 intervals, was determined so as to calculate the Jaccard index defined by the ratio JI=TPTP+FP+FN for a given threshold value.

To estimate the accuracy of classification, JIs were determined directly by comparing hand-annotated (ground truth) datasets obtained from fluorescence image of GFP (Myxocococus) and mCherry (E. coli).

Analysis of MiSiC performance in the presence of noise and comparison with Supersegger.

To illustrate MiSiC performances in the presence of noise and in comparison with SuperSegger (Stylianidou et al., 2016), datasets consisting of 141 E. coli microcolony images were retrieved from the SuperSegger website. These images were analysed with the provided parameters with SuperSegger and with the following parameters with MiSiC: Cell width = 9, Scaling factor = Auto, Noise = 0.001, Unsharp = 0.6 and Gamma = 0.1. To assess the segmentation robustness to noise for each program, datasets were normalized by the maximum intensity value recorded in the first frame of the dataset and Gaussian noise was added with varying variance. The resulting datasets were then analysed with the initial parameters used to compute reference segmented images except for MiSiC where the noise parameter was set to 0. The relative performance of each program was then evaluated by computing Dice indices (Zijdenbos et al., 1994).

Diceindex=2ABA+B

Morphological analyses

Request a detailed protocol

Classic morphological features: Area, Perimeter, Bounding Box (Width, Height), Circularity, Feret diameter, minimum Feret diameter, MajorAxis (ellipse), MinorAxis (ellipse), (n) number of objects.

Special calculated morphological features: Solidity = Area / Convex Area; AR = MajorAxis / MinorAxis; Extend = Area / (Width*Height).

Demograph construction

Request a detailed protocol

To construct the plot shown in Figure 4e (demograph), the cells bodies were obtained with MiSiC segmentation and the binary mask was analyzed with the MicrobeJ software (Ducret et al., 2016), with a cell model set to parameters (area >0.5 µm2, Circularity <0.8, ‘poles’ = 2) to filter all remaining segmented objects that do not correspond to cells. The localization of the centroid (cell middle) and length of longitudinal axis was then determined for each cell under MicrobeJ. The fluorescent clusters were detected with a local maxima filter and their position relative to the middle of the cell was plotted along the axis with a positive sign. Negative sign clusters are therefore the manifestation of rare cells with bi-polar foci. The fluorescent clusters are plotted as dots with a color scale based on spatial density.

Cell division detection

Request a detailed protocol

In Figure 4i, clusters of fluorescent protein FtsZ-NG were used as cell division markers. The clusters were detected by local maxima detection (scikits-image.peak_local_max(image = fluorescence image, label = MiSiC mask, num_peaks = 1)).

Calculation of cell division ratios

Request a detailed protocol

The cell division ratio in Figure 4j was calculated using the spatial 2D density derived from (i) the mask of the total fluorescent cell population across the entire image and (ii), the mask of the total number of fluorescent maxima (reflecting dividing cells) across the entire image. Spatial densities were calculated with sklearn.neighbors.KernelDensity(), with a bandwidth of 1% of the image width. The proportion of dividing cells was obtained by dividing the spatial density maps: density of fluorescent maxima/density of total cells.

Code availability

Request a detailed protocol

A MiSiC pip installable python package is available at https://github.com/pswapnesh/MiSiC (Swapnesh, 2021; copy archived at swh:1:rev:45f659124a8e207f78296d77664fb96de5472708).

A Graphic user Interface pip installable as python package is available at https://github.com/leec13/MiSiCgui (Espinosa, 2021; copy archived at swh:1:rev:97401bedd44aa7b28d22f8cf87e76b521f15f40a).

Data availability

The tensorflow model describe in this article is available in GitHub: https://github.com/pswapnesh/MiSiC (copy archived at https://archive.softwareheritage.org/swh:1:rev:45f659124a8e207f78296d77664fb96de5472708); https://github.com/leec13/MiSiCgui (copy archived at https://archive.softwareheritage.org/swh:1:rev:97401bedd44aa7b28d22f8cf87e76b521f15f40a). Source data files have been provided for Figures 2, 3, 4 and 5.

References

  1. Conference
    1. Yang Y
    2. Cao G
    (2013) 2013 10th International Conference on Fuzzy Systems and Knowledge Discovery
    Combining watersheds and conditional random fields for image classification. pp. 805–810.
    https://doi.org/10.1109/FSKD.2013.6816304

Decision letter

  1. Jie Xiao
    Reviewing Editor; Johns Hopkins University, United States
  2. Gisela Storz
    Senior Editor; National Institute of Child Health and Human Development, United States
  3. Zach Hensel
    Reviewer; ITQB NOVA, Portugal

Our editorial process produces two outputs: i) public reviews designed to be posted alongside the preprint for the benefit of readers; ii) feedback on the manuscript for the authors, including requests for revisions, shown below. We also include an acceptance summary that explains what the editors found interesting or important about the work.

Acceptance summary:

Your work of developing the MiSiC package to segment single cells in crowded bacterial colonies and identify different species in the colonies is of great importance to the community. The final version of the manuscript incorporated all reviewers' comments with improved readability. An updated user guide is provided. We are also pleased with your commitment to disseminate this important tool to other research labs in the community.

Decision letter after peer review:

Thank you for submitting your article "MiSiC, a general deep learning-based method for the high-throughput cell segmentation of complex bacterial communities" for consideration by eLife. Your article has been reviewed by 3 peer reviewers, one of whom is a member of our Board of Reviewing Editors, and the evaluation has been overseen by Gisela Storz as the Senior Editor. The following individual involved in review of your submission has agreed to reveal their identity: Zach Hensel (Reviewer #2).

The reviewers have discussed their reviews with one another, and the Reviewing Editor has drafted the following to help you prepare a revised submission.

Essential revisions:

All three reviewers recognize the significance and potential impact of the work and would like to see this work being implemented and disseminated to a wider bacterial cell biology field. However, the following essential revisions are required before the work could be considered for publication at eLife. These essential revisions were discussed and agreed upon among the reviewers and the reviewing editor during the consultation. In addition to these essential revisions, detailed comments from the three reviewers should also be addressed in the revision.

1. Please benchmark the performance of MiSiC against currently available segmentation methods using similar approaches such as Supersegger, DeepCell and DeLTA.

2. Please provide a detailed description of the working principle of MiSiC, and a ready-to-use workflow (or a handbook/guide) for users. These descriptions should include not only the essential steps, but also detailed parameters lists/ranges/considerations, such as cell density, size, pixel size, signal-to-noise ratio, and cell shape range etc. The goal of this essential revision is to ensure that the MiSiC tool can be easily disseminated and implemented by other non-technical-driven users in order to maximize the impact of the work.

Reviewer #1 (Recommendations for the authors):

1. It is surprising to see that the authors have not applied the same algorithm to oval or round-shaped cells. Based on the principle of SIM, I do not see why MiSiC cannot be applied to those cells. Could the authors comment further on the limitations or show some results of round cells?

Reviewer #2 (Recommendations for the authors):

The results applying U-Net to the SIM images are very interesting and I look forward to seeing if it will improve performance of the segmentation method (based on DeLTA) that we are using in our lab now; alone or input together with the unprocessed image.

Reviewer #3 (Recommendations for the authors):

– The authors should emphasize that MiSiC is a 2D image analysis tool. It cannot handle 3D image data and it cannot handle 3D+time image data natively at the moment. This needs to be clearly stated.

– The introduction is rather short and does not provide sufficient context for most readers in my opinion. Particularly the 2nd paragraph is so short that the arguments are not clear. The second sentence criticizes machine learning techniques for requiring large amounts of training data. But the method presented in this paper is also a machine learning technique that requires a large amount of training data. I recommend that the authors significantly expand this paragraph to clarify and motivate their methodology.

– More generally, the introduction would benefit by placing the paper into the context of other image analysis tools for bacterial segmentation and colonies. There has been a lot of activity in image analysis for microbiology recently, and I think it would be helpful to readers to learn about this context.

– Results, paragraph 1 +2: Can the authors explain why the images need to be scaled so that the average cell size is set to 10 pixels? I guess this is based on the implicit assumption that there are sufficient intensity gradients on the 2x2 pixel scale used by the Hessian. As the SIM approach is a critical component of the MiSiC method, a clear explanation is needed here.

– A major advantage of MiSiC is that it uses the SIM images as training data for the CNN, which seems to result in a trained CNN model that can produce accurate segmentations for phase contrast, brightfield and fluorescence images. Is this true? Can the authors please very clearly state if this is true? The statements regarding this point in the manuscript are not completely unambiguous in my opinion. If true, this would be a major advance for bacterial image segmentation. Therefore, the authors should also state if the trained CNN model results in equal performance on all 3 imaging modalities. Or does a different CNN model need to be trained for each of the 3 imaging modalities with the same MiSiC workflow?

– Figure 2: In order for MiSiC to perform better on non-rod-shaped bacteria (filamentous bacteria, spirochetes, or cocci) – would the user need to generate new training data and re-train the model? I think this needs to be clarified.

– The authors supplemented their manually annotated data with synthetic data created by images of model rods with synthetic noise. Can the authors explain why this was done? Is the training with the manual annotation not sufficient? If the authors only used this synthetic data, does MiSiC also produce accurate segmentations, or is the real data needed?

– The semantic segmentation obtained by MiSiC (Figure 4) is impressive and works well. It is unclear whether this semantic segmentation also works in cases of strong intermixing between the cell types. Can the authors comment on that?

– Any development of a single cell segmentation method should include a graph of the Jaccard coefficient (and/or Dice index) as a function of the intersection over union, with error bars. The authors need to add such a graph to the manuscript so that authors can judge the quality of the segmentation.

[Editors' note: further revisions were suggested prior to acceptance, as described below.]

Thank you for submitting your article "MiSiC, a general deep learning-based method for the high-throughput cell segmentation of complex bacterial communities" for consideration by eLife. Your article has been reviewed by 3 peer reviewers, including Jie Xiao as the Reviewing Editor and Reviewer #1, and the evaluation has been overseen by Gisela Storz as the Senior Editor. The following individual involved in review of your submission has agreed to reveal their identity: Zach Hensel (Reviewer #2).

The reviewers have discussed their reviews with one another, and wished the authors to provide the following before the work can be accepted for publication:

1. Update User Guide to address issues and miscellaneous bugs related to file installations, encoding, handling of multiple images and etc as specified in reviewers 1 and 2's comments.

2. Expand the introductory or Discussion section to include a more detailed comparison with existing algorithms with a focus on the strength and weakness of each method, as specified in Reviewers 1 and 3's comments.

Reviewer #1 (Recommendations for the authors):

The revised manuscript by Panigrahi et al. addresses our major concerns. We are especially pleased to see the handbook created for the github page. Not only will the handbook be a valuable resource to users, but also the handbook can be updated as improvements and additions are made to the MiSiC GUI. Also, the new supplemental figure S1 is a helpful illustration of the two user-set parameters. We understand the difficulty in comparing MiSiC to different methods and appreciate the quantification of MiSiC's performance. Outlining the appropriate uses and limitations of MiSiC in the discussion is also appreciated. We hope the authors could further address the following concerns. The goal is to maximize the usability of MiSic for the community.

1. Cell density should be explicitly addressed in 1a of the handbook to match the heading. The new example of Caulobacter in Figure 3a is less dense than the images from the previous manuscript version, indicating that MiSiC can segment less dense cell populations. Since the majority of the images are still very dense clusters, it is still worth addressing density in the handbook if not in the manuscript text.

2. Suggesting specific preprocessing methods is useful, but the names alone might not be enough detail for the target MiSiC user. Either referencing FIJI plugins that accomplish the recommended processing or adding citations to the methods section would clarify these suggestions.

3. MiSiC is not expected to be absolutely accurate, and sometimes the binary mask output will require manual edits. For example, two cells recognized as one might need to be manually separated by pixels. If the GUI could incorporate the ability to manually modify the mask, that would tremendously increase the functionality of MiSiC. However, if that is too labor intensive, the post-processing section of the handbook should address how to make these manual adjustments to the output. This could also be mentioned in the discussion.

4. Could the authors expand the comparison between MiSiC with DeltaT, DeeptCell and Supersegger in the third paragraph of introduction be incorporating some of the language in the rebuttal letter? The goal is to give a bird's eye view of the current field so readers will have a clear assessment of which is good for what and understand MiSiC's uniqueness better.

5. There are some errors in GUI. They are not related to the segmentation algorithm, but confusing for users sometimes. For instance, the cell width measuring function is not always returning the right measurement. The way to break this function is to draw an extremely long line before actually tracing the short axis of any cell. Furthermore, MiSiC should be able to analyze all images belong to the same directory with just "one click", in theory. However, this was not the case today when we fed the GUI with multiple images. New masks could not be generated unless previous/existing images and masks are all cleared from the workspace. We also realized that applying the same parameter settings based on only one image does not guarantee accurate segmentation for other images, even though these images belong to the same experiment/run. It would be great if the authors could fix these bugs to enhance user experience.

Reviewer #2 (Recommendations for the authors):

I think that the authors have sufficiently addressed issues raised in the reviews. The examples of preprocessing/parameter choices in the handbook will be very useful.

In my opinion, the availability workflows to generate synthetic data and train the models would strengthen the manuscript since some potential users will want to sacrifice general for specific performance. However, anyone with the expertise to do that also has the expertise to reinvent the wheel to some degree based on what is reported in the manuscript.

The only other issue I have now is installation instructions (MiSiCgui page) no longer work for me (Windows 10; following instructions for conda). I created a python environment as specified, installed the misic package, and tried to install the GUI. (1) I think that the "use package" instructions should be updated, because add_noise for example is now in extras.py; (2) The GUI pip install command raised an error regarding file encoding. I don't know whether the 2nd error is specific to my system and did not spend much time trying to diagnose it. Lastly, the screenshots on the github page are the old version (noise=0.0001 rather than 1).

Reviewer #3 (Recommendations for the authors):

Author's response to the general comment 1 (page 1+2 of the rebuttal letter): I now understand better how it can be difficult to benchmark MiSiC against the other segmentation software. I also appreciate that the authors now discuss these other tools in lines 73-84 of the manuscript. However, the essential points that make the other tools unsuitable for the analysis that was desired by the authors are not mentioned in the main text (only in the rebuttal letter). For me, as a potential user of all of these tools, and probably for anyone who reads such a paper, it is important to know the strengths and weaknesses of these tools and why the other tools are not suitable for the authors' application. Therefore, I recommend that the authors should expand further the relevant paragraph in the main text, to more clearly describe why the other tools are not suitable. This doesn't have to be overly critical of the other tools, but it would be helpful to the readers.

All other comments were addressed nicely by the authors in my opinion.

https://doi.org/10.7554/eLife.65151.sa1

Author response

Essential revisions:

All three reviewers recognize the significance and potential impact of the work and would like to see this work being implemented and disseminated to a wider bacterial cell biology field. However, the following essential revisions are required before the work could be considered for publication at eLife. These essential revisions were discussed and agreed upon among the reviewers and the reviewing editor during the consultation. In addition to these essential revisions, detailed comments from the three reviewers should also be addressed in the revision.

1. Please benchmark the performance of MiSiC against currently available segmentation methods using similar approaches such as Supersegger, DeepCell and DeLTA.

These software packages are machine (or Deep) learning-based but they do not address similar needs and none was designed or proven to be able to perform semantic segmentation on cells with different shapes. DeLTA was designed for the tracking of E. coli cells over many generations constrained in a so-called microfluidics device (called the mother machine) where cells are aligned on a microfluidics channel. Thus, DeLTa was not shown to work for semantic segmentation and is likely too specialized for a broad application.

Supersegger is an alternative to MiSiC for the segmentation of bacteria in dense colonies. Before developing MiSiC, we actually evaluated Supersegger extensively to attempt segmentation of Myxococcus cells in dense environments. During this experience, we observed that to produce acceptable results with Supersegger we had to train the network for each individual field-of-view (FOV). This was a lengthy process that required manual segmentations to produce reliable ground truths for each FOV and therefore made Supersegger unusable for the segmentation of large experiments with tens of FOVs displaying highly heterogeneous cell densities, as this would have required the separate training of each FOV for each experiment. We have never attempted to segment both Myxococcus and E. coli cells with Supersegger as this software tool was not designed to handle several cell types with different shapes at once.

DeepCell is a general-purpose tool developed mainly for the segmentation of eukaryotic cells, not of bacteria. Nevertheless, we installed DeepCell and attempted to segment a typical field of view containing Myxococcus and E. coli cells using the most closely applicable network provided by DeepCell (i.e. NuclearSegmentation). Unfortunately, the network was unable to converge to a prediction. The most likely reason is that DeepCell was developed to detect a single kind of eukaryotic cell at a time, not to perform semantic segmentation on cells with different shapes. We did not test DelTa as its design was specific to detect cells in microfluidic channels.

Our conclusion from this limited testing is that it is unlikely that software packages that were designed for very different purposes, and that were not designed or shown to work for semantic segmentation, would perform efficiently. Of course, it is likely that intense network retraining and/or parameter tuning would improve performance of these algorithms, but even in that case this would only demonstrate that these methods are ill suited for the semantic segmentation of cells in dense environments at high throughputs.

A major asset of MiSiC is its simple use, efficiency under several image modalities, ability to identify multiple bacterial species and robustness to noise and intensity variations for complex dataset segmentation. For these reasons, we think that will be a preferable alternative to the aforementioned softwares for these applications. To make this clear and rather than conducting disputable analyses between softwares for this revision, we rigorously quantified MiSiC performance by comparing it to the ground truth using an established statistical method as recommended by the reviewers. These analyses now demonstrate that MiSiC performs well under various modalities and on numerous species. In addition, it also allows comparing performance for the various conditions and species and thus an estimate of its advantages and limitations. This allows the user to evaluate the use of MiSiC for a given application, a type of analysis which to our knowledge has not been provided by the other softwares. The manuscript is now deeply modified in many instances to discuss the points raised in this response.

2. Please provide a detailed description of the working principle of MiSiC, and a ready-to-use workflow (or a handbook/guide) for users. These descriptions should include not only the essential steps, but also detailed parameters lists/ranges/considerations, such as cell density, size, pixel size, signal-to-noise ratio, and cell shape range etc. The goal of this essential revision is to ensure that the MiSiC tool can be easily disseminated and implemented by other non-technical-driven users in order to maximize the impact of the work.

Thank you for this recommendation. We now provide a detailed handbook for the correct segmentation of bacterial cells under MiSiC, including advice for installation under Napari, pre-processing, segmentation and post-segmentation analyses.

Reviewer #1 (Recommendations for the authors):

1. It is surprising to see that the authors have not applied the same algorithm to oval or round-shaped cells. Based on the principle of SIM, I do not see why MiSiC cannot be applied to those cells. Could the authors comment further on the limitations or show some results of round cells?

We now provide data showing that the quality of the segmentation decays as cell shape deviates from the rod shape (see Figure 3). This is clear for Bacillus filaments, which are well resolved but only partially septated and for Caulobacter, which is crescent shaped and segmented by MiSiC with an accuracy inferior to that of typical rod-shaped like E. coli, Myxococcus or Pseudomonas. As for round cells, the initial training data was such that round objects (see Author response image 1) would be excluded since it is frequent in images that there are spurious background objects that are also round. Thus, training the network to detect round cells is certainly feasible and will indeed make it useful to segment round bacteria, but it will also increase the spurious detections of background objects. For these reasons, we have not implemented round cells’ detection in this version.

Author response image 1
Note the presence of a round object (perhaps a round cell) in the PC image.

MiSiC excludes this object because it deviates too much from the training data set only to retain the rod shape objects.

Reviewer #3 (Recommendations for the authors):

– The authors should emphasize that MiSiC is a 2D image analysis tool. It cannot handle 3D image data and it cannot handle 3D+time image data natively at the moment. This needs to be clearly stated.

This is correct, we now make this unambiguously clear in the introduction and refer to other approaches for 3D analyses.

– The introduction is rather short and does not provide sufficient context for most readers in my opinion. Particularly the 2nd paragraph is so short that the arguments are not clear. The second sentence criticizes machine learning techniques for requiring large amounts of training data. But the method presented in this paper is also a machine learning technique that requires a large amount of training data. I recommend that the authors significantly expand this paragraph to clarify and motivate their methodology.

Agreed. We significantly expanded this part to contextualize the study better in terms of existing approaches and needs that motivated the development of MiSiC;

– More generally, the introduction would benefit by placing the paper into the context of other image analysis tools for bacterial segmentation and colonies. There has been a lot of activity in image analysis for microbiology recently, and I think it would be helpful to readers to learn about this context.

Agreed. We now discuss recent image analysis tools that allow bacterial cell segmentation in 2D but also 3D communities with added references to Hartmann, 2021 and Zhang et al. 2020.

– Results, paragraph 1 +2: Can the authors explain why the images need to be scaled so that the average cell size is set to 10 pixels? I guess this is based on the implicit assumption that there are sufficient intensity gradients on the 2x2 pixel scale used by the Hessian. As the SIM approach is a critical component of the MiSiC method, a clear explanation is needed here.

This is now explained in the text. Briefly, the cell size must be set to pixels to approximately match the width of the artificial cells used to enrich the training data set.

– A major advantage of MiSiC is that it uses the SIM images as training data for the CNN, which seems to result in a trained CNN model that can produce accurate segmentations for phase contrast, brightfield and fluorescence images. Is this true? Can the authors please very clearly state if this is true? The statements regarding this point in the manuscript are not completely unambiguous in my opinion. If true, this would be a major advance for bacterial image segmentation. Therefore, the authors should also state if the trained CNN model results in equal performance on all 3 imaging modalities. Or does a different CNN model need to be trained for each of the 3 imaging modalities with the same MiSiC workflow?

This comment was also raised by reviewer 1. We now provide quantification of how “true” this is. The results now provided in a new Figure 2b show that a single CNN resolves bacterial cells in CP, Fluo and BF with efficiency comparable to the naked eye and almost similar accuracy. Although clearly images are more accurately segmented in the following order: Fluo, CP and BF.

– Figure 2: In order for MiSiC to perform better on non-rod-shaped bacteria (filamentous bacteria, spirochetes, or cocci) – would the user need to generate new training data and re-train the model? I think this needs to be clarified.

As shown in the new Figure 3b, with our general training method, the accuracy of segmentation declines as cell shape deviates from rod-shapes. Nevertheless the system still accurately segments curved cells such as Caulobacter cells. For filamentous cells, as shown in Figure 3—figure supplement 1, using post processing methods like watershed, it is possible to refine to cell masks that also detect cell separations within the filaments. As discussed in our response to reviewer 1, round cell detection will require fine tuning of the network with a small number of new annotated data. In this version we omitted that data to avoid the detection of spurious non-cell objects. Given that training uses synthetic data, it is quite straightforward to implement new training data in MiSiC based on the needs, which we now explain in text.

– The authors supplemented their manually annotated data with synthetic data created by images of model rods with synthetic noise. Can the authors explain why this was done? Is the training with the manual annotation not sufficient? If the authors only used this synthetic data, does MiSiC also produce accurate segmentations, or is the real data needed?

Unet-based segmentation methods generally require annotated datasets that sample the diversity of images that may be encountered in ‘real-life’ for segmentation. In our case, synthetic data was mainly used for the following reasons:

a) Generalization of segmentation model to unseen imaging modalities and bacterial shapes. The ability to use synthetic data circumvents the necessity of a well annotated dataset that spans over different experimental conditions (different imaging modalities, bacterial shapes, background noise and spurious objects ). Such a dataset is necessary for an end-to-end deep learning model (raw image input and segmented image output). The synthetic data also adds examples of various cell densities that may be encountered in unseen images.

b) Smooth cell shapes: Secondly, the synthetic masks help the decoder side of U-Net to fit a smooth cell shape to the segmented bacteria.

The “real annotated” data is nevertheless necessary: when only trained with synthetic data, the segmentation works but it is not as accurate as with additional ‘real images’. This is presumably due to the fact that the synthetic data was generated by randomly throwing cell-shaped objects onto an image. In this case, the intricate patterns created by cell-cell interactions in ‘real-life’ situations are not sampled. Hence, ‘real’ images, even from a single imaging modality, are required to ‘capture’ the patterns created by dense cell communities. An alternative could have been to use a synthetic cell model with cell-cell interactions, but this is an active field of research and beyond the scope of this work.

We now explain the benefits of this approach in the methods section.

– The semantic segmentation obtained by MiSiC (Figure 4) is impressive and works well. It is unclear whether this semantic segmentation also works in cases of strong intermixing between the cell types. Can the authors comment on that?

We believe that by “strong intermixing” the reviewer refers to the areas where Myxococcus cells actively penetrate the E. coli colony and interact very tightly with the E. coli cells as shown for example in zoom inset in Figure 5c. Our analysis in a Figure 5b and Figure 5d shows that indeed classification works well for a large majority of the population. There are nevertheless ambiguous classifications which are observed as a minor overlap between the two populations. These ambiguities are also apparent in the zoomed image Figure 5c where it is visible that a few classifications events are uncertain. This is inevitable in the tight interaction areas due to low contrast in these areas. The ambiguous cells can be easily eliminated from analysis by filtering with discriminating parameters as shown in Figure 5d. We now discuss these limitations in the text.

– Any development of a single cell segmentation method should include a graph of the Jaccard coefficient (and/or Dice index) as a function of the intersection over union, with error bars. The authors need to add such a graph to the manuscript so that authors can judge the quality of the segmentation.

See our answer above for this particular point, which we believe is a major addition in this revision.

[Editors' note: further revisions were suggested prior to acceptance, as described below.]

Reviewer #1 (Recommendations for the authors):

The revised manuscript by Panigrahi et al. addresses our major concerns. We are especially pleased to see the handbook created for the github page. Not only will the handbook be a valuable resource to users, but also the handbook can be updated as improvements and additions are made to the MiSiC GUI. Also, the new supplemental figure S1 is a helpful illustration of the two user-set parameters. We understand the difficulty in comparing MiSiC to different methods and appreciate the quantification of MiSiC's performance. Outlining the appropriate uses and limitations of MiSiC in the discussion is also appreciated. We hope the authors could further address the following concerns. The goal is to maximize the usability of MiSic for the community.

1. Cell density should be explicitly addressed in 1a of the handbook to match the heading. The new example of Caulobacter in Figure 3a is less dense than the images from the previous manuscript version, indicating that MiSiC can segment less dense cell populations. Since the majority of the images are still very dense clusters, it is still worth addressing density in the handbook if not in the manuscript text.

Thank you for the comment. The handbook now addresses how noise adjustment can help segment images with low cell densities:

“MiSiC can segment bacterial cells at both high and low density. Importantly, in low cell density images, the smoothening of background noise after scaling (see main text ) can lead to spurious cell detection, which can be resolved by adding synthetic noise (see section 1d to set up the noise parameter).”

2. Suggesting specific preprocessing methods is useful, but the names alone might not be enough detail for the target MiSiC user. Either referencing FIJI plugins that accomplish the recommended processing or adding citations to the methods section would clarify these suggestions.

The handbook now links these methods to the FIJI software.

3. MiSiC is not expected to be absolutely accurate, and sometimes the binary mask output will require manual edits. For example, two cells recognized as one might need to be manually separated by pixels. If the GUI could incorporate the ability to manually modify the mask, that would tremendously increase the functionality of MiSiC. However, if that is too labor intensive, the post-processing section of the handbook should address how to make these manual adjustments to the output. This could also be mentioned in the discussion.

Incorporating these tools to the GUI could be interesting in a future version, but it is not immediately necessary as other existing softwares provide these means to correct the MiSiC masks.

We now mention these methods in more details in the handbook:

“The MiSiC mask can occasionally contain predicted cells with imperfect separation or poorly detected septa. […] Manual correction is also feasible in that software using a very practical GUI and associated toolbox that allows editing the contours directly on the mask.”

As well as in the Discussion section lines 443-449:

“Along similar lines, non-cell or improperly separated cell objects an appear in the MiSiC masks and while some can be removed by the introduction of noise, an easy way to do it is to apply a post-processing filter using morphometric parameters to remove objects that are not bacteria. […] MicrobeJ and Oufti are especially useful from downstream analyses of the masks because they both allow cell tracking as well as determining protein localization at sub-diffraction resolution.”

4. Could the authors expand the comparison between MiSiC with DeltaT, DeeptCell and Supersegger in the third paragraph of introduction be incorporating some of the language in the rebuttal letter? The goal is to give a bird's eye view of the current field so readers will have a clear assessment of which is good for what and understand MiSiC's uniqueness better.

We have now introduced these what these softwares provide and explained their merits as well as their current limitations justifying the development of MiSiC. This new text is now in the introduction Lines 72-101.

Machine-learning based techniques are powerful alternatives to overcome the above limitations of traditional segmentation approaches. […] Inspired by these methods, we decided to develop MiSiC (Microbial Segmentation in dense Colonies), a general CNN-based tool to segment bacteria in single or multispecies 2D bacterial communities at very high throughput.”

5. There are some errors in GUI. They are not related to the segmentation algorithm, but confusing for users sometimes. For instance, the cell width measuring function is not always returning the right measurement. The way to break this function is to draw an extremely long line before actually tracing the short axis of any cell.

We have not been able to reproduce this bug, so we could not fix it.

Rare bugs like these may be environment-specific, we now wrote in the handbook that upon such problems users should feel free to contact us directly.

Furthermore, MiSiC should be able to analyze all images belong to the same directory with just "one click", in theory. However, this was not the case today when we fed the GUI with multiple images. New masks could not be generated unless previous/existing images and masks are all cleared from the workspace.

For multiple images, we chose to provide a « process all » option that allows the segmentation of an image stack rather that multiple image files. This is especially convenient to process images of time lapse recordings. For multiple independent images the users just need to assemble them as an stack, for example under ImageJ/FIJI. We added a “Processing of multiple images” paragraph in the handbook to make this clear.

We also realized that applying the same parameter settings based on only one image does not guarantee accurate segmentation for other images, even though these images belong to the same experiment/run. It would be great if the authors could fix these bugs to enhance user experience.

Unfortunately, even when images belong to the same experiment they are not identical and setting up the parameters on a single image does not guarantee that all images will be segmented with similar accuracy. Nevertheless, we found that for an image stack from a time lapse or from images of a same experiment it is often possible to set “mean_width” and “noise” parameters that will be largely effective for the entire stack. For this several parameter combinations might need to be tested to find the best compromise. If the images vary a lot across a single experiment it might be a solution to split them in distinct related image sets for better prediction.

We added tips for the segmentation of image stacks in the “processing of multiple images” paragraph of the handbook to make this clear.

Reviewer #2 (Recommendations for the authors):

I think that the authors have sufficiently addressed issues raised in the reviews. The examples of preprocessing/parameter choices in the handbook will be very useful.

In my opinion, the availability workflows to generate synthetic data and train the models would strengthen the manuscript since some potential users will want to sacrifice general for specific performance. However, anyone with the expertise to do that also has the expertise to reinvent the wheel to some degree based on what is reported in the manuscript.

We also think that anyone with the computational skills to train the CNN with a new data set will be able to generate this set without being directed to a specific workflow.

The only other issue I have now is installation instructions (MiSiCgui page) no longer work for me (Windows 10; following instructions for conda). I created a python environment as specified, installed the misic package, and tried to install the GUI.

To correct this problem, we have now removed all dependencies of MiSiCgui on the MiSiC package. However the installation of the MiSiC package in the same environment as MiSiCgui may raise an incompatibility. Thus, in some cases it will be necessary to remove the old environments of MiSiC and MiSiCgui and force pip install to re-install the package:

pip install --upgrade --force-reinstall git+https://github.com/leec13/MiSiCgui.git

We have added this tip to the handbook and the GitHub page.

(1) I think that the "use package" instructions should be updated, because add_noise for example is now in extras.py;

Correct, the "use package » was updated to include the extras.py module in the imports that provide the add_noise function.

(2) The GUI pip install command raised an error regarding file encoding. I don't know whether the 2nd error is specific to my system and did not spend much time trying to diagnose it. Lastly, the screenshots on the github page are the old version (noise=0.0001 rather than 1).

Correct we fixed these issues.

Reviewer #3 (Recommendations for the authors):

Author's response to the general comment 1 (page 1+2 of the rebuttal letter): I now understand better how it can be difficult to benchmark MiSiC against the other segmentation software. I also appreciate that the authors now discuss these other tools in lines 73-84 of the manuscript. However, the essential points that make the other tools unsuitable for the analysis that was desired by the authors are not mentioned in the main text (only in the rebuttal letter). For me, as a potential user of all of these tools, and probably for anyone who reads such a paper, it is important to know the strengths and weaknesses of these tools and why the other tools are not suitable for the authors' application. Therefore, I recommend that the authors should expand further the relevant paragraph in the main text, to more clearly describe why the other tools are not suitable. This doesn't have to be overly critical of the other tools, but it would be helpful to the readers.

All other comments were addressed nicely by the authors in my opinion.

Thank you for the positive feedback and final recommendation. See our answer to reviewer 1 to see how we modified the text to introduce the available tools.

https://doi.org/10.7554/eLife.65151.sa2

Article and author information

Author details

  1. Swapnesh Panigrahi

    CNRS-Aix-Marseille University, Laboratoire de Chimie Bactérienne, Institut de Microbiologie de la Méditerranée and Turing Center for Living Systems, Marseille, France
    Contribution
    Conceptualization, Investigation, Methodology, Software, Visualization, Writing – original draft, Writing – review and editing
    Competing interests
    No competing interests declared
  2. Dorothée Murat

    CNRS-Aix-Marseille University, Laboratoire de Chimie Bactérienne, Institut de Microbiologie de la Méditerranée and Turing Center for Living Systems, Marseille, France
    Contribution
    Investigation, Data curation
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0001-5809-9267
  3. Antoine Le Gall

    Centre de Biochimie Structurale, CNRS UMR 5048, INSERM U1054, Université de Montpellie, Marseille, France
    Contribution
    Data curation, Visualization
    Competing interests
    No competing interests declared
  4. Eugénie Martineau

    CNRS-Aix-Marseille University, Laboratoire de Chimie Bactérienne, Institut de Microbiologie de la Méditerranée and Turing Center for Living Systems, Marseille, France
    Contribution
    Funding acquisition, Methodology
    Competing interests
    No competing interests declared
  5. Kelly Goldlust

    CNRS-Aix-Marseille University, Laboratoire de Chimie Bactérienne, Institut de Microbiologie de la Méditerranée and Turing Center for Living Systems, Marseille, France
    Contribution
    Conceptualization, Funding acquisition, Investigation, Methodology, Data curation, Visualization, Writing – original draft, Writing – review and editing
    Competing interests
    No competing interests declared
  6. Jean-Bernard Fiche

    Centre de Biochimie Structurale, CNRS UMR 5048, INSERM U1054, Université de Montpellie, Marseille, France
    Contribution
    Conceptualization, Funding acquisition, Project administration, Investigation, Methodology, Supervision, Validation, Data curation, Visualization, Writing – original draft, Writing – review and editing
    Competing interests
    No competing interests declared
  7. Sara Rombouts

    Centre de Biochimie Structurale, CNRS UMR 5048, INSERM U1054, Université de Montpellie, Marseille, France
    Contribution
    Conceptualization, Funding acquisition, Project administration, Investigation, Methodology, Supervision, Validation, Data curation, Writing – original draft, Writing – review and editing
    Competing interests
    No competing interests declared
  8. Marcelo Nöllmann

    Centre de Biochimie Structurale, CNRS UMR 5048, INSERM U1054, Université de Montpellie, Marseille, France
    Contribution
    Conceptualization, Funding acquisition, Project administration, Investigation, Methodology, Supervision, Validation, Data curation, Visualization, Writing – original draft, Writing – review and editing
    Competing interests
    No competing interests declared
  9. Leon Espinosa

    CNRS-Aix-Marseille University, Laboratoire de Chimie Bactérienne, Institut de Microbiologie de la Méditerranée and Turing Center for Living Systems, Marseille, France
    Contribution
    Conceptualization, Funding acquisition, Project administration, Investigation, Methodology, Supervision, Validation, Data curation, Visualization, Writing – original draft, Writing – review and editing
    Competing interests
    No competing interests declared
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0002-1923-2069
  10. Tâm Mignot

    CNRS-Aix-Marseille University, Laboratoire de Chimie Bactérienne, Institut de Microbiologie de la Méditerranée and Turing Center for Living Systems, Marseille, France
    Contribution
    Conceptualization, Funding acquisition, Project administration, Investigation, Methodology, Supervision, Validation, Data curation, Writing – original draft, Writing – review and editing
    For correspondence
    tmignot@imm.cnrs.fr
    Competing interests
    Reviewing editor, eLife
    ORCID icon "This ORCID iD identifies the author of this article:" 0000-0003-4338-9063

Funding

ERC advanced grant (JAWS 885145)

  • Tâm Mignot

AMIDEX

  • Eugénie Martineau

ANR (IBM (ANR-14-CE09-0025-01))

  • Marcelo Nöllmann

ANR (HiResBacs (ANR-15-CE11-0023))

  • Marcelo Nöllmann

CNRS 80-prime

  • Swapnesh Panigrahi

The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

Acknowledgements

We thank Anke Treuner-Lange in Lotte Sogaard-Andersen lab for sharing the plasmid pKA32 with us (Treuner-Lange et al., 2013). MiSiC robustness was tested on microscope images showing various bacterial strains that were kindly shared with us. For that, we thank Anne Galinier and Thierry Doan (Bacillus subtilis), Christophe Bordi (Pseudomonas aeruginosa), Aretha Fiebig (Caulobacter crescentus) and Romain Mercier (SgmX-GFP images of Myxococcus xanthus). We also thank Baptiste Piguet-Ruinet and Célia Jonas for constructing the pDM14 plasmid during their internship. We thank Jean Raphael Fantino for the design of the MiSiC Logo. SP, LE, and TM are funded by CNRS within a 80-Prime initiative. TM is funded by an ERC Advanced Grant (JAWS EM is funded by an AMIDEX-PhD program from Aix-Marseille University). MN, JBF, AL and SR are funded by two ANR grants IBM (ANR-14-CE09-0025-01) and HiResBacs (ANR-15-CE11-0023).

Senior Editor

  1. Gisela Storz, National Institute of Child Health and Human Development, United States

Reviewing Editor

  1. Jie Xiao, Johns Hopkins University, United States

Reviewer

  1. Zach Hensel, ITQB NOVA, Portugal

Publication history

  1. Preprint posted: October 7, 2020 (view preprint)
  2. Received: November 24, 2020
  3. Accepted: September 7, 2021
  4. Accepted Manuscript published: September 9, 2021 (version 1)
  5. Version of Record published: September 28, 2021 (version 2)

Copyright

© 2021, Panigrahi et al.

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 2,073
    Page views
  • 243
    Downloads
  • 4
    Citations

Article citation count generated by polling the highest count across the following sources: PubMed Central, Crossref, Scopus.

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Open citations (links to open the citations from this article in various online reference manager services)

Cite this article (links to download the citations from this article in formats compatible with various reference manager tools)

  1. Swapnesh Panigrahi
  2. Dorothée Murat
  3. Antoine Le Gall
  4. Eugénie Martineau
  5. Kelly Goldlust
  6. Jean-Bernard Fiche
  7. Sara Rombouts
  8. Marcelo Nöllmann
  9. Leon Espinosa
  10. Tâm Mignot
(2021)
Misic, a general deep learning-based method for the high-throughput cell segmentation of complex bacterial communities
eLife 10:e65151.
https://doi.org/10.7554/eLife.65151

Further reading

    1. Cancer Biology
    2. Computational and Systems Biology
    Erika K Ramos, Chia-Feng Tsai ... Huiping Liu
    Research Article

    Tumor-initiating cells with reprogramming plasticity or stem-progenitor cell properties (stemness) are thought to be essential for cancer development and metastatic regeneration in many cancers; however, elucidation of the underlying molecular network and pathways remains demanding. Combining machine learning and experimental investigation, here we report CD81, a tetraspanin transmembrane protein known to be enriched in extracellular vesicles (EVs), as a newly identified driver of breast cancer stemness and metastasis. Using protein structure modeling and interface prediction-guided mutagenesis, we demonstrate that membrane CD81 interacts with CD44 through their extracellular regions in promoting tumor cell cluster formation and lung metastasis of triple negative breast cancer (TNBC) in human and mouse models. In-depth global and phosphoproteomic analyses of tumor cells deficient with CD81 or CD44 unveils endocytosis-related pathway alterations, leading to further identification of a quality-keeping role of CD44 and CD81 in EV secretion as well as in EV-associated stemness-promoting function. CD81 is co-expressed along with CD44 in human circulating tumor cells (CTCs) and enriched in clustered CTCs that promote cancer stemness and metastasis, supporting the clinical significance of CD81 in association with patient outcomes. Our study highlights machine learning as a powerful tool in facilitating the molecular understanding of new molecular targets in regulating stemness and metastasis of TNBC.

    1. Cell Biology
    2. Computational and Systems Biology
    Julie Paxman, Zhen Zhou ... Nan Hao
    Research Article

    Chromatin instability and protein homeostasis (proteostasis) stress are two well-established hallmarks of aging, which have been considered largely independent of each other. Using microfluidics and single-cell imaging approaches, we observed that, during the replicative aging of S. cerevisiae, a challenge to proteostasis occurs specifically in the fraction of cells with decreased stability within the ribosomal DNA (rDNA). A screen of 170 yeast RNA-binding proteins identified ribosomal RNA (rRNA)-binding proteins as the most enriched group that aggregate upon a decrease in rDNA stability induced by inhibition of a conserved lysine deacetylase Sir2. Further, loss of rDNA stability induces age-dependent aggregation of rRNA-binding proteins through aberrant overproduction of rRNAs. These aggregates contribute to age-induced proteostasis decline and limit cellular lifespan. Our findings reveal a mechanism underlying the interconnection between chromatin instability and proteostasis stress and highlight the importance of cell-to-cell variability in aging processes.