To obtain an integrated view of how molecular machinery operates inside cells, biologists are increasingly combining structural data at different length scales, obtained using a range of techniques such as electron tomography, electron microscopy, NMR spectroscopy and X-ray crystallography. Structural data is held in public archives such as the Electron Microscopy Data Bank (EMDB; emdb-empiar.org; Tagari et al., 2002), the Electron Microscopy Public Image Archive (EMPIAR; empiar.org; Iudin et al., 2016), and the Protein Data Bank (PDB; wwpdb.org; Bernstein et al., 1977)

Integration between PDB and EMDB data is based on atomic models in the PDB that have been fitted to or built into EMDB volume maps. For purified biological molecules or larger defined complexes this approach is done routinely. Sequence information from the models can be used to link to other bioinformatics resources such as the Universal Protein Resource (UniProt; uniprot.org; UniProt Consortium, 2013). However, atomic models are not always available for a variety of reasons, such as when molecular averaging fails to obtain high-resolution features or the inherently lower resolution associated with molecules being imaged in more complex or even cellular environments. In such cases, the identification of features often relies on prior knowledge or correlation of structural data obtained at different scales.

Once features have been identified, segmentation (defined here as the decomposition of the 3D volume into regions that can be associated with defined objects) can be employed to facilitate and visualise the interpretation of the map. For example, in a recent study the segmentation of electron and soft X-ray tomography reconstructions was used to study leakage and breakage of the membranes in erythrocytes infected by Plasmodium falciparum, and documented the dramatic changes in the morphology of cells during egress (Hale et al., 2017). The soft X-ray tomograms provided overviews of the membrane compartments in intact, vitrified cells (Figure 1). It should be noted that the word 'segmentation' may have different interpretations: for example, in whole animal, pre-clinical and medical imaging, segmentation includes a concept of a model that is used for fitting of the features. In this manuscript we limit the definition to the separation of density into distinct sub-domains.

Figure 1

Download asset Open asset

In tomography, where multiple copies of nearly identical objects are found, 3D sub-tomogram averaging and 3D classification may be employed to obtain higher resolution reconstructions. This process often involves combining information from multiple tomograms. Since the higher resolution afforded by sub-tomogram averaging provides more structural detail, displaying sub-tomogram averages at the original tomogram positions and orientations may reveal important information about the organization and distribution of the object within a cellular and functional context. If properly annotated such data can be further mined with other questions in mind by other researchers. For example, researchers recently created composite maps of Lassa virus particles by inserting the sub-tomogram average structure of the Lassa virus glycoprotein spike back into the original tomographic reconstructions, revealing the organisation and copy number of the spikes on the virus surface (Figure 2; Li et al., 2016). Another example revealed the lateral clustering of viral membrane proteins mediating membrane fusion (Maurer et al., 2013).

Figure 2

Download asset Open asset

The archiving of segmentation data in EMDB entries was identified as an area requiring urgent attention in previous workshops on “Data-Management Challenges in 3D Electron Microscopy” in 2011 (Patwardhan et al., 2012) and “A 3D Cellular Context for the Macromolecular World” in 2012 (Patwardhan et al., 2014), as was the improved biological annotation of structural data to make it more accessible to the wider biological audience and to enable integration with structural and other bioinformatics resources. Crucially for data integration we need “structured biological annotation” which is here defined as the association of data with identifiers (e.g., accession codes from UniProt) and ontologies taken from well established bioinformatics resources. (Ontologies are formal collections of statements defining concepts, relationships and constraints; for example, the mitochondrial large and small ribosomal subunits are parts of the mitochondrial ribosome which, in turn, is a part of the mitochondrion). To our knowledge, none of the segmentation formats widely used in electron microscopy and related fields currently support structured biological annotation. Furthermore, spatial transformations relating sub-tomograms to their parent tomograms are not currently captured in EMDB. Moreover, wider usage of both segmentation and transformation data by non-expert users is hindered by a plurality of formats.

To discuss and address the challenges of representing and capturing segmentations and transformation data, the Protein Data Bank in Europe (PDBe) organised an expert workshop on “3D Segmentations and Transformations - Building Bridges between Cellular and Molecular Structural Biology” in December 2015. The objectives were:

• To identify data models and formats for representing segmentation and transformation data that could provide support for structured biological annotation, thus facilitating their use by EMDB and enabling data-exchange between different software packages

• To gain a better understanding of the challenges involved in the annotation of electron microscopy data and develop requirements in terms of tools and strategies to facilitate annotation.

Here we report and discuss the main outcomes of the workshop, which was attended by a range of participants including software developers, users of segmentation software, ontology experts, and experts in structure and data archiving.

Prior to the workshop, PDBe developed a draft data model to support segmentations and their annotations in EMDB that could accommodate segmentation descriptions from a range of existing formats and software packages as well as structured biological annotation. It supported the key features of major segmentation packages such as Amira (www.fei.com/software/amira), IMOD (Kremer et al., 1996) and Chimera (Pettersen et al., 2004), and provided scope for extension and flexibility as the field developed. However, the draft data model did not cover minor features (e.g., surface rendering parameters), especially those that are only relevant in the context of a particular software package. The data model was implemented in an XML schema with the following features:

1. Support for hierarchical segmentation description. This is important for representing segmentations from (semi-)automatic approaches that naturally result in a hierarchal segmentation, such as Segger (which iteratively groups the results of the initial watershed segmentation into a hierarchy; Pintilie et al., 2010).

2. Different representations of segmentations. Contours and simple geometric primitives such as spheres and lines are often used to delineate regions of interest (ROIs) when segmentation is performed manually. In automatic segmentation the segments are typically represented as surface meshes and/or 3D volume masks. In the latter case, run-length encoding and limited bit-depth are commonly used techniques to minimise memory requirements. It could be argued that it would be useful to have only one canonical representation and convert all the individual representations to it. However, representing geometric primitives such as spheres as surface meshes could lead to substantial increases in storage size and decreases in accuracy of the descriptions.

3. Support for externally defined (i.e., as separate files) 3D volume masks. It may be useful to allow separation between the metadata (annotations) and the actual segmentations (e.g., to lessen the burden on tools and web-services that only require the metadata). The data model accommodates links to external files (and locations within these files) for representing segments.

4. Segment colours. In some application areas, colour is used to identify objects of the same kind, so it is important that such information is not lost.

The draft data model was intended primarily for internal use in EMDB. However, the meeting participants strongly favoured a broader scope so that the format could serve the entire biological segmentation field. This would also make it easier to support the development of translators between different formats and possibly contribute to a reduction of the number of formats (or at least prevent further proliferation of formats). Representatives for several major software packages used for segmentation including IMOD (D.M.), Amira (R.B.) and Chimera (Tom Goddard, personal communication) have expressed a commitment to providing read/write capabilities for the developed format if standard libraries are made available.

The draft data model included support for various colour models including RGB, HSV and colour names. Participants argued that it would be sufficient to support only the most commonly used one, namely the RGB model, as the other models can be converted to it.

Participants also noted that it might be useful to allow quantification of the estimated certainty of a biological annotation, for example a score for the agreement between a sub-tomogram average and a corresponding region from an originating tomogram. There may also be alternative biological annotations in various combinations (logical OR, XOR, AND, etc.). The quantification of alternative annotations could become very complex to represent and use, and the participants agreed to initially limit the scope to a single annotation per segment and to let the need for more complex representations be driven by actual use cases.

Concerning the transformations between sub-tomogram averages and tomograms, the participants agreed that this information should be incorporated into the segmentation data model; it simply requires adding support for multiple transformations of the same 3D volume representation. It was agreed that the convention to define affine transformations should be well-defined in terms of the transformation, the order in which they are applied, the direction of the transformation, and the orientations and origins of coordinate systems.

With respect to correlative multi-modal imaging it was recognized that there would eventually be a need to go beyond affine transformations, for example to represent distortions and deformations of slice data, but the participants did not come to a conclusion about a coherent extensible format. Often, a segment consists of multiple spatially transformed copies of the same primitive. This is also relevant for sub-tomogram averages as the same volume is to be spatially transformed into multiple locations within a tomogram. To accommodate these situations, every segment can be associated with a list of transformations. This representation will also be useful in the context of template matching for describing the transformations between the template and the 3D volume.

The draft data model was developed in XSD (XML Schema Definition). The definition of data models is greatly facilitated by tools that enable GUI-based development of schemas such as Oxygen and XMLSpy. Code generators such as generateDS create object-model wrappers from schemas that enable reading, writing and manipulation of XML files, thus allowing for rapid prototyping. Various XML validators also allow the correctness of a file relative to a schema to be tested. However, concerns were raised about the verbosity of the XML format and the efficiency with which it can be used. Participants proposed that while XML may be the natural format for a schema defined in XSD, it would be useful to consider other more compact and efficient formats such as JSON and HDF5 (a binary format that allows for efficient representation of hierarchal metadata and data in a single container). Both JSON and HDF5 are now widely supported with libraries in most major programming languages, including Python and C/C++, to facilitate reading and writing. To this end, utilities to convert between the XML, JSON and HDF5 representations of the segmentation data model are currently in development at PDBe.

Future format development will be an iterative process involving extensive consultation with relevant stakeholders to obtain consensus in and support from the community of developers, yielding a format that they will support. A "Segmentation and transformation file format working group" has been established by a subset of the workshop participants, and other developers working on segmentation who are interested in joining the group are asked to contact AP.

PDBe has already modified the data model based on the feedback from the meeting, and this will continue in several rounds of consultation with the working group. The schema is versioned to keep track of changes. To facilitate adoption of the format, dubbed EMDB-SFF (SFF=Segmentation File Format), PDBe is developing translators to/from other commonly used formats. The code for these translators is provided as free open source and distributed via the CCP-EM SVN repository. Comments on the schema should be sent to AP.

As previously explained, structured biological annotation is the association of data with identifiers and ontologies taken from well-established bioinformatics resources. The use of structured biological annotation is not common practice in the electron microscopy or structural biology communities. Therefore, ontology experts were invited to the workshop to explain why these are useful and what resources and tools are available for assigning annotations. Use-cases such as mouse imaging data helped to explain the principles and practice of structured biological annotation. By the end of the meeting there was a clearer appreciation of the importance of structured biological annotation for searching and linking imaging data across different scales, between different imaging and structural databases and with other bioinformatics resources.

Structured annotation would enable the seamless integration of structural, imaging and bioinformatics data from different resources, thus making it possible to provide problem-centric views of biology that incorporate structural and imaging data and are easily accessible by the broader biological community (and in contrast to the highly specialised structure-centric resources that are available today and mainly serve domain-specific communities). However, there were concerns that many in the electron microscopy community would find navigating the landscape of ontologies challenging and that this approach would only gain traction in the community if tools were developed to simplify the biological annotation process.

It was also discussed whether annotation should be performed by the depositor or by EMDB curators. While curators could be trained to a high level of expertise in the use of ontologies, they would not necessarily have enough knowledge about the sample and the specifics of the biological system underlying the study. It was concluded that depositors should perform the annotation, with curators overseeing and checking annotations.

Structured biological annotation for electron microscopy will rely on a range of established ontologies such as Gene Ontology (GO; Gene Ontology Consortium, 2008), Experimental Factor Ontology (EFO; Malone et al., 2010), Protein Ontology (PRO; Natale et al., 2014), Cellular Microscopy Phenotype Ontology (CMPO; Jupp et al., 2016), NCBI organismal classification (NCBITaxon), integrated cross-species for anatomical structures (UBERON; Mungall et al., 2012), imaging modality and sample preparation from Fbbi (Orloff et al., 2013), Foundational Model of Anatomy (FMA) and Cell Ontology (CL; Diehl et al., 2016). It may also include identifiers from resources such as UniProt and the Complex Portal, which in turn contain cross-reference information to other useful standardised vocabularies and common terminology identifiers, such as the OMIM and KEGG (Kanehisa et al., 2016) pathways. This cross-reference information is useful when linking data coded with these terminologies to the ontologies.

Several of these resources provide application programming interfaces (APIs) that can be used to access the information programmatically and provide search functionality. The Samples, Phenotypes and Ontologies Team (SPOT) at EMBL-EBI has developed tools such as Zooma and the Ontology Lookup Service (OLS; Jupp et al., 2015), which aggregate information from a wide range of ontologies and provide APIs to access these tools. These APIs can be used when building tools for segmentation annotation to provide simplified views and search facilities for ontological terms.

At the workshop, PDBe presented mock-ups of a web-based segmentation annotation tool (SAT; Figure 3). This tool would allow a user to add structured biological annotation to segmentations obtained from a variety of different software packages and then output an annotated segmentation file in EMDB-SFF that could be deposited to EMDB or EMPIAR. Annotation could either be done during deposition, in which case the biological annotation from the segmentation file could be harvested by the deposition system to facilitate the deposition process, or it could be done post deposition. The workflow would consist of: (i) the user uploading segmentation files (there could be several if the segments have been saved as separate files) and the corresponding map (unless it is already released in EMDB or EMPIAR); (ii) conversion to an EMDB-SFF file; (iii) use of a GUI-based interface to view the segmentations overlaid on the map and to select segmentations and add annotation; (iv) output of a fully annotated EMDB-SFF file that could be uploaded to EMDB (Figure 4).

Figure 3

Download asset Open asset

Figure 4

Download asset Open asset

Two different options were presented for how annotation could take place (Figure 3). Many macromolecular systems for which data are deposited in EMDB fall into broad categories such as ribosomes, proteasomes, chaperonins and so on: for each of these categories, and with the added information about taxonomy, lists of likely components could be generated to facilitate annotation (Figure 3A). Similarly for cellular level annotation, lists of cellular components could be used. As it would not be possible to cover every potential scenario with pre-defined lists, the other option is to provide a search facility that offers potentially applicable terms from available ontologies (Figure 3B).

The workshop participants expressed strong support for the development of the SAT and the functionality depicted in the mock-ups but raised concerns about a number of issues: the upload of data to a web server – some users may find it challenging to upload large maps and segmentations; the need to annotate segmentations twice – users would typically add free text annotations in the software used for the segmentation and would the need to re-annotate in the SAT; finding the 'right' metadata terms (particularly in cases where a search yields more than one term, and it is not clear which is the most relevant term); annotating a hierarchical segmentation. (The SAT mock-up accommodates annotation on only one level of hierarchy: this might be sufficient in many cases, but it could become problematic as more automated segmentation techniques are developed and their usage expands.)

A desktop version of the SAT would help users concerned about the upload of large amounts of data to a web-server. Another option would be to integrate the functionality for structured biological annotation into existing packages such as IMOD, Chimera and Amira; this would also avoid the problem of users having to annotate the segmentations twice. This alternative would require the development of libraries and widgets that facilitate the use of ontologies and the EMDB-SFF by third parties. For example, the program for segmentation in IMOD already has a 'Name Wizard' plugin that helps the user to choose standardized object names from a CSV file: however, additional development would be need to provide access to on-line ontologies.

Participants agreed that PDBe should start by developing the web-based SAT because it could reuse a number of components that are already being used in other electron microscopy-related web services (such as the Volume slicer; Salavert-Torres et al., 2016), followed by the desktop version. Once the SAT reaches a certain level of maturity PDBe could work with third-party developers to integrate the annotation functionality into their packages.

By far the greatest challenge is developing the functionality to find the appropriate biological metadata (Malone et al., 2016) and tools such as Zooma and OLS will be useful for this purpose. A "Segmentation annotation working group" has been established by a subset of the workshop participants to provide data sets and use cases to aid the design of the SAT and to help with its testing. Members of the electron microscopy community and related communities who are interested in joining the group are asked to contact AP.

The EMDB-SFF data model has undergone a round of updates based on the feedback from the meeting. The development of the file format and the segmentation annotation tools will be iterative, with user-testing and feedback from the working groups being integral parts of the process. The file format and tools are expected to be ready by late 2017, although they might not offer all the features discussed above.

Wide acceptance and support of the EMDB-SFF format by software developers working on segmentations and transformations will be crucial. Providing well-documented open-source tools for working with the format will help in this regard, and the Collaborative Computational Project for Electron cryo-Microscopy (CCP-EM) has committed to distributing these tools, and including them in training events for users and developers. However, the scope of the format is not limited to the cryo-EM field. For example, segmentation is an essential element of the workflow for interpreting data in 3D scanning electron microscopy (3D-SEM; Patwardhan et al., 2014). It will also be possible to provide support for segmentations for other imaging modalities (and also for imaging on other length scales), although the range of biological ontologies and vocabularies will need to be expanded. It should also be possible to support techniques that combine imaging modalities (such as correlative light and electron microscopy), but this will involve extra work on the transformation model.

It was clear from the discussions regarding the annotation of segmentations that there are significant language barriers between the fields. Overcoming these barriers is a prerequisite for progress, as is the development of new tools that will facilitate annotation.

This workshop was an important milestone in that it defined concrete actionable outcomes to address the challenges involved in the integration of cellular and molecular structural data in the public archives. This integration will provide researchers with "problem-centric views" of data from many different sources, and will also help the wider biological and medical communities by making make structural data more accessible.

1. 1. Bennett A
   2. Liu J
   3. Van Ryk D
   4. Bliss D
   5. Arthos J
   6. Henderson RM
   7. Subramaniam S

   (2007) Cryoelectron tomographic analysis of an HIV-neutralizing protein and its complex with native viral gp120

   Journal of Biological Chemistry 282:27754–27759.

   https://doi.org/10.1074/jbc.M702025200

   • PubMed
   • Google Scholar
2. 1. Bennett AE
   2. Narayan K
   3. Shi D
   4. Hartnell LM
   5. Gousset K
   6. He H
   7. Lowekamp BC
   8. Yoo TS
   9. Bliss D
   10. Freed EO
   11. Subramaniam S

   (2009) Ion-abrasion scanning electron microscopy reveals surface-connected tubular conduits in HIV-infected macrophages

   PLoS Pathogens 5:e1000591.

   https://doi.org/10.1371/journal.ppat.1000591

   • PubMed
   • Google Scholar
3. 1. Bernstein FC
   2. Koetzle TF
   3. Williams GJ
   4. Meyer EF
   5. Brice MD
   6. Rodgers JR
   7. Kennard O
   8. Shimanouchi T
   9. Tasumi M

   (1977) The protein data bank: A computer-based archival file for macromolecular structures

   Journal of Molecular Biology 112:535–542.

   https://doi.org/10.1016/S0022-2836(77)80200-3

   • PubMed
   • Google Scholar
4. 1. Diehl AD
   2. Meehan TF
   3. Bradford YM
   4. Brush MH
   5. Dahdul WM
   6. Dougall DS
   7. He Y
   8. Osumi-Sutherland D
   9. Ruttenberg A
   10. Sarntivijai S
   11. Van Slyke CE
   12. Vasilevsky NA
   13. Haendel MA
   14. Blake JA
   15. Mungall CJ

   (2016) The Cell Ontology 2016: Enhanced content, modularization, and ontology interoperability

   Journal of Biomedical Semantics 7:44.

   https://doi.org/10.1186/s13326-016-0088-7

   • PubMed
   • Google Scholar
5. 1. Gene Ontology Consortium

   (2008) The Gene Ontology project in 2008

   Nucleic Acids Research 36:D440–D444.

   https://doi.org/10.1093/nar/gkm883

   • PubMed
   • Google Scholar
6. 1. Hale VL
   2. Watermeyer JM
   3. Hackett F
   4. Vizcay-Barrena G
   5. van Ooij C
   6. Thomas JA
   7. Spink MC
   8. Harkiolaki M
   9. Duke E
   10. Fleck RA
   11. Blackman MJ
   12. Saibil HR

   (2017) Parasitophorous vacuole poration precedes its rupture and rapid host erythrocyte cytoskeleton collapse in plasmodium falciparum egress

   PNAS 114:3439–3444.

   https://doi.org/10.1073/pnas.1619441114

   • PubMed
   • Google Scholar
7. 1. Iudin A
   2. Korir PK
   3. Salavert-Torres J
   4. Kleywegt GJ
   5. Patwardhan A

   (2016) EMPIAR: A public archive for raw electron microscopy image data

   Nature Methods 13:387–388.

   https://doi.org/10.1038/nmeth.3806

   • PubMed
   • Google Scholar
8. Conference

   1. Jupp S
   2. Burdett T
   3. Leroy C
   4. Parkinson H

   (2015)

   A New Ontology Lookup Service at EMBL-EBI

   8th International Conference on Semantic Web Applications and Tools for Life Sciences.

   • Google Scholar
9. 1. Jupp S
   2. Malone J
   3. Burdett T
   4. Heriche JK
   5. Williams E
   6. Ellenberg J
   7. Parkinson H
   8. Rustici G

   (2016) The cellular microscopy phenotype ontology

   Journal of Biomedical Semantics 7:28.

   https://doi.org/10.1186/s13326-016-0074-0

   • PubMed
   • Google Scholar
10. 1. Kanehisa M
    2. Sato Y
    3. Kawashima M
    4. Furumichi M
    5. Tanabe M

    (2016) KEGG as a reference resource for gene and protein annotation

    Nucleic Acids Research 44:D457–D462.

    https://doi.org/10.1093/nar/gkv1070

    • PubMed
    • Google Scholar
11. 1. Kremer JR
    2. Mastronarde DN
    3. McIntosh JR

    (1996) Computer visualization of three-dimensional image data using IMOD

    Journal of Structural Biology 116:71–76.

    https://doi.org/10.1006/jsbi.1996.0013

    • PubMed
    • Google Scholar
12. 1. Li S
    2. Sun Z
    3. Pryce R
    4. Parsy ML
    5. Fehling SK
    6. Schlie K
    7. Siebert CA
    8. Garten W
    9. Bowden TA
    10. Strecker T
    11. Huiskonen JT

    (2016) Acidic pH-induced conformations and LAMP1 binding of the Lassa virus glycoprotein spike

    PLOS Pathogens 12:e1005418.

    https://doi.org/10.1371/journal.ppat.1005418

    • PubMed
    • Google Scholar
13. 1. Liu J
    2. Bartesaghi A
    3. Borgnia MJ
    4. Sapiro G
    5. Subramaniam S

    (2008) Molecular architecture of native HIV-1 gp120 trimers

    Nature 455:109–113.

    https://doi.org/10.1038/nature07159

    • PubMed
    • Google Scholar
14. 1. Malone J
    2. Holloway E
    3. Adamusiak T
    4. Kapushesky M
    5. Zheng J
    6. Kolesnikov N
    7. Zhukova A
    8. Brazma A
    9. Parkinson H

    (2010) Modeling sample variables with an Experimental Factor Ontology

    Bioinformatics 26:1112–1118.

    https://doi.org/10.1093/bioinformatics/btq099

    • PubMed
    • Google Scholar
15. 1. Malone J
    2. Stevens R
    3. Jupp S
    4. Hancocks T
    5. Parkinson H
    6. Brooksbank C

    (2016) Ten simple rules for selecting a bio-ontology

    PLOS Computational Biology 12:e1004743.

    https://doi.org/10.1371/journal.pcbi.1004743

    • PubMed
    • Google Scholar
16. 1. Maurer UE
    2. Zeev-Ben-Mordehai T
    3. Pandurangan AP
    4. Cairns TM
    5. Hannah BP
    6. Whitbeck JC
    7. Eisenberg RJ
    8. Cohen GH
    9. Topf M
    10. Huiskonen JT
    11. Grünewald K

    (2013) The structure of herpesvirus fusion glycoprotein B-bilayer complex reveals the protein-membrane and lateral protein-protein interaction

    Structure 21:1396–1405.

    https://doi.org/10.1016/j.str.2013.05.018

    • PubMed
    • Google Scholar
17. 1. Mungall CJ
    2. Torniai C
    3. Gkoutos GV
    4. Lewis SE
    5. Haendel MA

    (2012) Uberon, an integrative multi-species anatomy ontology

    Genome Biology 13:R5.

    https://doi.org/10.1186/gb-2012-13-1-r5

    • PubMed
    • Google Scholar
18. 1. Müller A
    2. Beeby M
    3. McDowall AW
    4. Chow J
    5. Jensen GJ
    6. Clemons WM

    (2014) Ultrastructure and complex polar architecture of the human pathogen Campylobacter jejuni

    MicrobiologyOpen 3:702–710.

    https://doi.org/10.1002/mbo3.200

    • PubMed
    • Google Scholar
19. 1. Natale DA
    2. Arighi CN
    3. Blake JA
    4. Bult CJ
    5. Christie KR
    6. Cowart J
    7. D'Eustachio P
    8. Diehl AD
    9. Drabkin HJ
    10. Helfer O
    11. Huang H
    12. Masci AM
    13. Ren J
    14. Roberts NV
    15. Ross K
    16. Ruttenberg A
    17. Shamovsky V
    18. Smith B
    19. Yerramalla MS
    20. Zhang J
    21. AlJanahi A
    22. Çelen I
    23. Gan C
    24. Lv M
    25. Schuster-Lezell E
    26. Wu CH

    (2014) Protein Ontology: A controlled structured network of protein entities

    Nucleic Acids Research 42:D415–D421.

    https://doi.org/10.1093/nar/gkt1173

    • PubMed
    • Google Scholar
20. 1. Orloff DN
    2. Iwasa JH
    3. Martone ME
    4. Ellisman MH
    5. Kane CM

    (2013) The cell: an image library-CCDB: A curated repository of microscopy data

    Nucleic Acids Research 41:D1241–D1250.

    https://doi.org/10.1093/nar/gks1257

    • PubMed
    • Google Scholar
21. 1. Patwardhan A
    2. Carazo JM
    3. Carragher B
    4. Henderson R
    5. Heymann JB
    6. Hill E
    7. Jensen GJ
    8. Lagerstedt I
    9. Lawson CL
    10. Ludtke SJ
    11. Mastronarde D
    12. Moore WJ
    13. Roseman A
    14. Rosenthal P
    15. Sorzano CO
    16. Sanz-García E
    17. Scheres SH
    18. Subramaniam S
    19. Westbrook J
    20. Winn M
    21. Swedlow JR
    22. Kleywegt GJ

    (2012) Data management challenges in three-dimensional EM

    Nature Structural & Molecular Biology 19:1203–1207.

    https://doi.org/10.1038/nsmb.2426

    • PubMed
    • Google Scholar
22. 1. Patwardhan A
    2. Ashton A
    3. Brandt R
    4. Butcher S
    5. Carzaniga R
    6. Chiu W
    7. Collinson L
    8. Doux P
    9. Duke E
    10. Ellisman MH
    11. Franken E
    12. Grünewald K
    13. Heriche JK
    14. Koster A
    15. Kühlbrandt W
    16. Lagerstedt I
    17. Larabell C
    18. Lawson CL
    19. Saibil HR
    20. Sanz-García E
    21. Subramaniam S
    22. Verkade P
    23. Swedlow JR
    24. Kleywegt GJ

    (2014) A 3D cellular context for the macromolecular world

    Nature Structural & Molecular Biology 21:841–845.

    https://doi.org/10.1038/nsmb.2897

    • PubMed
    • Google Scholar
23. 1. Pettersen EF
    2. Goddard TD
    3. Huang CC
    4. Couch GS
    5. Greenblatt DM
    6. Meng EC
    7. Ferrin TE

    (2004) UCSF Chimera--A visualization system for exploratory research and analysis

    Journal of Computational Chemistry 25:1605–1612.

    https://doi.org/10.1002/jcc.20084

    • PubMed
    • Google Scholar
24. 1. Pintilie GD
    2. Zhang J
    3. Goddard TD
    4. Chiu W
    5. Gossard DC

    (2010) Quantitative analysis of cryo-EM density map segmentation by watershed and scale-space filtering, and fitting of structures by alignment to regions

    Journal of Structural Biology 170:427–438.

    https://doi.org/10.1016/j.jsb.2010.03.007

    • PubMed
    • Google Scholar
25. 1. Salavert-Torres J
    2. Iudin A
    3. Lagerstedt I
    4. Sanz-García E
    5. Kleywegt GJ
    6. Patwardhan A

    (2016) Web-based volume slicer for 3D electron-microscopy data from EMDB

    Journal of Structural Biology 194:164–170.

    https://doi.org/10.1016/j.jsb.2016.02.012

    • PubMed
    • Google Scholar
26. 1. Santarella-Mellwig R
    2. Pruggnaller S
    3. Roos N
    4. Mattaj IW
    5. Devos DP

    (2013) Three-dimensional reconstruction of bacteria with a complex endomembrane system

    PLoS Biology 11:e1001565.

    https://doi.org/10.1371/journal.pbio.1001565

    • PubMed
    • Google Scholar
27. 1. Tagari M
    2. Newman R
    3. Chagoyen M
    4. Carazo JM
    5. Henrick K

    (2002) New electron microscopy database and deposition system

    Trends in Biochemical Sciences 27:589.

    https://doi.org/10.1016/S0968-0004(02)02176-X

    • PubMed
    • Google Scholar
28. 1. UniProt Consortium

    (2013) Update on activities at the Universal Protein Resource (UniProt) in 2013

    Nucleic Acids Research 41:D43–47.

    https://doi.org/10.1093/nar/gks1068

    • PubMed
    • Google Scholar

Thank you for submitting your article "3D segmentation and transformation – building bridges between cellular and molecular structural biology" to eLife for consideration as a Feature Article. Your article has been reviewed by two peer reviewers, and the evaluation has been overseen by a Reviewing Editor and John Kuriyan as the Senior Editor.

The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission

Summary:

The manuscript by Patwardhan et al., reports on the outcome of a workshop at the European Bioinformatics Institute (EBI) in Hinxton nr. Cambridge, UK. The workshop brought together experienced users and stakeholders from twelve different research laboratories and companies around the world. The purpose of the workshop was to come up with a roadmap for the annotation of structural data on the sub-cellular to cellular scale (primarily electron cryo-tomography and correlated light-EM (CLEM)), and to develop formats and tools for annotation that would be made available to the communities of structural and cellular biologists. Of particular interest will be tools for the integration of structural data from tomography, high-resolution cryoEM, protein crystallography and NMR and the development for a structured biological annotation tool for interactive annotation of tomographic and similar volumes, before uploading them to the EMDB data bank. This is a very worthwhile endeavour that will greatly benefit the future of structural and molecular cell biology. The manuscript itself is a valuable resource in that it provides numerous links to tools and websites that will be useful to many in the community.

Essential revisions:

1) Segmentation is considered from the perspective of structural biology, PDB and EMDB data, as well as sub-tomogram averaging. While integration of structures in the biological context is certainly a worthwhile effort, these are not true segmentation tasks. In fact, true segmentation tasks as in brain connectomics or cell biology are not mentioned at all.

2) The cross-correlation coefficient between a sub-tomogram average and the original volume is not useful as a means of quantification, as is well documented in the literature.

3) The article should distinguish more clearly between segmentation and template matching, and between annotations and metadata.

4) The expression "structured biological annotation" is used repeatedly in the manuscript but is difficult to understand. The authors should explain what they mean more clearly.

Essential revisions:

1) Segmentation is considered from the perspective of structural biology, PDB and EMDB data, as well as sub-tomogram averaging. While integration of structures in the biological context is certainly a worthwhile effort, these are not true segmentation tasks. In fact, true segmentation tasks as in brain connectomics or cell biology are not mentioned at all.

We agree that the scope of the term segmentation and what it entails for different imaging fields can vary. We have now added a sentence explicitly explaining this to be the case and that the manuscript focuses on the narrower definition outlined in the manuscript.

2) The cross-correlation coefficient between a sub-tomogram average and the original volume is not useful as a means of quantification, as is well documented in the literature.

We agree that the scope of the term segmentation and what it entails for different imaging fields can vary. We have now added a sentence explicitly explaining this to be the case and that the manuscript focuses on the narrower definition outlined in the manuscript.

3) The article should distinguish more clearly between segmentation and template matching, and between annotations and metadata.

The manuscript as such does not deal with template matching but with relating sub-tomogram averages to tomograms. The file format could be used to deal with template matching and a line has been added explaining this. We have also gone through the manuscript and updated a couple of instances where it would be more appropriate to use the term ‘meta-data’ rather than annotations.

4) The expression "structured biological annotation" is used repeatedly in the manuscript but is difficult to understand. The authors should explain what they mean more clearly.

In the original manuscript we had defined “Structured biological annotation” explicitly as the first sentence in the section on “Structured biological annotation”. The term was used earlier on in the text and we understand that this could have been the source of confusion. We now define “Structured biological annotation” early on in the text before it is first used.

Author details

1. Ardan Patwardhan

   Cellular Structure and 3D Bioimaging, European Molecular Biology Laboratory, European Bioinformatics Institute, Wellcome Genome Campus, Hinxton, United Kingdom

   Contribution

   AP, Conceptualization, Resources, Supervision, Funding acquisition, Investigation, Visualization, Methodology, Writing—original draft, Project administration, Writing—review and editing, Orgainzed workshop and chaired sessions

   For correspondence

   ardan@ebi.ac.uk

   Competing interests

   No competing interests declared.

   "This ORCID iD identifies the author of this article:" 0000-0001-7663-9028

2. Robert Brandt

   Visualization Sciences Group, FEI, Mérignac, France

   Contribution

   RB, Conceptualization, Writing—review and editing

   Competing interests

   No competing interests declared.

3. Sarah J Butcher

   Institute of Biotechnology and the Department of Biosciences, University of Helsinki, Helsinki, Finland

   Contribution

   SJB, Conceptualization, Writing—review and editing, Chaired workshop sessions

   Competing interests

   No competing interests declared.

4. Lucy Collinson

   Electron Microscopy Science Technology Platform, Francis Crick Institute, London, United Kingdom

   Contribution

   LC, Conceptualization, Writing—review and editing, Chaired workshop session

   Competing interests

   No competing interests declared.

5. David Gault

   Centre for Gene Regulation and Expression, University of Dundee, Dundee, United Kingdom

   Contribution

   DG, Conceptualization, Writing—review and editing

   Competing interests

   No competing interests declared.

6. Kay Grünewald

   Division of Structural Biology, Wellcome Trust Centre for Human Genetics, University of Oxford, Oxford, United Kingdom

   Contribution

   KG, Conceptualization, Writing—review and editing, Chaired workshop session

   Competing interests

   No competing interests declared.

7. Corey Hecksel

   National Center for Macromolecular Imaging, Verna and Marrs McLean Department of Biochemistry and Molecular Biology, Baylor College of Medicine, Houston, United States

   Present address

   Electron Bio-Imaging Centre, Diamond Light Source Ltd, Didcot, United Kingdom

   Contribution

   CH, Conceptualization, Writing—review and editing

   Competing interests

   No competing interests declared.

8. Juha T Huiskonen

   Division of Structural Biology, Wellcome Trust Centre for Human Genetics, University of Oxford, Oxford, United Kingdom

   Contribution

   JTH, Conceptualization, Writing—review and editing, Chaired workshop session

   Competing interests

   No competing interests declared.

9. Andrii Iudin

   Cellular Structure and 3D Bioimaging, European Molecular Biology Laboratory, European Bioinformatics Institute, Wellcome Genome Campus, Hinxton, United Kingdom

   Contribution

   AI, Conceptualization, Visualization, Writing—review and editing

   Competing interests

   No competing interests declared.

10. Martin L Jones

    Electron Microscopy Science Technology Platform, Francis Crick Institute, London, United Kingdom

    Contribution

    MLJ, Conceptualization, Writing—review and editing

    Competing interests

    No competing interests declared.

    "This ORCID iD identifies the author of this article:" 0000-0003-0994-5652

11. Paul K Korir

    Cellular Structure and 3D Bioimaging, European Molecular Biology Laboratory, European Bioinformatics Institute, Wellcome Genome Campus, Hinxton, United Kingdom

    Contribution

    PKK, Conceptualization, Software, Visualization, Methodology, Writing—review and editing

    Competing interests

    No competing interests declared.

12. Abraham J Koster

    Department of Molecular Cell Biology, Leiden University Medical Center, Leiden, The Netherlands

    Contribution

    AJK, Conceptualization, Writing—review and editing, Chaired workshop session

    Competing interests

    No competing interests declared.

13. Ingvar Lagerstedt

    European Molecular Biology Laboratory, European Bioinformatics Institute, Wellcome Genome Campus, Hinxton, United Kingdom

    Present address

    Computational Chemistry and Cheminformatics, Lilly UK, Windlesham, United Kingdom

    Contribution

    IL, Conceptualization, Writing—review and editing

    Competing interests

    No competing interests declared.

14. Catherine L Lawson

    Center for Integrative Proteomics Research and the Research Collaboratory for Structural Bioinformatics, Rutgers, The State University of New Jersey, Piscataway, United States

    Contribution

    CLL, Conceptualization, Writing—review and editing

    Competing interests

    No competing interests declared.

15. David Mastronarde

    Department of Molecular, Cellular, and Developmental Biology, University of Colorado, Boulder, United States

    Contribution

    DM, Conceptualization, Writing—review and editing

    Competing interests

    No competing interests declared.

16. Matthew McCormick

    Kitware, Inc., Carrboro, United States

    Contribution

    MM, Conceptualization, Writing—review and editing

    Competing interests

    No competing interests declared.

17. Helen Parkinson

    Molecular Archival Resources, European Molecular Biology Laboratory, European Bioinformatics Institute, Wellcome Genome Campus, Hinxton, United Kingdom

    Contribution

    HP, Conceptualization

    Competing interests

    No competing interests declared.

18. Peter B Rosenthal

    Structural Biology of Cells and Viruses, Francis Crick Institute, London, United Kingdom

    Contribution

    PBR, Conceptualization, Writing—review and editing, Chaired workshop session

    Competing interests

    No competing interests declared.

19. Stephan Saalfeld

    Janelia Research Campus, Howard Hughes Medical Institute, Ashburn, United States

    Contribution

    SSaa, Conceptualization, Writing—review and editing

    Competing interests

    No competing interests declared.

    "This ORCID iD identifies the author of this article:" 0000-0002-4106-1761

20. Helen R Saibil

    Institute of Structural and Molecular Biology, Department of Crystallography, Birkbeck College, London, United Kingdom

    Contribution

    HRS, Conceptualization, Visualization, Writing—review and editing

    Competing interests

    No competing interests declared.

21. Sirarat Sarntivijai

    Molecular Archival Resources, European Molecular Biology Laboratory, European Bioinformatics Institute, Wellcome Genome Campus, Hinxton, United Kingdom

    Contribution

    SSar, Conceptualization, Writing—review and editing

    Competing interests

    No competing interests declared.

22. Irene Solanes Valero

    European Molecular Biology Laboratory, European Bioinformatics Institute, Wellcome Genome Campus, Hinxton, United Kingdom

    Present address

    University of Vic - Central University of Catalonia, Barcelona, Spain

    Contribution

    ISV, Conceptualization, Data curation, Writing—review and editing

    Competing interests

    No competing interests declared.

23. Sriram Subramaniam

    Laboratory for Cell Biology, Center for Cancer Research, National Cancer Institute, Bethesda, United States

    Contribution

    SS, Conceptualization, Visualization, Writing—review and editing

    Competing interests

    SS: Reviewing editor, eLife

24. Jason R Swedlow

    Centre for Gene Regulation and Expression and the Division of Computational Biology, University of Dundee, Dundee, United Kingdom

    Contribution

    JRS, Conceptualization, Writing—review and editing

    Competing interests

    No competing interests declared.

25. Ilinca Tudose

    Molecular Archival Resources, European Molecular Biology Laboratory, European Bioinformatics Institute, Wellcome Genome Campus, Hinxton, United Kingdom

    Contribution

    IT, Conceptualization, Writing—review and editing

    Competing interests

    No competing interests declared.

26. Martyn Winn

    Scientific Computing Department, Science and Technology Facilities Council, Research Complex at Harwell, Didcot, United Kingdom

    Contribution

    MW, Conceptualization, Funding acquisition, Writing—review and editing

    Competing interests

    No competing interests declared.

27. Gerard J Kleywegt

    Molecular and Cellular Structure Cluster, European Molecular Biology Laboratory, European Bioinformatics Institute, Wellcome Genome Campus, Hinxton, United Kingdom

    Contribution

    GJK, Conceptualization, Funding acquisition, Writing—review and editing

    For correspondence

    gerard@ebi.ac.uk

    Competing interests

    No competing interests declared.

• 3,266

  views

• 581

  downloads

• 14

  citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.