Microscopes can be used to view objects or structural details that are not visible with the naked eye. A type of microscope called an electron microscope—which uses beams of particles called electrons—is particularly useful for examining tiny objects or structures because it can produce images with a higher level of detail than microscopes that use light.

There are several ways to prepare biological samples for electron microscopy. One technique is called cryo-electron microscopy, or cryo-EM for short, where the sample is rapidly frozen and then viewed under the electron microscope. Using this technique it is possible to produce highly detailed images of viruses, individual compartments within cells and even single proteins.

To convert the images of proteins into three-dimensional models, high-performing clusters of computers are required. It can be difficult and expensive for many scientists to access these resources, which may limit the wider use of cryo-EM in research.

To address this problem and aid the spread of cryo-EM, Cianfrocco and Leschziner developed a publicly available ‘off the shelf’ system on Amazon's elastic cloud computing infrastructure. This provides users with software packages and the ability to create a cluster containing up to around 480 computers to analyze cryo-EM data.

Cianfrocco and Leschziner tested the system using a publicly available cryo-EM dataset of a structure in yeast cells called the 80S ribosome, which contains proteins and molecules of ribonucleic acid. This revealed that a highly detailed model of the 80S ribosome could be developed in a time frame similar to what it would have taken on a local high-performing computing cluster within a university. The cost of using this system was also competitive in price with that of maintaining a local computing cluster, with the added flexibility of its ‘pay-as-you-go’ structure.

These findings show that Amazon's cloud computing infrastructure may be a useful alternative to using clusters of computers based within a research institute or university. This will help the spread of cryo-EM as a general tool to reveal the three-dimensional structures of large molecules. Further work is required to make this cloud-based computing tool easily accessible to researchers who may have limited experience with using Linux software and computing clusters.

Cryo-electron microscopy (cryo-EM) has long served as an important tool to provide structural insights into biological samples. Recent advances in cryo-EM data collection and analysis, however, have transformed single particle cryo-EM (Kuhlbrandt, 2014; Bai et al., 2015), allowing it to achieve resolutions better than 5 Å for samples ranging in molecular weight from the 4 MDa eukaryotic ribosome (Bai et al., 2013) to the 170 kDa membrane protein γ-secretase (Lu et al., 2014). These high-resolution structures are the result of a new generation of cameras that detect electrons directly without the need of a scintillator, which results in a dramatic increase in the signal-to-noise ratio relative to CCD cameras, the previous most commonly used device (McMullan et al., 2009). In addition to direct electron detection, the high frame rate of these cameras allows each image to be recorded as a ‘movie’, dividing it into multiple frames. These fractionated images can be used to correct for sample movement during the exposure, further increasing the quality of the cryo-EM images (Campbell et al., 2012; Li et al., 2013; Scheres, 2014).

In addition to these technological developments in the detectors, improvements in computer software packages have played an equally important role in moving cryo-EM into the high-resolution era. Atomic or near-atomic structures have been obtained with software packages such as EMAN2 (Tang et al., 2007), Sparx (Hohn et al., 2007), FREALIGN (Grigorieff, 2007), Spider (Frank et al., 1996), and Relion (Scheres, 2012, 2014). In general, obtaining these structures involved computational approaches that sorted out the data into homogenous classes that could then be refined to high resolution.

While these advances in microscopy and analysis have been essential for the recent breakthroughs in cryo-EM, their implementation is computationally intensive and requires high-performance computing clusters. A recent survey of high-resolution single particle cryo-EM structures showed that refinement of these structures required processing times in excess of 1000 CPU-hours (Scheres, 2014). Therefore, computational time (i.e., access to high-performance clusters) may represent a bottleneck to determining high-resolution structures by single particle cryo-EM.

In order to address this limitation, we explored the possibility of using Amazon's elastic cloud computing (EC2) for processing cryo-EM data. To help others take advantage of this resource, we have created a publicly available ‘off-the-shelf’ software environment that allows new users to start up a cluster of Amazon CPUs preinstalled with cryo-EM software and we have used it to test the performance of Amazon's EC2 platform. We were able to determine a 4.6 Å structure of the 80S ribosome using a published dataset (Bai et al., 2013) for an overall cost of $100 USD within a timeframe comparable to that of a local cluster. Given the range of prices for accessing Amazon CPUs (users can bid for significantly reduced costs) and the accessibility statistics, we estimate that typical cryo-EM structures can be determined for $50–$1500 per structure.

EC2 through Amazon Web Services (AWS)

AWS is a division of Amazon that offers a variety of cloud-based solutions for website hosting and high-performance computing, amongst other services. Many different types of privately held companies take advantage of Amazon's computing infrastructure because of its affordability, flexibility, and security. Of note, global biotechnology companies such as Novartis (AWS, 2014a), Bristol-Myers-Squibb (AWS, 2013), and Pfizer (AWS, 2014b) have utilized the computing power of Amazon for scientific data processing. Many academic researchers have also begun to use Amazon's EC2 resources for analyzing datasets from super-resolution light microscopy (Hu et al., 2013), genomics (Krampis et al., 2012; Yazar et al., 2014), and proteomics (Mohammed et al., 2012; Trudgian and Mirzaei, 2012).

The overall workflow starts with users logging into a virtual machine (‘instance’) on AWS (Figure 1). AWS offers a variety of instance types that have been configured for different computing tasks. For example, instances have been optimized for computing performance, GPU-based calculations, or memory-intensive calculations. After logging onto an instance, storage drives are mounted onto it, allowing data, which can be encrypted for security, to be transferred onto the storage drives (Figure 1).

Figure 1

Download asset Open asset

While users can utilize a single instance for calculations, the maximum number of CPU cores per instance is 18. Therefore, creating a computing cluster with a larger number of CPUs on AWS requires additional steps. The Software Tools for Academics and Researchers (STAR) group at the Massachusetts Institute of Technology developed a straightforward package that allows users to group individual AWS instances into a cluster. The STARcluster program is a python-based, open source package that automatically creates a cluster preconfigured with the necessary software to manage a computer cluster (Ivica et al., 2009). This package allows users to specify the number of instances to be included in the clusters as well as the instance type. By taking advantage of this tool, private clusters can be built with sizes ranging from 16 to 480 CPUs (Figure 1).

Global availability of spot instances on Amazon EC2

While Amazon provides dedicated access to instances through ‘on-demand’ reservations, there are ‘spot instances’ that are 80–90% cheaper than the on-demand price. Spot instances are unused instances within Amazon EC2 that are open for competitive bidding, where users gain access to them by making offers above the current minimum bid. This means that while the on-demand rate for high-memory, 16-CPU instances (called ‘r3.8xlarge’) is $2.80/hr, spot instance prices can be as low as $0.25–$0.35/hr.

In order to determine if spot instances offer a consistent reduction in price, we analyzed the global availability of r3.8xlarge spot instances. Currently, Amazon has 9 regions worldwide within 7 countries: US-East-1 (United States), US-West-1 (United States), US-West-2 (United States), SA-East-1 (Brazil), EU-Central-1 (Germany), EU-West-1 (Ireland), AP-Northeast-1 (Japan), AP-Southeast-1 (Singapore), and AP-Southeast-2 (Australia). For each region, we retrieved spot instance prices for r3.8xlarge instances over the past 3 months and analyzed the time they spent at prices below $0.35–$0.65/hr (corresponding to discounts of 87.5–76.8% over the full on-demand rate of $2.80/hr) (Figure 2 and Figure 2—figure supplement 1). This analysis revealed that, globally, 49.8% of r3.8xlarge instances were below $0.35/hr, 12.5% the on-demand price (Figure 2). For $0.65/hr, 76.5% below full price, one could access 82.2% of the global r3.8xlarge spot instances. These data indicate that spot instances provide dependable, cost-effective access to Amazon's computing resources.

Figure 2 with 1 supplement see all

Download asset Open asset

Figure 2—source data 1

Global spot instance price data from 1 January 2015 to 1 April 2015.

https://doi.org/10.7554/eLife.06664.005

Download elife-06664-fig2-data1-v2.txt

Performance analysis of Amazon's EC2 environment with a 80S yeast ribosome dataset

To test the performance of Amazon's EC2 environment, we analyzed a previously published 80S Saccharomyces cerevisiae ribosome dataset (Bai et al., 2013) (EMPIAR 10002) on a 128 CPU cluster (8 × 16 CPUs; using the r3.8xlarge instance). After extracting 62,022 particles, we performed 2D classification within Relion. Subsequent 3D classification of the particles into four classes revealed that two classes adopted a similar structural state. We merged those two classes and used the associated particles to carry out a 3D refinement in Relion—we were able to obtain a structure with an overall resolution of 4.6 Å (Figure 3A–C).

Figure 3

Download asset Open asset

This structure, whose generation included particle picking, CTF estimation, 2D and 3D classification, and refinement, cost us $99.64 on Amazon's EC2 environment. This cost was achieved by bidding on spot instances for particle picking (m1.small at $0.02/hr), 2D classification (STARcluster of r3.8xlarge instances at $0.65/hr), and 3D classification and refinement (STARcluster of r3.8xlarge instances at $0.65/hr). Thus, even though obtaining this structure required 1266 total CPU-hours, Amazon's EC2 computing infrastructure provided the necessary resources to calculate it to near-atomic resolution at a reasonable price.

To further test the performance of Amazon instances, we carried out 3D classification and refinement on a variety of STARcluster configurations using Relion. As before, we ran our tests on clusters of r3.8xlarge high-memory instances (256 GiB RAM and 16 CPUs per instance). Comparing performance across cluster sizes showed that 256 CPUs had the fastest overall time and the highest speedup relative to a single CPU for both 3D classification and refinement (Figure 4A,B). However, cluster sizes of 128 and 64 CPUs were the most cost effective for 3D classification and refinement, respectively, as these were the cluster configurations where the speedup per dollar reached a maximum (Figure 4C). Importantly, the average time required to boot up these STARclusters was ≤ 10 min for all cluster sizes (Figure 4D) and, once booted up, the clusters do not have any associated job wait times. Therefore, these tests showed that Amazon's EC2 infrastructure was amenable to the analysis of single particle cryo-EM data using Relion over a range of STARcluster sizes.

Figure 4

Download asset Open asset

Figure 4—source data 1

Performance analysis statistics for Relion 3D classification and 3D refinement on STARcluster configurations.

https://doi.org/10.7554/eLife.06664.009

Download elife-06664-fig4-data1-v2.xlsx

From our analysis of the 80S yeast ribosome, we extrapolated the processing times and combined them with previously published 3D refinement times to estimate typical costs on Amazon's EC2. First, we estimated the cost for 3D refinement in Relion for previously published structures (Supplementary file 2A)—these calculated costs ranged from $12.65 to $379.03 per structure, depending on the spot instance price and required CPU-hours. We then combined these data with conservative estimates for particle picking, CTF estimation, particle extraction, 2D and 3D classification to predict the overall cost of structure determination on Amazon's EC2 (Supplementary file 2B). From these considerations, we estimated that published structures could be determined using Amazon's EC2 environment at costs of $50–$1500 per structure (Supplementary file 2B).

1. 1. Cianfrocco MA
   2. Leschziner AE

   (2014) Data from: Single particle cryo-electron microscopy image processing in the cloud: High performance at low cost

   Available at Dryad Digital Repository under a CC0 Public Domain Dedication.

   http://dx.doi.org/10.5061/dryad.9mb54

1. 1. Bai XC
   2. Fernandez IS
   3. McMullan G
   4. Scheres SH

   (2013) S.cereviseae 80S ribosome direct electron detetector dataset

   Publicly available at Electron Microscopy Pilot Image Archive.

   http://www.ebi.ac.uk/pdbe/emdb/empiar/entry/10002/

1. 1. Amdahl GM

   (1967)

   Proceedings of the April 18-20, 1967, spring joint computer conference

   483–485, Proceedings of the April 18-20, 1967, spring joint computer conference, Atlantic City, New Jersey, ACM.

   • Google Scholar
2. 1. AWS

   (2013) Bristol-Myers Squibb on AWS

   Date Accessed: April 20, 2015. http://aws.amazon.com/solutions/case-studies/bristol-myers-squibb/.

   http://aws.amazon.com/solutions/case-studies/bristol-myers-squibb/

   • Google Scholar
3. 1. AWS

   (2014) AWS case study: Novartis

   Date Accessed: April 20, 2015. http://aws.amazon.com/solutions/case-studies/novartis/.

   http://aws.amazon.com/solutions/case-studies/novartis/

   • Google Scholar
4. 1. AWS

   (2014) AWS case study: Pfizer

   Date Accessed: April 20, 2015. http://aws.amazon.com/solutions/case-studies/pfizer/.

   http://aws.amazon.com/solutions/case-studies/pfizer/

   • Google Scholar
5. 1. Bai XC
   2. Fernandez IS
   3. McMullan G
   4. Scheres SH

   (2013) Ribosome structures to near-atomic resolution from thirty thousand cryo-EM particles

   eLife 2:e00461.

   https://doi.org/10.7554/eLife.00461

   • Google Scholar
6. 1. Bai XC
   2. McMullan G
   3. Scheres SH

   (2015) How cryo-EM is revolutionizing structural biology

   Trends in Biochemical Sciences 40:49–57.

   https://doi.org/10.1016/j.tibs.2014.10.005

   • Google Scholar
7. 1. Campbell MG
   2. Cheng A
   3. Brilot AF
   4. Moeller A
   5. Lyumkis D
   6. Veesler D
   7. Pan J
   8. Harrison SC
   9. Potter CS
   10. Carragher B
   11. Grigorieff N

   (2012) Movies of ice-embedded particles enhance resolution in electron cryo-microscopy

   Structure 20:1823–1828.

   https://doi.org/10.1016/j.str.2012.08.026

   • Google Scholar
8. 1. Cianfrocco MA
   2. Leschziner AE

   (2015)

   Data from: single particle cryo-electron microscopy image processing in the cloud: high performance at low cost

   Dryad Data Repository, 10.5061/dryad.9mb54.

   • Google Scholar
9. 1. Frank J
   2. Radermacher M
   3. Penczek P
   4. Zhu J
   5. Li Y
   6. Ladjadj M
   7. Leith A

   (1996) SPIDER and WEB: processing and visualization of images in 3D electron microscopy and related fields

   Journal of Structural Biology 116:190–199.

   https://doi.org/10.1006/jsbi.1996.0030

   • Google Scholar
10. 1. Grigorieff N

    (2007) FREALIGN: high-resolution refinement of single particle structures

    Journal of Structural Biology 157:117–125.

    https://doi.org/10.1016/j.jsb.2006.05.004

    • Google Scholar
11. 1. Hohn M
    2. Tang G
    3. Goodyear G
    4. Baldwin PR
    5. Huang Z
    6. Penczek PA
    7. Yang C
    8. Glaeser RM
    9. Adams PD
    10. Ludtke SJ

    (2007) SPARX, a new environment for Cryo-EM image processing

    Journal of Structural Biology 157:47–55.

    https://doi.org/10.1016/j.jsb.2006.07.003

    • Google Scholar
12. 1. Hu YS
    2. Nan X
    3. Sengupta P
    4. Lippincott-Schwartz J
    5. Cang H

    (2013) Accelerating 3B single-molecule super-resolution microscopy with cloud computing

    Nature Methods 10:96–97.

    https://doi.org/10.1038/nmeth.2335

    • Google Scholar
13. 1. Ivica C
    2. Riley JT
    3. Shubert C

    (2009)

    Paper presented at: information technology interfaces, 2009 ITI '09 proceedings of the ITI 2009 31st international conference on

    Paper presented at: information technology interfaces, 2009 ITI '09 proceedings of the ITI 2009 31st international conference on.

    • Google Scholar
14. 1. Krampis K
    2. Booth T
    3. Chapman B
    4. Tiwari B
    5. Bicak M
    6. Field D
    7. Nelson KE

    (2012) Cloud BioLinux: pre-configured and on-demand bioinformatics computing for the genomics community

    BMC Bioinformatics 13:42.

    https://doi.org/10.1186/1471-2105-13-42

    • Google Scholar
15. 1. Kuhlbrandt W

    (2014) Cryo-EM enters a new era

    eLife 3:e03678.

    https://doi.org/10.7554/eLife.03678

    • Google Scholar
16. 1. Li X
    2. Mooney P
    3. Zheng S
    4. Booth CR
    5. Braunfeld MB
    6. Gubbens S
    7. Agard DA
    8. Cheng Y

    (2013) Electron counting and beam-induced motion correction enable near-atomic-resolution single-particle cryo-EM

    Nature Methods 10:584–590.

    https://doi.org/10.1038/nmeth.2472

    • Google Scholar
17. 1. Lu P
    2. Bai XC
    3. Ma D
    4. Xie T
    5. Yan C
    6. Sun L
    7. Yang G
    8. Zhao Y
    9. Zhou R
    10. Scheres SH
    11. Shi Y

    (2014) Three-dimensional structure of human gamma-secretase

    Nature 512:166–170.

    https://doi.org/10.1038/nature13567

    • Google Scholar
18. 1. Ludtke SJ
    2. Baldwin PR
    3. Chiu W

    (1999) EMAN: semiautomated software for high-resolution single-particle reconstructions

    Journal of Structural Biology 128:82–97.

    https://doi.org/10.1006/jsbi.1999.4174

    • Google Scholar
19. 1. McMullan G
    2. Faruqi AR
    3. Henderson R
    4. Guerrini N
    5. Turchetta R
    6. Jacobs A
    7. van Hoften G

    (2009) Experimental observation of the improvement in MTF from backthinning a CMOS direct electron detector

    Ultramicroscopy 109:1144–1147.

    https://doi.org/10.1016/j.ultramic.2009.05.005

    • Google Scholar
20. 1. Mindell JA
    2. Grigorieff N

    (2003) Accurate determination of local defocus and specimen tilt in electron microscopy

    Journal of Structural Biology 142:334–347.

    https://doi.org/10.1016/S1047-8477(03)00069-8

    • Google Scholar
21. 1. Mohammed Y
    2. Mostovenko E
    3. Henneman AA
    4. Marissen RJ
    5. Deelder AM
    6. Palmblad M

    (2012) Cloud parallel processing of tandem mass spectrometry based proteomics data

    Journal of Proteome Research 11:5101–5108.

    https://doi.org/10.1021/pr300561q

    • Google Scholar
22. 1. Pettersen EF
    2. Goddard TD
    3. Huang CC
    4. Couch GS
    5. Greenblatt DM
    6. Meng EC
    7. Ferrin TE

    (2004) UCSF Chimera–a visualization system for exploratory research and analysis

    Journal of Computational Chemistry 25:1605–1612.

    https://doi.org/10.1002/jcc.20084

    • Google Scholar
23. 1. Scheres SH

    (2012) RELION: implementation of a Bayesian approach to cryo-EM structure determination

    Journal of Structural Biology 180:519–530.

    https://doi.org/10.1016/j.jsb.2012.09.006

    • Google Scholar
24. 1. Scheres SH

    (2014) Beam-induced motion correction for sub-megadalton cryo-EM particles

    eLife 3:e03665.

    https://doi.org/10.7554/eLife.03665

    • Google Scholar
25. 1. Sorzano CO
    2. Marabini R
    3. Velazquez-Muriel J
    4. Bilbao-Castro JR
    5. Scheres SH
    6. Carazo JM
    7. Pascual-Montano A

    (2004) XMIPP: a new generation of an open-source image processing package for electron microscopy

    Journal of Structural Biology 148:194–204.

    https://doi.org/10.1016/j.jsb.2004.06.006

    • Google Scholar
26. 1. Tang G
    2. Peng L
    3. Baldwin PR
    4. Mann DS
    5. Jiang W
    6. Rees I
    7. Ludtke SJ

    (2007) EMAN2: an extensible image processing suite for electron microscopy

    Journal of Structural Biology 157:38–46.

    https://doi.org/10.1016/j.jsb.2006.05.009

    • Google Scholar
27. 1. Trudgian DC
    2. Mirzaei H

    (2012) Cloud CPFP: a shotgun proteomics data analysis pipeline using cloud and high performance computing

    Journal of Proteome Research 11:6282–6290.

    https://doi.org/10.1021/pr300694b

    • Google Scholar
28. 1. Yazar S
    2. Gooden GE
    3. Mackey DA
    4. Hewitt AW

    (2014) Benchmarking undedicated cloud computing providers for analysis of genomic datasets

    PLOS ONE 9:e108490.

    https://doi.org/10.1371/journal.pone.0108490

    • Google Scholar

Author details

1. Michael A Cianfrocco

   1. Department of Molecular and Cellular Biology, Harvard University, Cambridge, United States
   2. Department of Cell Biology, Harvard Medical School, Boston, United States

   Contribution

   MAC, Conception and design, Acquisition of data, Analysis and interpretation of data, Drafting or revising the article

   For correspondence

   mcianfrocco@fas.harvard.edu

   Competing interests

   The authors declare that no competing interests exist.

2. Andres E Leschziner

   Department of Molecular and Cellular Biology, Harvard University, Cambridge, United States

   Contribution

   AEL, Analysis and interpretation of data, Drafting or revising the article

   Competing interests

   The authors declare that no competing interests exist.

• 6,264

  views

• 934

  downloads

• 33

  citations

Views, downloads and citations are aggregated across all versions of this paper published by eLife.