Abstract
There is considerable potential for Xray free electron lasers (XFELs) to enable determination of macromolecular crystal structures that are difficult to solve using current synchrotron sources. Prior XFEL studies often involved the collection of thousands to millions of diffraction images, in part due to limitations of data processing methods. We implemented a data processing system based on classical postrefinement techniques, adapted to specific properties of XFEL diffraction data. When applied to XFEL data from three different proteins collected using various sample delivery systems and XFEL beam parameters, our method improved the quality of the diffraction data as well as the resulting refined atomic models and electron density maps. Moreover, the number of observations for a reflection necessary to assemble an accurate data set could be reduced to a few observations. These developments will help expand the applicability of XFEL crystallography to challenging biological systems, including cases where sample is limited.
https://doi.org/10.7554/eLife.05421.001eLife digest
Large biological molecules (or macromolecules) have intricate threedimensional structures. Xray crystallography is a technique that is commonly used to determine these structures and involves directing a beam of Xrays at a crystal that was grown from the macromolecule of interest. The macromolecules in the crystal scatter the Xrays to produce a diffraction pattern, and the crystal is rotated to provide further diffraction images. It is then possible to work backwards from these images and elucidate the structure of the macromolecule in three dimensions.
Xray beams are powerful enough to damage crystals, and scientists are developing new approaches to overcome this problem. One recent development uses ‘Xray free electron lasers’ to circumvent the damage caused to crystals. However, early applications of this approach required many crystals and thousands to millions of diffraction patterns to be collected—largely because methods to process the diffraction data were far from optimal.
Uervirojnangkoorn et al. have now developed a new dataprocessing procedure that is specifically designed for diffraction data obtained using Xray free electron lasers. This method was applied to diffraction data collected from crystals of three different macromolecules (which in this case were three different proteins). For all three, the new method required many fewer diffraction images to determine the structure, and in one case revealed more details about the structure than the existing methods.
This new method is now expected to allow a wider range of macromolecules to be studied using crystallography with Xray free electron lasers, including cases where very few crystals are available.
https://doi.org/10.7554/eLife.05421.002Introduction
Radiation damage often limits the resolution and accuracy of macromolecular crystal structures (Garman, 2010; Zeldin et al., 2013). Femtosecond Xray free electron laser (XFEL) pulses enable the possibility of visualizing molecular structures before the onset of radiation damage, and allow the dynamics of chemical processes to be captured (Solem, 1986; Neutze et al., 2000). Thus, from the first XFEL operation at the Linac Coherent Light Source (LCLS) in 2009, there has been considerable effort dedicated to the development of methods to utilize this rapid succession of bright pulses for macromolecular crystallography, with the aim of obtaining damagefree, chemically accurate structures. Most of the structures reported from XFELs to date use a liquid jet to inject small crystals into the beam (DePonte et al., 2008; Sierra et al., 2012; Weierstall et al., 2014), but diffraction data have also been measured from crystals placed in the beam with a standard goniometer setup (Cohen et al., 2014; Hirata et al., 2014). In both cases, the illuminated volume diffracts before suffering damage by a single XFEL pulse. Because the crystal is effectively stationary during the 10–50 fs exposure, ‘still’ diffraction patterns are obtained, in contrast to standard diffraction data collection where the sample is rotated through a small angle during the exposure.
Extracting accurate Bragg peak intensities from XFEL diffraction data is a substantial challenge. An XFEL data set comprises ‘still’ diffraction patterns generally containing only partially recorded reflections, typically from randomly oriented crystals. The full intensity then has to be estimated from the observed partial intensity observations. Most XFEL diffraction data processing approaches reported to date have approximated the full intensity by the socalled “Monte Carlo” method, in which thousands of partial intensity observations of a given reflection are summed and normalized by the number of observations, which assumes that these observations sample the full 3D Bragg volume. Because a single diffraction image—in which each observed reflection samples only part of each reflection intensity–contains much less information than a small continuous wedge of diffraction data (as used in conventional crystallography), this method requires a very large number of crystals to ensure convergence of the averaged partial reflection intensities to the full intensity value (Kirian et al., 2010). Moreover, shottoshot differences in pulse intensity and energy spectrum that arise from the selfamplified stimulated emission (SASE) process (Kondratenko and Saldin, 1979; Bonifacio et al., 1984), along with differences in illuminated crystal volume, mosaicity, and unitcell dimensions, contribute to intensity variation of the equivalent reflections observed on different images. These differences are assumed to be averaged out by the Monte Carlo method (Hattne et al., 2014). Thus, accurate determination of these parameters for each diffraction image should, in principle, provide more accurate integrated intensities, and converge with fewer measurements. Furthermore, it is desirable to assemble a data set from as few diffraction images as possible, since the potential of XFELs has been limited by the very large amounts of sample required for the Monte Carlo method, compounded by severe limitations in the availability of beamtime.
In the 1970's, the Harrison and Rossmann groups developed ‘postrefinement’ methods (Rossmann et al., 1979; Winkler et al., 1979), in which the parameters that determine the location and volume of the Bragg peaks are ‘post’refined against a reference set of fully recorded reflections following initial indexing and integration of rotation data. Accurate estimation of these parameters, including the unitcell lengths and angles, crystal orientation, mosaic spread, and beam divergence enables accurate calculation of what fraction of the reflection intensity was recorded on the image, i.e., its ‘partiality’, which is then used to correct the measurement to its fully recorded equivalent. Applied to virus crystals, for which only a few images can typically be collected before radiation damage becomes significant, postrefinement made it possible to obtain highquality diffraction data sets collected from many crystals (Rossmann et al., 1979; Winkler et al., 1979).
The implementation of postrefinement for XFEL diffraction data poses unique challenges. Firstly, since XFEL diffraction data generally do not contain fully recorded reflections, the initial scaling and merging of images is difficult. Secondly, since the XFEL diffraction images are stills rather than rotation data, different approaches are required for the correction of measurements to determine the full spot equivalent. Other schemes for implementing postrefinement of XFEL diffraction data have been described previously, but thus far they have been only applied to simulated XFEL data (White, 2014), and to pseudostill images collected using monochromatic synchrotron radiation (Kabsch, 2014).
We have developed a new postrefinement procedure specifically designed for diffraction data from still images collected from crystals in random orientations. We implemented our method in a new computer program, prime (postrefinement and merging), that postrefines the parameters needed for calculating the partiality of reflections recorded on each still image. We describe here our method and demonstrate that postrefinement greatly improves the quality of the diffraction data from XFEL diffraction experiments with crystals of three different proteins. We show that our postrefinement procedure allows complete data sets to be extracted from a much smaller number of diffraction images than that necessary when using the Monte Carlo method. Thus, this development will help make XFEL crystallography accessible to many challenging problems in biology, including those for which sample quantity is a major limiting factor.
Results
Notation
Units are arbitrary unless specified in parenthesis.
I_{obs}, observed intensity.
I_{ref}, reference intensity.
w, weighting term (inverse variance of the observed intensity).
G, function of linear scale (G_{0}) and resolutiondependent (B) factors that scales the different diffraction images to the reference set.
Eoc, Ewaldoffset correction function.
r_{h}, offset reciprocalspace distance from the center of the reflection to the Ewald sphere (Å^{−1}).
r_{p}, radius of the disc of intersection between the reciprocal lattice point and the Ewald sphere (Å^{−1}).
r_{s}, radius of the reciprocal lattice point (Å^{−1}).
θ_{x}, θ_{y}, θ_{z}, crystal rotation angles (see Figure 1A; °).
γ_{0}, parameter for Equation 3 (Å^{−1}).
γ_{e}, energy spread and unitcell variation (see Equation 3; Å^{−1}).
γ_{x} and γ_{y}, beam divergence (see Equation 4; Å^{−1}).
{uc}, unitcell dimensions (a, b, c (Å), α, β, and γ (°)).
V_{c}, reciprocallattice volume correction function (Å^{−3}).
x_{obs} and x_{calc}, observed and predicted spot positions on the detector (mm).
x, position of the reciprocal lattice point (Å^{−1}).
S, displacement vector from the center of the Ewald sphere to x (Å^{−1}).
S_{0}, incident beam vector with length 1/wavelength (Å^{−1}).
O, orthogonalization matrix.
R, rotation matrix.
f_{L} and f_{LN}, Lorentzian function and its normalized counterpart.
Γ, full width at half maximum (FWHM) of the Lorentzian function.
Postrefinement overview
Partiality can be modeled by describing the full reflection as a sphere (Figure 1A). In a still diffraction pattern, assuming a monochromatic photon source, the observed intensity I_{obs,h} for Miller index h is a thin slice through a threedimensional reflection. To calculate partiality, we assume that the measurement is an areal (i.e., infinitely thin) sample of the volume (Figure 1B). The maximum partial intensity that can be recorded for a given reflection will occur when its center lies exactly on the Ewald sphere. By definition, the center of the reflection will be offset from the Ewald sphere by r_{h}, and the corresponding disc will have a radius r_{p}. The offset r_{h} is determined by various experimental parameters, including the crystal orientation, unitcell dimensions, and Xray photon energy. The offset distance is used to calculate the Ewald offset correction, Eoc_{area,} defined as the ratio between the areas defined by r_{p} and r_{s} (implemented as a smoothed correction function Eoc_{h} as defined in ‘Materials and methods’). The Ewaldoffset corrected intensity is then converted to the full intensity in 3D by applying a volume correction factor, V_{c}.
We define the target T_{pr} for the postrefinement of a partiality and scaling model by:
which minimizes the difference between the observed reflections I_{obs} and a scaled and Ewaldoffset corrected full intensity ‘reference set’ I_{ref} using a leastsquares method. The sum is over all observed reflections with Miller indices h.
In alternate refinement cycles, we also minimize the deviations between predicted (x_{calc}) and observed (x_{obs}) spot positions on the detector using a subset of strong spots as has been suggested previously (Hattne et al., 2014; Kabsch, 2014):
Sets of parameters associated with each diffraction image, i.e., G_{0}, B, θ_{x}, θ_{y}, γ_{0}, γ_{e}, γ_{x}, γ_{y} and the unitcell constants, are iteratively refined in a series of ‘microcycles’ against the current reference set (Figure 2).
Procedures for generating the initial reference set I_{ref}(initial) are described below. After convergence of the microcycles, scaled full intensities are calculated from the observed partial intensities I_{obs} by multiplication of the inverse of the Ewaldoffset correction and the scale factor G, along with the volume correction factor V_{c}. These scaled full reflections are then merged for each unique Miller index, taking into account estimated errors of the observed intensities, σ(I_{obs}), and propagation of error estimates for the refined parameters. This merged and scaled set of full reflections is then used as the new reference set in the next round of postrefinement using the target functions (Equations 1 and 2, for details see ‘Materials and methods’). These ‘macrocycles’ are repeated until convergence is achieved, after which the merged and scaled set of full intensities is provided to the user.
The prime program controls postrefinement of specified parameters in a particular microcycle (Figure 2). One can refine all parameters together, or selectively refine groups of parameters iteratively, starting from (1) a linear scale factor and a Bfactor, (2) crystal orientations, (3) crystal mosaicity, beam divergence, and spectral dispersion, and (4) unitcell dimensions. Spacegroupspecific constraints are used to limit the number of free parameters for the unitcell refinement. A particular microcycle is completed when the target functions converge or when a specified number of iterations is reached; the program then generates the new reference intensity set to replace the current reference set for the next macrocycle. Finally, the program exits and outputs the latest merged reflection set either when the macrocycles converge or when a userspecified maximum number of cycles has been reached.
Preparation of the observed intensities
The starting point for our postrefinement method is a set of indexed and integrated partial intensities, along with their estimated errors, obtained from still images. For this study, diffraction data and their estimated errors were obtained from the cctbx.xfel package (Sauter et al., 2013; Hattne et al., 2014), although in principle integrated diffraction data from any other program can be used. Observed intensities on the diffraction image were classified as ‘spots’ by the program Spotfinder (Zhang et al., 2006), which identifies Bragg spots by considering connected pixels with area and signal height greater than userdefined thresholds. By trial and error, we accepted reflections larger than 25 pixels with individualpixel intensity more than 5 σ over background for myoglobin and hydrogenase (collected on a Rayonix MX325HE detector with pixel size of 0.08 mm and beam diameter [FWHM] of 50 μm). For thermolysin (collected on a CornellSLAC pixel array detector with pixel size of 0.1 mm and beam size of 2.25 μm^{2}), where reflections are generally smaller, these values were 1 pixel and 5 σ. A full list of parameters is available on the cctbx.xfel wiki (http://cci.lbl.gov/xfel). Separate resolution cutoffs for each image were applied by cctbx.xfel, at resolutions where the average I/σ(I) fell below 0.5 (Hattne et al., 2014).
Prior to postrefinement, the experimentally observed partial intensities need to be corrected by a polarization factor. The primary XFEL beam at LCLS is strongly polarized in the horizontal plane, and we calculate the correction factor as a function of the Bragg angle (θ) and the angle $\varphi $ between the sample reflection and the laboratory horizontal planes (Kahn et al., 1982; see ‘Materials and methods’). For a stationary crystal and a monochromatic beam, a Lorentz factor correction is not applicable; the spectral dispersion of the SASE beam (δE/E ∼ 3 × 10^{−3} for the data sets studied here) is accounted for by the γ_{e} term (see ‘Materials and methods’).
Generating the initial reference set and initial parameters
An essential step to initiate postrefinement is the generation of the initial reference set I_{ref} (initial). This reference set has to be estimated from the available unmerged and unscaled partial reflection intensities after application of the polarization correction. For the results presented here, linear scale factors for each diffraction image were chosen to make the mean intensities of each diffraction image equal. Since this procedure can be affected by outliers in the observed intensities, we select a subset of reflections with userspecified resolution range and signaltonoise ratio (I/σ(I)) cutoffs. From this selection, we calculate the mean intensity on each diffraction image and then scale each image to make the mean intensity of all images equal. We correct the scaled observed reflections to their Ewaldoffset corrected equivalents using the starting parameters, and then merge the observations, taking into account the experimental σ(I_{obs}), to generate the initial reference set.
The initial values for crystal orientation, unitcell dimensions, crystaltodetector distance, and spot position on the detector were obtained from the refinement of these parameters by cctbx.xfel. The photon energy was that provided by the LCLS endstation system and is not refined. Initial values for the parameters of the reflection width model are described in the ‘Materials and methods’ section.
Definition and comparison of data processing schemes
In order to separately assess the effects of scaling, the Ewald offset correction (Equation 1), and postrefinement, we refer to three alternative schemes for processing the diffraction data sets: (1) ‘Averaged merged’, in which intensities were generated by averaging all observed partial intensities from equivalent reflections without Ewaldoffset correction and scaling; (2) ‘Meanintensity partiality corrected’, in which intensities were generated by scaling the reflections to the mean intensity and also applying the Ewaldoffset correction determined from the initial parameters obtained from the indexing and integration program, followed by merging; and (3) ‘Postrefined’, in which intensities were from the final set of scaled and merged full reflections after the convergence of postrefinement. We note that although the ‘averaged merged’ process is similar to the original Monte Carlo method (Kirian et al., 2010), the integrated, unmerged partial intensities used in our tests were obtained from the program cctbx.xfel (Hattne et al., 2014), which also refines various parameters on an imagebyimage basis (Sauter et al., 2014).
Quality assessment of postrefined data
We tested our postrefinement method on experimental XFEL diffraction data sets from three different crystallized proteins of known structure: myoglobin, hydrogenase, and thermolysin (Table 1). For quality assessment, we performed molecular replacement (MR) with Phaser (McCoy et al., 2007) using models with selected parts of the known structures omitted, followed by atomic model refinement with phenix.refine (Afonine et al., 2012), and inspection of (mF_{o}DF_{c}) omit maps. We further used three different metrics: CC_{1/2}, and the crystallographic R_{work} and R_{free} of the fully refined atomic model. We then compared changes in the three quality metrics between merged XFEL diffraction data sets after scaling, partiality correction, and postrefinement. We also investigated the effect of reducing the number of images used by randomly selecting a subset from the full set of diffraction images and repeating the entire postrefinement, merging, MR and refinement processes using this subset.
Diffraction data for both myoglobin and hydrogenase were collected from frozen crystals mounted on a standard goniometer setup (Cohen et al., 2014), whereas the thermolysin data were collected using an electrospun liquid jet to inject nanocystals into a vacuum chamber (Sierra et al., 2012; Bogan, 2013). The completeness of each data set was better than 90% at the limiting resolution used in our tests (Tables 2, 3, 4). Each diffraction data set involved a different number of images due the differing diffraction quality of the crystals.
Myoglobin
For myoglobin, we used both an XFEL diffraction data set consisting of 757 diffraction images (Table 1) collected by the SSRLSMB group using a goniometermounted fixedtarget grid (Cohen et al., 2014), and a randomly selected subset of 100 diffraction images. The diffraction images were from crystals in random orientations, with a single still image collected from each crystal.
Convergence of postrefinement
Convergence properties for our postrefinement method for myoglobin are shown in Figures 3 and 4, and a representative example of the first macrocycle for a selected diffraction image is provided in Figure 3. The order of the three microcycle postrefinement iterations was: scale factors (SF—Equation 17), crystal orientation (CO—Equation 5), reciprocal spot size (RR—Equations 3 and 4), and unitcell dimensions (UC—Equation 5). The partiality model target function T_{pr} (Equation 1) markedly decreased in the first microcycle and fully converged in the last microcycle. The spot position residual T_{xy} (Equation 2), also decreased both during postrefinement of the crystal orientation and the unitcell parameters.
Figure 5 shows the results for five macrocycles for postrefinement using the subset of 100 randomly selected still images of the myoglobin XFEL diffraction data set. The partiality model target function T_{pr} (Equation 1) continually decreased in the first three macrocycles. The average spot position residual T_{xy} (Equation 2) decreased in the first cycle and converged in the next cycle. The quality metric CC_{1/2} also converged within the first three macrocycles.
Inaccuracies in the starting parameters obtained from indexing and integration of still images may limit the radius of convergence and the accuracy of the postrefined parameters. The sources of such errors will be the subject of future improvement in indexing and integration in cctbx.xfel. Nonetheless, for the systems studied here the postrefinements converged within 3–5 cycles.
Improvements due to postrefinement
For the myoglobin diffraction data set using all 757 images (Table 2, Figure 6A,B), the CC_{1/2} value improved after postrefinement, especially for those reflections in the lowresolution shells (Figure 5C; Table 2).
Omit maps were used to compare the quality of the diffraction data processed with the different methods. Specifically, we omitted the heme group from the molecular replacement search model (PDB ID: 3U3E) and in subsequent atomic model refinement, and calculated mF_{o}DF_{c} difference maps (Figure 6). The realspace correlation coefficient of the heme group to the difference maps calculated from the postrefined diffraction data sets is higher than that calculated from the corresponding averaged merged diffraction data sets using the same set of diffraction images (Figure 6A).
After initial model refinement with the heme group omitted, we included the heme group and welldefined water molecules and completed the atomic model refinement. The postrefined diffraction data set produced the best R_{free} and R_{work} values, followed by the meanscaled partiality corrected, with the averaged merged diffraction data sets yielding the poorest refinement statistics.
Overall, comparison of the CC_{1/2} (Figure 5), omit map quality, and R values (Figure 6B) shows that postrefinement substantially improves scaling and correction of the diffraction data with respect to the meanscaled partialitycorrected diffraction data set. Thus, postrefinement against the iteratively improved reference set is superior to methods that only consider each diffraction image individually, even when the reflections are scaled and corrected for partiality.
100 diffraction images are sufficient for myoglobin structure refinement
Given the significant improvements obtained by postrefining all available images, we tested whether accurate diffraction data and refined atomic models could be obtained using fewer diffraction images by postrefining the randomly selected subset of 100 myoglobin diffraction images. Since this subset is only 80% complete, the CC_{1/2} is poorer than that of the full diffraction data set consisting of 757 images, but it is nonetheless greatly improved relative to the corresponding nonpostrefined diffraction data set (Figure 5). Moreover, the realspace correlation coefficient of the heme group with the difference map obtained with the postrefined 100 diffraction images is better than that calculated from the averaged merged diffraction data set using all the 757 diffraction images (Figure 6A), despite the higher completeness and CC_{1/2} value of the latter data set (Figure 5C). Thus, postrefinement both improves diffraction data quality for a given set of images and reduces the number of diffraction images required for structure determination and refinement from serial diffraction data.
Comparison with a synchrotron data set
We also compared the postrefined XFEL difference map (using all 757 diffraction images) with that calculated from an isomorphous synchrotron data set and model (PDB ID: 1JW8, excluding reflections past 1.35 Å resolution to make the resolution of the diffraction data sets equivalent). The omit maps and realspace correlation coefficients for the heme group were of comparable quality (Figure 7).
Hydrogenase
XFEL diffraction data for Clostridium pasteurianum hydrogenase were measured from eight crystals by the Peters (University of Montana) and SSRLSMB groups using a goniometermounted fixedtarget grid (Cohen et al., 2014). This experiment generated 177 diffraction images that could be merged to a completeness of 91%, with more than half of the diffraction images containing reflections to 1.6 Å (each diffraction image typically has approximately 3000 spots). We also used a randomly selected subset of 100 diffraction images to assess the effect of postrefinement on a smaller number of images.
The CC_{1/2} value improved significantly with postrefinement (Table 3). For quality assessment, the FeS cluster was omitted from both the molecular replacement search model (PDB ID 3C8Y) and subsequent atomic model refinement. The omit map densities for the postrefined diffraction data sets using the complete set of 177 diffraction images and the randomly selected subset of 100 diffraction images (83% complete) clearly show the entire FeS cluster whereas the densities using the averaged merged data sets are much poorer (Figure 8A). Upon atomic model refinement with the FeS clusters and water molecules included, the R and R_{free} values for both postrefined data sets were significantly better than the averaged merged case (Figure 8B).
Thermolysin
For thermolysin, we tested the entire deposited XFEL diffraction data set consisting of 12,692 diffraction images (Table 1) (Hattne et al., 2014; the diffraction data are publicly archived in the Coherent Xray Imaging Data Bank, accession ID 23, http://cxidb.org), as well as a randomly selected subset of 2000 diffraction images. In this experiment, the crystaltodetector distance gave a maximum resolution of 2.6 Å at the edge and 2.1 Å at the corners of the detector. Thus, a large number of diffraction images were required to achieve reasonable completeness of the merged data set for reflections in the 2.1—2.6 Å resolution range.
As in the other two cases, postrefinement significantly improved the CC_{1/2} value (Table 4). For quality assessment, zinc and calcium ions were omitted from the thermolysin molecular replacement search model (PDB ID: 2TLI) and subsequent atomic model refinement. Postrefinement improved the peak heights of both the zinc and calcium ions (Table 4).
Anomalous difference Fourier peak heights
The thermolysin diffraction data were collected at a photon energy just above the absorption edge of zinc, so we compared the anomalous signals with and without postrefinement. We used the same four diffraction data sets (i.e., averagedmerged, postrefined, with 2000 and 12,692 diffraction images, respectively), but processed them keeping Friedel mates separate. We refined the atomic model of thermolysin lacking zinc and calcium ions, and calculated anomalous difference Fourier maps (Figure 9). We observed two anomalous difference peaks near the active site above 3 σ using the postrefined data sets. In contrast, the second, smaller peak is not visible in the anomalous difference map using the ‘averagedmerged’ data set with 2000 images, and it had not been clearly visible in the previous data analysis of the thermolysin XFEL data set (PDB ID: 4OW3; Hattne et al., 2014). A previous thermolysin structure (PDB ID: 1LND; Holland et al., 1995) reported two zinc sites in the active site that correspond to the two anomalousdifference peaks observed with our postrefined data set. Although the crystallization condition used in our case did not have the high concentration (10 mM) of zinc used in the Holland et al. study, the second anomalous difference peak suggests the presence of this second zinc site.
Difference map reveals a bound dipeptide
When the molecular replacement model of thermolysin was refined against the postrefined data, we observed a wellconnected electron density feature in the mF_{o}DF_{c} map near the active site. In contrast, in the deposited model refined against the original XFEL data (Hattne et al., 2014; PDB ID: 4OW3), weak density features in this region were interpreted as water molecules. We found several examples of deposited thermolysin structures that have a dipeptide in this region (e.g., PDB entry 2WHZ with Tyr–Ile, PDB entry 2WI0 with Leu–Trp, and PDB entry 8TLN with Val–Lys). We interpreted the shape of the difference density as a Leu–Lys dipeptide, superimposed its structure and calculated realspace correlation coefficients. The dipeptide had a higher realspace correlation coefficient (CC) with the maps calculated from the postrefined diffraction data than those calculated from the averaged merged diffraction data. The electron density for both postrefined diffraction data sets is also better connected than that of the averaged merged diffraction data set (Figure 10A). The R_{work} and R_{free} values of the refined complete model using the postrefined diffraction data are lower than those using the averaged merged data throughout the entire resolution range (Figure 10B).
Effect of completeness
The completeness of the merged data sets has a direct impact on the overall quality of the diffraction data set (CC_{1/2}), quality of the electron density maps and the refined structures (Tables 2–4, and Figure 6). When completeness is high, adding more images to increase the multiplicity of observations has only a modest impact on the quality of the final refined structures using the postrefined diffraction data. For example, when subsets ranging from 2000 to 12,000 thermolysin diffraction images (all subsets 100% complete at 2.6 Å) were postrefined the peak height in the omit map for the larger of the two anomalous sites (Figure 11C), the CC_{1/2} values, and the R values of the refined structures did not improve significantly when more than 8000 images were used.
Discussion
Diffraction data collection using conventional xray sources typically employs the rotation method, in which a single crystal is rotated through a contiguous set of angles, and the diffraction patterns are recorded on a 2D detector. If a full data set can be collected from a single crystal without a prohibitive level of radiation damage, diffraction data processing is a wellestablished and reliable process. In contrast, processing of XFEL diffraction data, which are collected from crystals in random orientations as ‘still’ diffraction images, requires new methods and implementations such as those described here. Improved data collection and processing methods, particularly those that can significantly reduce the amount of sample needed to assemble a complete and accurate diffraction data set, are important for making XFELs useful for certain challenging investigations in structural biology.
We developed a postrefinement method for still diffraction images, such as those obtained at XFELs, and implemented it in new computer program, prime, that applies a leastsquares minimization method to refine parameters as defined in our partiality model. Other postrefinement methods for XFEL diffraction data have been described recently (Kabsch, 2014; White, 2014), but our implementation differs from these reports. Kabsch uses a partiality model in which an Ewald offset correction is defined as a Gaussian function of angular distance from the Ewald sphere. White used the intersecting volume between the reflection and the limitingenergy Ewald spheres defined by the energy spectrum for the partiality calculation, and calculates the initial reference data set by averaging all observations without scaling. Neither report describes an application to experimental XFEL diffraction data, so we cannot compare these methods to the results presented here.
We have demonstrated here that our implementation of postrefinement substantially improves the quality of the diffraction data from three different XFEL experiments. Moreover, the resulting structures can be refined to significantly lower R_{free} and R values, with electron density maps that reveal novel features more clearly, than those using nonpostrefined XFEL data sets. A key feature of our method is that the parameters that define the diffracted spot are iteratively refined against the reference set. This approach is superior to methods that only consider each diffraction image individually. Moreover, our postrefinement procedure allows accurate diffraction data sets to be extracted from a much smaller number of images (average number of observations) than that necessary without postrefinement. Thus, this development will make XFEL crystallography accessible to many challenging problems in biology for which sample quantity is a major limiting factor.
At present, it is difficult to assess the relative quality of postrefined XFEL data studied here with conventional rotation data measured at a synchrotron. The comparison of myoglobin omit maps (Figure 7) suggests that the SR data are perhaps somewhat better, but more systematic studies will be needed to understand the relative merits of the different data sets. We suspect that rotation data would be better due to the ability to directly measure full reflections (at least by summation of partials) without modeling partiality, which is still a relatively crude process (see below). However, a comparison between still data sets measured at a synchrotron and an XFEL is needed to deconvolute the effect of rotation vs other differences between these sources.
Our formulation of postrefinement employs the simplifying assumption that reflections are spherical volumes. More sophisticated models consider crystal mosaicity to have three components, each with a distinct effect on the reciprocal lattice point (Juers et al., 2007; Nave, 1998, 2014). First, the domain size (the average size of the coherently scattering mosaic blocks) produces reciprocal lattice points of constant, finite size: small domains produce largesized spots, while large domains produce small spots, as there is an inverse (Fourier) relation between spot size and domain size. Second, unitcell variation among domains produces reflections that are spheres whose radii increase with distance from the origin. In cctbx.xfel, mosaicity (modeled as isotropic parameter) and effective domain size are taken into account when predicting which reflections are in diffracting position prior to integration (Sauter et al., 2014; Sauter, 2015). Third, orientational spread among mosaic domains produces spots shaped like spherical caps. Each cap subtends a solid angle that depends on the magnitude of the spread. In addition, anisotropy in crystal mosaicity is not considered; this would require refining separate parameters along each lattice direction. Finally, the rugged energy spectrum that results from the SASE process of the XFEL is not yet considered in our current model. These issues will require future investigation.
Materials and methods
Partiality model
Request a detailed protocolThe observed intensity I_{h}(i) for observation i of Miller index h is a thin slice through a threedimensional reflection. To calculate partiality, we assume that the measurement is an infinitely thin, circular sample of a spherical volume (Figure 1B). We assume a monochromatic beam as the starting point to define the Ewald offset correction Eoc_{area}. The Eoc_{area} of any reflection centered on the Ewald sphere is defined as 1; this position corresponds to the maximum partial intensity that could be measured for the reflection. The Eoc_{area} for any other position is defined as a function of the normal distance from the Ewald sphere to the center of the reciprocal lattice point (the offset distance, r_{h}), and of the reciprocallattice radius of the spot r_{s}, which is a function of the crystal mosaicity and spectral dispersion (Figure 1B). The Eoc_{area} can be described by the ratio of the observed area (A_{p}) with a radius r_{p} to the Ewaldoffset corrected area (A_{s}) with a radius r_{s} (Figure 1B).
The SASE spectrum emitted by the XFEL is broad and varies from shottoshot (Zhu et al., 2012). To calculate the Ewald sphere, we set the wavelength to be the centroid of the SASE spectrum recorded with each shot. For XFEL data measured with a seeded beam (Amann et al., 2012), the spectrum is narrow and constant from shottoshot, and this single value can be used in this case.
In order to model spectral dispersion and the possible effects of asymmetric beam divergence, we adapt the rocking curve model described in Winkler et al. (1979). The fourparameter function used for the rocking curve is ${r}_{s}({\gamma}_{0},\hspace{0.17em}{\gamma}_{e},{\gamma}_{x},\hspace{0.17em}{\gamma}_{y})=\hspace{0.17em}{r}_{s}\left(\theta \right)+\hspace{0.17em}{r}_{s}\left(\alpha \right),$ where the first term includes the contribution by spectral dispersion and the second term models beam anisotropy. Specifically,
where γ_{0} is a parameter that is initially set to the r.m.s.d. of the Ewald offset calculated for all the reflections on a given image, γ_{e} represents the width of the energy spread and the unitcell variation (the initial value of γ_{e} is calculated from the average energy spread), and θ is the Bragg angle. The second term is provided by:
where α is the azimuthal angle going from meridional (α = 0) to equatorial (α = π/2) . The values of γ_{y} and γ_{x} are initially set to 0.
The distribution of r_{h} values for the myoglobin case with 757 images after postrefinement is shown in Figure 12. The parameters γ_{e}, γ_{y}, γ_{x}, γ_{0} are refined within a microcycle (Figure 2).
Calculating the reciprocal lattice point offset
Request a detailed protocolThe crystal orientation is described in a righthanded coordinate system with the zaxis pointing to the source of the incident beam and the yaxis vertical (Figure 1A). We define the crystal orientation by rotations in the order θ_{z}, θ_{y}, θ_{x} about these axes. For each Miller index h(i), the reciprocal lattice point vector x(i) is obtained by applying orthogonalization and rotation matrixes O and R:
where
where ${R}_{{\theta}_{i}}$ is the rotation matrix for a rotation around the ith axis, ${a}^{\ast},\hspace{0.17em}{b}^{\ast},\hspace{0.17em}{c}^{\ast},\hspace{0.17em}{\alpha}^{\ast},\hspace{0.17em}{\beta}^{\ast},\hspace{0.17em}{\gamma}^{\ast}\hspace{0.17em}$ are the reciprocal unitcell parameters, and $\mathrm{cos}\left({c}^{\ast},\hspace{0.17em}c\right)={\left(1+2\hspace{0.17em}\mathrm{cos}\hspace{0.17em}{\alpha}^{\ast}\hspace{0.17em}\mathrm{cos}\hspace{0.17em}{\beta}^{\ast}\hspace{0.17em}\mathrm{cos}\hspace{0.17em}{\gamma}^{\ast}{\mathrm{cos}}^{2}{\alpha}^{\ast}{\mathrm{cos}}^{2}{\beta}^{\ast}{\mathrm{cos}}^{2}{\gamma}^{\ast}\right)}^{1/2}/\mathrm{sin}\hspace{0.17em}{\gamma}^{\ast}$.
As shown in Figure 1A, the displacement to x(i) from the center of the Ewald sphere is given by:
where S_{0} = (0, 0, −1/λ). The offset distance is thus the difference between the length of S(i) and the Ewaldsphere radius,
The Ewaldoffset correction function Eoc
Request a detailed protocolWe introduce a smooth approximation of the area ratio Eoc_{area} (see ‘Results’) in order to circumvent the undefined first derivative when the ratio is zero. We use a Lorentzian function (f_{L}) to model the radius as function of distance from the Ewald sphere:
The function is normalized so that f_{L}(r_{h} = 0) = 1.0 when the reciprocallattice point is centered on the Ewald sphere, so that
We then use the ratio of the observed area (A_{p}) with a radius r_{p} to the Ewaldoffset corrected area (A_{s}) with a radius r_{s} (Figure 1B) that corresponds to the full width at half maximum (FWHM), Γ, in the Lorentzian function. Using the Lorentzian function to describe the falloff in radius as we move away from the Ewald sphere makes the Eoc function differentiable at r_{h} = r_{s}. For the reciprocal lattice volume being bound by a sphere of radius r_{s} centered on the reciprocal lattice point, the intersecting area of the volume is given by:
where
The Eoc is then given by the ratio of this intersecting area to the area when this reflection is centered on the Ewald sphere (A_{s}),
By setting the FWHM of Γ proportional to the radius, r_{s}, at half Eoc_{area},
we arrive at the Ewaldoffset correction function (Figure 13A)
The use of this Lorentzian approximation to derive the Eoc function vs an actual sphere function, Eoc_{area}, is illustrated in Figure 13B.
Correction to full intensity
Request a detailed protocolTo adjust the observed still intensity to its equivalent at zero offset, we apply the Ewaldoffset correction to the observed intensity,
where I_{h}(i) is the observed partial intensity i of Miller index h on image m, Eoc_{h}(i) is the Ewaldoffset correction, and G_{m} is a scale function for image m. We then convert this maximum partial intensity to a full intensity estimate by correcting for the volume of the spot, a factor of $\frac{\frac{4}{3}\pi {r}_{s}^{3}}{\pi {r}_{s}^{2}}\hspace{0.17em}=\frac{4}{3}{r}_{s}$:
where
Note that I_{full,h}(i) will be on an arbitrary scale, and appropriate scaling methods may be applied to place the data on a quasiabsolute scale prior to structure determination and refinement, as is done for conventional rotation data.
Refinement of crystal orientation, reflection width, and unitcell parameters
Request a detailed protocolWe refine image m by first minimizing the target function:
where
and the scale function G_{m} comprises a linear scale factor G_{0} and a Bfactor:
We apply a spot position restraint as a second target function in subsequent steps during a microcycle using the x, y positions determined by the spotfinding step of data processing (Hattne et al., 2014; Kabsch, 2014).
where ${x}_{h}^{obs}\left(i\right)$ and ${x}_{h}^{calc}\left(i\right)$ are the observed and calculated spot centroids, respectively.
The Levenberg–Marquardt (LM) algorithm from the scipy python library (Oliphant, 2007), which is a combination of the gradient descent and the Gauss–Newton iteration, is used to minimize the target function residuals. The refinement of the unitcell parameters (a, b, c, α, β, γ) takes crystal symmetry constraints into account to make the procedure more robust.
After these iterative refinement cycles are complete, we apply the refined parameters to the reflection intensities of each still, and then merge the same reduced Miller indices (from all stills) to obtain the zerooffset still intensities, which are used for the new reference intensity set (see next section).
Reflection selection criteria
Request a detailed protocolAt each step in a microcycle, the user can select reflections that are used for postrefinement of a parameter group using the following criteria: resolution range, signal strength (I/σ(I)), and the Ewald offset correction value. In addition to these selection criteria, deviations from the target unitcell dimensions (specified as a fraction of each dimension) can also be used in the merging step so that only diffraction patterns with acceptable unitcell dimension values are included in the merged reflection set. Each postrefinement parameter group can have its own separate set of reflection selection criteria.
Merging procedure
Request a detailed protocolStarting from the observed intensities, we obtain the fullvolume intensity, I_{full,h}(i), from I_{Eoc,h}(i) by first applying the Ewald offset correction (Equation 15) and then the fullintensity correction (Equation 16). Prior to merging equivalent observations, we detect outliers using an iterative rejection scheme, discarding reflections with intensity more or less than a userspecified cutoff (3 σ default, where σ is defined as the standard deviation of the distribution of the full reflections I_{full,h}). Finally, in order to obtain the merged reflection set, we calculate 〈I_{h}〉 from the intensity of reflections with the same reduced Miller indices using the sigmaweighted average:
where
and $\sigma \left(i\right)\left[{I}_{\hspace{0.17em}full,h}\left(i\right)\right]$ is derived from the calculation of error:
Since G is a function of G_{0} and B, and Eoc is a function of crystal orientation, mosaicity, and unitcell parameters, the error estimates for G can be further calculated as:
and ΔEoc^{2} can be calculated similarly by summing all over products of partial derivatives and errors estimated for each parameter in the Eoc function (square root of the diagonal elements of the covariance matrix).
We use CC_{1/2} as a quality indicator for the diffraction data sets (Diederichs and Karplus, 2013). We calculate CC_{1/2} by randomly partitioning all (partial) intensity observations of a given reflection into two groups. We reject any reflections with fewer than four observations; for all other reflections, we merge the observations in each group using Equation 20. CC_{1/2} is then calculated as the correlation between these two independently merged diffraction data sets.
Partial derivatives of the diffraction parameters
Let
for observed partial intensity i of miller index h.
Scale factor, G_{0} and B.
Request a detailed protocolThe derivatives of function g with respect to G_{0}:
The derivatives of function g with respect to B:
Crystal rotation angles (θ_{x}, θ_{y}, θ_{z}).
Request a detailed protocolAlthough three rotation angles θ_{x}, θ_{y}, θ_{z} can be refined, a rotation around the beam direction (zaxis) has no component on the reciprocallattice offset (r_{h}) from the Ewald sphere—therefore, the derivative with respect to θ_{z} is 0. The partial derivatives with respect to the remaining parameters can be derived in a similar way—here, only the derivatives with respect to are θ_{y} given.
where
and R is the rotation matrix of the still image. The derivatives of the g function (Equation 24) with respect to θ_{x} and the unitcell parameters can be calculated by substituting the last partial derivatives of R with the appropriate ones.
Unitcell parameters $\left(a,\hspace{0.17em}b,\hspace{0.17em}c,\hspace{0.17em}\alpha ,\hspace{0.17em}\beta ,\hspace{0.17em}\gamma \right)$
Request a detailed protocolFor unitcell parameters, constraints imposed by crystallographic space groups are applied during the refinement—e.g., tetragonal systems only have two free parameters (a and c) since a = b and α = β = γ = 90. Other restraint conditions, such as allowable refinement limits of the unitcell dimensions, can also be applied as a ‘penalty terms’ in the leastsquares refinement. The partial derivatives with respect to each unitcell parameter in reciprocal units (here, ${a}^{\ast}$ is given and ${a}^{\ast}=1/a$):
where $\frac{\partial g}{\partial Eoc}$, $\frac{\partial \mathit{Eoc}}{\partial {r}_{h}}$, and $\frac{\partial {r}_{h}}{\partial \mathit{x}}$ are as derived in (2) and
Reflection radius, r_{s}
Request a detailed protocolThe reflection radius that accommodates effects of crystal mosaicity and spectral dispersion, described by the four parameters, γ_{0}, γ_{y}, γ_{x}, and γ_{e}, has following derivatives:
For ${\gamma}_{y}$,
where $\frac{\partial g}{\partial Eoc}$ is derived in (Equation 27) and
For γ_{x} and γ_{e}, the $\frac{\partial g}{\partial p}$ and $\frac{\partial p}{\partial {r}_{s}}$ are the same as derived for γ_{y} and
Polarization correction
Request a detailed protocolThe XFEL beam is nearly 100% polarized in the horizontal direction. The optics at both the LCLS XPP and CXI stations do not introduce additional polarization. To account for the polarization of the primary beam, for a given reflection, we consider the angle $\varphi $ between the sample reflection plane formed by the h vector and the zaxis, and the laboratory horizontal (Figure 14).
As described in Kahn et al. (1982), the beam I_{0} incident on the sample crystal can be described in terms of two components, one parallel (σ) and the other perpendicular (π) to the plane of reflection:
Each of these components is affected by the polarization of the primary beam in both the horizontal (x) and vertical (y) directions. Using f_{x} and f_{y} as the fractions horizontal and vertical in the laboratory frame (f_{x} + f_{y} = 1),
and
where f_{x} and f_{y} are the polarization fractions in the x and y directions.
After reflection, only I_{σ} is attenuated:
By substituting I_{σ} and I_{π} from Equations 30 and 31 in Equation 32, we arrive at
where the bracketed expression is P (Kahn et al., 1982).
Molecular replacement and atomic model refinement protocol
Request a detailed protocolTo ensure atomic model refinements against the various diffraction data sets were as comparable as possible, we used a standard semiautomated solution and refinement protocol. First, we performed molecular replacement phasing with known structures as search models (PDB ID 3U3E for myoglobin, 3C8Y for hydrogenase, and 2TLI for thermolysin) with all heteroatoms, water molecules, and ligands removed. Molecular replacement was carried out with Phaser (McCoy et al., 2007) using default settings, with r.m.s.d. set to 0.8. The resulting solutions were then refined using phenix.refine (Afonine et al., 2012) in two cycles. In the first cycle, we carried out rigid body refinement, positional (xyz) refinement with automatic correction of Asn, Gln and His sidechain orientations, and atomic displacement parameter (ADP) refinement. We then used the difference density maps for missing ligands and heteroatoms obtained from this cycle to calculate realspace correlation coefficients using phenix.get_cc_mtz_pdb from the PHENIX software suite (Adams et al., 2010) for myoglobin and thermolysin and the program ‘Map Correlation’ from the CCP4 software (Winn et al., 2011) for hydrogenase. These omit difference density maps are shown in Figures 6, 7, 8, 10. In the second cycle, all ligands and heteroatoms were placed in the difference density maps and combined with the refined structure from the first cycle using Coot (Emsley et al., 2010). The second cycle employed positional and ADP refinement with target weights optimization and water update was carried out with these complete models. The structures were validated by MolProbity (Chen et al., 2010). Final refinement statistics (Tables 2, 3, 4) were analyzed with phenix.polygon (Urzhumtseva et al., 2009) and found to be within acceptable range for other structures at similar resolutions. For the thermolysin structure obtained from anomalous diffraction data (processed keeping Friedel pairs separate), only one cycle of atomic model refinement was carried out. All figures were made in PyMOL (The PyMOL Molecular Graphics System, Version 1.5.0.4 Schrödinger, LLC.).
Computer program
Request a detailed protocolThe computer program, prime, is implemented as a part of the cctbx computational crystallography toolbox (GrosseKunstleve et al., 2002). Download and installation instructions are available on the cctbx website (http://cctbx.sourceforge.net).
Note added at proof
Request a detailed protocolSubsequent to acceptance of this article, a paper was published by Ginn et al. (2015) describing an alternative method for orientation refinement as compared to the method of Sauter et al. (2014), and partiality estimation for each individual image, but without postrefinement.
Data availability

Accurate macromolecular structures using minimal measurements from Xray free electron lasers. Nat Methods 11:545548Publicly available at the CXIDB Coherent Xray Imaging Data Bank ID23.
References

PHENIX: a comprehensive Pythonbased system for macromolecular structure solutionActa Crystallographica Section D, Biological Crystallography 66:213–221.https://doi.org/10.1107/S0907444909052925

Towards automated crystallographic structure refinement with phenix.refineActa Crystallographica Section D, Biological Crystallography 68:352–367.https://doi.org/10.1107/S0907444912001308

Demonstration of selfseeding in a hardXray freeelectron laserNature Photonics 6:693–698.https://doi.org/10.1038/nphoton.2012.180

Collective instabilities and highgain regime in a free electron laserOptics Communications 50:373–378.https://doi.org/10.1016/00304018(84)901056

MolProbity: allatom structure validation for macromolecular crystallographyActa Crystallographica Section D, Biological Crystallography 66:12–21.https://doi.org/10.1107/S0907444909042073

Goniometerbased femtosecond crystallography with Xray free electron lasersProceedings of the National Academy of Sciences of USA 111:17122–17127.https://doi.org/10.1073/pnas.1418733111

Gas dynamic virtual nozzle for generation of microscopic droplet streamsJournal of Physics D 41:195505.https://doi.org/10.1088/00223727/41/19/195505

Better models by discarding data?Acta Crystallographica Section D Biological Crystallography 69:1215–1222.https://doi.org/10.1107/S0907444913001121

Features and development of cootActa Crystallographica Section D, Biological Crystallography 66:486–501.https://doi.org/10.1107/S0907444910007493

Radiation damage in macromolecular crystallography: what is it and why should we care?Acta Crystallographica Section D Biological Crystallography 66:339–351.https://doi.org/10.1107/S0907444910008656

The computational crystallography toolbox: Crystallographic algorithms in a reusable software frameworkJournal of Applied Crystallography 35:126–136.https://doi.org/10.1107/S0021889801017824

Changes to crystals of Escherichia coli betagalactosidase during roomtemperature/lowtemperature cycling and their relation to cryoannealingActa Crystallographica Section D, Biological Crystallography 63:1139–1153.https://doi.org/10.1107/S0907444907045040

Processing of Xray snapshots from crystals in random orientationsActa Crystallographica Section D Biological Crystallography 70:2204–2216.https://doi.org/10.1107/S1399004714013534

Macromolecular crystallography with synchrotron radiation: photographic data collection and polarization correctionJournal of Applied Crystallography 15:330–337.https://doi.org/10.1107/S0021889882012060

Femtosecond protein nanocrystallographydata analysis methodsOptics Express 18:5713–5723.https://doi.org/10.1364/OE.18.005713

Generation of coherent radiation by a relativistic electron beam in an undulatorSoviet Physics Doklady 24:986.

Phaser crystallographic softwareJournal of Applied Crystallography 40:658–674.https://doi.org/10.1107/S0021889807021206

A description of imperfections in protein crystalsActa Crystallographica Section D Biological Crystallography 54:848–853.https://doi.org/10.1107/S0907444998001875

Matching Xray beam and detector properties to protein crystals of different perfectionJournal of Synchrotron Radiation 21:537–546.https://doi.org/10.1107/S1600577514003609

Python for scientific computingComputing in Science & Engineering 9:10–20.https://doi.org/10.1109/MCSE.2007.58

Processing and postrefinement of oscillation camera dataJournal of Applied Crystallography 12:570–581.https://doi.org/10.1107/S0021889879013273

XFEL diffraction: developing processing methods to optimize data qualityJournal of Synchrotron Radiation 22:239–248.https://doi.org/10.1107/S1600577514028203

Improved crystal orientation and physical properties from singleshot XFEL stillsActa Crystallographica Section D Biological Crystallography 70:3299–3309.https://doi.org/10.1107/S1399004714024134

New Pythonbased methods for data processingActa Crystallographica Section D, Biological Crystallography 69:1274–1282.https://doi.org/10.1107/S0907444913000863

Nanoflow electrospinning serial femtosecond crystallographyActa Crystallographica Section D, Biological Crystallography 68:1584–1587.https://doi.org/10.1107/S0907444912038152

Imaging biological specimens with highintensity soft x raysJournal of the Optical Society of America B 3:1551.https://doi.org/10.1364/JOSAB.3.001551

Crystallographic model quality at a glanceActa Crystallographica Section D, Biological Crystallography 65:297–300.https://doi.org/10.1107/S0907444908044296

Postrefinement method for snapshot serial crystallography Postrefinement method for snapshot serial crystallographyPhilosophical Transactions of the Royal Society of London Series B, Biological Sciences 369:20130330.https://doi.org/10.1098/rstb.2013.0330

The oscillation method for crystals with very large unit cellsActa Crystallographica Section A, Foundations of Crystallography 35:901–911.https://doi.org/10.1107/S0567739479002035

Overview of the CCP4 suite and current developmentsActa Crystallographica Section D, Biological Crystallography 67:235–242.https://doi.org/10.1107/S0907444910045749

Predicting the Xray lifetime of protein crystalsProceedings of the National Academy of Sciences of USA 110:20551–20556.https://doi.org/10.1073/pnas.1315879110

Automated diffraction image analysis and spot searching for highthroughput crystal screeningJournal of Applied Crystallography 39:112–119.https://doi.org/10.1107/S0021889805040677

A singleshot transmissive spectrometer for hard xray free electron lasersApplied Physics Letters 101:034103.https://doi.org/10.1063/1.4736725
Decision letter

Stephen C HarrisonReviewing Editor; Harvard Medical School, Howard Hughes Medical Institute, United States
eLife posts the editorial decision letter and author response on a selection of the published articles (subject to the approval of the authors). An edited version of the letter sent to the authors after peer review is shown, indicating the substantive concerns or comments; minor concerns are not usually shown. Reviewers have the opportunity to discuss the decision before the letter is sent (see review process). Similarly, the author response typically shows only responses to the major concerns raised by the reviewers.
Thank you for sending your work entitled “Enabling Xray Free Electron Laser Crystallography for Challenging Biological Systems from a Limited Number of Crystals” for consideration at eLife. Your article has been favorably evaluated by John Kuriyan (Senior editor) and three reviewers, one of whom, Stephen Harrison, is a member of our Board of Reviewing Editors.
The Reviewing editor and the other reviewers discussed their comments before we reached this decision, and the Reviewing editor has assembled the following comments to help you prepare a revised submission.
The manuscript represents a substantial advance in the processing of XFEL diffraction data, through the introduction of postrefinement methods. Although it provides no direct comparison with other procedures (from Kabsch or White), data for the three test cases considered show considerably improved statistics. Moreover, the total number of frames required to compile a data set is very much smaller than with the socalled “Monte Carlo” method. The manuscript is well written and thorough (especially in the provision of equations in the Methods section). A possible, but forgivable, omission was the inclusion of a more challenging test case that could demonstrate a genuine increase in the effective resolution of a dataset from the new procedures.
The reviewers had the following modest concerns, questions and requests:
A. Theory:1) Sphere model for Eoc_{h}. The basic idea of postrefinement is that the ratio of the intensity of a partially recorded reflection to its value in a properly corrected and scaled reference data set is a very sensitive measure of the orientation and unitcell parameters of the crystal and of diffractingrange parameters such as mosaic spread, energy dispersion, and range of unitcell parameters within the diffracting volume. Equations (1) and (2) summarize the application to XFEL stills, with Eoc_{h} as the critical function requiring definition and evaluation. The authors choose a sphere model for Eoc, which is reasonable for cases in which the diffracting range is dominated by energy dispersion and unitcell variation. For a mosaic crystal with no variation in unit cell across the diffracting volume and mosaicity higher than the energy dispersion, the reciprocalspace shape of the diffraction spot will not be spherical; it will intersect the Ewald sphere as an arc, since the Bragg angle and hence the distance of any component of the spot from the origin of reciprocal space will be (ex hypothesi) invariant, while the range of azimuthal angles of the spot on the detector will depend on the mosaic spread (assumed to be nonzero). With cryopreserved crystals, the assumption that a combination of unitcell variation and energy dispersion dominates is almost certainly a good one, but it may not hold for tiny crystals at ambient temperature in an injected beam. Anisotropy of some of the parameters may also make other shapes a better fit. The approach in the paper is, of course, generalizable to other shapes (with much “hairier” expressions for Eoc_{h} and its derivatives). In any case, the authors should discuss the assumptions that go into the sphere approximation.
2) Lorentz factor. A clear discussion of Lorentz factor is important, to give the paper full archival value as a complete treatment of the intensity correction problem. Formally, there is no Lorentz factor for a still. This statement is easy to prove using the “sinc” formula given in Equation 1 of the cited article by Kirian et al. (2010). If two different relps lie precisely on the Ewald sphere, then the value of the sinc function is simply equal to the square of the number of unit cells, regardless of resolution or any other geometric factor. All that remains is the polarization (which is not a Lorentz factor) and the incident intensity, which is the same for every spot. The only terms that remain hkldependent are the structure factor F, and the solid angle subtended by a pixel. The latter has some semblance to a Lorentz factor, but disappears upon pixel integration if the detector is corrected to be spherical. The spreading out of the spot due to mosaic spread and spectral dispersion in reciprocal space could be considered a Lorentz factor, but in the context of the present work, this should be part of the “partiality”.
B. Questions:
1) In the test cases, the data quality for the subset of images (e.g. 2,000 for thermolysin) is clearly lower than using the entire dataset. Is there any indication of convergence when considering data quality metrics vs the number of images included, or does inclusion of all images always give the best data?
2) The Discussion section is relatively brief. Even with the improved processing, the data quality falls significantly short of what would be expected for conventional SR rotation data collection. Does the analysis provide any pointers to the remaining major sources of error?
3) Figure 6 (myoglobin data): For the high resolution terms, postrefinement appears to make the data worse as judged by the R and R_{free} metrics. Why?
4) There are many fewer spots per image for thermolysin than for the other two datasets. What is the definition of a “spot” in this context?
5) It is not clear if a separate resolution limit is applied to each image during the final merging step. Can this be clarified?
6) Figure 9: What is the second peak that is clearly visible when all images are used? Perhaps it would be useful to quote the largest “noise” peak as well as that for the Zn.
7) Table 3: The hydrogenase data were collected with a seeded beam, and yet the term representing the energy dispersion γe is larger than that for thermolysin and almost as large as for the myoglobin data. Why?
C. Request:
The paper should have a complete list of all the parameters and symbols in the equations and their definitions (as Acta Cryst may still do and certainly used to do). Many of the parameters (such as theta(x) and theta(y)) were defined only in the figures, and it might indeed clutter the text to define each of them immediately after their first appearance in equation (1).
https://doi.org/10.7554/eLife.05421.021Author response
A. Theory:
1) Sphere model for Eoc_{h}. The basic idea of postrefinement is that the ratio of the intensity of a partially recorded reflection to its value in a properly corrected and scaled reference data set is a very sensitive measure of the orientation and unitcell parameters of the crystal and of diffractingrange parameters such as mosaic spread, energy dispersion, and range of unitcell parameters within the diffracting volume. Equations (1) and (2) summarize the application to XFEL stills, with Eoc_{h} as the critical function requiring definition and evaluation. The authors choose a sphere model for Eoc, which is reasonable for cases in which the diffracting range is dominated by energy dispersion and unitcell variation. For a mosaic crystal with no variation in unit cell across the diffracting volume and mosaicity higher than the energy dispersion, the reciprocalspace shape of the diffraction spot will not be spherical; it will intersect the Ewald sphere as an arc, since the Bragg angle and hence the distance of any component of the spot from the origin of reciprocal space will be (ex hypothesi) invariant, while the range of azimuthal angles of the spot on the detector will depend on the mosaic spread (assumed to be nonzero). With cryopreserved crystals, the assumption that a combination of unitcell variation and energy dispersion dominates is almost certainly a good one, but it may not hold for tiny crystals at ambient temperature in an injected beam. Anisotropy of some of the parameters may also make other shapes a better fit. The approach in the paper is, of course, generalizable to other shapes (with much “hairier” expressions for Eoc_{h} and its derivatives). In any case, the authors should discuss the assumptions that go into the sphere approximation.
The spherical model used in this work is indeed a crude approximation of the diffraction spots. In the Discussion, we now describe the factors that contribute to spot shape and the consequent limitations of our model that will need to be addressed in the future.
2) Lorentz factor. A clear discussion of Lorentz factor is important, to give the paper full archival value as a complete treatment of the intensity correction problem. Formally, there is no Lorentz factor for a still. This statement is easy to prove using the “sinc” formula given in Equation 1 of the cited article by Kirian et al. (2010). If two different relps lie precisely on the Ewald sphere, then the value of the sinc function is simply equal to the square of the number of unit cells, regardless of resolution or any other geometric factor. All that remains is the polarization (which is not a Lorentz factor) and the incident intensity, which is the same for every spot. The only terms that remain hkldependent are the structure factor F, and the solid angle subtended by a pixel. The latter has some semblance to a Lorentz factor, but disappears upon pixel integration if the detector is corrected to be spherical. The spreading out of the spot due to mosaic spread and spectral dispersion in reciprocal space could be considered a Lorentz factor, but in the context of the present work, this should be part of the “partiality”.
We agree that there is no Lorentz correction for a stationary crystal and monochromatic beam. We had tried to follow the discussion of Kabsch (2014) on this point, but now we explicitly note that there is no Lorentz correction for a still, and have removed the discussion of the Kabsch paper on this topic.
B. Questions:
1) In the test cases, the data quality for the subset of images (e.g. 2,000 for thermolysin) is clearly lower than using the entire dataset. Is there any indication of convergence when considering data quality metrics vs the number of images included, or does inclusion of all images always give the best data?
In the last section of Results we now describe a comparison of thermolysin diffraction data sets merged using 2,00012,000 images. To avoid potential differences arising from different levels of completeness, which would confound this analysis, we truncated the diffraction data at 2.6 Å to insure that each of these sets was 100% complete. Comparison of CC_{1/2} values, electron density maps and model R values shows that there is little improvement beyond 8,000 images.
2) The Discussion section is relatively brief. Even with the improved processing, the data quality falls significantly short of what would be expected for conventional SR rotation data collection. Does the analysis provide any pointers to the remaining major sources of error?
This is an excellent question, but we do not feel that we can directly compare SR rotation and XFEL data at this point. The one comparison that we present (Figure 7) suggests that the SR data are at least somewhat better, but it is difficult to quantify. It is likely that rotation data would be better due to the ability to directly measure full reflections (at least by summation of partials) without modeling partiality, which is still a relatively crude process. However, we believe that a comparison between still diffraction data sets collected at SR and XFELs is needed to deconvolute the effect of rotation vs. other differences between SR and XFEL sources. This will be a subject of future investigation. We have added a brief discussion of these issues to the new Discussion.
3) Figure 6 (myoglobin data): For the high resolution terms, postrefinement appears to make the data worse as judged by the R and R_{free} metrics. Why?
We assume that the question refers to the 100 image subset; the full postrefined 757 image set has lower or comparable R values in all bins. The 100image set is the minimum number of images required for successful molecular replacement and an interpretable omit map (the heme group). While the R and R_{free} values of the 757image set (97.7% completeness) improved in all resolution shells, they improved only to approximately 1.7 Å, where the completeness drops below 90%. We observed that completeness has an impact on postrefinement procedure and the postrefined data sets. We now note this effect in the last section of Results, “Effect of completeness”.
4) There are many fewer spots per image for thermolysin than for the other two datasets. What is the definition of a “spot” in this context?
We now describe the criteria for “spot” definition in the subsection headed “Preparation of the observed intensities”.
5) It is not clear if a separate resolution limit is applied to each image during the final merging step. Can this be clarified?
The cctbx.xfel program applies separate resolution cutoffs on each image. This is now noted in the Results section.
6) Figure 9: What is the second peak that is clearly visible when all images are used? Perhaps it would be useful to quote the largest “noise” peak as well as that for the Zn.
We thank the reviewer for pointing out this feature, which turns out not to be noise. We suggest that the second anomalous peak may indicate a second zinc ion: indeed, a previous thermolysin structure (PDB ID: 1LND; Holland et al., 1995) has two zinc ions bound to the same active site, and their locations match with the anomalous peaks observed in the postrefined maps of Figure 9. We rerefined the thermolysin structure with two zinc ions (the refined Bfactors for the first zinc ion with occupancy 1.0 and the second zinc ion with occupancy 0.5 are 24.4 and 30.9, respectively). Interestingly, adding the second zinc ion resulted in an improvement of difference density of the dipeptide near the zinc sites (see Table 4 for updated refinement statistics). Thus, in addition to the missing dipeptide in the original structure (PDB ID: 4OW3; Hattne et al., 2014), this adds another feature that was not clearly visible before the postrefinement procedures.
7) Table 3: The hydrogenase data were collected with a seeded beam, and yet the term representing the energy dispersion γe is larger than that for thermolysin and almost as large as for the myoglobin data. Why?
We mistakenly thought that these data had been measured with a seeded beam, as a single energy value was present in the header records of each frame. However, after conferring with the experimental team that collected the data (Cohen et al., 2014), we discovered that in fact the data were collected with the usual SASE spectrum; there was a hardware problem that prevented recording the energy spectrum per frame. We have revised our manuscript accordingly.
C. Request:
The paper should have a complete list of all the parameters and symbols in the equations and their definitions (as Acta Cryst may still do and certainly used to do). Many of the parameters (such as theta(x) and theta(y)) were defined only in the figures, and it might indeed clutter the text to define each of them immediately after their first appearance in equation (1).
We have added a full list of parameters and symbols with their definitions in the Notation section.
https://doi.org/10.7554/eLife.05421.022Article and author information
Author details
Funding
National Institute of General Medical Sciences (NIGMS) (GM103393)
 William I Weis
National Institute of General Medical Sciences (NIGMS) (GM095887)
 Aaron S Brewster
 Nicholas K Sauter
Howard Hughes Medical Institute (HHMI) (Collaborative Innovation Award)
 Axel T Brunger
 William I Weis
U.S. Department of Energy (Department of Energy) (DEAC0205CH11231)
 Aaron S Brewster
 Nicholas K Sauter
National Institute of General Medical Sciences (NIGMS) (GM102520)
 Aaron S Brewster
 Nicholas K Sauter
The funders had no role in study design, data collection and interpretation, or the decision to submit the work for publication.
Acknowledgements
We thank Henrik Lemke, Sebastien Boutet, and Ralf GrosseKunstleve for discussions. We thank S Michael Soltis, Aina E Cohen, Ana González, Yingssu Tsai, Winnie Brehmer, Laura Aguila, Jinhu Song, Scott McPhillips, and Henrik Lemke for providing the myoglobin XFEL diffraction data set. We thank John W Peters, Stephen Keable, Oleg A Zadvornyy, Aina E Cohen, S Michael Soltis, Jinhu Song, Scott McPhillips, Clyde Smith, and Henrik Lemke for providing the Cpl hydrogenase XFEL diffraction data set. Portions of this research were carried out at the Linac Coherent Light Source (LCLS) at the SLAC National Accelerator Laboratory. LCLS is an Office of Science User Facility operated for the U.S. Department of Energy Office of Science by Stanford University. ASB and NKS were supported by National Institutes of Health grants GM095887 and GM102520 and Director, Office of Science, Department of Energy under contract DEAC0205CH11231. WIW was supported in part by National Institutes of Health grant P41 GM103393. This work is supported by a HHMI Collaborative Innovation Award (HCIA) to ATB and WIW.
Reviewing Editor
 Stephen C Harrison, Harvard Medical School, Howard Hughes Medical Institute, United States
Publication history
 Received: October 31, 2014
 Accepted: March 16, 2015
 Accepted Manuscript published: March 17, 2015 (version 1)
 Version of Record published: April 15, 2015 (version 2)
Copyright
This is an openaccess article, free of all copyright, and may be freely reproduced, distributed, transmitted, modified, built upon, or otherwise used by anyone for any lawful purpose. The work is made available under the Creative Commons CC0 public domain dedication.
Metrics

 3,806
 Page views

 853
 Downloads

 80
 Citations
Article citation count generated by polling the highest count across the following sources: Crossref, Scopus, PubMed Central.