Registered report: The common feature of leukemia-associated IDH1 and IDH2 mutations is a neomorphic enzyme activity converting alpha-ketoglutarate to 2-hydroxyglutarate

The Reproducibility Project: Cancer Biology seeks to address growing concerns about reproducibility in scientific research by conducting replications of selected experiments from a number of high-profile papers in the field of cancer biology. The papers, which were published between 2010 and 2012, were selected on the basis of citations and Altmetric scores (Errington et al., 2014). This Registered Report describes the proposed replication plan of key experiments from “The common feature of leukemia-associated IDH1 and IDH2 mutations is a neomorphic enzyme activity converting alpha-ketoglutarate to 2-hydroxyglutarate” by Ward and colleagues, published in Cancer Cell in 2010 (Ward et al., 2010). The experiments that will be replicated are those reported in Figures 2, 3 and 5. Ward and colleagues demonstrate the mutations in isocitrate dehydrogenase 2 (IDH2), commonly found in acute myeloid leukemia (AML), abrogate the enzyme’s wild-type activity and confer to the mutant neomorphic activity that produces the oncometabolite 2-hydroxyglutarate (2-HG) (Figures 2 and 3). They then show that elevated levels of 2-HG are correlated with mutations in IDH1 and IDH2 in AML patient samples (Figure 5). The Reproducibility Project: Cancer Biology is a collaboration between the Center for Open Science and Science Exchange and the results of the replications will be published by eLife. DOI: http://dx.doi.org/10.7554/eLife.12626.001


Introduction
Mutations in the metabolic enzymes isocitrate dehydrogenase 1 (IDH1) and IDH2 genes, which catalyze the production of a-ketoglutarate (a-KG) from isocitrate, have been associated with numerous forms of cancer (Krell et al., 2013) leading to exploration of how changes in their function could be linked to the development of tumors. All known mutations alter key residues in both proteins that decrease the enzyme's affinity for isocitrate, leading to the theory that the loss of IDH function perturbs the equilibrium of a-KG, negatively affecting various a-KG dependent enzymes (Zhao et al., 2009). However, work from the Thompson group determined that the tumor-associated mutations actually created a neomorphic function; rather than catalyzing the production of a-KG, mutant IDH proteins produce the oncometabolite 2-hydroxyglutarate (2-HG) . Dang and colleagues first described this neomorphic function and demonstrated a correlation between 2-HG levels and glioma samples harboring IDH mutations (Dang et al., 2009). In their 2010 Cancer Cell paper, Ward and colleagues further confirm these findings and extend the association of 2-HG levels and IDH mutations to acute myeloid leukemia (AML) .
In Figure 2, Ward and colleagues transfected 293T cells with either wild type or mutant forms of IDH2. They assessed cell lysates for their ability to generate NDPH in the presence of isocitrate (Figure 2A) or to consume NADPH in the presence of a-KG ( Figure 2B). Their data indicated that cells transfected with IDH2 WT generated NADPH in the presence of isocitrate, and did not consume much NADPH in the presence of a-KG, consistent with its canonical function of converting isocitrate to a-KG. However, IDH2 R172K displayed the opposite effect, indicating that it was able to consume NADPH in an a-KG dependent manner. These data were the first suggesting that the mutant form of IDH2 might have a neomorphic function. This key experiment will be replicated in Protocol 1.
In Figure 3, Ward and colleagues use gas-chromatography mass spectrometry (GC-MS) to identify a novel function of IDH2 R172K . They identified a unique peak in the lysates of cells transfected with IDH2 R172K that corresponded to the retention time of the metabolite 2-hydroxyglutarate (2-HG). They confirmed the metabolite identity by mass spectrometry. These data provide evidence that the mutant form of IDH2 leads to 2-HG production. This key experiment will be replicated in Protocol 2.
In Figure 5, Ward and colleagues examined the correlation between AML patient samples carrying IDH mutations and the levels of 2-HG found in those samples. They showed that patient samples carrying IDH mutations contained higher levels of 2-HG than samples from patients with WT IDH genes. This key experiment will be replicated in Protocol 3.
Several groups' work has supported the results of Ward and colleagues, who themselves confirmed and extended their initial findings in subsequent reports (Ward et al., 2011;. Leonardi and colleagues confirmed that mutant forms of IDH, specifically IDH1, did not perform the canonical forward reaction converting isocitrate to a-KG (Leonardi et al., 2012). Using magnetic resonance spectroscopy, Izquierdo-Garcia and colleagues confirmed that transfection of cells with mutant IDH forms increased the levels of 2-HG (Izquierdo-Garcia et al., 2015), while Jin and colleagues demonstrated similar findings for IDH1 and IDH2 mutants (Jin et al., 2011). Evaluating 2-HG levels in astrocytomas and gliomas harboring various IDH1 mutations, Pusch and colleagues also showed that any mutations in IDH1 correlated with increased levels of 2-HG in human patient samples (Pusch et al., 2014), a trend also observed by Juratli and colleagues (Juratli et al., 2013).
Discovery of IDH neomorphic function, resulting in the production of the 'oncometabolite' 2-HG, opened many avenues of research into how the production of excess 2-HG could impact tumorigenesis. Figueroa and colleagues expanded upon the foundation laid by Ward and colleagues and determined that excess 2-HG was correlated with changes in global methylation patterns (Figueroa et al., 2010). Xu and colleagues showed that 2-HG was able to competitively inhibit many a-KG dependent enzymes, including several histone demethylases, and that exogenous 1-HG was able to inhibit histone demethylation (Xu et al., 2011). Lu and colleagues also observed this correlation between 2-HG levels and perturbations in global histone methylation patterns, and went on to show that this resulted in impaired cellular differentiation .

Materials and methods
Unless otherwise noted, all protocol information was derived from the original paper, references from the original paper, or information obtained directly from the authors.
Protocol 1: Assessing the a-ketoglutarate dependent NADPH consumption of wild-type or mutant IDH2 In this protocol, 293T cells are transfected with empty vector, IDH2 WT , or IDH2 R172K . Lysates are generated from these cells and their ability to produce NADPH from NADP+ and isocitrate is assayed ( Figure 2A). The same lysates are also assayed for their ability to consume NADPH in the presence of 0.5 mM a-ketoglutarate (a-KG) ( Figure 2B). Expression of the transfected protein will be confirmed by Western blot ( Figure 2C).

Sampling
Oxidative and reductive activity (Figures 2A and B): . Experiment has three conditions. Each will be performed with seven biological replicates and three technical replicates of each condition at each time point for a final power of at least 80%.
.  Figure 2C) . This is a quality control experiment and is not being powered to detect a specific effect size.
Western blots will be performed alongside each biological replicate.
. Western blotting of each lysate will be performed for the following proteins . IDH2 . IDH1 . Actin [additional] Materials and reagents a. As per manufacture's instructions 1 mg plasmid DNA per well in a 6-well plate for 70-80% confluent 293T cells. b. Transfect 1 well (or plate if reaction needs to be scaled up) for each construct i. This will be one biological replicate 6. 48 hr after transfection, remove medium from cells, wash with PBS, and lyse in 1 ml/well of mammalian protein extraction reagent containing protease inhibitor cocktail (aprotinin, AEBSF, leupeptin and pepstatin A, all at 1:1000) and phosphatase inhibitor cocktails (NaOV, Pepstatin A, Leupeptin, AEBSF, NaF, aprotinin) at 4˚C or on ice. 7. Collect lysate and sonicate.
a. Perform test for optimal conditions as follows. i. Sonication for 5 min ii. Sonication for 10 min b. Centrifuge lysate in refrigerated microcentrifuge at 14000xg at 4˚C for 10 min. c. Collect supernatants and measure the protein concentration of each using the DC Protein Assay Kit II according to the manufacturer's instructions. d. Will need >50 mg total protein to proceed i. If 50 mg total protein is not achieved the reaction will be scaled to a 25 cm plate. These conditions will be used for the subsequent replicates without any further optimization. ii. If further optimization is needed, the experiment will not proceed to step 7 until this is achieved. e. Aliquot lysate protein for measuring IDH oxidative (Step 9) and reductive activity (step 10) and for examining expression of IDH2 WT , IDH2 R172K by western blot (step 11). 8. Measuring IDH oxidative activity: a. Mix 0.3 mg of each protein lysate with 200 ml of assay buffer solution in a 96-well plate. Each condition should be plated in triplicate. i. Assay buffer solution: 100 mM Tris-HCl buffer (pH 7.5), 1.3 mM MnCl 2 , 0.33 mM EDTA, 0.1 mM ß-NADP + , 0.1 mM D-(+)-threo-isocitrate ii. Include buffer lacking lysate protein to determine background reading. b. Put mixtures in spectrometer and measure absorbance at 340 nm every 20 s for 30 min. c. Use absorbance readings at 5 min intervals for analysis.
i. An exploratory investigation of all data will be used in the analysis as well. 9. Measuring IDH reductive activity: a. Mix 3 mg of each protein lysate with 200 ml of assay buffer solution in a 96-well plate. Each condition should be plated in triplicate. i. Assay buffer solution: 100 mM Tris-HCl buffer (ph 7.5), 1.3 mM MnCl 2 , 0.01 mM ß-NADPH, 0.5 mM a-ketoglutarate ii. Include buffer lacking lysate protein to determine background reading. b. Put mixtures in spectrometer and measure absorbance at 340 nm every 20 min for 3 hr. 10. Western blot to confirm protein expression: a. Add sample buffer and boil lysates to prepare for loading. i. Sample buffer: 0.5 mL 1 M TrisCl, pH 6.8, 1 mL glycerol, 0.5 mL ß-mercaptoethanol, 0.24 g SDS, 0.1 mL 1% bromophenol blue. ii. Add 30 mg of protein per well by diluting protein to same concentrations (based on protein quantification results) in 10 mL of lyse buffer and added 20 mL of sample buffer iii. Place at 65˚C for 15 min. b. Separate 20-30 mg of protein per lane on an 8% SDS-PAGE gel with protein ladder.
i. Run through the stacker at 45 mAmp/gel, then increase to 300 V for 3 hr. c. Transfer to nitrocellulose membrane.
i. Transfer at 100 A for 1 hr 40 min in 2.5 mM Tris, 19 mM glycine in 20% methanol.
ii. Wash membrane in deionized water then wash in 1X TBST.
iii. Confirm protein transfer with Ponceau stain. d. Block membrane with 5% milk/0.2% azide in TBST for 30 min at room temperature. e. Incubate with the following primary antibodies using the manufacturer's recommended dilution. Following antibodies will be probed at one time i. Mouse anti-IDH2; 37 kDa ii. Goat anti-IDH1; 47 kDa f. Incubate with appropriate secondary antibodies using manufacture's recommended dilutions i. HRP-conjugated sheep anti-mouse ii. HRP-conjugated rabbit anti-goat 1. The anti-actin antibody is HRP conjugated and a secondary antibody incubation is not necessary. g. Treat membranes with ECL reagent according to manufacturer's recommendations and image. h. Between antibody incubations, inactivate HRP activity by incubating with a final concentration of 1mM sodium azide in blocking buffer. i. Shake at room temp for 1 hr.
iii. Incubate with ECL reagent as directed by the manufacturer and image at a time point of at least 5 min to confirm HRP inactivation iv. Save blank image i. Incubate with Rabbit anti-actin-HRP; 45 kDa [additional] to evaluate loading control j. Treat membranes with ECL reagent according to manufacturer's recommendations and image. 11. Repeat steps 6-9 independently six additional times.
Deliverables . Data to be collected: . Sequencing reads and agarose gel images confirming vector identity and integrity . Raw data from plate reader for reduced NADP + and oxidated NADPH . Background subtracted readings . Full western images, including ladder . Ponceau stains confirming protein transfer . ECL negative control from step 9-hr

Confirmatory analysis plan
Statistical Analysis of the Replication Data: . Note: At the time of analysis, we will perform the Shapiro-Wilk test and generate a quantilequantile plot to assess the normality of the data. We will also perform Levene's test to assess homoscedasticity. If the data appears skewed, we will perform the appropriate transformation to proceed with the proposed statistical analysis. If this is not possible, we will perform the equivalent non-parametric Wilcoxon-Mann-Whitney test.
. This is a quality control experiment and is not powered to detect a specific effect.
. Meta-analysis of original and replication attempt: . This replication attempt will perform the statistical analysis listed above, compute the effects sizes, compare them against the reported effect size in the original paper and use a meta-analytic approach to combine the original and replication effects, which will be presented as a forest plot.
Although not performed by the original authors, actin was added as internal loading control for Western blots and will be added to the resulting data. Details of the Western blot protocol and possible stripping/sodium azide treatment were unspecified; information was added by the replicating lab. The details of the transfection specifics were unspecified and that information is provided by the replicating lab. Additionally, these experiments will be conducted in 6-well dishes, however, if total protein yield is not sufficient, the replicating lab will scale up to 25 cm dishes.

Provisions for quality control
All data obtained from the experiment -raw data, data analysis, control data, and quality control data -will be made publicly available, either in the published manuscript or as an open access dataset available on the Open Science Framework (https://osf.io/8l4ea/).
. STR profiling and mycoplasma testing results . Sequencing reads and agarose gel images confirming vector identity and integrity . Ponceau stains confirming protein transfer for Western Blot . Confirmation of HRP inactivation prior to proceeding with the following antibodies.

Protocol 2: Production of 2-HG from IDH2 WT and mutant transfected cells
In this protocol, the production of 2-HG from 293T cells transfected with vectors expressing IDH2 WT or IDH2 R172K is measured by gas chromatography-mass spectrometry (as seen in Figures 3A-C). The amount of 2-HG relative to glutamate is quantified, as seen in Figure 3D.
Sampling & Experiment will be performed with at least three biological replicates for a final power of at least 80%. The original data are qualitative, thus to determine an appropriate number of replicates to initially perform, sample sizes based on a range of potential variance was determined.
. i. Weigh out methoxyamine hydrochloride in 1.5 ml Eppendorf tube on balance and add appropriate amount of pyridine. b. Vortex MeOX solution and sonicate at 60˚C for 15 min to dissolve. c. Add 10 ml of 40 mg/ml MeOX solution to each dried sample. d. Shake at maximum speed at 60˚C for 1 hr. e. To 1 ml of MSTFA, add 10 ml of FAME marker.
i. Vortex for 10 s. f. Add 91 ml of MSTFA + FAME mixture to each sample and standard. Cap immediately.
i. Shake at maximum speed at 37˚C. g. Transfer contents to glass vials with micro-inserts and cap immediately.
i. Submit to GCTOF MS analysis. 12. Inject samples into GC-MS.
a. Operate the detector in spitless mode using electron impact ionization.  Figure 3A) and cells transfected with IDH2 R172K ( Figure 3B).
Mass spectrum confirmation of metabolite identity as 2-HG. . 48 hr run GC traces for all samples run Close-up of the time range showing metabolite abundance for aspartate, glutamate, and 2-HG for cells transfected with IDH2 WT ( Figure 3A) and cells transfected with IDH2 R172K ( Figure 3B).
Quantification of the relative intensity of the 2-HG signal to the glutamate signal, graphed as seen in Figure 3D.

Confirmatory analysis plan
. Statistical Analysis of the Replication Data: . Note: At the time of analysis, we will perform the Shapiro-Wilk test and generate a quantilequantile plot to assess the normality of the data. We will also perform Levene's test to assess homoscedasticity. If the data appears skewed, we will perform the appropriate transformation to proceed with the proposed statistical analysis. If this is not possible, we will perform the equivalent non-parametric test.
. Two-way ANOVA performed on 2-HG/glutamate ratios followed by Fisher's LSD for the following comparisons: . Vector vs. IDH2 WT . IDH2 WT vs. IDH2 R172K o Analyses will be performed on both 24 and 48 hr runs.
. Meta-analysis of original and replication attempt: . The replication data will be presented as a mean with 95% confidence intervals and will include the original data point, calculated directly from the graph, as a single point on the same plot for comparison.
. The GC-MS sample preparation protocol was modified by the replicating lab including a shaking incubation step at 11f. However, this protocol was taken from Bennett et al. which the authors reference in the original manuscript.

Provisions for quality control
All data obtained from the experiment -raw data, data analysis, control data and quality control data -will be made publicly available, either in the published manuscript or as an open access dataset available on the Open Science Framework (https://osf.io/8l4ea/).
. STR profiling and mycoplasma testing results. . Mass spectrum of the metabolite peak for derivatized 2HG to confirm identity.

Protocol 3: Assessing the correlation of IDH status with 2-HG levels in samples from patients with AML
In this protocol, samples from patients with acute myeloid leukemia (AML) are examined for their IDH mutational status and their level of 2-HG, as seen in Figure 5.

Sampling
. This experiment will use four samples per group for a final power of at least 80%. . See Power Calculations section for details. . This experiment has three genetically distinct groups: . AML patients with no IDH mutations . AML patients with mutant IDH1 . AML patients with mutant IDH2, including both R172K and R140Q mutants . All samples will come from Roswell Park Cancer Institute and are ficoll separated in media with 10% DMSO and prescreened for IDH genotypic status.
. Each patient sample will be assessed for their ratio of 2-HG/glutamate. a. If using frozen cells, warm cells to 37˚C in a 37˚C water bath for 10 min b. Centrifuge cells for 5 min at 1000xg to form a pellet i. If necessary, transfer cells to a conical or microcentrifuge tube c. Gently remove freezing medium from MNCs d. Proceed with metabolite extraction and GC-MS analysis as detailed in protocol 2 Steps 5 through 12. e. For each sample, divide the GC signal intensity of their 2-HG peak by the signal intensity of their glutamate peak and graph.

Materials and reagents
Deliverables . Data to be collected: . Tabulated patient data (age, sex, IDH mutation status, 2-HG/glutamate ratio) (as seen in Table 1) . GC traces for all samples . Graph of 2-HG/glutamate ratio for samples by mutational status, as seen in Figure 5C.

Confirmatory analysis plan
. Statistical Analysis of the Replication Data: . Note: The authors report WT IDH ratios were less than 1% which we are using as the constant for the comparisons below.
. Bonferroni Correct one-sample t-test for 3 comparisons (alpha corrected for 2 test groups = 0.025) . Constant vs. IDH1 mutant . Constant vs. IDH2 mutant . Constant vs IDH1/2 mutants . Meta-analysis of original and replication attempt: . This replication attempt will perform the statistical analysis listed above, compute the effects sizes, compare them against the reported effect size in the original paper and use a meta-analytic approach to combine the original and replication effects, which will be presented as a forest plot.
Known differences from the original study . The GC-MS sample preparation protocol was modified by the replicating lab including a shaking incubation step at 11f, protocol 2. However, this protocol was taken from Bennett et al. which the authors reference in the original manuscript.

Provisions for quality control
All data obtained from the experiment -raw data, data analysis, control data and quality control data -will be made publicly available, either in the published manuscript or as an open access dataset available on the Open Science Framework (https://osf.io/8l4ea/). This includes confirmation of the GCMS peaks and elution times as well as MS QC data.

Power calculations
For details of power calculations, see spreadsheet and additional files at https://osf.io/9jkpg/

Protocol 1
Summary of original data estimated from graph reported in Figure 2A: . SD was calculated using formula SD = SEM*(SQRT n=3). Summary of original data estimated from graph reported in Figure 2B: . SD was calculated using formula SD = SEM*(SQRT(n)), where n = 3.  (Faul et al., 2007). Protocol 2: Figure 3D Summary of original data . In order to produce quantitative replication data, we will run the experiment three times. We will determine the standard deviation across the biological replicates and combine this with the reported value from the original study to simulate the original effect size. We will use this simulated effect size to determine the number of replicates necessary to reach a power of at least 80%. We will then perform additional replicates, if required, to ensure that the experiment has more than 80% power to detect the original effect.

Figure 2A (NADPH production) values
. Note: Simulation analysis was also conducted using randomly generated values based on the SD and variance desired. These data are comparable to what is seen above when using a parametric model approach. Also there may be a need to appropriately transform these data based on the scale of Figure 3D, and we have assumed that this is one representative sample and not averages of all the data showing no variance. This simulation will be loaded to the OSF (https://osf.io/8l4ea/).

Power calculations
Power calculations were performed using R software version 3.2.2 and G*Power (version 3.1.7) (Faul et al., 2007). Bonferroni corrected one-sample t-tests compared to. 01 (threshed as reported by original authors). . Because of the inherent complications that can occur when using primary patient cell lines, we have adjusted our sample size to four samples/group even though we achieve >90% power when using three samples/group.