1. Biochemistry
  2. Cell Biology
Download icon

Registered report: The common feature of leukemia-associated IDH1 and IDH2 mutations is a neomorphic enzyme activity converting alpha-ketoglutarate to 2-hydroxyglutarate

  1. Oliver Fiehn
  2. Megan Reed Showalter
  3. Christine E Schaner-Tooley
  4. Reproducibility Project: Cancer Biology Is a corresponding author
  1. University of California, Davis, United States
  2. University of Louisville School of Medicine, United States
Registered Report
Cited
1
Views
906
Comments
0
Cite as: eLife 2016;5:e12626 doi: 10.7554/eLife.12626

Abstract

The Reproducibility Project: Cancer Biology seeks to address growing concerns about reproducibility in scientific research by conducting replications of selected experiments from a number of high-profile papers in the field of cancer biology. The papers, which were published between 2010 and 2012, were selected on the basis of citations and Altmetric scores (Errington et al., 2014). This Registered Report describes the proposed replication plan of key experiments from “The common feature of leukemia-associated IDH1 and IDH2 mutations is a neomorphic enzyme activity converting alpha-ketoglutarate to 2-hydroxyglutarate” by Ward and colleagues, published in Cancer Cell in 2010 (Ward et al., 2010). The experiments that will be replicated are those reported in Figures 2, 3 and 5. Ward and colleagues demonstrate the mutations in isocitrate dehydrogenase 2 (IDH2), commonly found in acute myeloid leukemia (AML), abrogate the enzyme’s wild-type activity and confer to the mutant neomorphic activity that produces the oncometabolite 2-hydroxyglutarate (2-HG) (Figures 2 and 3). They then show that elevated levels of 2-HG are correlated with mutations in IDH1 and IDH2 in AML patient samples (Figure 5). The Reproducibility Project: Cancer Biology is a collaboration between the Center for Open Science and Science Exchange and the results of the replications will be published by eLife.

https://doi.org/10.7554/eLife.12626.001

Introduction

Mutations in the metabolic enzymes isocitrate dehydrogenase 1 (IDH1) and IDH2 genes, which catalyze the production of α-ketoglutarate (α-KG) from isocitrate, have been associated with numerous forms of cancer (Krell et al., 2013) leading to exploration of how changes in their function could be linked to the development of tumors. All known mutations alter key residues in both proteins that decrease the enzyme’s affinity for isocitrate, leading to the theory that the loss of IDH function perturbs the equilibrium of α-KG, negatively affecting various α-KG dependent enzymes (Zhao et al., 2009). However, work from the Thompson group determined that the tumor-associated mutations actually created a neomorphic function; rather than catalyzing the production of α-KG, mutant IDH proteins produce the oncometabolite 2-hydroxyglutarate (2-HG) (Ward et al., 2012). Dang and colleagues first described this neomorphic function and demonstrated a correlation between 2-HG levels and glioma samples harboring IDH mutations (Dang et al., 2009). In their 2010 Cancer Cell paper, Ward and colleagues further confirm these findings and extend the association of 2-HG levels and IDH mutations to acute myeloid leukemia (AML) (Ward et al., 2010).

In Figure 2, Ward and colleagues transfected 293T cells with either wild type or mutant forms of IDH2. They assessed cell lysates for their ability to generate NDPH in the presence of isocitrate (Figure 2A) or to consume NADPH in the presence of α-KG (Figure 2B). Their data indicated that cells transfected with IDH2WT generated NADPH in the presence of isocitrate, and did not consume much NADPH in the presence of α-KG, consistent with its canonical function of converting isocitrate to α-KG. However, IDH2R172K displayed the opposite effect, indicating that it was able to consume NADPH in an α-KG dependent manner. These data were the first suggesting that the mutant form of IDH2 might have a neomorphic function. This key experiment will be replicated in Protocol 1.

In Figure 3, Ward and colleagues use gas-chromatography mass spectrometry (GC-MS) to identify a novel function of IDH2R172K. They identified a unique peak in the lysates of cells transfected with IDH2R172K that corresponded to the retention time of the metabolite 2-hydroxyglutarate (2-HG). They confirmed the metabolite identity by mass spectrometry. These data provide evidence that the mutant form of IDH2 leads to 2-HG production. This key experiment will be replicated in Protocol 2.

In Figure 5, Ward and colleagues examined the correlation between AML patient samples carrying IDH mutations and the levels of 2-HG found in those samples. They showed that patient samples carrying IDH mutations contained higher levels of 2-HG than samples from patients with WT IDH genes. This key experiment will be replicated in Protocol 3.

Several groups’ work has supported the results of Ward and colleagues, who themselves confirmed and extended their initial findings in subsequent reports (Ward et al., 2011; 2013). Leonardi and colleagues confirmed that mutant forms of IDH, specifically IDH1, did not perform the canonical forward reaction converting isocitrate to α-KG (Leonardi et al., 2012). Using magnetic resonance spectroscopy, Izquierdo-Garcia and colleagues confirmed that transfection of cells with mutant IDH forms increased the levels of 2-HG (Izquierdo-Garcia et al., 2015), while Jin and colleagues demonstrated similar findings for IDH1 and IDH2 mutants (Jin et al., 2011). Evaluating 2-HG levels in astrocytomas and gliomas harboring various IDH1 mutations, Pusch and colleagues also showed that any mutations in IDH1 correlated with increased levels of 2-HG in human patient samples (Pusch et al., 2014), a trend also observed by Juratli and colleagues (Juratli et al., 2013).

Discovery of IDH neomorphic function, resulting in the production of the 'oncometabolite' 2-HG, opened many avenues of research into how the production of excess 2-HG could impact tumorigenesis. Figueroa and colleagues expanded upon the foundation laid by Ward and colleagues and determined that excess 2-HG was correlated with changes in global methylation patterns (Figueroa et al., 2010). Xu and colleagues showed that 2-HG was able to competitively inhibit many α-KG dependent enzymes, including several histone demethylases, and that exogenous 1-HG was able to inhibit histone demethylation (Xu et al., 2011). Lu and colleagues also observed this correlation between 2-HG levels and perturbations in global histone methylation patterns, and went on to show that this resulted in impaired cellular differentiation (Lu et al., 2012).

Materials and methods

Unless otherwise noted, all protocol information was derived from the original paper, references from the original paper, or information obtained directly from the authors.

Protocol 1: Assessing the α-ketoglutarate dependent NADPH consumption of wild-type or mutant IDH2

In this protocol, 293T cells are transfected with empty vector, IDH2WT, or IDH2R172K. Lysates are generated from these cells and their ability to produce NADPH from NADP+ and isocitrate is assayed (Figure 2A). The same lysates are also assayed for their ability to consume NADPH in the presence of 0.5 mM α-ketoglutarate (α-KG) (Figure 2B). Expression of the transfected protein will be confirmed by Western blot (Figure 2C).

Sampling

Oxidative and reductive activity (Figures 2A and B):

  • Experiment has three conditions. Each will be performed with seven biological replicates and three technical replicates of each condition at each time point for a final power of at least 80%.

      • Condition 1: 293T cells expressing IDH2WT

      • Condition 2: 293T cells expressing IDH2R172K

      • Condition 3: 293T cells expressing empty pCDNA3 vector

    • o Each lysate will be assessed for cell’s ability to reduce NADP+ and oxidate NADPH

    • o See Power Calculations section for details.

Confirmatory Western Blot (Figure 2C)

  • This is a quality control experiment and is not being powered to detect a specific effect size. Western blots will be performed alongside each biological replicate.

  • Western blotting of each lysate will be performed for the following proteins

    • IDH2

    • IDH1

    • Actin [additional]

Materials and reagents

ReagentTypeManufacturerCatalog #Comments
293T cellsCellsATCCCRL-3216Original source unspecified
Dulbecco’s modified
Eagle’s medium (DMEM)
MediaInvitrogen11965118Original unspecified
FBSReagentHycloneSH30071.03Replaces FBS from CellGro
IDH2WT ORF in pCMV6PlasmidOrigeneRC201152
IDH2R172K ORF in pCMV6PlasmidOrigeneRC400103
pCDNA3PlasmidInvitrogenV790-20
Lipofectamine 2000ReagentInvitrogen11668027
M-Per Mammalian protein
extraction reagent
ReagentPierce78503
AprotininReagentSigma248614Original protease
inhibitor cocktail
unspecified
AEBSFReagentEMD Millipore101500-100MG
LeupeptinReagentSigmaL2884-100mg
Pepstatin AReagentEMD Millipore516481-100MG
NaOVReagentSigma450243-50GOriginal unspecified
NaFReagentSigma215309-50G
SonicatorEquipmentVCR75HTOriginal unspecified
Refrigerated
microcentrifuge
EquipmentLabnet International, IncPrismROriginal unspecified
Tris-HClReagentBioRadBR0011Original unspecified
MnCl2ReagentM87-100FisherOriginal unspecified
EDTAReagentVWREM-4050Original unspecified
ß-NADP+ReagentMP BiomedicalsICN10116680Original unspecified
ß-NADPHReagentSigma10107824001Original unspecified
D-(+)-threo-isocitrateReagentSigmaI1252
SpectrophotometerInstrumentMolecular DevicesFilter Max F5 Multi-mode Microplate ReaderOriginal unspecified
6-well tissue
culture plates
MaterialsE& K Scientific27160Original unspecified
96 well platesMaterialsFisher (Costar)07-200-656Original unspecified
Tric-HClReagentBioRadBR0011Original unspecified
GlycerolReagentVWREM-4760Original unspecified
ß-mercaptoethanolReagentSigmaM6250-250mLOriginal unspecified
Sodium dodecyl sulfate (SDS)ReagentSigmaL3771-100GOriginal unspecified
Bromophenol blueReagentSigmaB0126-25GOriginal unspecified
ProtogelReagentFisher/National Diagnostics50-899-90119Original unspecified
APSReagentSigma248614Original unspecified
TEMEDReagentFisherBP150-100Original unspecified
nitrocelluloseMaterialsBioRad162-0112Original unspecified
Anti-IDH2 antibody
(mouse monoclonal)
Primary AntibodyAbcamab55271
Anti-IDH1 antibody
(goat polyclonal)
Primary AntibodySanta Cruzsc49996
Anti-Actin antibody
(rabbit monoclonal)-
HRP conjugated
Primary AntibodyCell Signaling12620Not included in original.
ECL Mouse IgG,
HRP-linked whole Ab
(from sheep)
Secondary AntibodyGE HealthcareNA931V
HRP conjugated rabbit
anti-goat antibody
Secondary AntibodyInvitrogen811620Original unspecified
Protein laddersReagentCell Signaling Tech.7727LOriginal unspecified
Gold Biotechp007-1500Original unspecified
ECL reagentReagentFisher ScientificPI34096Original unspecified
Endo-free maxiprep kitReagentQiagen12362Original unspecified
α-ketoglutarateReagentSigma75892-25GOriginal unspecified
DC Protein Assay KitKitBioRad5000112Original unspecified
Alpha innotech imagerEquipmentAlpha InnotechAlphaimager 2200
sodium azideReagentSigmaS2002-5GOriginal Unspecified
Ponceau stainReagentQuality Biological50-751-6798Additional reagent

Procedure

Notes

  • 293T cells are grown in DMEM supplemented with 10% FBS at 37°C in 5% CO2

  • Cells will be sent for STR profiling and mycoplasma testing.

  1. Confirm insert identity by sequencing.

    1. Origene clones are shipped with two sequencing primers.

  2. Sub-clone IDH2WT and IDH2R172K from the Origene pCMV6-Entry vectors into pcDNA3.

    1. Confirm insert identity by sequencing.

    2. Confirm vector integrity by agarose gel electrophoresis.

  3. Grow up and use an endo-free maxiprep kit to prep the following vectors:

    1. pcDNA3

    2. pcDNA3-IDH2WT

    3. pcDNA3-IDH2R172K

  4. Seed 0.25-1x106 293T cells per well of a 6-well plate in growth medium without antibiotics.

    1. Grow overnight.

    2. Confirm cells at 70–80% confluency by light microscopy at time of transfection.

  5. Transfect 293T cells with pcDNA3, pcDNA3-IDH2WT, pcDNA3-IDH2R172K with Lipofectamine 2000 according to manufacturer instructions for a 6-well plate.

    1. As per manufacture’s instructions 1 µg plasmid DNA per well in a 6-well plate for 70–80% confluent 293T cells.

    2. Transfect 1 well (or plate if reaction needs to be scaled up) for each construct

      1. This will be one biological replicate

  6. 48 hr after transfection, remove medium from cells, wash with PBS, and lyse in 1 ml/well of mammalian protein extraction reagent containing protease inhibitor cocktail (aprotinin, AEBSF, leupeptin and pepstatin A, all at 1:1000) and phosphatase inhibitor cocktails (NaOV, Pepstatin A, Leupeptin, AEBSF, NaF, aprotinin) at 4°C or on ice.

  7. Collect lysate and sonicate.

    1. Perform test for optimal conditions as follows.

      1. Sonication for 5 min

      2. Sonication for 10 min

    2. Centrifuge lysate in refrigerated microcentrifuge at 14000xg at 4°C for 10 min.

    3. Collect supernatants and measure the protein concentration of each using the DC Protein Assay Kit II according to the manufacturer’s instructions.

    4. Will need >50 µg total protein to proceed

      1. If 50 µg total protein is not achieved the reaction will be scaled to a 25 cm plate. These conditions will be used for the subsequent replicates without any further optimization.

      2. If further optimization is needed, the experiment will not proceed to step 7 until this is achieved.

    5. Aliquot lysate protein for measuring IDH oxidative (Step 9) and reductive activity (step 10) and for examining expression of IDH2WT, IDH2R172K by western blot (step 11).

  8. Measuring IDH oxidative activity:

    1. Mix 0.3 µg of each protein lysate with 200 µl of assay buffer solution in a 96-well plate. Each condition should be plated in triplicate.

      1. Assay buffer solution: 100 mM Tris-HCl buffer (pH 7.5), 1.3 mM MnCl2, 0.33 mM EDTA, 0.1 mM ß-NADP+, 0.1 mM D-(+)-threo-isocitrate

      2. Include buffer lacking lysate protein to determine background reading.

    2. Put mixtures in spectrometer and measure absorbance at 340 nm every 20 s for 30 min.

    3. Use absorbance readings at 5 min intervals for analysis.

      1. An exploratory investigation of all data will be used in the analysis as well.

  9. Measuring IDH reductive activity:

    1. Mix 3 µg of each protein lysate with 200 µl of assay buffer solution in a 96-well plate. Each condition should be plated in triplicate.

      1. Assay buffer solution: 100 mM Tris-HCl buffer (ph 7.5), 1.3 mM MnCl2, 0.01 mM ß-NADPH, 0.5 mM α-ketoglutarate

      2. Include buffer lacking lysate protein to determine background reading.

    2. Put mixtures in spectrometer and measure absorbance at 340 nm every 20 min for 3 hr.

  10. Western blot to confirm protein expression:

    1. Add sample buffer and boil lysates to prepare for loading.

      1. Sample buffer: 0.5 mL 1 M TrisCl, pH 6.8, 1 mL glycerol, 0.5 mL ß-mercaptoethanol, 0.24 g SDS, 0.1 mL 1% bromophenol blue.

      2. Add 30 µg of protein per well by diluting protein to same concentrations (based on protein quantification results) in 10 µL of lyse buffer and added 20 µL of sample buffer

      3. Place at 65˚C for 15 min.

    2. Separate 20–30 µg of protein per lane on an 8% SDS-PAGE gel with protein ladder.

      1. Run through the stacker at 45 mAmp/gel, then increase to 300 V for 3 hr.

    3. Transfer to nitrocellulose membrane.

      1. Transfer at 100 A for 1 hr 40 min in 2.5 mM Tris, 19 mM glycine in 20% methanol.

      2. Wash membrane in deionized water then wash in 1X TBST.

      3. Confirm protein transfer with Ponceau stain.

    4. Block membrane with 5% milk/0.2% azide in TBST for 30 min at room temperature.

    5. Incubate with the following primary antibodies using the manufacturer’s recommended dilution. Following antibodies will be probed at one time

      1. Mouse anti-IDH2; 37 kDa

      2. Goat anti-IDH1; 47 kDa

    6. Incubate with appropriate secondary antibodies using manufacture’s recommended dilutions

      1. HRP-conjugated sheep anti-mouse

      2. HRP-conjugated rabbit anti-goat

        1. The anti-actin antibody is HRP conjugated and a secondary antibody incubation is not necessary.

    7. Treat membranes with ECL reagent according to manufacturer’s recommendations and image.

    8. Between antibody incubations, inactivate HRP activity by incubating with a final concentration of 1mM sodium azide in blocking buffer.

      1. Shake at room temp for 1 hr.

      2. Wash membrane 3 x 5 min in 1X TBST.

      3. Incubate with ECL reagent as directed by the manufacturer and image at a time point of at least 5 min to confirm HRP inactivation

      4. Save blank image

    9. Incubate with Rabbit anti-actin-HRP; 45 kDa [additional] to evaluate loading control

    10. Treat membranes with ECL reagent according to manufacturer’s recommendations and image.

  11. Repeat steps 6–9 independently six additional times.

Deliverables

  • Data to be collected:

    • Sequencing reads and agarose gel images confirming vector identity and integrity

      • pcDNA3

      • pcDNA3-IDH2WT

      • pcDNA3-IDH2R172K

    • Raw data from plate reader for reduced NADP+ and oxidated NADPH

    • Background subtracted readings

    • Full western images, including ladder

      • Ponceau stains confirming protein transfer

      • ECL negative control from step 9-hr

Confirmatory analysis plan

Statistical Analysis of the Replication Data:

  • Note: At the time of analysis, we will perform the Shapiro-Wilk test and generate a quantile-quantile plot to assess the normality of the data. We will also perform Levene’s test to assess homoscedasticity. If the data appears skewed, we will perform the appropriate transformation to proceed with the proposed statistical analysis. If this is not possible, we will perform the equivalent non-parametric Wilcoxon-Mann-Whitney test.

    • For oxidative activity assays:

      • Bonferroni corrected ANOVA followed by two-tailed Bonferroni corrected planned contrasts:

        • Vector vs. IDH2WT

        • Vector vs. IDH2R172K

    • For reductive activity assays

      • Bonferroni corrected ANOVA followed by two-tailed Bonferroni corrected planned contrasts:

        • Vector vs. IDH2WT

        • Vector vs. IDH2R172K

    • Western blot:

      • This is a quality control experiment and is not powered to detect a specific effect.

  • Meta-analysis of original and replication attempt:

    • This replication attempt will perform the statistical analysis listed above, compute the effects sizes, compare them against the reported effect size in the original paper and use a meta-analytic approach to combine the original and replication effects, which will be presented as a forest plot.

Known differences from the original study

Although not performed by the original authors, actin was added as internal loading control for Western blots and will be added to the resulting data. Details of the Western blot protocol and possible stripping/sodium azide treatment were unspecified; information was added by the replicating lab. The details of the transfection specifics were unspecified and that information is provided by the replicating lab. Additionally, these experiments will be conducted in 6-well dishes, however, if total protein yield is not sufficient, the replicating lab will scale up to 25 cm dishes.

Provisions for quality control

All data obtained from the experiment - raw data, data analysis, control data, and quality control data - will be made publicly available, either in the published manuscript or as an open access dataset available on the Open Science Framework (https://osf.io/8l4ea/).

  • STR profiling and mycoplasma testing results

  • Sequencing reads and agarose gel images confirming vector identity and integrity

  • Ponceau stains confirming protein transfer for Western Blot

  • Confirmation of HRP inactivation prior to proceeding with the following antibodies.

Protocol 2: Production of 2-HG from IDH2 WT and mutant transfected cells

In this protocol, the production of 2-HG from 293T cells transfected with vectors expressing IDH2WT or IDH2R172K is measured by gas chromatography-mass spectrometry (as seen in Figures 3A–C). The amount of 2-HG relative to glutamate is quantified, as seen in Figure 3D.

Sampling

  • Experiment will be performed with at least three biological replicates for a final power of at least 80%. The original data are qualitative, thus to determine an appropriate number of replicates to initially perform, sample sizes based on a range of potential variance was determined.

    • See Power Calculations section for details.

  • Experiment has three conditions:

    • Condition 1:293T cells expressing IDH2WT

    • Condition 2: 293T cells expressing IDH2R172K

    • Condition 3: 293T cells expressing empty pCDNA3 vector

  • For each condition, lysates will be analyzed for 2-HG/glutamate levels

Materials and reagents

ReagentTypeManufacturerCatalog #Comments
293T cellsCellsATCCCRL-3216Original source unspecified
Dulbecco’s modified
Eagle’s medium (DMEM)
MediaInvitrogen11965118Original unspecified
Pen/StrepReagentFisher15140-122Original unspecified
FBSReagentHycloneSH30071.03Replaces FBS from CellGro
pcDNA-IDH2WTPlasmidGenerated in
Protocol 1
pcDNA-IDH2R172KPlasmidGenerated in
Protocol 1
Lipofectamine 2000ReagentInvitrogen11668027
MethanolReagentFisherA452SK-4Original unspecified
Refrigerated centrifugeEquipmentLabnet International, IncPrismROriginal unspecified
Nitrogen gasReagentGenerated in labOriginal unspecified
AG-1 X8 100-200
anion exchange column
ReagentBio-Rad731-6211Poly-Prep Columns, AG 1-X8, chloride form
HClReagentFisherSA56-1Original unspecified
N-methyl-N-tert-
butyldimethylsily
trifluoroacetamide (MTBSTFA; Regis)
ReagentRegis1-270243-200
Gas Chromatograph with
an HP-5MS capillary column
and Mass selective detector
EquipmentAgilent 7890A
with 7693 Autosampler
Cold trap concentratorEquiptmentLabconco Centrivap
R(-)-2-HGReagentSigma-AldrichH8378-100MGOriginal unspecified

Procedure

Notes

  • 293T cells grown in DMEM supplemented with 10% FBS at 37°C in 5% CO2.

  • All cells will be sent for STR profiling and mycoplasma testing

  1. Seed 0.25–1 x 106 293T cells per well of a 6-well plate in growth medium without antibiotics.

    1. Grow overnight.

    2. Confirm cells at 70–80% confluence by light microscopy at time of transfection.

  2. Transfect 293T cells with pCDNA3, pCDNA3-IDH2WT, or pCDNA3-IDH2R172K with Lipofectamine 2000 according to manufacturer instructions.

    1. Transfect 1 µg of plasmid DNA per well in 6-well plate at 70–80% confluence.

    2. Generate duplicate plates for each transfection:

      1. Harvest one plate at 24 hr.

      2. Harvest one plate at 48 hr.

  3. 24 hr later, replace with fresh media with 1x pen/strep

  4. 24 or 48 hr later, gently remove medium from proliferating cells.

    1. Note: from this point on this protocol contains information as described in (Bennett et al., 2008).

  5. Rapidly quench cells with 1–2 ml per well of -80°C methanol.

    1. Chill cells to -80°C and incubate at -80°C for 15 min.

  6. Scrape cells off the dish and transfer the cell suspension to a 15 ml conical tube.

    1. Centrifuge for 5 min at 2000xg at 4°C to pellet cellular debris.

    2. Transfer supernatant to a fresh 15 ml tube.

  7. Resuspend the pellet in 500 µl of -80°C 80% methanol in water by vortexing.

    1. Incubate at 4°C for 15 min.

    2. Centrifuge for 5 min at 2000xg at 4°C.

    3. Combine supernatant with supernatant from Step 6b.

    4. Repeat step 7 for a third round of extraction and combine all supernatants.

  8. Evaporate to dryness using a cold trap concentrator.

  9. Elute through an AG-1 X8 100–200 anion exchange resin according to the manufacturer’s instructions.

    1. Wash with five column volumes of wash buffer.

    2. Elute in 3N HCl.

  10. Evaporate to dryness using cold trap concentrator

  11. Redissolve sample in MSTFA + FAME.

    1. Prepare 40 mg/mL Methoxyamine hydrochloride (MeOX) solution in pyridine.

      1. Weigh out methoxyamine hydrochloride in 1.5 ml Eppendorf tube on balance and add appropriate amount of pyridine.

    2. Vortex MeOX solution and sonicate at 60°C for 15 min to dissolve.

    3. Add 10 µl of 40 mg/ml MeOX solution to each dried sample.

    4. Shake at maximum speed at 60˚C for 1 hr.

    5. To 1 ml of MSTFA, add 10 µl of FAME marker.

      1. Vortex for 10 s.

    6. Add 91 µl of MSTFA + FAME mixture to each sample and standard. Cap immediately.

      1. Shake at maximum speed at 37°C.

    7. Transfer contents to glass vials with micro-inserts and cap immediately.

      1. Submit to GCTOF MS analysis.

  12. Inject samples into GC-MS.

    1. Operate the detector in spitless mode using electron impact ionization.

      1. Ionizing voltage: -70 eV

      2. Electron multiplier: 1060 V

    2. GC temperature ramp:

      1. Hold at 100°C for 3 min.

      2. Ramp to 230°C at 4°C/min.

      3. Hold for 4 min.

      4. Ramp to 300°C.

      5. Hold for 5 min.

    3. Record mass range of 50–500 amu and record 2.71 scans/s.

  13. Repeat steps 1–12 independently three additional times.

Deliverables

  • Data to be collected:

    • 24 hr samples:

      • GC traces for all samples run

        • Close-up of the time range showing metabolite abundance for aspartate, glutamate, and 2-HG for cells transfected with IDH2WT (Figure 3A) and cells transfected with IDH2R172K (Figure 3B).

          • Mass spectrum confirmation of metabolite identity as 2-HG.

    • 48 hr run

      • GC traces for all samples run

        • Close-up of the time range showing metabolite abundance for aspartate, glutamate, and 2-HG for cells transfected with IDH2WT (Figure 3A) and cells transfected with IDH2R172K (Figure 3B).

      • Quantification of the relative intensity of the 2-HG signal to the glutamate signal, graphed as seen in Figure 3D.

Confirmatory analysis plan

  • Statistical Analysis of the Replication Data:

  • Note: At the time of analysis, we will perform the Shapiro-Wilk test and generate a quantile-quantile plot to assess the normality of the data. We will also perform Levene’s test to assess homoscedasticity. If the data appears skewed, we will perform the appropriate transformation to proceed with the proposed statistical analysis. If this is not possible, we will perform the equivalent non-parametric test.

    • Two-way ANOVA performed on 2-HG/glutamate ratios followed by Fisher’s LSD for the following comparisons:

        • Vector vs. IDH2WT

        • IDH2WT vs. IDH2R172K

      • o Analyses will be performed on both 24 and 48 hr runs.

  • Meta-analysis of original and replication attempt:

    • The replication data will be presented as a mean with 95% confidence intervals and will include the original data point, calculated directly from the graph, as a single point on the same plot for comparison.

Known differences from the original study

  • The GC-MS sample preparation protocol was modified by the replicating lab including a shaking incubation step at 11f. However, this protocol was taken from Bennett et al. which the authors reference in the original manuscript.

Provisions for quality control

All data obtained from the experiment - raw data, data analysis, control data and quality control data - will be made publicly available, either in the published manuscript or as an open access dataset available on the Open Science Framework (https://osf.io/8l4ea/).

  • STR profiling and mycoplasma testing results.

  • Mass spectrum of the metabolite peak for derivatized 2HG to confirm identity.

Protocol 3: Assessing the correlation of IDH status with 2-HG levels in samples from patients with AML

In this protocol, samples from patients with acute myeloid leukemia (AML) are examined for their IDH mutational status and their level of 2-HG, as seen in Figure 5.

Sampling

  • This experiment will use four samples per group for a final power of at least 80%.

    • See Power Calculations section for details.

  • This experiment has three genetically distinct groups:

    • AML patients with no IDH mutations

    • AML patients with mutant IDH1

    • AML patients with mutant IDH2, including both R172K and R140Q mutants

  • All samples will come from Roswell Park Cancer Institute and are ficoll separated in media with 10% DMSO and prescreened for IDH genotypic status.

  • Each patient sample will be assessed for their ratio of 2-HG/glutamate.

Materials and reagents

ReagentTypeManufacturerCatalog #Comments
Samples of peripheral blood, bone marrow,
or pheresis from patients
with karyotypically normal AML
Patient sampleNANABanked RPCI samples
DMSOReagentFisherBP231-1Original Unspecified
MethanolReagentFisherA452SK-4Original unspecified
Refrigerated centrifugeEquipmentLabnet International, IncPrismROriginal unspecified
AG-1 X8 100-200
anion exchange column
ReagentBio-Rad731-6211Poly-Prep Columns, AG 1-X8, chloride form
HClReagentFisherSA56-1Original unspecified
N-methyl-N-tert-
butyldimethylsily
trifluoroacetamide (MTBSTFA; Regis)
ReagentRegis1-270243-200
Gas Chromatograph with an
HP-5MS capillary column and Mass selective detector
EquipmentAgilent 7890A with
7693 Autosampler
Cold trap concentratorEquiptmentLabconco Centrivap

Procedure

  1. GC-MS analysis of 2-HG levels.

    1. If using frozen cells, warm cells to 37˚C in a 37˚C water bath for 10 min

    2. Centrifuge cells for 5 min at 1000xg to form a pellet

      1. If necessary, transfer cells to a conical or microcentrifuge tube

    3. Gently remove freezing medium from MNCs

    4. Proceed with metabolite extraction and GC-MS analysis as detailed in protocol 2 Steps 5 through 12.

    5. For each sample, divide the GC signal intensity of their 2-HG peak by the signal intensity of their glutamate peak and graph.

Deliverables

  • Data to be collected:

    • Tabulated patient data (age, sex, IDH mutation status, 2-HG/glutamate ratio) (as seen in Table 1)

    • GC traces for all samples

    • Graph of 2-HG/glutamate ratio for samples by mutational status, as seen in Figure 5C.

Confirmatory analysis plan

  • Statistical Analysis of the Replication Data:

  • Note: The authors report WT IDH ratios were less than 1% which we are using as the constant for the comparisons below.

    • Bonferroni Correct one-sample t-test for 3 comparisons (alpha corrected for 2 test groups = 0.025)

      • Constant vs. IDH1mutant

      • Constant vs. IDH2mutant

      • Constant vs IDH1/2mutants

  • Meta-analysis of original and replication attempt:

    • This replication attempt will perform the statistical analysis listed above, compute the effects sizes, compare them against the reported effect size in the original paper and use a meta-analytic approach to combine the original and replication effects, which will be presented as a forest plot.

Known differences from the original study

  • The GC-MS sample preparation protocol was modified by the replicating lab including a shaking incubation step at 11f, protocol 2. However, this protocol was taken from Bennett et al. which the authors reference in the original manuscript.

Provisions for quality control

All data obtained from the experiment - raw data, data analysis, control data and quality control data - will be made publicly available, either in the published manuscript or as an open access dataset available on the Open Science Framework (https://osf.io/8l4ea/). This includes confirmation of the GCMS peaks and elution times as well as MS QC data.

Power calculations

For details of power calculations, see spreadsheet and additional files at https://osf.io/9jkpg/

Protocol 1

Summary of original data estimated from graph reported in Figure 2A:

  • SD was calculated using formula SD = SEM*(SQRT n=3).

SampleTimeMeanSEMSD
IDH2WT000.08200.1421
50.2250.08200.1421
100.450.10250.1776
150.6790.15380.2664
200.9170.19740.3419
251.1290.25120.4352
301.3420.30.5196
IDH2R172K000.08200.1421
50.0380.08200.1421
100.0620.08200.1421
150.0620.08200.1421
200.0620.08200.1421
250.10.08200.1421
300.0960.08200.1421
Vector000.05640.0977
50.0210.05640.0977
100.0210.05640.0977
150.0170.05640.0977
200.0170.05640.0977
250.0330.05640.0977
300.0210.05640.0977

Linear regression to determine slopes from estimate values.

Calculations performed with R software (version 3.2.2) (R Core Team, 2015)

SampleMean slopeSDN
IDH2WT0.010.0903
IDH2R172K0.060.1403
Vector0.670.2803

Summary of original data estimated from graph reported in Figure 2B:

  • SD was calculated using formula SD = SEM*(SQRT(n)), where n = 3.

SampleTimeOriginal_Value_MeanSEMSD
IDH2WT000.00390.0067
17-0.0030.00600.0105
33-0.0040.00730.0126
50-0.0050.01020.0177
71-0.0060.01040.0181
90-0.0080.01140.0198
112-0.0090.01170.0202
131-0.010.00750.0130
171-0.0140.01210.0211
IDH2R172K000.00390.0067
17-0.0060.00390.0067
33-0.0090.00650.0114
50-0.0160.00850.0147
71-0.0240.00800.0139
90-0.0280.00870.0152
112-0.0360.00950.0164
131-0.0430.01040.0181
171-0.0550.00950.0164
Vector000.00260.0046
170.0010.00260.0046
3300.00260.0046
5000.00260.0046
7100.00260.0046
9000.00260.0046
112-0.0020.00260.0046
131-0.0020.00260.0046
171-0.0030.00260.0046

Linear regression to determine slopes from estimates values.

Calculations performed with R software (version 3.2.2) (R Core Team, 2015)

SampleMean slopeSDN
IDH2WT-0.00060.0053
IDH2R172K-0.02410.0133
Vector-0.00650.0163

Test family

  • One-way ANOVA: Fixed effects, omnibus, one-way: Bonferroni correction: alpha error = 0.025.

Power calculations

  • Power calculations were performed using G*Power, version 3.1.7 (Faul et al., 2007).

  • ANOVA F test statistic and partial η2performed with R software, version 3.2.2 (R Core Team, 2015).

GroupsF test statisticPartial η2Effect size fA priori powerTotal sample size
Slopes of NADPH
production from
IDH2WT, IDH2R172,
or Vector (Figure 2A)
F(2,6) = 10.80.78261.89763699.99%1211
(3 groups)
Slopes of NADP+
production from
IDH2WT, IDH2R172,
or Vector (Figure 2B)
F(2,6) = 3.020.50231.004894.13%1211 (3 groups)
  1. 1 7 samples per group will be used based on the planned comparisons making the power at least 80%.

Test family

  • 2 tailed t test, Wilcoxon-Mann-Whitney test, Bonferroni’s correction: alpha error = 0.0125

Power Calculations performed with G*Power software, version 3.1.7 (Faul et al., 2007).

Figure 2A (NADPH production) values

Group 1Group 2Effect size dA priori powerGroup 1 sample sizeGroup 2 sample size
VectorIDH2WT3.0513498.8%17171
VectorIDH2R172K2.12463280.0%277
  1. 1 7 samples per group will be used based on the Vector vs IDH2R172K NADP+ planned comparison making the power 98.8%.

  2. 2 A sensitivity calculation was performed since the original data showed a non-significant effect. This is the effect size that can be detected with 80% power and the indicated sample size. The original effect size reported was 0.49386.

Figure 2B (NADP+ production) values

Group 1Group 2Effect size dA priori powerGroup 1 sample sizeGroup 2 sample size
VectorIDH2WT2.12463180.0%177
VectorIDH2R172K2.2147189.3%77
  1. 1 A sensitivity calculation was performed since the original data showed a non-significant effect. This is the effect size that can be detected with 80% power and the indicated sample size. The original effect size reported was 0.47369.

Test family

  • Due to the large variance, these parametric tests are only used for comparison purposes. To ensure an adequate sample size is used, the number is based on the non-parametric tests listed above.

  • 2 tailed t test, difference between two independent means, Bonferroni’s correction: alpha error = 0.0125

Power Calculations performed with G*Power software, version 3.1.7 (Faul et al., 2007).

Figure 2A (NADPH production) values

Group 1Group 2Effect size dA priori powerGroup 1 sample sizeGroup 2 sample size
VectorIDH2WT3.0513499.2%17171
VectorIDH2R172K2.03280.0%277
  1. 1Seven samples per group will be used based on the Vector vs IDH2R172K NADP+ planned comparison making the power 98.8%.

  2. 2 A sensitivity calculation was performed since the original data showed a non-significant effect. This is the effect size that can be detected with 80% power and the indicated sample size. The original effect size reported was 0.33972.

Figure 2B (NADP+ production) values

Group 1Group 2Effect size dA priori powerGroup 1 sample sizeGroup 2 sample size
VectorIDH2WT2.05829180.0%177
VectorIDH2R172K2.0390.4%77
  1. 1 A sensitivity calculation was performed since the original data showed a non-significant effect. This is the effect size that can be detected with 80% power and the indicated sample size. The original effect size reported was 0.51213.

Protocol 2: Figure 3D

Summary of original data

  • Note: data estimated from published graphs

SampleMean intracellular 2-HG/glutamateAssumed N
Vector0.01053
IDH2WT0.01023
IDH2R172K1.23

Test family

  • One way ANOVA followed by Bonferroni corrected planed comparisons:

    • Power calculations:

      • Vector vs. IDH2R172K

      • IDH2WT vs. IDH2R172K

    • Sensitivity Calculations

      • Vector vs. IDH2WT

Power calculations

  • Power calculations were performed using GraphPad PRISM v6 and G*Power (version 3.1.7) (Faul et al., 2007)

  • Because the data did not display variance, we have performed power calculations with a range of variances and an assumed N of 3 per group.

  • 2% variance

ANOVA; α=0.05
F(2,6)Partial eta2Effect size fPowerTotal N
73700.99959349.55807>99.99%6*
Power calculations; α=0.05
Group 1Group 2Effect size dPowerN/group
VectorIDH2WT70.10710478>99.99%2*
IDH2WTIDH2R172K70.08927663>99.99%2*
Sensitivity Calculations; α=0.05, powered to 80%
Group 1Group 2Effect size dDetectable dN/group
VectorIDH2R172K1.4491231830.27748443
  1. *With a minimum of 3 per group (9 total), achieved power is >99.99%.

  • 15% variance

ANOVA; α=0.05
F(2,6)Partial eta2Effect size fPowerTotal N
1310.9776126.60808599.99%6*
Power calculations; α=0.05
Group 1Group 2Effect size dPowerN/group
VectorIDH2WT9.34761397198.65%2*
IDH2WTIDH2R172K9.34523688498.65%2*
Sensitivity Calculations; α=0.05, powered to 80%
Group 1Group 2Effect size dDetectable dN/group
VectorIDH2R172K0.1932164240.05398263
  1. *With a minimum of 3 per group (9 total), achieved power is >99.99%.

  • 28% variance

ANOVA; α=0.05
F(2,6)Partial eta2Effect size fPowerTotal N
37.600.9261083.54023598.61%6*
Power calculations; α=0.05
Group 1Group 2Effect size dPowerN/group
VectorIDH2WT5.00765034299.28%3
IDH2WTIDH2R172K5.00637690299.28%3
Sensitivity Calculations; α=0.05, powered to 80%
Group 1Group 2Effect size dDetectable dN/group
VectorIDH2R172K0.1035087990.05114193
  1. *With a minimum of 3 per group (9 total), achieved power is 99.99%.

  • 40% variance

ANOVA; α=0.05
F(2,6)Partial eta2Effect size fPowerTotal N
18.430.8600092.47857185.73%6*
Power calculations; α=0.05
Group 1Group 2Effect size dPowerN/group
VectorIDH2WT3.50535523988.73%3
IDH2WTIDH2R172K4.20528577196.37%3
Sensitivity Calculations; α=0.05, powered to 80%
Group 1Group 2Effect size dDetectable dN/group
VectorIDH2R172K0.0724561590.05055943
  1. *With a minimum of 3 per group (9 total), achieved power is 99.92%.

  • In order to produce quantitative replication data, we will run the experiment three times. We will determine the standard deviation across the biological replicates and combine this with the reported value from the original study to simulate the original effect size. We will use this simulated effect size to determine the number of replicates necessary to reach a power of at least 80%. We will then perform additional replicates, if required, to ensure that the experiment has more than 80% power to detect the original effect.

  • Note: Simulation analysis was also conducted using randomly generated values based on the SD and variance desired. These data are comparable to what is seen above when using a parametric model approach. Also there may be a need to appropriately transform these data based on the scale of Figure 3D, and we have assumed that this is one representative sample and not averages of all the data showing no variance. This simulation will be loaded to the OSF (https://osf.io/8l4ea/).

Protocol 3: Figure 5C

Summary of original data

  • Note: data estimated from published graphs and log transformed. Data includes IDHWT (no mutations in IDH1 or IDH2), IDH1R132C/G, IDH2Mutant (IDH2R172K and IDH2R140Q)

Sample2HG/glutamatelog(2HG/glut)
IDHWT(Constant)0.01-4.605
IDH1Mutant0.600-0.511
IDH1Mutant1.2000.182
IDH1Mutant1.6000.470
IDH1Mutant1.8000.588
IDH1Mutant3.0001.099
IDH1Mutant0.600-0.511
IDH2Mutant0.140-1.966
IDH2Mutant0.160-1.832
IDH2Mutant0.290-1.237
IDH2Mutant0.300-1.204
IDH2Mutant0.310-1.171
IDH2Mutant0.470-0.755
IDH2Mutant0.590-0.528
IDH2Mutant0.310-1.171

Test family

  • One sample t-test comparing Constant and mutant IDH groups:

    • Constant vs. IDH1R132C/G

    • Constant vs. IDH2mutant (grouped)

    • Constant vs. IDH1/2mutant (grouped)

Power calculations

Power calculations were performed using R software version 3.2.2 and G*Power (version 3.1.7) (Faul et al., 2007). Bonferroni corrected one-sample t-tests compared to. 01 (threshed as reported by original authors).

ConstantGroupEffect size dA priori powerGroup sample size
0.01IDH1R132C/G8.40499.99%4
0.01IDH2Mutant6.74699.99%4
0.01IDH1/2Mutant4.36199.99%4
  • Because of the inherent complications that can occur when using primary patient cell lines, we have adjusted our sample size to four samples/group even though we achieve >90% power when using three samples/group.

References

  1. 1
  2. 2
  3. 3
    An open investigation of the reproducibility of cancer biology research
    1. TM Errington
    2. E Iorns
    3. W Gunn
    4. FE Tan
    5. J Lomax
    6. BA Nosek
    (2014)
    eLife, 3, 10.7554/eLife.04333.
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
    R: A Language and Environment for Statistical Computing
    1. R Core Team
    (2015)
    Vienna, Austria: R Foundation for Statistical Computing.
  14. 14
  15. 15
  16. 16
  17. 17
  18. 18

Decision letter

  1. Jessica K Tyler
    Reviewing Editor; Weill Cornell Medical College, United States

In the interests of transparency, eLife includes the editorial decision letter and accompanying author responses. A lightly edited version of the letter sent to the authors after peer review is shown, indicating the most substantive concerns; minor comments are not usually included.

Thank you for submitting your work entitled "Registered report: The common feature of leukemia-associated IDH1 and IDH2 mutations is a neomorphic enzyme activity converting α-ketoglutarate to 2-hydroxyglutarate" for consideration by eLife. Your article has been reviewed by three peer reviewers, and the evaluation has been overseen by Jessica Tyler as Reviewing Editor and Vivek Malhotra as the Senior Editor.

The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you perform your study, and then submit your revised submission once the experiments are complete.

Summary:

The submitted report plan will replicate the results of the key experiments in Ward et al., 2010, using three protocols to replicate the experiments. Power calculations and confirmatory analysis plans are described. In general the authors have done a great job, and the proposed analyses are appropriate and accurate, but we have the following suggestions:

Essential revisions:

1) In the confirmatory analysis plan of the Protocols 2 and 3, One-way ANOVA is followed by planned comparisons using Fisher's LSD and in the corresponding power calcualtions for the t-tests of Protocol 2, a 0.05 α error is used. Fisher's LSD however does not control the family wise error rate (see Hayter, 1986) and it is useful only for the calculation of the effect size d. If that is the same statistical analysis method that was used in the original paper, we would like you to do it, but we would also like you to perform a Bonferroni correction (α = 0.025), as this is superior.

Anthony J. Hayter. The maximum familywise error rate of fisher's least significant difference test. Journal of the American Statistical Association, 81(396):

1000-1004, 1986. doi: 10.1080/01621459.1986.10478364.

2) Rather than combining the IDH1 mutant and IDH2 mutant patient samples into a single group (table after “Power calculations” in subheading “Protocol 3: Figure 5C”), it will be useful to also consider them as two separate groups in addition to IDH1+IDH2, as the difference in subcellular localization may lead to differences in D2HG abundance, and it would be worth knowing if there are differences in IDH1 mutants vs. IDH2 mutants, in addition to control comparisons. This would obviously change the statistical comparison needed to analyze the data.

For your interest, we wanted to communicate to you some insightful suggestions that the reviewers had to improve the impact of the findings of the analyses beyond the original study. We appreciate that that is not the intent of the reproduction goal of the Registered report, but we include those ideas here as they may be worthy of your consideration for increasing the impact of this study and/or your future work:

1) It may be useful to include IDH2 R140Q mutants in the cell based studies, as it is likely that some AML patient samples will harbor that specific mutation.

2) An estimation of non-IDH1,2-formed 2-HG levels would be useful.

3) Distinction between the R and S chiral forms of 2-HG would be useful.

4) Concerning the Protocol 1, i.e. replication of the Figure 2 experiments from the original article, this test represents biochemical assays of somewhat "reconstituted" WT or mutant IDH2 forms and is of the appropriate design. The design could be improved only by a few variations of the overexpressed IDH2, either WT or mutant, to better correlate the possible variations due to the different levels of expression between WT or mutant. The statistics to be performed is sufficient to encompass the quality of the data and results even if the overexpression strength is not varied.

5) Concerning the Protocol 2, i.e. replication of the Figure 3 experiments there is a problem whether "mere reproducibility" is tested or whether a scientific design of this experiment is to be scrutinized. In the former case, assessment is similar as for the Protocol 1. In the latter case however, the experimental design cannot exclude the possibility that WT enzyme does not form 2-HG, since conditions for this counter-Krebs cycle direction of reaction are not set up in the experimental design. Actually such conditions were set up if 2-HG was assayed in experiments similar to those in protocol 1, but long-term - where both directions of reaction are assayed in parallel. The Protocol 2 also cannot rule out the possibility, that mutant enzyme occurring under conditions leading to the forward-Krebs cycle direction is as inefficient in synthesizing 2-HG as the WT.

6) Concerning the Protocol 3, i.e. replication of the Figure 5 experiments analyzing the AML patient samples the precise error-free identification of mutations is the key to good reproducibility. Since this effort should rather mimic a clinical study, you should do the best to maximize the quality of these replication analyses and should substantially increase the number of samples per group. The suggested 4 samples per group are fulfilling a power of 80%, which however only represents a single repeat of the figure from the original publication but not validation of the possible diagnostic testing.

7) It would potentially be more beneficial to also focus on other aspects related to D or L2HG and IDHs such as the effects on epigenetics and the therapeutic potential of targeting 2HG-mediated epigenetic alterations.

https://doi.org/10.7554/eLife.12626.002

Author response

1) In the confirmatory analysis plan of the Protocols 2 and 3, One-way ANOVA is followed by planned comparisons using Fisher's LSD and in the corresponding power calcualtions for the t-tests of Protocol 2, a 0.05 α error is used. Fisher's LSD however does not control the family wise error rate (see Hayter, 1986) and it is useful only for the calculation of the effect size d. If that is the same statistical analysis method that was used in the original paper, we would like you to do it, but we would also like you to perform a Bonferroni correction (α = 0.025), as this is superior.

Anthony J. Hayter. The maximum familywise error rate of fisher's least significant difference test. Journal of the American Statistical Association, 81(396): 1000-1004, 1986. doi: 10.1080/01621459.1986.10478364.

With regard to Protocol 2, we agree with the reviewers’ comment on the use of a correction, such as Bonferroni or the modification of LSD by Hayter to control for the MFWER, however as Hayter describes in his 1986 paper, this applies in situations where the ANOVA is unbalanced or with a balanced design with four or more populations. Since the proposed analysis is balanced with three population groups, the LSD is sufficiently conservative and powerful to account for the multiple comparisons in this specific situation. This is further explained by Levin et al., 1994 and discussed in Maxwell and Delaney, 200 (Chapter 5) and Cohen, 2001 (Chapter 12).

References:

Levin, J.R., Serline, R.C., & Seaman M.A. (1994). A controlled, powerful multiple-comparison strategy for several situations. Psychological Bulletin, 115, 153-159.

Maxwell, S.E. & Delaney, H.D. (2004). Designing experiments and analyzing data: a model comparison perspecitive. Lawrence Erlbaum Associates, Mahwah, N.J., 2nd edition.

Cohen, B.H. (2001). Explaining psychological statistics. John Wiley and Sons, New York, 2nd edition.

As for Protocol 3, we have clarified this section. We will perform 3 one sample t-tests compared to the constant (a threshold of 0.01(WT) as defined by Ward et al.). The comparisons will be constant vs. IDH1 mutant, constant vs. IDH2 mutant, and constant vs. IDH1/2 pooled mutants.

2) Rather than combining the IDH1 mutant and IDH2 mutant patient samples into a single group (table after “Power calculations” in subheading “Protocol 3: Figure 5C”), it will be useful to also consider them as two separate groups in addition to IDH1+IDH2, as the difference in subcellular localization may lead to differences in D2HG abundance, and it would be worth knowing if there are differences in IDH1 mutants vs. IDH2 mutants, in addition to control comparisons. This would obviously change the statistical comparison needed to analyze the data.

We apologize for the confusion here. There was a typo in the table. Only IDH2 mutants will be combined for all analyses because of the likely low percentage of samples that will harbor the IDH2R172 allele. Further we plan on combining all IDH mutation and compare them to WT. With regard to the IDH1 vs. IDH2 mutants, because the Ward et al. paper never made these comparisons this analysis would be outside of the scope of the proposal. However, all data from this project will be made publically available to allow for further exploratory analysis such as this.

For your interest, we wanted to communicate to you some insightful suggestions that the reviewers had to improve the impact of the findings of the analyses beyond the original study. We appreciate that that is not the intent of the reproduction goal of the Registered report, but we include those ideas here as they may be worthy of your consideration for increasing the impact of this study and/or your future work:

1) It may be useful to include IDH2 R140Q mutants in the cell based studies, as it is likely that some AML patient samples will harbor that specific mutation.

We apologize if this was not made clear initially. Any IDH2 R140Q mutants that are identified in our sampling will be included in the replication of original Figure 5C. An addition has been made to the Sampling section of Protocol 3 to clarify this.

2) An estimation of non-IDH1,2-formed 2-HG levels would be useful.

We agree with the reviewers that this would be an interesting investigation. While outside of the scope of this specific project, all data from GC-MS will be made publically available to allow for further exploratory analysis.

3) Distinction between the R and S chiral forms of 2-HG would be useful.

We agree with the reviewers that this would be an interesting investigation. While outside of the scope of this specific project, all data from GC-MS will be made publically available to allow for further exploratory analysis.

4) Concerning the Protocol 1, i.e. replication of the Figure 2 experiments from the original article, this test represents a biochemical assays of somewhat "reconstituted" WT or mutant IDH2 forms and is of the appropriate design. The design could be improved only by a few variations of the overexpressed IDH2, either WT or mutant, to better correlate the possible variations due to the different levels of expression between WT or mutant. The statistics to be performed is sufficient to encompass the quality of the data and results even if the overexpression strength is not varied.

We agree that investigating the dose dependent impact of IDH overexpression on NADP-NADPH and NADPH-NADP would be interesting, however it is outside of the scope of this project as the original authors did not perform this experiment.

5) Concerning the Protocol 2, i.e. replication of the Figure 3 experiments there is a problem whether "mere reproducibility" is tested or whether a scientific design of this experiment is to be scrutinized. In the former case, assessment is similar as for the Protocol 1. In the latter case however, the experimental design cannot exclude the possibility that WT enzyme does not form 2-HG, since conditions for this counter-Krebs cycle direction of reaction are not set up in the experimental design. Actually such conditions were set up if 2-HG was assayed in experiments similar to those in protocol 1, but long-term - where both directions of reaction are assayed in parallel. The Protocol 2 also cannot rule out the possibility, that mutant enzyme occurring under conditions leading to the forward-Krebs cycle direction is as inefficient in synthesizing 2-HG as the WT.

We agree with the reviewers that this could be the case and we have done our best to design the experiments detailed in this protocol to be a “direct” replication. Study design is always an aspect to be investigated but like the other analyses, all data will be made public so that future analyses can take into account these aspects.

6) Concerning the Protocol 3, i.e. replication of the Figure 5 experiments analyzing the AML patient samples the precise error-free identification of mutations is the key to good reproducibility. Since this effort should rather mimic a clinical study, you should do the best to maximize the quality of these replication analyses and should substantially increase the number of samples per group. The suggested 4 samples per group are fulfilling a power of 80%, which however only represents a single repeat of the figure from the original publication but not validation of the possible diagnostic testing.

We understand the reviewers’ concerns about identification of mutational status and believe this has essential clinical implications. Here, our replication attempt is meant specifically to address the difference in 2HG/glutamate in IDH mutants vs. controls as reported in the original paper and not necessarily to investigate the precision of using 2HG/glutamate to identify IDH mutations, which we agree would require a larger sample size. The cell lines that we are obtaining are Roswell Park Cancer Institute and mutations have been confirmed multiple times on different platforms. Based on our power analyses, 4 samples should achieve 99% power to detect the published differences in 2HG/glutamate between WT and mutant IDH, as the effect size is quite large. Please keep in mind that the scale of Figure 5C is logarithmic.

7) It would potentially be more beneficial to also focus on other aspects related to D or L2HG and IDHs such as the effects on epigenetics and the therapeutic potential of targeting 2HG-mediated epigenetic alterations.

We agree that these are interesting avenues of investigation, however, they are outside of the scope of this project.

https://doi.org/10.7554/eLife.12626.003

Article and author information

Author details

  1. Oliver Fiehn

    West Coast Metabolomics Center, University of California, Davis, Davis, United States
    Contribution
    OF, Drafting or revising the article
    Competing interests
    OF: West Coast Metabolomics Center is a Science Exchange associated laboratory
  2. Megan Reed Showalter

    West Coast Metabolomics Center, University of California, Davis, Davis, United States
    Contribution
    MRS, Drafting or revising the article
    Competing interests
    MRS: West Coast Metabolomics Center is a Science Exchange associated laboratory
  3. Christine E Schaner-Tooley

    University of Louisville School of Medicine, Louisville, United States
    Contribution
    CES-T, Drafting or revising the article
    Competing interests
    No competing interests declared.
  4. Reproducibility Project: Cancer Biology

    Contribution
    RP:CB, Conception and design, Drafting or revising the article
    For correspondence
    stephen@cos.io
    Competing interests
    RP:CB: EI, FT, JL, NP: Employed by and hold shares in Science Exchange Inc.
    1. Elizabeth Iorns, Science Exchange, Palo Alto, United States
    2. William Gunn, Mendeley, London, United Kingdom
    3. Fraser Tan, Science Exchange, Palo Alto, United States
    4. Joelle Lomax, Science Exchange, Palo Alto, United States
    5. Stephen Williams, Center for Open Science, Charlottesville, Virginia
    6. Nicole Perfito, Science Exchange, Palo Alto, United States
    7. Timothy Errington, Center for Open Science, Charlottesville, Virginia

Funding

The Reproducibility Project: Cancer Biology is funded by the Laura and John Arnold Foundation, provided to the Center for Open Science in collaboration with Science Exchange. The funder had no role in study design or the decision to submit the work for publication.

Acknowledgements

The Reproducibility Project: Cancer Biology core team would like to thank Courtney Soderberg at the Center for Open Science for assistance with statistical analyses. We would also like to thank Kermit L. Carraway III, Kacey Vandervorst, and Jason Hatakeyama from the Department of Biochemistry and Molecular Medicine at UC Davis and the UC Davis Comprehensive Cancer Center for methods consultation. The following companies generously donated reagents to the Reproducibility Project: Cancer Biology; American Type and Tissue Collection (ATCC), Applied Biological Materials, BioLegend, Charles River Laboratories, Corning Incorporated, DDC Medical, EMD Millipore, Harlan Laboratories, LI-COR Biosciences, Mirus Bio, Novus Biologicals, Sigma-Aldrich, and System Biosciences (SBI).

Reviewing Editor

  1. Jessica K Tyler, Reviewing Editor, Weill Cornell Medical College, United States

Publication history

  1. Received: October 30, 2015
  2. Accepted: February 13, 2016
  3. Version of Record published: February 26, 2016 (version 1)

Copyright

© 2016, Fiehn et al.

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 906
    Page views
  • 172
    Downloads
  • 1
    Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.

Comments

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Download citations (links to download the citations from this article in formats compatible with various reference manager tools)

Open citations (links to open the citations from this article in various online reference manager services)

Further reading

  1. PS Ward et al.
    1
    1. Biochemistry
    2. Cancer Biology
    Megan Reed Showalter et al.
    Replication Study

    In 2016, as part of the Reproducibility Project: Cancer Biology, we published a Registered Report (Fiehn et al., 2016), that described how we intended to replicate selected experiments from the paper "The common feature of leukemia-associated IDH1 and IDH2 mutations is a neomorphic enzyme activity converting alpha-ketoglutarate to 2-hydroxyglutarate" (Ward et al., 2010). Here, we report the results of those experiments. We found that cells expressing R172K mutant IDH2 did not display isocitrate-dependent NADPH production above vector control levels, in contrast to the increased production observed with wild-type IDH2. Conversely, expression of R172K mutant IDH2 resulted in increased alpha-ketoglutarate-dependent consumption of NADPH compared to wild-type IDH2 or vector control. These results are similar to those reported in the original study (Figure 2; Ward et al., 2010). Further, expression of R172K mutant IDH2 resulted in increased 2HG levels within cells compared to the background levels observed in wild-type IDH2 and vector control, similar to the original study (Figure 3D; Ward et al., 2010). In primary human AML samples, the 2HG levels observed in samples with mutant IDH1 or IDH2 status were higher than those observed in samples without an IDH mutation, similar to what was observed in the original study (Figure 5C; Ward et al., 2010). Finally, we report meta-analyses for each result.

    1. Biochemistry
    2. Biophysics and Structural Biology
    G Elif Karagöz et al.
    Research Article Updated