1. Biochemistry
  2. Human Biology and Medicine
Download icon

Registered report: Oncometabolite 2-hydroxyglutarate is a competitive inhibitor of α-ketoglutarate-dependent dioxygenases

  1. Brad Evans
  2. Erin Griner
  3. Reproducibility Project: Cancer Biology Is a corresponding author
  1. Donald Danforth Plant Science Center, United States
  2. University of Virginia, United States
Registered Report
Cited
1
Views
915
Comments
0
Cite as: eLife 2015;4:e07420 doi: 10.7554/eLife.07420

Abstract

The Reproducibility Project: Cancer Biology seeks to address growing concerns about reproducibility in scientific research by conducting replications of selected experiments from a number of high-profile papers in the field of cancer biology. The papers, which were published between 2010 and 2012, were selected on the basis of citations and Altmetric scores (Errington et al., 2014). This Registered report describes the proposed replication plan of key experiments from ‘Oncometabolite 2-hydroxyglutarate is a competitive inhibitor of α-ketoglutarate-dependent dioxygenases’ by Xu and colleagues, published in Cancer Cell in 2011 (Xu et al., 2011). The key experiments being replicated include Supplemental Figure 3I, which demonstrates that transfection with mutant forms of IDH1 increases levels of 2-hydroxyglutarate (2-HG), Figures 3A and 8A, which demonstrate changes in histone methylation after treatment with 2-HG, and Figures 3D and 7B, which show that mutant IDH1 can effect the same changes as treatment with excess 2-HG. The Reproducibility Project: Cancer Biology is a collaboration between the Center for Open Science and Science Exchange, and the results of the replications will be published by eLife.

https://doi.org/10.7554/eLife.07420.001

Introduction

Mutations in IDH1 and IDH2 are found in gliomas and in acute myeloid leukemia. All mutations are heterozygous and result in changes to one of two amino acids: arginine 132 in IDH1, or either arginine 172 or arginine 140 in IHD2. Wild-type IDH1 catalyzes the conversion of isocitrate to α-ketoglutarate (α-KG). The arginine mutations abolish its normal activity and instead mutant IDH1 and IDH2 reduce α-KG to generate the oncometabolite 2-hydroxyglutarate (2-HG) (Ward et al., 2010), which in turn affects the function of multiple α-KG dependent dioxygenases, including the TET family of 5-methylcytosine (5 mC) hydroxylases (Kinney and Pradhan, 2012; McKenney and Levine, 2013). In their Cancer Cell 2011 paper, Xu and colleagues examined the effects of excess production of 2-HG on downstream processes that could affect cancer progression. They showed that 2-HG could act as a competitive inhibitor for α-KG-dependent DNA demethylases, specifically Tet2. Ectopic expression of the mutant forms of IDH1 and IDH2 inhibited histone demethylation and 5mC hydroxylation. Examination of glioma samples from patients also showed that mutations in IDH1 were associated with increased histone methylation and decreased 5-hydroxymethylcytosine (5hmC) levels (Xu et al., 2011).

In Supplemental Figure 3I, Xu and colleagues demonstrated that transfection of U-87 MG cells with the mutant IDH1R132H increased the amount of 2-HG in the cells, as compared to transfection with wild-type IDH1 (Xu et al., 2011). This is evidence that mutant IDH1 changes the physiological levels of 2-HG, and is replicated in Protocol 1.

Xu and colleagues first showed that 2-HG can occupy the same binding pocket as α-KG in Caenorhabditis elegans KDM7A, indicating it acts as a competitive inhibitor of α-KG. Importantly, they also presented evidence that 2-HG may outcompete α-KG, since 2-HG levels affected many enzymatic functions normally dependent on α-KG. In Figure 3A, they treated U-87 MG cells with cell permeable versions of α-KG and 2-HG, and examined levels of histone methylation by Western Blot. Treatment with increasing amounts of 2-HG led to increases in H3K9me2 and H3K79me2, consistent with the idea that 2-HG inhibited histone demethylases. This effect was abolished by co-treatment with α-KG, confirming a competitive relationship between the two metabolites (Xu et al., 2011). This experiment is replicated in Protocol 2. Xu and colleagues also examined the effect of 2-HG on the TET family of 5 mC hydroxylases using an in vitro system of purified TET2 and double-stranded oligos containing a 5mC restriction digestion site in Figure 8A. Adding increasing concentrations of 2-HG abolished the ability of TET2 to convert 5 mC to 5hmC (Xu et al., 2011). This experiment will be replicated in Protocol 5.

In addition to demonstrating that the metabolite 2-HG can affect the activity of α-KG-dependent enzymes, Xu and colleagues showed that treatment with mutant forms of IDH1 and IDH2 resulted in similar outcomes. In Figure 3D, they transfected U-87 MG cells with IDH1R132H and assessed levels of histone methylation by Western blot. Transfection with IDH1R132H increased histone methylation, and treatment with α-KG abolished this increase in histone methylation, consistent with the idea that α-KG and 2-HG are competitive metabolites (Xu et al., 2011). This experiment will be replicated in Protocol 3. In Figure 7B, they also examined TET activity in the presence of mutant IDH1. While 5hmC levels are normally undetectable in HEK293 cells, transfection with TET catalytic domain (CD)-expressing plasmids increased 5hmC levels to detectable amounts. Co-transfection of TET-CD and wild-type IDH1 or IDH2 increased levels of 5hmC, as expected, while co-transfection of TET-CD with mutant forms of IDH1 and IDH2 decreased 5hmC levels (Xu et al., 2011). This experiment is replicated in Protocol 4.

The work of Xu and colleagues (Xu et al., 2011), along with work from Figueroa and colleagues (Figueroa et al., 2010) and Lu and colleagues (Lu et al., 2012), has generated much interest in the role of altered metabolites in the changing methylation patterns seen in various types of cancer. Using a different cell line than Xu and colleagues, Lu and colleagues demonstrated that mutations in IDH2, similar to mutations in IDH1, also generated abnormal levels of 2-HG which correlated with increased global methylation levels (Lu et al., 2012). Kernystsky and colleagues, Duncan and colleagues and Turcan and colleague have also shown that expression of exogenous mutated IDH genes in immortalized human cancer cell lines or in erythroid progenitor cells caused increased production of 2HG and increased levels of methylation (Duncan et al., 2012; Turcan et al., 2012; Kernytsky et al., 2015). Sasaki and colleagues extended these inquiries by generating conditional knock-in IDH1 mutant mice. These mice displayed elevated serum levels of 2HG and similar patterns of hypermethylation as observed in AML patients (Sasaki et al., 2012). Akbay and colleagues generated IDH2 mutant mice and also observed an increase in global methylation in heart tissue. They also demonstrated that mice carrying IDH mutant xenograft tumors displayed higher serum levels of 2HG (Akbay et al., 2014). Recently, 2-HG production has also been associated with MYC activation in some breast cancers, which also displayed increased levels of methylation as compared to tumors with lower levels of 2-HG (Terunuma et al., 2013).

Materials and methods

Unless otherwise noted, all protocol information was derived from the original paper, references from the original paper, or information obtained directly from the authors. An asterisk (*) indicates data or information provided by the Reproducibility Project: Cancer Biology core team. A hashtag (#) indicates information provided by the replicating lab.

Protocol 1: Gas chromatography-mass spectrometry measurement of cellular α-KG and 2-HG concentrations in U87MG cells ectopically expressing mutant IDH1

This protocol describes how to transfect cells with exogenous wild-type IDH1 or mutant IDH1R132H and assess levels of α-KH and 2-HG by gas chromatography-mass spectrometry (GC-MS), as seen in Supplemental Figure 3I.

Sampling

  • This experiment will be repeated independently 5 times for a final power of at least 92%.

    • ○ See Power calculations for details.

  • Each experiment consists of three cohorts:

    • ○ Cohort 1: U-87 MG cells transfected with vector alone.

    • ○ Cohort 1: U-87 MG cells transfected with wild-type IDH1.

    • ○ Cohort 1: U-87 MG cells transfected with mutant IDH1.

    • Each cohort will be assessed via GC–MS for:

      • ■ α-KG levels.

      • ■ 2-HG levels.

Materials and reagents

ReagentTypeManufacturerCatalog #Comments
U-87 MG cellsCellsATCCHTB-14
N-methyl-N-[tert-butyldimethylsilyl] trifluoroacetamideChemicalSigma–Aldrich375934-10× 1 M
Agilent 6890-5973 gas chromatograph-mass spectrometerInstrumentAgilent6890-5973
HP-5MS columnMaterialAgilent19091S-433I
60 mm tissue culture dishesMaterialCorning430166
DMEM; high glucoseMediumSigma–AldrichD5671Original unspecified
Opti-MEM Reduced Serum MediumMediumLife Technologies31985-062Original unspecified
Vector only plasmid (GFP)PlasmidProvided by the original authors
IDH1-IRES-GFP vectorPlasmidProvided by the original authors
IDH1R132H-IRES-GFP vectorPlasmidProvided by the original authors
FBSReagentSigma–AldrichF2442Original unspecified
Trypsin-EDTA solution, 1×ReagentATCCATCC-30-2101
Penicillin-streptomycin solutionReagentATCCATCC-30-2300
TransIT®-LT1 transfection reagentReagentMirus BioMIR 2300Replaces SunBio-EZ (SunBio)
Methoxyamine hydrochlorideReagentSigma–Aldrich226904
PyridineReagentSigma–Aldrich33,553
GenElute Endotoxin-free Plasmid Maxiprep KitKitSigma–AldrichPLEX15-1KT
α-KGChemicalSigma–Aldrich75892
L-2-HGChemicalSigma–Aldrich90790
0.2 µm filter vialsMaterialRestek25893
CentrivapEquipmentLabonco
Anti-GAPDH-HRPAntibodyAbcamab9385
Mouse monoclonal IgG1 α IDH1AntibodyAbcamab117976The original catalog number was not specified

Procedure

Notes

  • U-87 MG cells are maintained in DMEM supplemented with 10% FBS at 37°C/5% CO2.

  • All cells will be sent for mycoplasma testing and STR profiling.

  1. Transform, grow up and maxiprep vector only (GFP), IDH1-IRES-GFP, and IDH1R132H-IRES-GFP plasmids using a Endo-free Maxiprep kit following manufacturer's instructions.

    • a. Confirm plasmid identity by sequencing.

  2. Plate U-87 MG cells in 60 mm dishes.

    • a. First, run an optimization to determine the growth rate of the cells and optimal number of cells per plate for transfection.

  3. 24 hr after plating, transfect cells with plasmids using TransIT-LT1 transfection reagent and Opti-MEM medium according to the manufacturer's protocol.

    • a. Transfect 8 µg of DNA per construct using appropriate amount of transfection reagent.

      • i. Vector only (GFP).

      • ii. Wild-type IDH1 (IDH1-IRES-GFP).

      • iii. Mutant IDH1 (IDH1R132H-IRES-GFP).

    • b. Prepare two plates per cohort; one will be harvested for Western blot confirmation of protein expression (Step 4), the other will be used for metabolite analyses (Step 5).

  4. For Western blot: 48 hr after transfection, confirm protein expression by Western blot.

    Note: perform each time cells are transfected.

    • a. Run Western blot as outlined in Protocol 2 Steps 3 through 17 with the following modifications:

      • i. Blots do not need to be stripped and re-probed.

      • ii. Blots will be probed with:

        1. Anti-IDH1; diluted according to the manufacturer's recommendation.

        2. Anti-GAPDH-HRP; 1:5000.

          • a. Loading control.

  5. For metabolite analysis: 24 hr after transfection, remove culture medium, wash cells with cold PBS and immediately add 10 ml of pre-chilled (−80°C) 80% (vol/vol) methanol. Harvest cells by scraping and lyophilize following the manufacturer's instructions.

    • a. Samples will be lyophilized in a speedvac with no heating to keep samples frozen throughout. Immediately after drying remove samples from the speedvac for derivitization.

  6. Oximate lyophilized samples with 20 µl 20 mg/ml methoxyamine hydrochloride in pyridine at 30°C for 60 min.

  7. Derivatize samples for 30 min at 70°C in 80 μl pyridine and 20 μl N-methyl-N-[tert-butyldimethylsilyl] trifluoroacetamide.

  8. Filter samples using 0.2 µm filter vials (PTFE).

  9. Inject 3 µl of samples for gas chromatography-mass spectrometry analysis (GC–MS) into Agilent 6890-5973 GC–MS. Use a HP-5MS column (30 m 0.25 mm 0.25 μm) for analysis. Program GC oven temperature from 60°C to 180°C at 5°C/min and from 180°C to 260°C at 10°C/min. Set the flow rate of carrier gas at 1 ml/min. Operate the mass spectrometer in the electron impact (EI) mode at 70 eV.

  10. Calculate relative α-KG and 2-HG concentrations by normalizing α-KG (29.86 min) and 2-HG (30.10 min) peak areas to the average of L-threonine (29.58 min), L-serine (29.96 min) and L-phenylalanine (30.74 min) peak areas.

  11. Repeat independently four additional times.

Deliverables

  • Data to be collected:

    • ○ Chromatograms and sequence files confirming plasmid identity.

    • ○ Data generated determining growth optimization.

    • ○ Full image of Western blot showing protein expression and loading controls.

    • ○ Mass spectra readouts of all samples.

    • ○ Raw values of peak areas for α-KG (29.86 min), 2-HG (30.10 min), L-threonine (29.58 min), L-serine (29.96 min) and L-phenylalanine (30.74 min).

    • ○ Quantification of average of peak areas for L-threonine (29.58 min), L-serine (29.96 min) and L-phenylalanine (30.74 min).

    • ○ Quantification of relative α-KG and 2-HG concentrations by normalization to average peak areas for L-threonine (29.58 min), L-serine (29.96 min) and L-phenylalanine (30.74 min).

    • ○ Bar graphs of relative α-KG or 2-HG concentrations (in percent) for each cell line (as in Supplemental Figure 3I).

Confirmatory analysis plan

  • Statistical analysis of the replication data:

    • ○ Note: At the time of analysis we will perform the Shapiro–Wilk test and generate a quantile–quantile plot to assess the normality of the data. We will also perform Levene's test to assess homoscedasticity. If the data appears skewed we will perform the appropriate transformation in order to proceed with the proposed statistical analysis. If this is not possible we will perform the equivalent non-parametric test.

    • ○ One-way MANOVA of α-KG and 2-HG levels in vector-transfected, IDH1-wildtype transfected, and IDH1R132H-transfected cells with the following Bonferroni corrected comparisons:

      • ■ α-KG levels planned comparisons:

        • vector vs IDH1WT.

        • vector vs IDHR132H.

        • IDH1WT vs IDHR132H.

      • ■ 2-HG levels planned comparisons:

        • vector vs IDH1WT.

        • vector vs IDHR132H.

        • IDH1WT vs IDHR132H.

  • Meta-analysis of original and replication attempt effect sizes:

    • ○ Compute the effect sizes of each comparison, compare them against the reported effect size in the original paper and use a meta-analytic approach to combine the original and replication effects, which will be presented as a forest plot.

Known differences from the original study

  • Aspects of the Western blot protocol are provided by the replicating lab; complete details of the original protocol were unavailable.

  • Since the cell density during transfection is unknown in the original paper, the replicating lab will optimize growth conditions and cell density for transfection.

Provisions for quality control

All data obtained from the experiment—raw data, data analysis, control data and quality control data—will be made publicly available, either in the published manuscript or as an open access dataset available on the Open Science Framework (https://osf.io/kvshc/).

  • Sequence data confirming plasmid identity.

  • Western blots confirming exogenous protein expression.

  • STR profiling confirming cell line authenticity.

  • Mycoplasma testing confirming lack of contamination.

  • Growth characteristics of the cells will be optimized.

Protocol 2: Western blot to assess histone methylation in U-87 MG cells following treatment with oct-2-HG and/or oct-α-KG

This protocol describes how to treat U-87 MG cells with cell permeable versions of 2-HG and α-KG and assess histone methylation via Western blot, as seen in Figure 3A and Supplemental Figure 3F.

Sampling

  • The experiment will be repeated independently 3 times for a final power of 84%.

    • ○ See Power calculations for details.

  • Each experiment consists of four cohorts:

    • ○ Cohort 1: untreated U-87 MG cells.

    • ○ Cohort 2: U-87 MG cells treated with 10 mM racemic Oct-2-HG.

    • ○ Cohort 3: U-87 MG cells treated with 20 mM racemic Oct-2-HG.

    • ○ Cohort 4: U-87 MG cells treated with 20 mM racemic Oct-2-HG and 5 mM oct-α-KG.

    • Each sample will be blotted for:

      • ■ H3K9me2.

      • ■ H3K79me2.

      • ■ H3.

Materials and reagents

ReagentTypeManufacturerCatalog #Comments
Mouse monoclonal anti-H3K9me2AntibodyAbcamAb1220The original catalog number was not specified
Mouse monoclonal anti -H3K79me2AntibodyAbcamAb3594The original catalog number was not specified
Mouse monoclonal anti -H3AntibodyAbcamab10799The original catalog number was not specified
Goat Anti-Mouse IgG H&L (HRP)AntibodyAbcamab97023We will use this for all mouse primaries
U-87 MG cellsCellsATCCHTB-14
60 mm tissue culture dishesMaterialCorning430166
DMEM; high glucoseMediumSigma–AldrichD5671Original unspecified
FBSReagentSigma–AldrichF2442Original unspecified
Oct-α-KGReagentCayman Chemical11970
2S(L)-Oct-2-HGReagentTRCH942596Original synthesized in house
2R(L)-Oct-2-HGReagentTRCH942595
Protease inhibitor cocktail (mammalian)ReagentSigma–AldrichP8340-1MLOriginal not specified
TruPAGE TEA-tricine SDS running buffer (20×)ReagentSigma–AldrichPCG3001-500 MLOriginal not specified
TruPAGE LDS sample buffer (4×)ReagentSigma–AldrichPCG3009-10 MLOriginal not specified
TruPAGE DTT sample reducer (10×)ReagentSigma–AldrichPCG3005-1MLOriginal not specified
TruPAGE transfer buffer (20×)ReagentSigma–AldrichPCG3011-500 MLOriginal not specified
PBS, without MgCl2 and CaCl2ReagentSigma–AldrichD8537Original not specified
Hybond ECL nitrocellulose membranes; 20 cm × 20 cmReagentGE Healthcare (Sigma–Aldrich)GERPN2020DOriginal not specified
Ponceau S solution; 0.1% (wt/vol) in 5% acetic acidReagentSigma–AldrichP7170Original not specified
Tris Buffered Saline (TBS); 10× solutionReagentSigma–AldrichT5912Original not specified
Bradford reagentReagentSigma–AldrichB6916Original not specified
ECL DualVue Western Blotting MarkersReagentGE Healthcare (Sigma–Aldrich)GERPN810Original not specified
ECL Prime Western blotting systemReagentGE Healthcare (Sigma–Aldrich)GERPN2232Original not specified
ImageQuantSoftwareMolecular DynamicsVersion 5.2
Typhoon scannerEquipmentGE Healthcare

Procedure

Notes

  • U-87 MG cells are maintained in DMEM supplemented with 10% FBS at 37°C/5% CO2.

  • All cells will be sent for mycoplasma testing and STR profiling.

  1. Plate U-87 MG cells in 60 mm dishes.

  2. 24 hr after plating, treat cells with 10 or 20 mM racemic Oct-2-HG or 5 mM Oct-α-KG or vehicle (DMSO) for 4–6 hr.

    • a. To form racemic mixtures of Oct-2-HG, mix equal amounts of the L and R enantiomers.

  3. Wash cells once with cold PBS, then lyse cells in 0.5 mL of SDS loading buffer.

    • a. 4× SDS-PAGE loading buffer: 50 mM Tris pH 6.8, 2% SDS, 10% glycerol, 1% B-ME, 12.5 mM EDTA, 0.02% bromophenol blue.

    • b. #Measure protein concentration using a CBX assay.

  4. Heat lysates at 99°C for 10 min.

  5. Run equal amounts of protein per well on a 4–20% SDS-PAGE gel at 220V until ladder marker reaches the bottom of the gel.

  6. #Equilibrate gel in transfer buffer for 15 min.

  7. #Meanwhile, cut membrane and 4 pieces 3 MM filter paper to size of gel.

    • a. Soak membrane in MeOH for a few seconds, then wash with H2O.

    • b. Soak membrane, 3 MM filter paper and pads in transfer buffer.

      • i. Transfer buffer: 38 mM glycine, 47 mM Tris, 11 mM SDS, 20% MeOH.

  8. #Assemble transfer cassette:

    • a. red pole (+) < clear plate < pad < 2 × 3 MM filter paper < membrane < gel < 2 × 3 MM filter paper < pad < black pole (−).

  9. #Add stirring bar and ice box to transfer box and fill box with transfer buffer until cassette is submerged.

    • a. Run at 100 V for 1 hr.

  10. #Wash membrane in wash buffer for 2 × 5 min.

    • a. Wash buffer: 1× PBS with 0.05% Tween-20 and 0.1% sodium azide.

  11. #Incubate membrane in blocking buffer for 30 min.

    • a. Blocking buffer: 3% non-fat milk in PBS.

  12. #Incubate membrane with one of the following primary antibody in blocking buffer for 2 hr at RT or O/N at 4°C (use manufacturer's suggested dilution in blocking buffer).

    • a. H3K9me2.

    • b. H3K79me2.

    • c. H3.

      • i. See Step 17 to strip and re-probe the blot with subsequent antibodies.

  13. #Wash 5 min 2× with wash buffer.

  14. #Incubate membrane with secondary antibody for 90 min at RT (use manufacturer's suggested dilution in blocking buffer).

    • a. HRP-conjugated Goat Anti-Mouse IgG H&L: 1: 2000.

  15. #Wash 3 × 5 min in wash buffer.

  16. #Detect HRP-conjugated secondary antibodies with chemiluminescent detection according to the manufacturer's protocol and image on the Typhoon scanner.

  17. Strip the blot in between probes:

    • a. Wash the membrane with 100 ml stripping buffer (100 mM beta-mercaptoethanol, 1% SDS 25 mM glycine pH 2.0) for 30 min with agitation.

    • b. Wash the stripped membrane twice with Western blotting wash buffer, 600 ml each wash, for 10 min with agitation.

    • c. Go to the blocking step of the western blot protocol.

    • d. Check that stripping was successful by repeating the detection step (without re-probing). Record image of the stripped gel. This will confirm the first antibody-HRP conjugate is removed and/or inactivated. If the stripping procedure is successful, wash the membrane with washing buffer and repeat the blocking-probing and detection steps for the second antibody.

      • i. Note: if stripping is unsuccessful, individual blots will be performed.

  18. Quantify intensity of bands on western blots using ImageQuant 5.2. Normalize H3k9me2 and H3K79me2 values to total H3 protein level.

  19. Repeat independently 2 additional times.

Deliverables

  • Data to be collected:

    • ○ Full scans of western blots for H3K9me2, H3K79me2 and H3 including ladder.

    • ○ Raw values of intensity of western blot bands.

    • ○ Quantification of H3K9me2 or H3K79me2 values normalized to total protein level. Levels of H3K9me2 and H3K79me2 in vehicle treated cells are set to relative intensity = 1 and all other conditions are expressed as fold change relative to the values for vehicle treated cells.

    • ○ Quantification of average values and standard deviations for each condition for triplicate experiments.

    • ○ Bar graph of average ± standard deviation of H3K9me2 and H3K79me2 levels normalized to H3 for each condition. Fold change in intensity relative to vehicle treated cells is plotted on the y axis (as seen in Supp. Figure 3F).

Confirmatory analysis plan

  • Statistical analysis of the replication data:

    • ○ Note: At the time of analysis we will perform the Shapiro–Wilk test and generate a quantile–quantile plot to assess the normality of the data. We will also perform Levene's test to assess homoscedasticity. If the data appears skewed we will perform the appropriate transformation in order to proceed with the proposed statistical analysis. If this is not possible we will perform the equivalent non-parametric test.

    • ○ One-way MANOVA of normalized H3K9me2 and H3K79me3 levels in U-87 MG cells untreated or treated with 10 mM Oct-2-HG, 20 mM Oct-2-HG, or 20 mM Oct-2-HG and 5 mM alpha-KG with the following Bonferroni corrected comparisons:

      • ■ H3K9me2 planned comparisons:

        • 0 mM 2-HG vs 10 mM 2-HG.

        • 0 mM 2-HG vs 20 mM 2-HG.

        • 20 mM 2-HG vs 5 mM α-KG + 20 mM 2-HG.

      • ■ H3K79me3 planned comparisons:

        • 0 mM 2-HG vs 10 mM 2-HG.

        • 0 mM 2-HG vs 20 mM 2-HG.

        • 20 mM 2-HG vs 5 mM α-KG + 20 mM 2-HG.

  • Additional statistical analysis for comparison to the original reported data:

    • ○ Bonferroni corrected one-sample t-tests of normalized H3K9me2 levels of the following conditions compared to 1 (0 mM 2-HG):

      • ■ 10 mM 2-HG.

      • ■ 20 mM 2-HG.

    • ○ Bonferroni corrected one-sample t-tests of normalized H3K79me3 levels of the following conditions compared to constant (0 mM 2-HG set to 1):

      • ■ 10 mM 2-HG.

      • ■ 20 mM 2-HG.

  • Meta-analysis of original and replication attempt effect sizes:

    • ○ Compute the effect sizes of each comparison, compare them against the reported effect size in the original paper and use a meta-analytic approach to combine the original and replication effects, which will be presented as a forest plot.

Known differences from the original study

  • The original racemic mixture of Oct-2-HG was synthesized in house by the original lab. The replicating lab is purchasing both L and R enantiomers and mixing them in equal amounts to form a racemic mixture.

  • Aspects of the Western blot protocol are provided by the replicating lab; complete details of the original protocol were unavailable.

Provisions for quality control

All data obtained from the experiment—raw data, data analysis, control data and quality control data—will be made publicly available, either in the published manuscript or as an open access dataset available on the Open Science Framework (https://osf.io/kvshc/).

  • STR profiling confirming cell line authenticity.

  • Mycoplasma testing confirming lack of contamination.

  • Images of stripped gel membranes confirming stripping was successful.

Protocol 3: Transfection of U-87 MG cells and determination of histone methylation by western blot

This protocol describes the transfection of U-87 MG cells with the mutant form of IDH1 and assessing methylation by Western blot, as seen in Figure 3D and Supplemental Figure 3J.

Sampling

  • This experiment will be repeated independently 6 times for a final power of 94%.

    • ○ See Power calculations for details.

  • Each experiment consists of 5 cohorts:

    • ○ Cohort 1: untransfected cells [additional control].

    • ○ Cohort 2: Vector transfected cells [additional control].

    • ○ Cohort 3: Vector transfected cells + vehicle.

    • ○ Cohort 4: IDH1R132H transfected cells + vehicle.

    • ○ Cohort 5: IDH1R132H transfected cells + 5 mM oct-α-KG.

    • ○ Each cohort is probed with antibodies against:

      • ■ H3.

      • ■ IDH1.

      • ■ H3K4me1.

      • ■ H3K4me3.

      • ■ H3K9me2.

      • ■ H3K27me2.

Materials and reagents

ReagentTypeManufacturerCatalog #Comments
Mouse monoclonal IgG3 α H3AntibodyAbcamab10799The original catalog number was not specified
Mouse monoclonal IgG1 α IDH1AntibodyAbcamab117976The original catalog number was not specified
Rabbit α H3K4me1AntibodyAbcamab8895The original catalog number was not specified
Mouse monoclonal IgG2b α H3K4me3AntibodyAbcamab6000The original catalog number was not specified
Mouse monoclonal IgG2a α H3K9me2AntibodyAbcamab1220The original catalog number was not specified
Rabbit α H3K27me2AntibodyAbcamab24684The original catalog number was not specified
Rabbit α H3K79me2AntibodyAbcamab3594The original catalog number was not specified
U-87 MG cellsCellsATCCHTB-14
60 mm tissue culture dishesMaterialCorning430166Or equivalent
DMEM; high glucoseMediumSigma–AldrichD5671Original unspecified
FBSReagentSigma–AldrichF2442Original unspecified
Empty vector plasmidPlasmidProvided by original authors
IDH1R132H expression vectorPlasmidProvided by original authors
TransIT®-LT1 transfection reagentReagentMirus BioMIR 2300Replaces SunBio-EZ (SunBio)
Oct-α-KGReagentCayman Chemical11970
Typhoon scannerEquipmentGE Healthcare
ImageQuantSoftwareMolecular DynamicsVersion 5.2
Protease inhibitor cocktail (mammalian)ReagentSigma–AldrichP8340-1MLOriginal not specified
TruPAGE TEA-tricine SDS running buffer (20×)ReagentSigma–AldrichPCG3001-500 MLOriginal not specified
TruPAGE LDS sample buffer (4×)ReagentSigma–AldrichPCG3009-10 MLOriginal not specified
TruPAGE DTT sample reducer (10×)ReagentSigma–AldrichPCG3005-1MLOriginal not specified
TruPAGE transfer buffer (20×)ReagentSigma–AldrichPCG3011-500 MLOriginal not specified
PBS, without MgCl2 and CaCl2ReagentSigma–AldrichD8537Original not specified
Hybond ECL nitrocellulose membranes; 20 cm × 20 cmReagentGE Healthcare (Sigma–Aldrich)GERPN2020DOriginal not specified
Ponceau S solution; 0.1% (wt/vol) in 5% acetic acidReagentSigma–AldrichP7170Original not specified
Tris buffered saline (TBS); 10× solutionReagentSigma–AldrichT5912Original not specified
Bradford reagentReagentSigma–AldrichB6916Original not specified
ECL DualVue Western blotting markersReagentGE Healthcare (Sigma–Aldrich)GERPN810Original not specified
ECL prime Western blotting systemReagentGE Healthcare (Sigma–Aldrich)GERPN2232Original not specified
Goat Anti-Rabbit IgG H&L (HRP)AntibodyAbcamab97051
Goat Anti-Mouse IgG H&L (HRP)AntibodyAbcamab97023

Procedure

Notes

  • U-87 MG cells are maintained in DMEM supplemented with 10% FBS at 37°C/5% CO2.

  • All cells will be sent for mycoplasma testing and STR profiling.

  1. Plate U-87 MG cells in 60 mm dishes.

  2. 24 hr after plating, transfect cells with plasmids (maxiprepped in Protocol 1) using TransIT-LT1 Transfection Reagent according to manufacturer's protocol.

    • a. #Transfect 8 µg of DNA per construct using appropriate volume of transfection reagent.

      • i. Empty vector.

      • ii. IDH1R132H vector.

  3. 48 hr after transfection, treat cells with vehicle or 5 mM Oct-α-KG for 6 hr.

    • a. Vehicle is DMSO.

  4. Wash cells once with cold PBS, then lyse cells in 0.5 ml of SDS loading buffer.

    • a. #4 SDS-PAGE loading buffer: 50 mM Tris pH 6.8, 2% SDS, 10% glycerol, 1% B-ME, 12.5 mM EDTA, 0.02% bromophenol blue.

  5. Heat lysates at 99°C for 10 min.

  6. Run SDS-PAGE gel until ladder marker reaches the bottom of the gel.

  7. #Equilibrate gel in transfer buffer for 15 min.

  8. #Meanwhile, cut membrane and 4 pieces 3 MM filter paper to size of gel.

    • a. Soak membrane in MeOH for a few seconds, then wash with H2O.

    • b. Soak membrane, 3 MM filter paper and pads in transfer buffer.

      • i. Transfer buffer: 38 mM glycine, 47 mM Tris, 11 mM SDS, 20% MeOH.

  9. #Assemble transfer cassette:

    • a. red pole (+) < clear plate < pad < 2 × 3 MM filter paper < membrane < gel < 2 × 3 MM filter paper < pad < black pole (−).

  10. #Add stirring bar and ice box to transfer box and fill box with transfer buffer until cassette is submerged.

    • a. Run at 100 V for 1 hr.

  11. #Wash membrane in wash buffer for 2 × 5 min.

    • a. Wash buffer: 1× PBS with 0.05% Tween-20 and 0.1% sodium azide.

  12. #Incubate membrane in blocking buffer for 30 min.

    • a. Blocking buffer: 3% non-fat milk in PBS.

  13. #Incubate membrane with primary antibody in blocking buffer for 2 hr at room temperature (RT) or overnight at 4°C (use manufacturer's suggested dilution in blocking buffer).

    • a. H3.

    • b. IDH1.

    • c. H3K4me1.

    • d. H3K4me3.

    • e. H3K9me2.

    • f. H3K27me2.

    • g. H3K79me2.

  14. #Wash 5 min 2× with wash buffer.

  15. #Incubate membrane with secondary antibody for 90 min at RT (use manufacturer's suggested dilution in blocking buffer).

    • a. HRP-conjugated Goat Anti-Mouse IgG H&L: 1:2000.

    • b. HRP-conjugated Goat Anti-Rabbit IgG H&L: 1:2000.

  16. #Wash 3 × 5 min in wash buffer.

  17. # Detect HRP-conjugated secondary antibodies with chemiluminescent detection according to the manufacturer's protocol and image on the Typhoon scanner.

  18. Strip the blot in between probes:

    • a. Wash the membrane with 100 ml stripping buffer (100 mM betamercaptoethanol, 1% SDS 25 mM glycine pH 2.0) for 30 min with agitation.

    • b. Wash the stripped membrane twice with Western blotting wash buffer, 600 ml each wash, for 10 min with agitation.

    • c. Go to the blocking step of the western blot protocol.

    • d. Check that stripping was successful by repeating the detection step (without re-probing). Record image of the stripped gel. This will confirm the first antibody-HRP conjugate is removed and/or inactivated. If the stripping procedure is successful, wash the membrane with washing buffer and repeat the blocking-probing and detection steps for the second antibody.

      • i. Note: if stripping is unsuccessful, individual blots will be performed.

  19. Quantify intensity of bands on western blots using ImageQuant 5.2. Normalize levels of methylated histones to total H3 protein level. Normalize IDH1R132H + vehicle and IDH1R132H + oct-α-KG treated samples to vector + vehicle samples for each normalized methylated histone.

  20. Repeat independently 5 additional times.

Deliverables

  • Data to be collected:

    • ○ Full scans of western blots for H3, IDH1, H3K4me1, H3K4me3, H3K9me2, H3K27me2, and H3K79me2 (as seen in Figure 3D) including ladder.

    • ○ Raw values of intensity of western blot bands as measured by ImageQuant 5.2 software.

    • ○ Quantification of methylated histone values normalized to total protein level.

    • ○ Quantification of average values and standard deviations for each condition. Levels of methylated histone in vector control cells are set to 100% and levels of methylated histone for other conditions are relative to vector control.

    • ○ Table of average ± standard deviation of methylated histone levels normalized to H3 for each condition and relative to vector control cells (as seen in Supplemental Figure 3J).

Confirmatory analysis plan

  • Statistical analysis of the replication data:

    • ○ Note: At the time of analysis we will perform the Shapiro–Wilk test and generate a quantile–quantile plot to assess the normality of the data. We will also perform Levene's test to assess homoscedasticity. If the data appears skewed we will perform the appropriate transformation in order to proceed with the proposed statistical analysis. If this is not possible we will perform the equivalent non-parametric test.

    • ○ One-way MANOVA of normalized H3K4me1, H3K4me3, H3K9me2, H3K27me2, and H3K79me2 levels from IDH1R132H + vehicle and IDH1R132H + oct-α-KG cells with the following Bonferroni corrected comparisons:

      • ■ H3K4me1 levels of IDH1R132H vs IDH1R132H + oct-α-KG.

      • ■ H3K4me3 levels of IDH1R132H vs IDH1R132H + oct-α-KG.

      • ■ H3K9me2 levels of IDH1R132H vs IDH1R132H + oct-α-KG.

      • ■ H3K27me2 levels of IDH1R132H vs IDH1R132H + oct-α-KG.

      • ■ H3K79me2 levels of IDH1R132H vs IDH1R132H + oct-α-KG.

    • ○ Bonferroni corrected one-sample t-tests (outside the MANOVA framework) of normalized levels from IDH1R132H + vehicle of the following conditions compared to constant (vector + vehicle set to 100):

      • ○ H3K4me1.

      • ○ H3K4me3.

      • ○ H3K9me2.

      • ○ H3K27me2.

      • ○ H3K79me2.

  • Meta-analysis of original and replication attempt effect sizes:

    • ○ Compute the effect sizes of each comparison, compare them against the reported effect size in the original paper and use a meta-analytic approach to combine the original and replication effects, which will be presented as a forest plot.

Known differences from the original study

  • While the manufacturer was specified for antibodies used, the exact catalog number was not. The RP:CB core team chose the most appropriate antibody from the manufacturer based on manufacturer's recommended applications and user reviews of the antibody.

  • Aspects of the Western blot protocol are provided by the replicating lab; complete details of the original protocol were unavailable.

Provisions for quality control

All data obtained from the experiment—raw data, data analysis, control data and quality control data—will be made publicly available, either in the published manuscript or as an open access dataset available on the Open Science Framework (https://osf.io/kvshc/).

  • STR profiling confirming cell line authenticity.

  • Mycoplasma testing confirming lack of contamination.

  • Images of stripped gel membranes confirming stripping was successful.

Protocol 4: Dot blot to measure of levels of 5hmC in genomic DNA

This protocol describes how to transfect HEK293 cells with vectors expressing the catalytic domain of TET2 (TET2-CD) and wild-type or mutant forms of IHD1 and IDH2 and then assess genomic DNA hydroxymethylation by dot blot, as seen in Figure 7B and Supplemental Figure 7C.

Sampling

  • This experiment will be conducted independently 4 times for a final power of 96%.

    • ○ See Power calculations for details.

  • Each experiment consists of 9 cohorts:

    • ○ Cohort 1: Untransfected cells [additional control].

    • ○ Cohort 2: Vector transfected cells.

    • ○ Cohort 3: FLAG-TET2-CD transfected cells.

      • ■ The catalytic domain of TET2.

    • ○ Cohort 4: FLAG-TET2-CM transfected cells.

      • ■ CM: mutant version of the TET2 catalytic domain.

    • ○ Cohort 5: FLAG-TET2-CD + FLAG-IDH1 transfected cells.

    • ○ Cohort 6: FLAG-TET2-CD + FLAG-IDH1R132H transfected cells.

    • ○ Cohort 7: FLAG-TET2-CD + FLAG-IDH2 transfected cells.

    • ○ Cohort 8: FLAG-TET2-CD + FLAG-IDH2R140Q transfected cells.

    • ○ Cohort 9: FLAG-TET2-CD + FLAG-IDH2R172K transfected cells.

    • ○ Each cohort will have gDNA spotted out at 5, 10, 25, 50, 100 and 250 ng and probed with anti-5hmC antibody.

Materials and reagents

ReagentTypeManufacturerCatalog #Comments
Mouse monoclonal IgG2a α Anti-5hmCAntibodyActive Motif40000Original catalog number unspecified
Mouse monoclonal IgG1 α FLAGAntibodySigma–AldrichF3165Original catalog number unspecified
Goat Anti-Mouse IgG H&L (HRP)AntibodyAbcamab97023
HEK293 cellsCellsATCCCRL-1573Original unspecified
Typhoon scannerEquipmentAmersham/ GE Health Sciences9410
Hybond ECL nitrocellulose membranes; 20 cm × 20 cmReagentGE Healthcare (Sigma–Aldrich)GERPN2020DOriginal not specified
DMEM; high glucoseMediumSigma–AldrichD5671Original unspecified
FBSReagentSigma–AldrichF2442Original unspecified
Vector alonePlasmidProvided by original authors
FLAG-TET2-CDPlasmidProvided by original authors
FLAG-TET2-CMPlasmidProvided by original authors
FLAG-IDH1PlasmidProvided by original authors
FLAG-IDH1R132HPlasmidProvided by original authors
Flag-IDH2PlasmidProvided by original authors
FLAG-IDH2R140QPlasmidProvided by original authors
FLAG-IDH2R172KPlasmidProvided by original authors
TransIT-LT1 transfection reagentReagentMirus BioMIR 2300Replaces SunBio-EZ (SunBio)
Nonfat-dried milk bovineReagentSigma–AldrichM7409
ECL prime Western blotting systemReagentGE Healthcare (Sigma–Aldrich)GERPN2232Original not specified
Image Quant 5.2SoftwareGEVersion 5.2
Protease inhibitor cocktail (mammalian)ReagentSigma–AldrichP8340-1MLOriginal not specified
TruPAGE TEA-Tricine SDS running buffer (20×)ReagentSigma–AldrichPCG3001-500 MLOriginal not specified
TruPAGE LDS sample buffer (4×)ReagentSigma–AldrichPCG3009-10 MLOriginal not specified
TruPAGE DTT sample reducer (10×)ReagentSigma–AldrichPCG3005-1MLOriginal not specified
TruPAGE transfer buffer (20×)ReagentSigma–AldrichPCG3011-500 MLOriginal not specified
PBS, without MgCl2 and CaCl2ReagentSigma–AldrichD8537Original not specified
Ponceau S solution; 0.1% (wt/vol) in 5% acetic acidReagentSigma–AldrichP7170Original not specified
Tris buffered saline (TBS); 10× solutionReagentSigma–AldrichT5912Original not specified
Bradford reagentReagentSigma–AldrichB6916Original not specified
QIAamp DNA mini kitKitQiagen51304

Procedure

Notes

  • This protocol contains information from Ito and colleagues (Ito et al., 2010).

  • HEK293 cells are maintained in DMEM supplemented with 10% FBS at 37°C/5% CO2.

  • All cells will be sent for mycoplasma testing and STR profiling.

  1. Transform, grow up and maxiprep plasmids using an Endo-free Maxiprep kit following the manufacturer's instructions.

    • a. Confirm plasmid identity by sequencing.

  2. Plate 6 × 105 − 1.2 × 106 HEK293 cells per 60 mm dish.

  3. 24 hr after plating, transfect cells with indicated plasmids.

    • a. #Transfect cells with 8 µg of DNA per construct using TransIT-LT1 Transfection Reagent according to manufacturer's protocol#.

      • i. Cohort 1: Untransfected cells.

      • ii. Cohort 2: Vector only.

      • iii. Cohort 3: FLAG-TET2-CD.

      • iv. Cohort 4: FLAG-TET2-CM.

      • v. Cohort 5: FLAG-TET2-CD + FLAG-IDH1.

      • vi. Cohort 6: FLAG-TET2-CD + FLAG-IDH1R132H.

      • vii. Cohort 7: FLAG-TET2-CD + FLAG-IDH2.

      • viii. Cohort 8: FLAG-TET2-CD + FLAG-IDH2R140Q.

      • ix. Cohort 9: FLAG-TET2-CD + FLAG-IDH2R172K.

  4. *For each cohort, transfect two parallel plates; harvest genomic DNA from one plate (proceed to Step 5) and protein from the second plate (proceed to Step 7).

  5. 36–40 hr after transfection, isolate genomic DNA from cells on the first plate using the QIAamp kit according to the manufacturer's instructions.

    • a. Determine DNA concentration and purity.

  6. Dot blot to assess levels of 5hmC:

    • a. Quantify gDNA concentration using a NanoDrop. #Spot genomic DNA onto nitrocellulose membrane using a pipet, then crosslink the DNA to the membrane by UV irradiation for 2 min.

      • i. The following amounts of genomic DNA should be spotted: 250 ng, 100 ng, 50 ng, 25 ng, 10 ng, and 5 ng.

    • b. Bake nitrocellulose membrane at 80°C for #1 hr.

    • c. Block membrane with 5% skim milk in TBS with 0.1% Tween 20 (TBST) for 1 hr.

    • d. Perform western blot on spotted nitrocellulose with the following antibody: anti-5hmC. Incubate membrane with primary antibody diluted 1:10,000 overnight at 4°C.

    • e. Wash membrane three times with TBST.

    • f. Incubate membrane with secondary antibody (HRP-conjugated anti-rabbit IgG) diluted 1:2000 for 1 hr at room temperature.

    • g. Wash membrane three times with TBST, then treat with ECL and scan with a Typhoon scanner.

    • h. Quantify dot-blot using Image-Quanta software.

  7. Check expression of exogenous proteins by Western blot using the second plate.

    • a. Wash cells once with cold PBS, then lyse cells in 0.5 ml of SDS loading buffer.

      • i. #4× SDS-PAGE loading buffer: 50 mM Tris pH 6.8, 2% SDS, 10% glycerol, 1% B-ME, 12.5 mM EDTA, 0.02% bromophenol blue.

    • b. Heat lysates at 99°C for 10 min.

    • c. Run SDS-PAGE gel until ladder marker reaches the bottom of the gel.

    • d. #Equilibrate gel in transfer buffer for 15 min.

    • e. #Meanwhile, cut membrane and 4 pieces 3 MM filter paper to size of gel.

      • i. Soak membrane in MeOH for a few seconds, then wash with H2O.

      • ii. Soak membrane, 3 mM filter paper and pads in transfer buffer.

      • iii. Transfer buffer: 38 mM glycine, 47 mM Tris, 11 mM SDS, 20% MeOH

    • f. #Assemble transfer cassette:

      • i. red pole (+) < clear plate < pad < 2 × 3 MM filter paper < membrane < gel < 2 × 3 MM filter paper < pad < black pole (−).

    • g. #Add stirring bar and ice box to transfer box and fill box with transfer buffer until cassette is submerged.

      • i. Run at 100 V for 1 hr.

    • h. #Wash membrane in wash buffer for 2 × 5 min.

      • i. Wash buffer: 1× PBS with 0.05% Tween-20 and 0.1% sodium azide.

    • i. #Incubate membrane in blocking buffer for 30 min.

      • i. Blocking buffer: 3% non-fat milk in PBS.

    • j. #Incubate membrane with primary antibody in blocking buffer for 2 hr at RT or O/N at 4°C (use manufacturer's suggested dilution in blocking buffer).

      • i. α FLAG.

    • k. #Wash 5 min 2× with wash buffer.

    • l. #Incubate membrane with secondary antibody for 90 min at RT (use manufacturer's suggested dilution in blocking buffer).

      • i. HRP-conjugated Goat Anti-Mouse IgG H&L: 1:2000.

    • m. #Wash 3 × 5 min in wash buffer.

    • n. # Detect HRP-conjugated secondary antibodies with chemiluminescent detection according to the manufacturer's protocol and image on the Typhoon scanner.

    • o. Quantify intensity of dots on western blots using ImageQuant 5.2.

      • i. Normalize values to FLAG-TET2-CD transfected cells.

  8. Repeat independently three additional times.

Deliverables

  • Data to be collected:

    • ○ Chromatograms and sequence files confirming plasmid identity.

    • ○ DNA concentration and purity data.

    • ○ Full scans of dot blots for anti-5hmC and western blots for anti-FLAG (as seen in Figure 7B).

    • ○ Raw values of intensity of dot blot as measured by Image-Quanta software.

    • ○ Quantification of 5hmc values relative to TET2-CD.

    • ○ Quantification of average values and standard deviations for each condition for all experiments.

    • ○ Bar graph and table of average values and standard deviations relative to TET2-CD samples (as seen in Figure 7B and Supplemental Figure 7C).

Confirmatory analysis plan

  • Statistical analysis of the replication data:

    • ○ Note: At the time of analysis we will perform the Shapiro–Wilk test and generate a quantile–quantile plot to assess the normality of the data. We will also perform Levene's test to assess homoscedasticity. If the data appears skewed we will perform the appropriate transformation in order to proceed with the proposed statistical analysis. If this is not possible we will perform the equivalent non-parametric test.

    • ○ Comparison of the various genotypes for each of the DNA concentrations.

      • ■ Bonferonni corrected one-sample t-test of normalized 5hmC levels of the following cohorts compared to constant (TET2-CD set to 1):

        • TET-2CD + IDH1.

        • TET-2CD + IDH1R132H.

        • TET-2CD + IDH2.

        • TET-2CD + IDH2R140Q.

        • TET-2CD + IDH2R172K.

  • Meta-analysis of original and replication attempt effect sizes:

    • ○ Compute the effect sizes of each comparison, compare them against the reported effect size in the original paper and use a meta-analytic approach to combine the original and replication effects, which will be presented as a forest plot.

Known differences from the original study

  • Aspects of the Western blot protocol are provided by the replicating lab; complete details of the original protocol were unavailable.

Provisions for quality control

All data obtained from the experiment—raw data, data analysis, control data and quality control data—will be made publicly available, either in the published manuscript or as an open access dataset available on the Open Science Framework (https://osf.io/kvshc/).

  • Sequence data confirming plasmid identity.

  • Western blots confirming exogenous protein expression.

  • STR profiling confirming cell line authenticity.

  • Mycoplasma testing confirming lack of contamination.

Protocol 5: Radiolabeled 5mC-5hmC conversion assay

This protocol describes how to run the in vitro assay to examine the effect of 2-HG on the TET family of methyl hydroxylases, as seen in Figure 8A.

Sampling

  • This experiment will be performed independently a total of 6 times for a final power of ≥80%.

    • ○ The original data is qualitative, thus to determine an appropriate number of replicates to initially perform, sample sizes based on a range of potential variance was determined.

    • ○ See Power calculations for details.

  • Each experiment consists of 8 cohorts:

    • ○ No recombinant protein.

    • ○ FLAG-TET2-CD + vehicle.

    • ○ FLAG-TET2-CD + 10 mM D-2-HG.

    • ○ FLAG-TET2-CD + 25 mM D-2-HG.

    • ○ FLAG-TET2-CD + 50 mM D-2-HG.

    • ○ FLAG-TET2-CD + 10 mM L-2-HG.

    • ○ FLAG-TET2-CD + 25 mM L-2-HG.

    • ○ FLAG-TET2-CD + 50 mM L-2-HG.

    • ○ Each cohort will detect:

      • ■ 5m-dCMP.

      • ■ 5hm-dCMP.

Materials and reagents

ReagentTypeManufacturerCatalog #Comments
D-2-HGReagentSigma–AldrichH8378
L-2-HGReagentSigma–Aldrich90790
Sf9 cellsCellsATCCCRL-1711Original unspecified
Shrimp alkaline phosphataseReagentNew England BiolabsMO371S
T4 polynucleotide kinaseReagentSigma–AldrichKEM0006
DNase IReagentSigma–AldrichAMPD1
Phosphodiesterase IReagentSigma–AldrichP3243
PEI-cellulose TLC plateMaterialSigma–AldrichZ122882
FLAG-TET2-CD viral particlesVirusProvided by the original authors
Anti-Flag M2 antibody agarose affinity gelReagentSigma–AldrichA2220
Flag peptideReagentSigma–AldrichF4799
α-KGReagentSigma–Aldrich75,892
GenElute PCR Clean-Up KitKitSigma–AldrichNA1020-1KTReplaces Qiagen cat no. 28304
[γ-32]ATPReagentPerkin ElmerBLU502H/NEG502H
MspI methyltransferaseReagentNEBM0215L
MspI restriction endonucleaseReagentNEBR0106T
DNA duplex oligonucleotide substrateoligoIntegrated DNA Technologiescustom 5′-GTGTTCTTTCAGCTCCGGTCACGCTGACCAGC-3′ as a duplex oligo, HPLC purified at 1 umole scale maybe higher depending on recovery
M13-F primeroligoIntegrated DNA TechnologiesCCAGTCACGACGTTGTAAAACG
M13-R primeroligoIntegrated DNA TechnologiesCCAGTCACGACGTTGTAAAACG
JumpStart REDTaq DNA PolymeraseReagentSigmaD8189-50UN
dNTP mix 10 mMReagentSigmaD7295-.2 ML
BlueView TAE bufferBufferSigmaT8935-1L
Molecular biology grade waterReagentSigmaW4502-1L

Procedure

Note: This protocol contains information from Ito and colleagues (2010).

  1. Generate recombinant FLAG-TET2-CD virus from supplied virus stock.

    • a. #Infect a 5 ml culture with 0.1 ml of virus stock supplied.

      • i. Grow in a stationary tissue culture flask at 27°C.

    • b. #After 5 days, collect the virus. Simultaneously, start a 50 ml suspension culture at 27°C with 140 rpm shaking.

      • i. Confirm viral insert identity by sequencing using M13F and R primers and REDTaq polymerase, followed by gel purification and sequencing of PCR product.

    • c. #After culturing for 3 days, infected the suspension culture with 2.5 ml of virus stock.

    • d. #4 days after infection collect virus. Simultaneously, start new 50 ml suspension cultures for protein expression.

      • e. #After 3 days of culture, the suspension cultures are infected with 2.5 ml virus.

    • e. #After 3 days of infection the cells expressing recombinant protein are collected by centrifugation and stored at −80°C until the protein is to be purified.

      • i. #More round of expression may be required depending on expression level.

    • f. Purify baculovirus expressed recombinant FLAG-TET2-CD from insect Sf9 cells with anti-Flag M2 antibody agarose affinity gel and elute with buffer containing 10 mM Tris–HCl pH 8.0, 150 mM NaCl, 1 mM DTT, 15% glycerol and 0.2 μg/µl Flag peptide.

    • g. Note; generate sufficient recombinant protein to use in a total of 6 replicates of this protocol.

  2. #Prepare methylated oligonucleotide substrate.

    • a. Treat unmethylated DNA duplex oligo with MspI methyltransferase for 2 hr at 37°C following manufacturer's instructions.

    • b. Purify with a QiaQuick Nucleotide Removal kit following manufacturer's instructions.

  3. Incubate 5 µg of the purified recombinant TET2-CD protein and various concentrations of vehicle only, D-2-HG, or L-2-HG with 0.5 μg methylated oligonucleotide substrate in vehicle (50 mM HEPES (pH 8), 75 μM Fe(NH4)2(SO4)2, 2 mM ascorbate) and 0.1 mM α-KG for 3 hr at 37°C.

    • a. See cohorts for detailed concentrations to use.

    • b. #If necessary, concentrate protein to ensure the final reaction volume is between 100–1000 µl.

    • c. Purify oligonucleotide substrates using a GenElute PCR Clean-Up Kit following manufacture's instructions.

  4. Digest oligonucleotides with 1 U/μg MspI restriction endonuclease at 37°C for #2 hr following manufacturer's instructions.

  5. Treat digested DNA with 1U/μmol shrimp alkaline phosphatase at 37°C for #2 hr.

    • a. #Heat inactivate at 65°C for 10 min.

  6. Label DNA with [γ-32]ATP and polynucleotide kinase.

    • a. #Add 1 µl of [γ-32]ATP at 3000 Ci/mmol, 5 mCi/ml and 1 µl polynucleotide kinase to the previous reaction.

    • b. #Incubate for 1 hr at 37°C.

  7. Ethanol precipitate labeled fragments.

    • i. #Add 3M NaOAc to a final concentration of 0.3M.

    • ii. #Add 2 vol 100% EtOH.

    • iii. #Incubate mixture at on dry ice for 20 min.

    • iv. #Centrifuge in a microfuge at 4°C at maximum speed for 10 min.

    • v. #Remove supernatant and air dry pellet.

    • vi.#Resuspend.

  8. Digest labeled fragments with 10 µg DNAse I and 10 μg phosphodiesterase I in the presence of 15 mM MgCl2 and 2 mM CaCl2 at 37 °C for #2 hr.

  9. Spot 1 µl of digestion product from step 8 onto a PEI-cellulose TLC plate and separate in an isobutyric acid/water/ammonium hydroxide (66:20:2) buffer.

  10. Dry the TLC plate and then expose to film.

  11. Quantify intensity of 5hmC bands.

    • a. Normalize values to FLAG-TET2-CD + vehicle.

  12. Repeat independently five additional times starting at Step 2.

Deliverables

  • Data to be collected:

    • ○ Sequencing data confirming viral insert identity.

    • ○ Data about viral titer and amount of and quality of protein generated.

    • ○ Scans of films exposed to TLC plate (as in Figure 8A, left).

    • ○ Raw values of intensity of 5hm-dCMP (5hmC) spots.

    • ○ Quantification of 5hmC intensity relative to FLAG-TET2-CD (recombinant protein) + vehicle sample.

    • ○ Quantification of average values and standard deviations for each condition for triplicate experiments.

    • ○ Bar graph of relative 5hmC intensity for each sample with standard deviations (as in Figure 8A, right).

Confirmatory analysis plan

  • Statistical Analysis of the Replication Data:

    • ○ Note: At the time of analysis we will perform the Shapiro–Wilk test and generate a quantile–quantile plot to assess the normality of the data. We will also perform Levene's test to assess homoscedasticity. If the data appears skewed we will perform the appropriate transformation in order to proceed with the proposed statistical analysis. If this is not possible we will perform the equivalent non-parametric test.

    • ○ Two-way ANOVA of normalized 5hmC levels of TET2-CD protein treated with D-2-HG or L-2-HG with the following Bonferroni corrected comparisons:

      • 10 mM D-2-HG vs 10 mM L-2-HG.

      • 50 mM D-2-HG vs 50 mM L-2-HG.

      • 10 mM D-2-HG vs 50 mM D-2-HG.

    • ○ Bonferroni corrected one-sample t-tests (outside the ANOVA framework) of normalized 5hmC levels of TET2-CD protein treated with the following concentrations of D-2-HG compared to constant (TET2-CD + vehicle set to 1):

      • 10 mM D-2-HG.

      • 50 mM D-2-HG.

  • Meta-analysis of original and replication attempt effect sizes:

    • ○ The replication data (mean and 95% confidence interval) will be plotted with the original reported data value plotted as a single point on the same plot for comparison.

Known differences from the original study

  • The lab provided the protocol for expansion of the viral aliquot shared by the original authors for generation of the recombinant FLAG-TET2 protein.

Provisions for quality control

All data obtained from the experiment—raw data, data analysis, control data and quality control data—will be made publicly available, either in the published manuscript or as an open access dataset available on the Open Science Framework (https://osf.io/kvshc/).

  • Sequence data confirming viral insert identity.

  • Data about viral titer and amount of and quality of protein generated.

Power calculations

Power calculations are performed to calculate the number of samples required to achieve at least 80% power and the indicated alpha error. For a detailed breakdown of all power calculations, please see spreadsheet at https://osf.io/gnsti/wiki/home/.

Protocol 1

Summary of original data

  • Note: Data estimated from published figures.

Supp. Figure 3I: Levels of α-KG with WT or mutant IDH1MeanSDN
Vector-transfected U-87 MG cells840*2
WT IDH-transfected U-87 MG cells1207.82
IDHR132H-transfected U-87 MG cells41142
Supp. Figure 3I: Levels of 2-HG with WT or mutant IDH1.MeanSDN
Vector-transfected U-87 MG cells900*2
WT IDH-transfected U-87 MG cells1400*2
Mutant IDH-transfected U-87 MG cells1730142
  1. *

    Because the original data reported null variances, the calculations below used the average of the non-null variances, 11.9, in place of a SD of 0.

Test family

  • Due to a lack of raw original data, we are unable to perform power calculations using a MANOVA. We are determining sample size calculations using a two-way ANOVA.

  • Two-way ANOVA followed by Bonferroni corrected comparisons.

Power calculations

ANOVA calculations; α = 0.05
F(1,6) metabolitePartial η2Effect size fPowerTotal sample size
6702.30.99910633.4300599.9%7*
  1. *

    With 5 samples per group (30 samples total), power achieved is 99.9%.

Corrected t-test sample size calculations; α = 0.0083333
Group 1Group 2Effect size dPowerSample size per group
α-KGVectorIDH1-WT3.5781580.1%*4*
VectorIDH1-R132H3.3096092.0%5
IDH1-WTIDH1-R132H6.9712597.4%3
2HGVectorIDH1-WT4.2016893.1%4
VectorIDH1-R132H126.2266999.9%§2§
IDH1-WTIDH1-R132H122.3783299.9%#2#
  1. *

    With a sample size of 5 per group, the achieved power is 95.7%.

  2. With a sample size of 5 per group, the achieved power is 99.9%.

  3. With a sample size of 5 per group, the achieved power is 99.2%.

  4. §

    With a sample size of 5 per group, the achieved power is 99.9%.

  5. #

    With a sample size of 5 per group, the achieved power is 99.9%.

Sensitivity calculations

  • Comparing 2-HG levels from Vector to IDH1 WT:

    • ○ Based on a sample size of 4 per group, we will be able to see an effect size of 3.3710662 with α = 0.01 and a power of 80%.

Protocol 2

Summary of original data

  • Note: Data estimated from published figures.

Supp. Fig. 3F: Quantification of Figure 3A Western BlotsMeanSDN
Untreated cellsH3K9me2/H3 ratio103
H3K79me2/H3 ratio103
10 mM oct-2-HG treated cellsH3K9me2/H3 ratio3.80.53
H3K79me2/H3 ratio8.51.53
20 mM oct-2-HG treated cellsH3K9me2/H3 ratio5.50.33
H3K79me2/H3 ratio17.22.43
20 mM oct-2-HG + 5 mM oct-α-KG treated cellsH3K9me2/H3 ratio0.60.33
H3K79me2/H3 ratio0.90.33

Test family

  • Due to a lack of raw original data, we are unable to perform power calculations using a MANOVA. We are determining sample size calculations using a two-way ANOVA.

  • Two-way ANOVA followed by Bonferroni corrected comparisons.

Power calculations

ANOVA calculations; α = 0.05
F(1,16) histonePartial η2Effect size fA priori powerTotal sample size
235.02000.9362603.8325999.9%*10*
  1. *

    With 3 samples per group (12 total), achieved power is 99.9%.

Corrected t-tests sample size calculations; α = 0.0083
Group 1Group 2Effect size dPowerSample size per group
H3K9me2Vehicle treated cells10 mM Oct-2-HG treated cells11.0593499.9%3
H3K79me28.9228899.9%3
H3K9me2Vehicle treated cells20 mM Oct-2-HG treated cells23.5540899.9%3
H3K79me214.2406999.9%3
H3K9me220 mM Oct-2-HG treated cells20 mM Oct-2-HG + 5 mM oct-α-KG treated cells28.8235399.9%3
H3K79me216.4612999.9%3

Test family

  • This is an additional analysis to allow a direct comparison with the original study.

  • Bonferroni corrected one-sample t-tests compared to 1 (vehicle treated cells).

Power calculations

  • Calculations were performed with G*Power software, version 3.1.7 (Faul et al., 2007).

Bonferroni corrected t-tests; α = 0.0083
GroupConstantEffect size dA Priori powerSample size per group
H3K9me210 mM Oct-2-HG treated cells19.6551790.3%3
H3K79me28.6206984.4%3
H3K9me220 mM Oct-2-HG treated cells126.4705999.9%3
H3K79me211.65468o 96.6%3

Protocol 3

Summary of original data

  • Note: Data estimated from published figure.

Supp. Figure 3J: quantification of Western blot band intensities from Figure 3D normalized to vector controlMeanSDN
With vector + vehicle
H3K4me1/H3 ratio100Unspecified3
H3K4me3/H3 ratio100unspecified3
H3K9me3/H3 ratio100unspecified3
H3K27me2/H3 ratio100unspecified3
H3K79me2/H3 ratio100unspecified3
With IDH1R132H + vehicle
H3K4me1/H3 ratio209363
H3K4me3/H3 ratio466643
H3K9me3/H3 ratio283563
H3K27me2/H3 ratio232243
H3K79me2/H3 ratio267473
With IDH1R132H and oct-α-KG
H3K4me1/H3 ratio105163
H3K4me3/H3 ratio274253
H3K9me3/H3 ratio126213
H3K27me2/H3 ratio9993
H3K79me2/H3 ratio130203

Test family

  • Due to a lack of raw original data, we are unable to perform power calculations using a MANOVA. We are determining sample size calculations using a two-way ANOVA.

  • Two-way ANOVA followed by Bonferroni corrected comparisons.

Power calculations

ANOVA calculations; α = 0.05
F(1,20) cell treatmentsPartial η2effect size fPowerTotal Sample size
119.56290.856702.4450297.1%*12*
  1. *

    With 6 samples per group (for a total of 60 samples), the power achieved is 99.9%.

Corrected t-test sample size calculations; α = 0.005
Group 1Group 2HistoneEffect size dPowerSample size per group
IDH1R132H + vehicleIDH1R132H + oct-α-KGH3K4me1/H3 ratio3.7333894.2%*5*
H3K4me3/H3 ratio3.9518482.1%4
H3K9me3/H3 ratio3.7124093.9%5
H3K27me2/H3 ratio7.3381195.0%§3§
H3K79me2/H3 ratio3.7931494.9%#5#
  1. *

    With a sample size of 6 per group, the achieved power is 98.9%.

  2. With a sample size of 6 per group, the achieved power is 99.5%.

  3. With a sample size of 6 per group, the achieved power is 98.8%.

  4. §

    With a sample size of 6 per group, the achieved power is 99.9%.

  5. #

    With a sample size of 6 per group, the achieved power is 99.1%.

Test family

  • Outside the ANOVA framework

  • Bonferroni corrected one-sample t-tests compared to 1 (vector + vehicle).

Power calculations

  • Calculations were performed with G*Power software, version 3.1.7 (Faul et al., 2007).

Corrected t-test sample size calculations; α = 0.005
Group 1ConstantHistoneEffect size dPowerSample size per group
IDH1R132H + vehicle100H3K4me1/H3 ratio3.0277894.2%6
H3K4me3/H3 ratio5.7187592.2%*4*
H3K9me3/H3 ratio3.2678682.9%5
H3K27me2/H3 ratio5.5000090.2%4
H3K79me2/H3 ratio3.5531988.8%§5§
  1. *

    With a sample size of 6 per group, the achieved power is 99.9%.

  2. With a sample size of 6 per group, the achieved power is 96.9%.

  3. With a sample size of 6 per group, the achieved power is 99.9%.

  4. §

    With a sample size of 6 per group, the achieved power is 98.7%.

Protocol 4

Summary of original data

  • ○ Note: Values estimated from published figure.

Figure 7B: Relative 5hmC intensityMeanSDN
50 ng Genomic DNA
Vector00.013
TET2-CD103
TET2-CM00.013
TET2-CD + IDH12.50.33
TET2-CD + IDH1R132H0.290.13
TET2-CD + IDH22.60.113
TET2-CD + IDH2R40Q0.310.073
TET2-CD + IDH2R172K0.310.093

Test family

  • Bonferroni corrected one-sample t-tests compared to 1 (TET2-CD).

Power calculations

  • Power calculations were performed using G*Power software, version 3.1.7 (Faul et al., 2007).

Corrected t-test sample size calculations; α = 0.01
Group 1: TET2 +ConstantEffect size dPowerSample size per group
IDH115.0000095.9%4
IDH1R132H17.1000099.9%4
IDH2114.5454599.8%*3*
IDH2R140Q19.8571494.6%3
IDH2R172K17.6666782.9%3
  1. *

    With a sample size of 4 per group, the achieved power is 99.9%.

  2. With a sample size of 4 per group, the achieved power is 99.9%.

  3. With a sample size of 4 per group, the achieved power is 99.9%.

Protocol 5

Summary of original data

  • Note: Data estimated from published figures.

Figure 8A: TLC blot intensitiesMean
TET2 + vehicle1
TET2 + 10 mM D-2-HG0.67
TET2 + 25 mM D-2-HG0.45
TET2 + 50 mM D-2-HG0.17
TET2 + 10 mM L-2-HG0.05
TET2 + 25 mM L-2-HG0.03
TET2 + 50 mM L-2-HG0.03

Test family

  • One way ANOVA followed by Bonferroni corrected comparisons.

  • Outside the ANOVA framework

    • ○ Bonferroni corrected one-sample t-tests compared to 1 (TET2 + vehicle).

Power calculations

  • Because the original data presented does not have variance (s.e.m. or s.d.), we have performed power calculations using several different levels of calculated variance and an assumed number of replicates to determine a suitable number of replications to perform.

  • Calculations were performed with R software, version 3.1.2 (R Core Team, 2014) and G*Power software, version 3.1.7 (Faul et al., 2007).

Calculated variances and assumed N
Figure 8A: dot blot intensitiesMeanN2%15%28%40%
TET2 + vehicle13n/a*n/a*n/a*n/a*
TET2 + 10 mM D-2-HG0.6730.01340.10050.18760.268
TET2 + 25 mM D-2-HG0.4530.0090.06750.1260.18
TET2 + 50 mM D-2-HG0.1730.00340.02550.04760.068
TET2 + 10 mM L-2-HG0.0530.0010.00750.0140.02
TET2 + 25 mM L-2-HG0.0330.00060.00450.00840.012
TET2 + 50 mM L-2-HG0.0330.00060.00450.00840.012
  1. *

    Because each replicate will be normalized to TET2 + vehicle this will not have a variance associated with it. And thus the TET2 + vehicle is also not include in the ANOVA calculation.

2% variance

ANOVA calculations; α = 0.05
F(2,12) interactionPartial η2Effect size fPowerTotal sample size
1910.60.9968717.843498.2%*9*
  1. *

    With 12 total samples, the power achieved is 99.9%.

Corrected t-test sample size calculations; α = 0.01
Group 1Group 2Effect size dPowerSample size per group
10 mM D-2-HG10 mM L-2-HG65.2523199.9%2
50 mM D-2-HG50 mM L-2-HG57.3462399.9%2
10 mM D-2-HG50 mM D-2-HG51.1483999.9%2
Corrected t-test sample size calculations; α = 0.01
Group 1:ConstantEffect size dPowerSample size per group
10 mM D-2-HG124.6268797.7%2
50 mM D-2-HG1244.1176599.9%2

15% variance

ANOVA calculations; α = 0.05
F(2,12) interactionPartial η2Effect size fPowerTotal sample size
2.379300.849872.3793093.8%12*
  1. *

    With 12 total samples, the power achieved is 99.9%.

Corrected t-test sample size calculations; α = 0.01
Group 1Group 2Effect size dPowerSample size per group
10 mM D-2-HG10 mM L-2-HG8.7003199.9%3
50 mM D-2-HG50 mM L-2-HG7.6461699.4%3
10 mM D-2-HG50 mM D-2-HG6.8197897.9%3
Corrected t-test sample size calculations; α = 0.01
Group 1:ConstantEffect size dPowerSample size per group
10 mM D-2-HG13.2835887.2%6
50 mM D-2-HG132.5490299.5%3

28% variance

ANOVA calculations; α = 0.05
F(2,12) interactionPartial η2Effect size fPowerTotal sample size
9.75480.619161.2750786.1%12
Corrected t-test sample size calculations; α = 0.01
Group 1Group 2Effect size dPowerSample size per group
10 mM D-2-HG10 mM L-2-HG4.6608897.9%4
50 mM D-2-HG50 mM L-2-HG4.0961693.6%4
10 mM D-2-HG50 mM D-2-HG3.6534686.6%4
Corrected t-test sample size calculations; α = 0.01
Group 1:ConstantEffect size dPowerSample size per group
10 mM D-2-HG11.7590680.3%14
50 mM D-2-HG117.4369794.0%4

40% variance

ANOVA calculations; α = 0.05
F(2,12) interactionPartial η2Effect size fPowerTotal sample size
4.77650.443230.89223782.2%17*
  1. *

    With 18 total samples, the power achieved is 85.3%.

Corrected t-test sample size calculations; α = 0.01
Group 1Group 2Effect size dPowerSample size per group
10 mM D-2-HG10 mM L-2-HG3.2626292.8%5
50 mM D-2-HG50 mM L-2-HG2.8673183.9%5
10 mM D-2-HG50 mM D-2-HG2.5574286.3%6
Corrected t-test sample size calculations; α = 0.01
Group 1:ConstantEffect size dPowerSample size per group
10 mM D-2-HG11.2313480.8%27
50 mM D-2-HG112.2058884.6%5
  1. In order to produce quantitative replication data, we will run the experiment six times. Each time we will quantify band intensity. We will determine the standard deviation of band intensity across the biological replicates and combine this with the reported value from the original study to simulate the original effect size. We will use this simulated effect size to determine the number of replicates necessary to reach a power of at least 80%. We will then perform additional replicates, if required, to ensure that the experiment has more than 80% power to detect the original effect.

References

  1. 1
  2. 2
  3. 3
    An open investigation of the reproducibility of cancer biology research
    1. TM Errington
    2. E Iorns
    3. W Gunn
    4. FE Tan
    5. J Lomax
    6. BA Nosek
    (2014)
    Elife, 3, 10.7554/eLife.04333.
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
    Ten Eleven Translocation Enzymes and 5-Hydroxymethylation in Mammalian Development and Cancer
    1. SRM Kinney
    2. S Pradhan
    (2012)
    In: AR Karpf, editors. Epigenetic alterations in oncogenesis. New York, NY: Springer New York. pp. 57–79.
  9. 9
  10. 10
  11. 11
    R: a language and environment for statistical computing
    1. R Core Team
    (2014)
    R Foundation for Statistical Computing, http://www.R-project.org/.
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16

Decision letter

  1. Irwin Davidson
    Reviewing Editor; Institut de Génétique et de Biologie Moléculaire et Cellulaire, France

eLife posts the editorial decision letter and author response on a selection of the published articles (subject to the approval of the authors). An edited version of the letter sent to the authors after peer review is shown, indicating the substantive concerns or comments; minor concerns are not usually shown. Reviewers have the opportunity to discuss the decision before the letter is sent (see review process). Similarly, the author response typically shows only responses to the major concerns raised by the reviewers.

Thank you for submitting your work entitled “Registered report: Oncometabolite 2-Hydroxyglutarate Is a Competitive Inhibitor of α-Ketoglutarate-Dependent Dioxygenases” for peer review at eLife. Your submission has been favorably evaluated by Michael Marletta (Senior editor), Irwin Davidson (Reviewing editor), and four reviewers.

The reviewers have discussed the reviews with one another and the Reviewing editor has drafted this decision to help you prepare a revised submission. The reviewers of this paper have raised several issues and we would ask you to specifically take into account the comments concerning the statistical analyses.

Summary:

The article outlines the detailed protocol to reproduce a report published in Cancer Cell linking mutations in IDH1/2 to cellular levels of Ketoglutarate and changes in histone and DNA methylation. This initial work had a major impact on linking metabolites to chromatin, but also raised a number of questions that justify a rigorous replication.

Overall, the proposed study covers the major aspects with the required detail and rigour.

Specific points to address are:

1) The referees suggest that the authors consider using mass spectrometry to measure 5hmC in addition to immune dot-blot. Mass spectrometry is a more quantitative measure and while it would go beyond replicating the published findings it might give a clearer answer.

2) In protocol 1 a 2-way ANOVA is proposed, however as 2 quantitative variables are measured and there is only one qualitative factor (with three possible values) influencing these measures, an MANOVA would be more suited.

3) There may be confusion between groups and variables in setting the degrees of freedom for ANOVA analyses. In protocol 1, how was (2, 6) obtained? The same question applies to protocols 3, 4 and 5.

4) In addition to t-tests for the comparison of means where both variances are equal, F-tests should be added when variances are significantly different.

5) Referees raised concerns about null variances that appear in the power calculation tables. Although these values are not always available, variance values can change the conclusion of the tests. When variances are not available, preliminary experiments in order to estimate them are proposed. More generally, variance values used in this paper are estimated from published figures using a low number of replicates, so they are not robust. A way to increase robustness would be to increase measured values by a pre-determined factor and then relax the expected power if too many replicates are required.

Also, non-rounded computed sample sizes are requested to have an idea of how close we are to the theoretical value after rounding.

6) For protocol 2, in subsection “Confirmatory analysis plan”, a MANOVA is proposed, whereas a one-way ANOVA is suggested for the same protocol in subsection “Test family”. Please correct this inconsistency.

7) Protocol 5: a mean across several groups is compared with the mean of a single group. This should not be done with a simple t-test to take into account the fact that the number of measures is different in the two groups being compared.

https://doi.org/10.7554/eLife.07420.002

Author response

1) The referees suggest that the authors consider using mass spectrometry to measure 5hmC in addition to immune dot-blot. Mass spectrometry is a more quantitative measure and while it would go beyond replicating the published findings it might give a clearer answer.

We agree mass spectrometry analysis would be a more quantitative approach to measure 5hmC levels, however feel it is beyond the scope of this project, which is to perform a direct replication of the original experiment(s). Aspects of an experiment not included in the original study are occasionally added to ensure the quality of the research, but by no means is a requirement of this project; rather, it is an extension of the original work. We know that the exclusion of certain experiments limits the scope of what can be analyzed by the project, but we are attempting to identify a balance of breadth of sampling for general inference with sensible investment of resources on replication projects to determine to what extent the included experiments are reproducible.

2) In protocol 1 a 2-way ANOVA is proposed, however as 2 quantitative variables are measured and there is only one qualitative factor (with three possible values) influencing these measures, an MANOVA would be more suited.

We agree and have included this in the confirmatory analysis section. However, as we do not have the raw data needed to perform a power calculation for the MANOVA test, we performed it with a 2-way ANOVA to estimate the needed sample size and adjusted the alpha error for the planned contrasts that will be performed to ensure the sample size is sufficient.

3) There may be confusion between groups and variables in setting the degrees of freedom for ANOVA analyses. In protocol 1, how was (2, 6) obtained? The same question applies to protocols 3, 4 and 5.

Thank you for catching these inconsistencies, we have checked and adjusted each protocol in regards to degrees of freedom. Some of the analysis sections have changed to address other questions below, but a link to all scripts has been provided below and in the revised manuscript.

4) In addition to t-tests for the comparison of means where both variances are equal, F-tests should be added when variances are significantly different.

We have added a note in the analysis section that at the time of analysis, we will assess the normality and homoscedasticity of the data. If necessary, we will perform the appropriate transformation in order to proceed with the proposed statistical analysis. We will note any changes or transformations made. We have updated the manuscript to address this point.

5) Referees raised concerns about null variances that appear in the power calculation tables. Although these values are not always available, variance values can change the conclusion of the tests. When variances are not available, preliminary experiments in order to estimate them are proposed. More generally, variance values used in this paper are estimated from published figures using a low number of replicates, so they are not robust. A way to increase robustness would be to increase measured values by a pre-determined factor and then relax the expected power if too many replicates are required.

Also, non-rounded computed sample sizes are requested to have an idea of how close we are to the theoretical value after rounding.

We agree about the concern about null variances and have reanalyzed the proposed analysis plans and power calculations to reflect this. In many cases the reason for the null variance was due to normalization to a common factor within each replicate – thus making the variance zero on purpose. In some cases we will need to repeat this as well (protocol 3, 4, and 5). In other cases (protocol 1 and 2) we used the other variances that were reported as an estimate for the null variances. And where possible (protocol 2) we included additional analysis to allow a direct comparison to the original analysis.

Full details of all power calculations are available through the study’s page on the Open Science Framework (https://osf.io/gnsti/?view_only=f3a48d5a355f429fa2264ee7c17e9705). Unfortunately, the program we use to calculate sample sizes, G*Power, only returns whole integers for recommended sample sizes.

6) For protocol 2, in subsection “Confirmatory analysis plan”, a MANOVA is proposed, whereas a one-way ANOVA is suggested for the same protocol in subsection “Test family”. Please correct this inconsistency.

The Confirmatory analysis plan is in reference to the proposed statistical analyses of the replication data. However, as we do not have the raw data needed to perform a power calculation for the MANOVA test, we performed it with a 2-way ANOVA to estimate the needed sample size and adjusted the alpha error for the planned contrasts that will be performed to ensure the sample size is sufficient.

7) Protocol 5: a mean across several groups is compared with the mean of a single group. This should not be done with a simple t-test to take into account the fact that the number of measures is different in the two groups being compared.

We had originally intended to perform a weighted planned contrast by comparing several groups to a single group – and agree an F test is properly suited. However, in the revised manuscript we are including multiple independent t-tests and one-sample t-tests (due to the necessary normalization described in point 5 above) to more thoroughly analyze the data as originally reported and interpreted.

https://doi.org/10.7554/eLife.07420.003

Article and author information

Author details

  1. Brad Evans

    Proteomics and Mass Spectrometry Facility, Donald Danforth Plant Science Center, St. Louis, Missouri, United States
    Contribution
    BE, Drafting or revising the article
    Competing interests
    No competing interests declared.
  2. Erin Griner

    University of Virginia, Charlottesville, Virginia, United States
    Contribution
    EG, Drafting or revising the article
    Competing interests
    No competing interests declared.
  3. Reproducibility Project: Cancer Biology

    Contribution
    RP:CB, Conception and design, Drafting or revising the article
    For correspondence
    fraser@scienceexchange.com
    Competing interests
    RP:CB: We disclose that EI, FT, and JL are employed by and hold shares in Science Exchange Inc. The experiments presented in this manuscript will be conducted by BE at the Proteomics and Mass Spectrometry Facility, which is a Science Exchange lab.
    1. Elizabeth Iorns, Science Exchange, Palo Alto, California
    2. William Gunn, Mendeley, London, United Kingdom
    3. Fraser Tan, Science Exchange, Palo Alto, California
    4. Joelle Lomax, Science Exchange, Palo Alto, California
    5. Timothy Errington, Center for Open Science, Charlottesville, Virginia

Funding

Laura and John Arnold Foundation

  • Reproducibility Project: Cancer Biology

The Reproducibility Project: Cancer Biology is funded by the Laura and John Arnold Foundation, provided to the Center for Open Science in collaboration with Science Exchange. The funder had no role in study design or the decision to submit the work for publication.

Acknowledgements

The Reproducibility Project: Cancer Biology core team would like to thank the original authors, in particular Dr. Yue Xiong, for generously sharing reagents to ensure the fidelity and quality of this replication attempt. We would also like to thanks the following companies for generously donating reagents to the Reproducibility Project: Cancer Biology; American Tissue Culture Collection (ATCC), Applied Biological Materials, BioLegend, Charles River Laboratories, Corning Incorporated, DDC Medical, EMD Millipore, Harlan Laboratories, LI-COR Biosciences, Mirus Bio, Novus Biologicals, Sigma–Aldrich, and System Biosciences (SBI).

Reviewing Editor

  1. Irwin Davidson, Reviewing Editor, Institut de Génétique et de Biologie Moléculaire et Cellulaire, France

Publication history

  1. Received: March 11, 2015
  2. Accepted: July 9, 2015
  3. Version of Record published: July 31, 2015 (version 1)

Copyright

© 2015, Evans et al.

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 915
    Page views
  • 200
    Downloads
  • 1
    Citations

Article citation count generated by polling the highest count across the following sources: Crossref, Scopus, PubMed Central.

Comments

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Download citations (links to download the citations from this article in formats compatible with various reference manager tools)

Open citations (links to open the citations from this article in various online reference manager services)

Further reading

  1. U Xu et al.
    1
    1. Cancer Biology
    Curated by Roger Davis et al.
    Collection Updated

    Investigating reproducibility in preclinical cancer research.

    1. Biochemistry
    2. Biophysics and Structural Biology
    G Elif Karagöz et al.
    Research Article Updated