1. Biochemistry
Download icon

Registered report: Transcriptional amplification in tumor cells with elevated c-Myc

  1. David Blum
  2. Haiping Hao
  3. Michael McCarthy
  4. Reproducibility Project: Cancer Biology Is a corresponding author
  1. University of Georgia, Georgia
  2. Johns Hopkins University
  3. University of Oxford, United Kingdom
Registered Report
Cited
2
Views
1,637
Comments
0
Cite as: eLife 2015;4:e04024 doi: 10.7554/eLife.04024

Abstract

The Reproducibility Project: Cancer Biology seeks to address growing concerns about reproducibility in scientific research by conducting replications of 50 papers in the field of cancer biology published between 2010 and 2012. This Registered report describes the proposed replication plan of key experiments from ‘Transcriptional amplification in tumor cells with elevated c-Myc’ by Lin et al. (2012), published in Cell in 2012. The experiments that will be replicated are those reported in Figures 3E and 3F. In these experiments, elevated levels of c-Myc in the P493-6 cell model of Burkitt's lymphoma results in an increase of the total level of RNA using UV/VIS spectrophotometry (Figure 3E; Lin et al., 2012) and on the mRNA levels/cell for a large set of genes using digital gene expression technology (Figure 3F; Lin et al., 2012). The Reproducibility Project: Cancer Biology is a collaboration between the Center for Open Science and Science Exchange, and the results of the replications will be published in eLife.

https://doi.org/10.7554/eLife.04024.001

Introduction

The proto-oncogene MYC is frequently amplified in human cancers and encodes the transcription factor c-Myc, which is associated with a variety of cellular processes such as cell growth and proliferation (Dang, 2013). While overexpressed c-Myc is known to contribute to tumorigenesis, an understanding of how this process occurs is complicated by a number of issues, including the large number of binding sites and the diversity between systems. While this was thought to occur by regulation of a specific subset of genes, Lin et al. (2012) present findings that c-Myc functions by globally amplifying the expression of actively transcribed genes.

The system used is the human P493-6 B cell model of Burkitt's lymphoma, which contains a tetracycline-repressible MYC transgene, allowing for titration of c-Myc protein (Schuhmacher et al., 1999; Pajic et al., 2000). The levels of c-Myc can be reduced and subsequently re-induced in a gradual time-dependent manner, as determined by western blot, which is shown in Figure 1B (Lin et al., 2012). As soon as 1 hr after re-induction, the protein levels of c-Myc increased above the repressed levels, which by 24 hr were similar to the tetracycline-free condition (Lin et al., 2012). This system has been used in other studies with similar results observed (Schuhmacher et al., 1999; Pajic et al., 2000). This experiment is important to replicate because it assesses the level of c-Myc re-induction with this system that will be used in the following experiments. This experiment is replicated in Protocol 1.

In Figure 3E the total levels of RNA, in P493-6 cells before and after re-induction of c-Myc, was determined by UV/VIS spectrophotometry (Lin et al., 2012). Lin et al. (2012) reported an increase in levels of absolute RNA over the timecourse of c-Myc re-induction. This experiment shows that c-Myc increases RNA content per cell, indicating c-Myc functions primarily in transcriptional amplification. Related experiments, using mouse B cells, also observed the same c-Myc-dependent amplification of cellular RNA content (Nie et al., 2012; Sabò et al., 2014). However, similar experiments in 3T9 fibroblasts or U2OS cells expressing an inducible c-Myc did not observe an increase in cellular RNA content (Sabò et al., 2014; Walz et al., 2014). Interestingly, an increase in total cellular RNA content was observed in 3T9 fibroblasts following serum stimulation, which was not observed when c-Myc was deleted in these cells (Sabò et al., 2014). This experiment is replicated in Protocol 2.

In Figure 3F, Lin et al. (2012) examined the transcriptional profile of a large number of genes from multiple functional categories in cells before and after re-induction of c-Myc using digital gene expression. Re-induction of c-Myc increased the number of active genes (defined as greater than 1 transcript/cell), but did not alter silent genes (defined as less than 0.5 transcript/cell) (Lin et al., 2012). This key finding suggests that elevated c-Myc levels lead to an amplification of the existing transcriptional profile. This finding was observed in other experiments using mouse B cells treated with the Myc-Max dimerization inhibitor 10058-F4 and analyzed by ChIP-Seq (Nie et al., 2012). Another report using P493-6 cells also found an increase in transcription of c-Myc target genes when c-Myc re-induction was titrated to different concentrations by microarray analysis (Schuhmacher and Eick, 2013). This experiment is replicated in Protocols 3 and 4. Recently, two papers were published that used primarily RNA-seq and ChIP-seq to focus on assessing if the transcriptional effects of c-Myc are direct or indirect (Alderton, 2014). These studies found that RNA amplification and promoter/enhancer invasion by c-Myc were separable events in 3T9 fibroblasts and U2OS cells, suggesting that c-Myc regulates a distinct subset of genes, which indirectly lead to RNA amplification (Sabò et al., 2014; Walz et al., 2014).

Materials and methods

Protocol 1: Western blot of c-Myc reactivation in tetracyclin-repressible system

This experiment tests the expression of the tetracycline (tet)-repressible Myc transgene after repression by tet and re-induction by removal into tet-free medium. This is a replication of the data presented in Figure 1B and assesses the levels of c-Myc protein. One major variable in this system is serum, which has been reported to stimulate the expression of a majority of genes independently from c-Myc (Schlosser et al., 2005). P493-6 cells will be cultured with two separate lots of serum that will be maintained throughout all the experiments to assess if there is variability between different batches of serum.

Sampling

  • Experiment to be repeated a total of three times.

    1. This replication attempt will be repeated three times to analyze if this effect is detected and to assess the variance of this system, since it will be repeated three times in the other replication protocols.

  • Each experiment has two cohorts:

    1. Cohort 1: P493-6 cells cultured in FBS lot #1.

    2. Cohort 2: P493-6 cells cultured in FBS lot #2.

  • Each cohort has four conditions:

    1. Cells cultured in tet-free media.

    2. Cells cultured 0 hr after tet induction.

    3. Cells cultured 1 hr after tet induction.

    4. Cells cultured 24 hr after tet induction.

Materials and reagents

ReagentTypeManufacturerCatalog #Comments
P493-6 modified Burkitt's lymphoma cellsCell lineOriginal labFrom original lab
RPMI 1640 medium with sodium bicarbonate, without L-glutamineCell cultureSigma–AldrichR0883No catalog # listed in original paper
Tetracycline system approved FBSSerumClontech631105Two separate lots will be used
Ala-Gln (L-glutamine substitute)Cell cultureSigma–AldrichG8541Original lab used GlutaMAX from Invitrogen
T75 flaskLabwareSigma–AldrichZ707503Originally not specified
TetracyclinePharmacological agentSigma–AldrichT7660
C-chip disposable hemocytometerLabwareDigital BioDHC-N01No catalog # listed in original paper
Refrigerated centrifugeEquipmentEppendorf5810ROriginal lab used a Sorvall Legend centrifuge
PBS, without MgCl2 and CaCl2BuffersSigma–AldrichD8537Originally not specified
RIPA lysis bufferBuffersSigma–AldrichR0278
SIGMAFAST Protease Inhibitor TabletsInhibitorsSigma-AldrichS8820Original lab used Halt Protease Inhibitor cocktail from Thermo-Fisher
Phosphatase inhibitor cocktail 2InhibitorsSigma–AldrichP5726
Phosphatase inhibitor cocktail 3InhibitorsSigma–AldrichP0044
Bradford ReagentReporter assaySigma–AldrichB6916Original lab used Bradford Reagent from Bio-Rad
TruPAGE LDS sample buffer (4X)BuffersSigma-AldrichPCG3009Originally not specified
ß-mercaptoethanolChemicalsSigma–AldrichM3148Originally not specified
Nuclease-free waterChemicalsSigma–AldrichW4502Original lab used nuclease-free water from Ambion (AM9938)
Mini-PROTEAN Electrophoresis SystemEquipmentBio-RadOriginally not specified
TruPAGE Precast Gels 4–20%Western materialsSigma-AldrichPCG2012Original lab used 4–12% Bis-Tris Midi gels from Invitrogen
TruPAGE TEA-Tricine SDS Running Buffer (20X)BuffersSigma-AldrichPCG3001Originally not specified
ECL DualVue Western Markers (15–150 kDa)Western materialsSigma–AldrichGERPN810Originally not specified
Hybond P Western blotting membranes; PVDFWestern materialsSigma–AldrichGE10600029No catalog # listed in original paper
TruPAGE Transfer Buffer (20X)BuffersSigma-AldrichPCG3011Originally not specified
MethanolChemicalsSigma–Aldrich494437Originally not specified
Semi-dry TransBlot SD SystemEquipmentBio-Rad170–3940
5% milk powderWestern materialsSigma–AldrichM7409Originally not specified
10X TBS buffered salineBuffersSigma–AldrichT5912Originally not specified
Tween-20ChemicalsSigma–AldrichP1379Originally not specified
Rabbit c-MycAntibodiesEpitomics1472-1Dilute 1:5000; 57 kDa
Mouse ß-actinAntibodiesSigma–AldrichA5441Dilute 1:10,000; 42 kDa
Goat anti-rabbit-HRPAntibodiesSigma–AldrichGERPN2124Originally not specified. Dilute 1:10,000
Goat anti-mouse-HRPAntibodiesSigma–AldrichGERPN2124Originally not specified. Dilute 1:10,000
ECL Prime Chemiluminescent reagentWestern materialsSigma–AldrichGERPN2232Originally not specified
G:BOX iChemi XTEquipmentSyngeneOriginal lab used a Gel Dox XR+ System from Bio-Rad
GeneSnapSoftwareSyngeneVersion 6.00.19Original lab used Image Lab software from Bio-Rad

Procedure

Notes

  • All cells will be sent for mycoplasma testing and STR profiling.

  • Tet-free media: RPMI1640 supplemented with 10% tet system approved FBS and 1% Ala-Gln.

  • Cells maintained at 37°C in a humidified atmosphere at 5% CO2.

  • P493-6 cells are maintained in two separate FBS lots (lot #1 and lot #2).

    1. Prepare stocks in upright T75 flasks of P493-6 modified Burkitt's lymphoma cells cultured in tet-free media.

    2. On day 0, seed Flask A with ∼1 × 107 P493-6 cells in 20 ml tet-free media, and Flask B with ∼2 × 107 P493-6 cells in 40 ml tet-free media with 0.1 µg/ml tet for a concentration of 0.5 × 106 cells/ml.

      • a. Count cells in each flask using a C-chip disposable hemocytometer.

    3. 72 hr later, spin cells down from flasks, and wash cells three times in tet-free media.

    4. From Flask A cells, seed 1 new flask with 1.2 × 107 in fresh tet-free media.

      • a. Count cells using a C-chip disposable hemocytometer.

    5. From Flask B cells, seed three new flasks with 1.2 × 107 cells in 20 ml of fresh tet-free media.

      • a. Count cells using a C-chip disposable hemocytometer.

    6. At time points 0 hr (immediately), 1 hr, and 24 hr, harvest protein lysates from 1 × 107 cells from a Flask B cell suspension.

      • a. Count cells in each flask using a C-chip disposable hemocytometer.

      • b. To harvest lysates:

        • i. Pellet 1 × 107 cells at 4°C at 1200 rpm for 5 min in a Sorvall Legend centrifuge.

        • ii. Wash pellets once with ice-cold 1X PBS.

          • Cell pellets can be snap frozen in liquid nitrogen and stored at −80°C until needed.

        • iii. Resuspend pellets in 100 µl RIPA lysis buffer containing 2× SIGMAFAST Protease inhibitors and 2× Phosphatase inhibitor cocktails 2 and 3.

    7. At 24 hr, harvest protein lysate from tet-free cells (Flask A) as described in step 6.

    8. Quantify total protein concentration in each lysate using the Bradford assay, according to the manufacturer's instructions.

      • a. Use a bovine serum albumin standard curve for the quantification.

    9. Run an electrophoresis gel under denaturing conditions, loading the same volume of 50 µg of total protein lysate, with 1× SDS loading buffer, 2.5% ß-mercaptoethanol, and nuclease-free water for each condition, on a 4–20% TruPAGE precast gel. Run at 200 V until the dye front reaches the reference line following manufacturer's instructions.

      • a. Samples run per gel:

        • i. Protein molecular weight marker.

        • ii. 0 hr from tet release (FBS lot #1).

        • iii. 1 hr from tet release (FBS lot #1).

        • iv. 24 hr from tet release (FBS lot #1).

        • v. tet-free control (FBS lot #1).

        • vi. 0 hr from tet release (FBS lot #2).

        • vii. 1 hr from tet release (FBS lot #2).

        • viii. 24 hr from tet release (FBS lot #2).

        • ix. tet-free control (FBS lot #2).

    10. Transfer gel to a PVDF membrane, using the semi-dry TransBlot SD system according to manufacturer's instructions.

    11. Block with 5% milk powder in TBST buffer.

      • a. TBST buffer: TBS with 0.2% Tween-20.

    12. Probe membrane with the following primary antibodies diluted in 5% milk powder in TBST buffer:

      • a. rabbit c-myc; use at 1:5000; 57 kDa.

      • b. mouse beta-actin; use at 1:10,000; 42 kDa.

    13. Wash membranes in TBST buffer.

    14. Detect primary antibodies with the following appropriate secondary antibodies diluted in 5% milk powder in TBST buffer.

      • a. goat anti-rabbit-HRP; use at 1:10,000.

      • b. goat anti-mouse-HRP; use at 1:10,000.

    15. Detect signal with chemiluminescent reagent.

    16. Image the chemiluminescent signal using a charge-coupled device (CCD) detection system.

    17. Repeat independently two additional times.

Deliverables

  • Data to be collected:

    1. STR profile and result of mycoplasma testing of P493-6 cells.

    2. Images of the chemiluminescence signal from the c-myc and beta-actin probed membrane (full images with ladder). (Compare to Figure 1B).

    3. Raw values and bar graphs of mean signal intensities of the bands normalized for beta-actin levels.

Confirmatory analysis plan

This experiment assesses the relative levels of c-Myc protein after re-induction, which was showed as a representative image in the original paper. The original paper showed an increase in c-Myc as soon as 1 hr after re-induction, which by 24 hr was similar to the tet-free condition. This replication attempt will perform the following statistical analysis listed below for each cohort.

  • Statistical Analysis:

Note: At the time of analysis we will perform the Shapiro–Wilk test and generate a quantile–quantile (q–q) plot to assess the normality of the data and also perform Levene's test to assess homoscedasticity. If the data appear skewed, we will perform an appropriate transformation in order to proceed with the proposed statistical analysis. If this is not possible, we will perform the equivalent non-parametric test.

  1. One-way ANOVA comparing the normalized c-Myc levels in cells cultured 0 hr, 1 hr, and 24 hr from tet release and tet-free control.

    • Planned comparisons with the Bonferroni correction:

      1. Normalized c-Myc levels in cells cultured 0 hr from tet release compared to cells cultured 24 hr from tet release.

      2. Normalized c-Myc levels in cells cultured in tet-free control medium compared to cells cultured 24 hr from tet release.

  • Meta-analysis of effect sizes:

    1. Compare the effect sizes of the two cohorts (separate FBS lots) and use a meta-analytic approach to combine the two cohort effects, which will be presented as a forest plot.

Known differences from the original study

The replication will only include the time points used in further studies instead of the full time-course shown in Figure 1B. A separate lot of serum was included to assess if there is variability between different batches of serum. All known differences are listed in the materials and reagents section above, with the originally used item listed in the comments section. All differences have the same capabilities as the original and are not expected to alter the experimental design.

Provisions for quality control

The cell line used in this experiment will undergo STR profiling to confirm its identity and will be sent for mycoplasma testing to ensure that there is no contamination. All of the raw data, including the image files and quantified bands from the western blot, will be uploaded to the project page on the OSF (https://osf.io/mokeb/) and made publically available. This experiment is also the quality control for the other replication protocols as it assesses the level of c-Myc re-induction with this system.

Protocol 2: total RNA expression during c-Myc re-activation

This experiment tests the effect of elevated levels of c-Myc on the total levels of RNA in P493-6 cells. This is a replication of the data presented in Figure 3E, which assess RNA levels by UV/VIS spectrophotometry.

Sampling

  • Experiment to be repeated a total of three times for a minimum power of 96%.

    1. See Power calculations section for details.

  • Each experiment has two cohorts:

    1. Cohort 1: P493-6 cells cultured in FBS lot #1.

    2. Cohort 2: P493-6 cells cultured in FBS lot #2.

  • Each cohort has three conditions:

    1. Cells cultured 0 hr after tet induction.

    2. Cells cultured 1 hr after tet induction.

    3. Cells cultured 24 hr after tet induction.

Materials and reagents

ReagentTypeManufacturerCatalog #Comments
P493-6 modified Burkitt's lymphoma cellsCell lineOriginal labFrom original lab
RPMI 1640 medium with sodium bicarbonate, without L-glutamineCell cultureSigma–AldrichR0883No catalog # listed in original paper
Tetracycline system approved FBSSerumClontech631105Two separate lots will be used
Ala-Gln (L-glutamine substitute)Cell cultureSigma–AldrichG8541Original lab used GlutaMAX from Invitrogen
T75 flaskLabwareSigma–AldrichZ707503Originally not specified
TetracyclinePharmacological agentSigma–AldrichT7660
C-chip disposable hemocytometerLabwareDigital BioDHC-N01No catalog # listed in original paper
Refrigerated centrifugeEquipmentEppendorf5810ROriginal lab used a Sorvall Legend centrifuge
TRI reagentChemicalSigma–AldrichT9424Included during communication with original authors. Original lab used Trizol
1-Bromo-3-chloropropaneChemicalSigma–AldrichB9673Included during communication with original authors
miRVana miRNA extraction kitBufferAmbionAM1561Original lab used #AM1560 with phenol, which is not used in protocol
Nuclease-free waterChemicalsSigma–AldrichW4502Original lab used nuclease-free water from Ambion (AM9938)
NanoDrop UV/Vis SpectrophotometerEquipmentThermo ScientificND-1000
NanoDrop OperatingSoftwareThermo ScientificVersion 3.3

Procedure

  • All cells will be sent for mycoplasma testing and STR profiling.

  • Tet-free media: RPMI1640 supplemented with 10% tet system approved FBS and 1% Ala-Gln.

  • Cells maintained at 37°C in a humidified atmosphere at 5% CO2.

  • P493-6 cells are maintained in two separate FBS lots (lot #1 and lot #2).

    1. Prepare stocks in upright T75 flask of P493-6 modified Burkitt's lymphoma cells cultured in tet-free media.

    2. On day 0, seed a flask with ∼2 × 107 P493-6 cells in 40 ml tet-free media with 0.1 µg/ml tet.

      • a. Count cells in each flask using a C-chip disposable hemocytometer.

    3. 72 hr later, spin cells down from both flasks, and wash cells three times in tet-free media.

    4. Seed three new flasks with 1.2 × 107 cells in 20 ml of fresh tet-free media.

      • a. Count cells using a C-chip disposable hemocytometer.

    5. At time points 0 hr (immediately), 1 hr, and 24 hr, harvest one flask, and prepare an aliquot of 1 × 107 cells.

      • a. Count cells in each flask using a C-chip disposable hemocytometer.

      • b. Homogenize sample in 1 ml Tri Reagent according to manufacturer's instructions.

      • c. Store at −80°C until further processing.

      • d. Do not exceed 1 × 107 cells per 1 ml Tri Reagent.

    6. For each sample, add 1/10 vol (100 µl per 1 ml Tri Reagent) miRNA Homogenate Additive, vortex, and incubate on ice for 10 min.

    7. Add 100 µl bromochloropropane per 1 ml Tri Reagent, vortex for 15–30 s, incubate the homogenate for 5 min at RT, then centrifuge at 12,000×g for 10 min at 4°C.

      • a. Avoid using chloroform containing isoamyl alcohol or water soluble organic solvents (ethanol, DMSO) as this will reduce DNA contamination downstream.

    8. Recover the aqueous phase and proceed directly to step F.I (Total RNA Isolation Procedure) in the miRVana isolation kit manufacturer's instructions.

      • a. Expect to take 312 µl of aqueous phase, add 390 µl of 100% ethanol, and place the total volume (702 µl) onto the filter cartridge.

      • b. Resuspend RNA in 100 µl of nuclease-free water for a total concentration of 100,000 cells/µl.

    9. Quantify RNA concentrations in each sample using a spectrometer.

      • a. Convert total RNA to ng per 1000 cells.

      • b. Record sample purity (A260/280 and A260/280 ratios).

    10. Repeat independently two additional times.

Deliverables

  • Data to be collected:

    1. STR profile and result of mycoplasma testing of P493-6 cells.

    2. Nanodrop measurements of RNA concentrations in total RNA preparations for each sample. Include A260/280 and A260/230 ratios.

    3. Bar graph of total RNA (ng) per 1000 cells for each condition. (Compare to Figure 3E).

Confirmatory analysis plan

This experiment assesses the relative levels of total RNA before and after re-induction of c-Myc. The original paper reported an increase in levels of absolute RNA over the timecourse of c-Myc re-induction. This replication attempt will perform the following statistical analysis listed below for each cohort.

  • Statistical analysis:

Note: At the time of analysis we will perform the Shapiro–Wilk test and generate a q–q plot to assess the normality of the data and also perform Levene's test to assess homoscedasiticity. If the data appear skewed we will perform an appropriate transformation in order to proceed with the proposed statistical analysis. If this is not possible we will perform the equivalent non-parametric test.

  1. One-way ANOVA comparing total RNA (ng/1000 cells) in cells cultured 0 hr, 1 hr, and 24 hr from tet release.

    • a. Planned comparisons:

      • i. Total RNA in cells cultured 0 hr from tet release compared to cells cultured 24 hr from tet release.

  • Meta-analysis of effect sizes:

    1. Compare the effect sizes of the original data to the two cohorts (separate FBS lots) and use a meta-analytic approach to combine the original and replication effects, which will be presented as a forest plot.

Known differences from the original study

A separate lot of serum was included to assess if there is variability between different batches of serum. All known differences are listed in the materials and reagents section above, with the originally used item listed in the comments section. All differences have the same capabilities as the original and are not expected to alter the experimental design.

Provisions for quality control

The cell line used in this experiment will undergo STR profiling to confirm its identity and will be sent for mycoplasma testing to ensure there is no contamination. The level of c-Myc re-induction in this system will be established in Protocol 1 as a comparison to the original paper. The sample purity (A260/280 and A260/230 ratios) of the isolated RNA from each sample will be reported. All of the raw data will be uploaded to the project page on the OSF (https://osf.io/mokeb/) and made publically available.

Protocol 3: transcript levels during c-Myc re-activation

This experiment, which includes Protocol 3 and 4, tests the effect of elevated levels of c-Myc on the mRNA levels/cell for a large set of genes in P493-6 cells. The data were analyzed to test the effect of c-Myc on transcript levels of genes that were expressed at reliable levels (>1 transcript/cell) or genes that were not expressed (<0.5 transcript/cell). This is a replication of the data presented in Figure 3F and Table S1, which assess the quantified mRNA levels/cells by digital gene expression (NanoString).

Sampling

  • Each experiment has two cohorts:

    1. Cohort 1: P493-6 cells cultured in FBS lot #1.

    2. Cohort 2: P493-6 cells cultured in FBS lot #2.

  • Each cohort has three conditons:

    1. Cells cultured 0 hr after tet induction.

    2. Cells cultured 1 hr after tet induction.

    3. Cells cultured 24 hr after tet induction.

  • Experiment to be repeated a total of three times.

    1. This replication attempt will be repeated three times to obtain averaged values of transcript/cell since it will be repeated three times in the other replication protocols.

Materials and reagents

ReagentTypeManufacturerCatalog #Comments
P493-6 modified Burkitt's lymphoma cellsCell lineOriginal labFrom original lab
RPMI 1640 medium with sodium bicarbonate, without L-glutamineCell cultureSigma–AldrichR0883No catalog # listed in original paper
Tetracycline system approved FBSSerumClontech631105Two separate lots will be used
Ala-Gln (L-glutamine substitute)Cell cultureSigma–AldrichG8541Original lab used GlutaMAX from Invitrogen
T75 flaskLabwareSigma–AldrichZ707503Originally not specified
TetracyclinePharmacological agentSigma–AldrichT7660
C-chip disposable hemocytometerLabwareDigital BioDHC-N01No catalog # listed in original paper
Refrigerated centrifugeEquipmentEppendorf5810ROriginal lab used a Sorvall Legend centrifuge
Buffer RLTBufferQiagen79216Original lab used the Buffer RLT from the RNeasy kit (#74104)
ß-mercaptoethanolChemicalsSigma–AldrichM3148Included during communication with original authors
Nuclease-free waterChemicalsSigma–AldrichW4502Original lab used nuclease-free water from Ambion (AM9938)
NanoDrop UV/Vis SpectrophotometerEquipmentThermo ScientificND-1000
NanoDrop OperatingSoftwareThermo ScientificVersion 3.3

Procedure

  • All cells will be sent for mycoplasma testing and STR profiling.

  • Tet-free media: RPMI1640 supplemented with 10% tet system approved FBS and 1% Ala-Gln.

  • Cells maintained at 37°C in a humidified atmosphere at 5% CO2.

  • P493-6 cells are maintained in two separate FBS lots (lot #1 and lot #2).

    1. Prepare stocks of upright T75 flask of P493-6 modified Burkitt's lymphoma cells cultured in tet-free media.

    2. On day 0, seed a flask with ∼2 × 107 P493-6 cells in 40-ml tet-free media with 0.1 µg/ml tet.

      • a. Count cells in each flask using a C-chip disposable hemocytometer.

      • b. This is performed for cells grown in both lots of FBS.

    3. 72 hr later, spin cells down from both flasks, and wash cells three times in tet-free media.

    4. Seed three new flasks with 1.2 × 107 cells in 20 ml of fresh tet-free media.

      • a. Count cells using a C-chip disposable hemocytometer.

    5. At time points 0 hr (immediately), 1 hr, and 24 hr, harvest one flask, and collect 1 × 106 cells.

      • a. Count cells in each flask using a C-chip disposable hemocytometer.

      • b. Lyse cells directly in 100 µl Buffer RLT supplemented with ß-mercaptoethanol (10 µl ß-ME per 1 ml Buffer RLT) for a concentration of 10,000 cells per µl.

      • c. Make multiple 4 µl aliquots and store RNA at −80°C until shipment. Avoid freeze/thaw cycles. Ship one aliquot/sample and store others for backup.

    6. Repeat independently two additional times.

    7. Ship cell lysate samples on dry ice to lab for NanoString nCounter gene expression assay (Protocol 4).

Deliverables

  • Data to be collected:

    1. STR profile and result of mycoplasma testing of P493-6 cells.

  • Sample delivered for further analysis:

    1. Cell lysate samples for digital gene expression assay (Protocol 4).

Confirmatory analysis plan

This experiment assesses if c-Myc re-induction alters the number of transcripts/cell in genes that were active (>1 transcript/cell) or silent (<0.5 transcript/cell) in cells with lower levels of c-Myc (0 hr after tet induction). The original paper reported an upregulation of active genes upon c-Myc re-induction, while genes not expressed remained silent. This replication attempt will perform the statistical analysis listed in Protocol 4 for each cohort.

Known differences from the original study

A separate lot of serum was included to assess if there is variability between different batches of serum. All known differences are listed in the materials and reagents section above, with the originally used item listed in the comments section. All differences have the same capabilities as the original and are not expected to alter the experimental design.

Provisions for quality control

The cell line used in this experiment will undergo STR profiling to confirm its identity and will be sent for mycoplasma testing to ensure there is no contamination. The level of c-Myc re-induction in this system will be established in Protocol 1 as a comparison to the original paper. All of the raw data will be uploaded to the project page on the OSF (https://osf.io/mokeb/) and made publically available.

Protocol 4: NanoString nCounter digital gene expression assay

This experiment, which includes Protocol 3 and 4, tests the effect of elevated levels of c-Myc on the mRNA levels/cell for a large set of genes in P493-6 cells. The data were analyzed to test the effect of c-Myc on transcript levels of genes that were expressed at reliable levels (>1 transcript/cell) or genes that were not expressed (<0.5 transcript/cell). This is a replication of the data presented in Figure 3F and Table S1, which assess the quantified mRNA levels/cells by digital gene expression (NanoString).

Sampling

  • Each experiment has two cohorts:

    1. Cohort 1: P493-6 cells cultured in FBS lot #1.

    2. Cohort 2: P493-6 cells cultured in FBS lot #2.

  • Each cohort has three conditons:

    1. Cells cultured 0 hr after tet induction.

    2. Cells cultured 1 hr after tet induction.

    3. Cells cultured 24 hr after tet induction.

  • Experiment will analyze transcript levels from 1409 unique genes for a minimum power of 80%.

    1. Based on the estimation of detecting 795 active genes and 541 silent genes, which is 56.5% and 38.4%, respectively, of total unique genes, similar to the original data.

    2. See Power Calculations section for details.

Materials and reagents

ReagentTypeManufacturerCatalog #Comments
nCounter Custom CodeSet (CS-1, CS-2, CS-MYC combined)ProbeNanoStringSee Table S1 of original paper for RefSeq IDs
nCounter GX Human Immunology KitProbeNanoStringGXA-HIM1-12
nCounter GX Human Kinase KitProbeNanoStringGXA-P2K1-12
nCounter Analysis SystemEquipmentNanoStringNCT-PREP-120Includes Prep Station and Digital Analyzer
nSolver AnalysisSoftwareNanoStringVersion 1.1

Procedure

  1. Process samples (from Protocol 3) according to manufacturer's instructions for ‘Cell Lysate Protocol’ (nCounter Gene Expression Assay Manual).

    • a. Thaw samples on ice, mix well, and briefly spin down contents of the tubes.

    • b. Incubate 4 µl of cell lysate overnight at 65°C in nCounter Reporter CodeSet, Capture ProbeSet and hybridization buffer.

    • c. Following hybridization, process samples immediately with the nCounter PrepStation and subsequently analyze on an nCounter Digital Analyzer.

    • d. Approximately 40,000 cells/hybridization assay.

  2. Analyze data for all target genes in all samples/flow cells using the nSolver Analysis Software.

    • a. Normalize the counts for all target genes in all samples/flow cells based on the positive spike-in controls to account for differences in hybridization efficiency and post-hybridization processing.

      • i. Sum the counts for the positive spike-in controls for each sample/flow cell to estimate overall assay efficiency.

      • ii. Calculate a normalization factor for each sample/flow cell based on the relative number of positive control counts.

      • iii. For each sample/flow cell, multiply the counts for each target gene and the control genes by the normalization factor for that sample/flow cell.

    • b. Calculate the average normalized background count for a sample/flow cell from the average of the eight negative control counts. Subtract this value from the normalized count for each gene target in that sample/flow cell to yield normalized, background-subtracted counts for each target gene.

    • c. Average triplicates for each sample.

    • d. For each gene, estimate its transcripts per cell level through linear interpolation of RNA spike-in positive controls using the approximation that the signal detected from a 0.5 femtomolar RNA spike in is equivalent to the signal detected for a transcript expressed at the equivalent of 1 transcript/cell.

Deliverables

  • Data to be collected:

    1. RCC data output files from nCounter Digital Analyzer before and after normalization, background subtraction, and approximation of transcripts/cell for each gene. Include positive and negative controls for each sample/flow cell.

    2. List of all genes with the transcript/cell estimations for each individual sample and the average across the triplicate samples. (Compare to Table S1).

    3. Box-and-whisker plot of transcript/cell estimates for the active (>1 transcript/cell) and silent (<0.5 transcript/cell) genes at 0, 1, and 24 hr. (Compare to Figure 3F).

Confirmatory analysis plan

To determine if the normalized, background-subtracted counts are statistically above background, a t-test or Wilcoxon rank-sum test, will be used to test against the average counts of the negative control genes. Genes are defined as transcriptionally active (if average expression across replicates is >1 transcript/cell at 0 hr) or transcriptionally silent (if average expression across replicates is <0.5 transcript/cell). Some genes are represented on multiple nCounter Reporter CodeSets, so the average expression will be computed from replicates of all CodeSets with the same RefSeq ID. Genes with an average expression between 0.5 and 1 transcript/cell at 0 hr will be excluded from the analysis of silent genes, similar to how the original paper analyzed the data, but will be included in an additional analysis of non-active genes.

This experiment assesses if c-Myc re-induction alters the number of transcripts/cell in genes that were active (>1 transcript/cell) or silent (<0.5 transcript/cell) in cells with lower levels of c-Myc (0 hr after tet induction). The original paper reported an upregulation of active genes upon c-Myc re-induction, while genes not expressed remained silent. This replication attempt will perform the following statistical analysis listed below.

  • Statistical analysis:

    1. Two-tailed Wilcoxon signed rank test of transcript levels of active genes in cells for the following comparisons with the Bonferroni correction:

      • a. Cultured 0 hr from tet release compared to 1 hr from tet release.

      • b. Cultured 0 hr from tet release compared to 24 hr from tet release.

      • c. Cultured 1 hr from tet release compared to 24 hr from tet release.

    2. Two-tailed Wilcoxon signed rank test of transcript levels of silent genes in cells for the following comparisons with the Bonferroni correction:

      • a. Cultured 0 hr from tet release compared to 1 hr from tet release.

      • b. Cultured 0 hr from tet release compared to 24 hr from tet release.

      • c. Cultured 1 hr from tet release compared to 24 hr from tet release.

  • Meta-analysis of effect sizes:

    1. Compare the effect sizes of the original data to the two cohorts (separate FBS lots) and use a meta-analytic approach to combine the original and replication effects, which will be presented as a forest plot.

Known differences from the original study

The original study used three separate custom code sets (CS-1, CS-2, and CS-MYC) and the replication will combine these into one custom code set. The three original custom code sets had a total of 495 genes, of which only 479 were unique, so the replication custom code set will only contain these unique genes. The nCounter GX human immunology kit from NanoString has been updated to a new version. In the second version, there are 106 new genes not present in the first version, of which 101 are unique to all genes examined in this experiment, while 38 genes that were in the first version are excluded in the second version. Thus, 63 unique genes will be added to the total gene set compared with the original study changing the total examined genes from 1346 to 1409. A separate lot of serum was included to assess if there is variability between different batches of serum. The original study analyzed these data using the Wilcoxon sum rank test, however since expressions of the same gene across different conditions are not independent, the Wilcoxon signed rank test, a non-parametric test for comparing two paired samples, will be used instead.

Provisions for quality control

Each of the nCounter Reporter CodeSets contain internal positive and negative controls that are used to normalize/plot a regression line to quantify relative levels of expression and to estimate the non-specific background, respectively. Genes will be tested to determine if the average normalized, background-subtracted counts are statistically above background. All of the raw data, including the data output files from the nCounter Digital Analyzer, will be uploaded to the project page on the OSF (https://osf.io/mokeb/) and made publically available.

Power calculations

Protocol 1

Power calculations are not applicable.

Protocol 2

Summary of original data (shared by original authors).

Data set being analyzedMeanSDN
Total RNA 0 hr from tet release4.2520.16173
Total RNA 1 hr from tet release4.0370.29463
Total RNA 24 hr from tet release5.4710.37523

Test family

  • ANOVA: Fixed effects, omnibus, one-way, alpha error = 0.05.

    1. Power calculations were performed with G*Power software (version 3.1.7) (Faul et al., 2007).

    2. ANOVA F statistic calculated with Graphpad Prism 6.0.

    3. Partial η2 calculated from Lakens (2013).

Power calculations for replication

GroupsF test statisticPartial η2Effect size fA priori powerTotal sample size
0 hr, 1 hr, 24 hrF(2, 6) = 21.21830.87612.6591499.9%9 (3/group)

Test family

  • Two-tailed t-test, difference between two independent means, alpha error = 0.05.

    1. Power calculations were performed with G*Power software (version 3.1.7) (Faul et al., 2007).

Power calculations for replication

Group 1Group 2Effect size dA priori powerGroup 1 sample sizeGroup 2 sample size
0 hr24 hr4.1926894796.3%33

Protocol 3 and 4

Summary of original data (obtained from Table S1)

Data set being analyzedNMeanSD
Active genes 0 hr from tet release76023.66788.104
Active genes 1 hr from tet release76030.418117.36
Active genes 24 hr from tet release76054.959179.67
Silent genes 0 hr from tet release5170.06460.1225
Silent genes 1 hr from tet release5170.08370.2810
Silent genes 24 hr from tet release5170.29811.3787
  1. Note: Active genes and silent genes are defined as >1 transcript/cell and <0.5 transcript/cell as in the original paper.

Test family

  • Two-tailed Wilcoxon signed rank test, alpha error = 0.01667.

    1. Power calculations were performed with G*Power software (version 3.1.7) (Faul et al., 2007).

Power calculations for replication (active genes)

Group 1Group 2Effect size dzA priori powerTotal sample size
0 hr from tet release1 hr from tet release0.223189780.2%*224*
0 hr from tet release24 hr from tet release0.303179380.3%*123*
1 hr from tet release24 hr from tet release0.289814780.2%*134*
  1. *

    795 is used based on the estimation of 56.5% of total unique genes being active making the power 99.9%.

Test family

  • Two-tailed Wilcoxon signed rank test, Bonferroni's correction, alpha error = 0.01667.

    1. Power calculations were performed with G*Power software (version 3.1.7) (Faul et al., 2007).

Power calculations for replication (silent genes)

Group 1Group 2Effect size dzA priori powerTotal sample size
0 hr from tet release1 hr from tet release0.1427513*80.0%*541*
0 hr from tet release24 hr from tet release0.174370780.1%364
1 hr from tet release24 hr from tet release0.168616080.0%389
  1. *

    This is a sensitivity calculation. The original effect size is 0.0807517.

  2. 541 is used based on the estimation of 38.4% of total unique genes being silent making the power 94.0%.

  3. 541 is used making the power 92.3%.

References

  1. 1
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12

Decision letter

  1. Michael R Green
    Reviewing Editor; Howard Hughes Medical Institute, University of Massachusetts Medical School, United States

eLife posts the editorial decision letter and author response on a selection of the published articles (subject to the approval of the authors). An edited version of the letter sent to the authors after peer review is shown, indicating the substantive concerns or comments; minor concerns are not usually shown. Reviewers have the opportunity to discuss the decision before the letter is sent (see review process). Similarly, the author response typically shows only responses to the major concerns raised by the reviewers.

Thank you for sending your work entitled “Registered report: Transcriptional Amplification in Tumor Cells with Elevated c-Myc” for consideration at eLife. Your article has been evaluated by Sean Morrison (Senior editor), a Reviewing editor, and 3 reviewers, one of whom is a biostatistician.

The Reviewing editor and the reviewers discussed their comments before we reached this decision, and the Reviewing editor has assembled the following comments to help you prepare a revised submission.

As detailed below, the reviewers raised a number of major concerns that need to be addressed in a revised Registered report.

Major comments:

1) Any replication of the Lin et al. paper needs to include both RNA profiling and ChIP-seq experiments. The Lin et al. paper has been quite controversial and was based in very large part on ChIP-seq data and its interpretation. Many of the conclusions were based on very subtle changes in data profiles. Subsequently, two papers were published in Nature (Walz et al. and Sabo et al.) that challenge the global claims of Lin et al. In both papers, the authors worked hard to accumulate comprehensive ChIP-seq and RNA expression data in carefully designed experimental systems. While some aspects of the Lin et al. paper may be correct, both Nature papers conclude that there is a set of defined target genes that are far more Myc responsive than others. Hence, reproducing only a subset of the Lin et al. experiments is unlikely to add anything new or resolve controversial claims.

The authors do not propose to reproduce the critical ChIP-seq data and they do not propose any analysis of RNAPII profiles that would support or conflict with the conclusions of the Lin et al. paper, namely that Myc promotes genome-wide transcriptional elongation. The proposal only focuses on RNA profiling without integration with binding of Myc and RNAPII.

2) More comprehensive RNA-seq analysis would determine global RNA expression in response to Myc and not be limited to a subset of genes represented by NanoString.

3) The cell line P493-6 has been established 15 years ago. The proliferation of these cells depends on c-Myc expression and the presence of serum. Serum is a major variable in this system and the majority of genes in stimulated cells are regulated by serum and not by c-Myc (Schlosser et al., Oncogene, 2005). The impact of different serum changes on P493-6 cells is highly significant. In some serum batches the cells barely grow after c-Myc activation. Unfortunately, the impact of various serum batches on P493-6 cells has never been systematically analyzed. Moreover, meanwhile many batches of P493-6 cells are distributed worldwide. These cells have been cultured with different types of serum in different laboratories. Exposure to different sera probably has altered the epigenetic state of P493-6 cells further contributing to variation in gene expression.

From the scientific view, it would be more helpful to study the stability of this biological system, e.g. by culturing P493-6 cells over longer periods of time in different batches of serum followed by a subsequent transcriptome analysis +/- Myc. At minimum, the authors should perform their experiments using multiple batches of serum to assess whether this significantly alters their results.

Statistical comments to the authors:

4) For protocol 1 and 2, authors propose to use ANOVA to analyze the data. Please make sure that the data do not violate the assumptions of the ANOVA: normality and homoscedasiticity. If the data do not fit the assumptions well enough, try to find a data transformation that makes them fit. If this doesn't work, you will need to apply a nonparametric counterpart of ANOVA such as Kruskal-Wallis test. In addition, performing contrast within the framework of ANOVA is more powerful than performing a separate t-test if the assumption of ANOVA is valid.

5) Authors used G*Power to calculate the power. I think that power calculation for protocol 3 & 4 is probably based on the test family t-test implemented in G*Power since there is no Wilcoxon sum rank test implemented in the G*Power. I suggest using t-tests as test family and matched pairs as statistical test to recalculate the power for protocol 3 and 4 (see below for justification). You will need to re-compute the effect size by calculating the SD for paired design, although mean difference between two groups will stay the same regardless.

6) Authors propose to use two-tailed Wilcoxon sum rank test, which has been used in the original paper. I suggest use either two-tailed Wilcoxon signed rank test or two-tailed paired t-test. If you prefer use G* Power to calculate power, then you will be left with two-tailed paired t-test option. The reason why paired analysis is needed is that expressions of the same gene across different conditions are not independent.

7) One major conclusion from the original paper (Lin et al., 2012) is that elevated c-Myc in tumor cells leads to amplification of the expression of actively transcribed genes, but has no effect on silent genes. I am wondering whether the authors will perform the same test to the silent genes, as well as the actively transcribed genes, to confirm the results from the original paper.

8) While it is very useful to leverage the previously reported effects to compute minimum power a priori, what you really need is to guarantee a minimum power on your own data. This can be done, a priori, by including some cross-study variation. This will be helpful for you to plan on the number of replicates and so forth. Papers by Giovanni Parmigiani and collaborators at the Dana–Farber provide some estimates about cross-study variation that could be used for this purpose. Worst case, you should budget some additional variability because of cross-study reproducibility, and increase the sample size as appropriate. We also want you to compute and report power post-hoc/on-the-fly on your own data. Some minimum power should be guaranteed using summaries of your own data.

[Editors' note: further revisions were requested prior to acceptance, as described below.]

Thank you for resubmitting your work entitled “Registered report: Transcriptional Amplification in Tumor Cells with Elevated c-Myc” for further consideration at eLife. Your revised article has been favorably evaluated by Sean Morrison (Senior editor), a Reviewing editor, and the original reviewers. As you might expect, there was a mixed response from the reviewers regarding the changes. On balance, we would like to move forward but would ask you to make one additional change. Different serum batches have only been included for the c-Myc off situation. To complete this control, please also include different serum batches for the c-Myc on situation.

https://doi.org/10.7554/eLife.04024.002

Author response

1) Any replication of the Lin et al. paper needs to include both RNA profiling and ChIP-seq experiments. The Lin et al. paper has been quite controversial and was based in very large part on ChIP-seq data and its interpretation. Many of the conclusions were based on very subtle changes in data profiles. Subsequently, two papers were published in Nature (Walz et al. and Sabo et al.) that challenge the global claims of Lin et al. In both papers, the authors worked hard to accumulate comprehensive ChIP-seq and RNA expression data in carefully designed experimental systems. While some aspects of the Lin et al. paper may be correct, both Nature papers conclude that there is a set of defined target genes that are far more Myc responsive than others. Hence, reproducing only a subset of the Lin et al. experiments is unlikely to add anything new or resolve controversial claims.

The authors do not propose to reproduce the critical ChIP-seq data and they do not propose any analysis of RNAPII profiles that would support or conflict with the conclusions of the Lin et al. paper, namely that Myc promotes genome-wide transcriptional elongation. The proposal only focuses on RNA profiling without integration with binding of Myc and RNAPII.

We agree that all of the experiments included in the original study are important, and choosing which experiments to replicate has been one of the great challenges of this project. The Reproducibility Project: Cancer Biology (RP:CB) aims to replicate experiments that are impactful, but does not necessarily aim to replicate all the impactful experiments in any given paper. In this case, the RP:CB core team felt that the RNA profiling experiment in Lin et al., 2012, was critical as it was a key part of the reported finding that Myc induction increased the expression of active genes but not silent genes, indicating the predominant effect of substantially elevated levels of Myc is amplified transcription of the existing gene expression program. We also agree the ChIP-seq experiments are a critical component of the overall reported finding, however these types of experiments (sequencing) are excluded from all articles. As this was only a part of the paper, the other experiments were still eligible. These exclusion criteria are outlined on the project page (https://osf.io/e81xl/wiki/studies) and in a Feature article describing the project (recently accepted for publication by eLife). We agree that the exclusion of certain experiments limits the scope of what can be analyzed by the project, but we are attempting to identify a balance of breadth of sampling for general inference with sensible investment of resources on replication projects to determine to what extent the included experiments are reproducible. Therefore, we will restrict our analysis to the experiments being replicated and will not include discussion of experiments not being replicated in this study.

We have also updated the Introduction to include the two papers (Walz et al. and Sabo et al.) that also examine transcriptional regulation by Myc.

2) More comprehensive RNA-seq analysis would determine global RNA expression in response to Myc and not be limited to a subset of genes represented by NanoString.

We agree RNA-seq analysis would be an informative approach to determine the global RNA expression in response to Myc, however Lin and colleagues did not use this approach. The Reproducibility Project: Cancer Biology aims to perform direct replications using the same methodology reported in the original paper. The use of RNA-seq analysis would be a conceptual replication, which we agree is a useful approach to test the experiment’s underlying hypothesis, but which is not an aim of the project.

3) The cell line P493-6 has been established 15 years ago. The proliferation of these cells depends on c-Myc expression and the presence of serum. Serum is a major variable in this system and the majority of genes in stimulated cells are regulated by serum and not by c-Myc (Schlosser et al., Oncogene, 2005). The impact of different serum changes on P493-6 cells is highly significant. In some serum batches the cells barely grow after c-Myc activation. Unfortunately, the impact of various serum batches on P493-6 cells has never been systematically analyzed. Moreover, meanwhile many batches of P493-6 cells are distributed worldwide. These cells have been cultured with different types of serum in different laboratories. Exposure to different sera probably has altered the epigenetic state of P493-6 cells further contributing to variation in gene expression.

From the scientific view, it would be more helpful to study the stability of this biological system, e.g. by culturing P493-6 cells over longer periods of time in different batches of serum followed by a subsequent transcriptome analysis +/- Myc. At minimum, the authors should perform their experiments using multiple batches of serum to assess whether this significantly alters their results.

This project focuses on direct replication of the experiments as detailed in the original report and with information provided by the original authors. Aspects of an experiment not included in the original study are occasionally added to ensure the quality of the research, but by no means is a requirement of this project; rather, it is an extension of the original work. Adding additional aspects not included in the original study can be of scientific interest, and can be included, if it is possible, to balance them with the main aim of this project: to perform a direct replication of the original experiment(s).

Therefore, we agree with the reviewers that there is scientific interest in better understanding the stability of this biological system. We will be using the same P493-6 cells used by the original authors (generously provided by Charles Lin) and will use the same source of Tet System Approved FBS as originally reported. However, even if we were to know the lot number of FBS originally used, it is unlikely we would be able to obtain it. We will use the same lot for all the experiments described, but have included an additional cohort to all experiments that will use a different lot of FBS. This cohort will be harvested in the same manner described for the cohort harvested 0 hr after tetracycline induction. These two cohorts, both 0 hr after tetracycline induction, but with different lots of FBS, will be compared to each other to assess if there is variability between different batches of serum.

Statistical comments to the authors:

4) For protocol 1 and 2, authors propose to use ANOVA to analyze the data. Please make sure that the data do not violate the assumptions of the ANOVA: normality and homoscedasiticity. If the data do not fit the assumptions well enough, try to find a data transformation that makes them fit. If this doesn't work, you will need to apply a nonparametric counterpart of ANOVA such as Kruskal-Wallis test. In addition, performing contrast within the framework of ANOVA is more powerful than performing a separate t-test if the assumption of ANOVA is valid.

Thank you for this suggestion. At the time of analysis, we will perform the Shapiro-Wilk test and generate a quantile-quantile (q-q) plot, to assess the normality of the data, and also perform the Brown-Forsythe test to assess homoscedasiticity. If the data appears skewed, we will perform the appropriate transformation in order to proceed with the proposed statistical analysis. We will note any changes or transformations made. If this doesn’t work we will perform the Kruskal-Wallis test and if necessary the Wilcoxon-Mann-Whitney test. We have also updated the manuscript to address this point.

Also, in protocol 1 we made the intended planned comparison (contrast) explicit to clarify that it is not a separate t-test.

5) Authors used G*Power to calculate the power. I think that power calculation for protocol 3 & 4 is probably based on the test family t-test implemented in G*Power since there is no Wilcoxon sum rank test implemented in the G*Power. I suggest using t-tests as test family and matched pairs as statistical test to recalculate the power for protocol 3 and 4 (see below for justification). You will need to re-compute the effect size by calculating the SD for paired design, although mean difference between two groups will stay the same regardless.

G*Power does have the Wilcoxon sum rank test as an option. It is called Wilcoxon-Mann-Whitney test, which is another name for this test. However, we agree with the recalculation for protocols 3 and 4 suggested below and now include the two-tailed Wilcoxon signed rank test instead of the Wilcoxon sum rank test.

6) Authors propose to use two-tailed Wilcoxon sum rank test, which has been used in the original paper. I suggest use either two-tailed Wilcoxon signed rank test or two-tailed paired t-test. If you prefer use G* power to calculate power, then you will be left with two-tailed paired t-test option. The reason why paired analysis is needed is that expressions of the same gene across different conditions are not independent.

We agree with this assessment. A two-tailed Wilcoxon signed rank test is the appropriate test because the data is paired and the normality assumption does not hold. We have recalculated power and sample size for protocols 3 and 4 accordingly. We also changed the language in the analysis plan and added this as a known difference from the original study.

7) One major conclusion from the original paper (Lin et al., 2012) is that elevated c-Myc in tumor cells leads to amplification of the expression of actively transcribed genes, but has no effect on silent genes. I am wondering whether the authors will perform the same test to the silent genes, as well as the actively transcribed genes, to confirm the results from the original paper.

We agree and have included the analysis of silent genes in the analysis plan and performed sensitivity calculations to determine the effect size that will be detected with 80% power. In the original paper (Lin et al., 2012), the authors determined actively transcribed genes as > 1 transcript/cell and silent genes as < 0.5 transcript/cell in cells with lower levels of c-Myc (0 hr after tet induction). However, this excludes 69 genes from the analysis that fall between 1 and 0.5. Thus, in the analysis plan we have included silent genes as defined by the original authors (< 0.5 transcript/cell) and also non-active genes (< 1 transcript/cell).

8) While it is very useful to leverage the previously reported effects to compute minimum power a priori, what you really need is to guarantee a minimum power on your own data. This can be done, a priori, by including some cross-study variation. This will be helpful for you to plan on the number of replicates and so forth. Papers by Giovanni Parmigiani and collaborators at the Dana–Farber provide some estimates about cross-study variation that could be used for this purpose. Worst case, you should budget some additional variability because of cross-study reproducibility, and increase the sample size as appropriate. We also want you to compute and report power post-hoc/on-the-fly on your own data. Some minimum power should be guaranteed using summaries of your own data.

We thank the reviewers for these suggestions. The cross-study variation, such as approaches that utilize the 95% confidence interval of the effect size, can be useful in conducting power calculations when planning adequate sample sizes for detecting the true population effect size, which requires a range of possible observed effect sizes. However, the Reproducibility Project: Cancer Biology is designed to conduct replications that have 80% power to detect the point estimate of the originally reported effect size. While this has the limitation of being underpowered to detect smaller effects than what is originally reported, this standardizes the approach across all studies to be designed to detect the originally reported effect size with at least 80% power. Also, while the minimum power guarantee is beneficial for observing a range of possible effect sizes, the experiments in this replication, and all experiments in the project, are designed to detect the originally reported effect size with a minimum power of 80%. Thus, performing power calculations during or after data collection is not necessary in this replication attempt as all studies included are already designed to meet a minimum power or are identified beforehand as being underpowered and thus are not included in the confirmatory analysis plan. The papers by Giovanni Parmigiani and collaborators highlight the importance of accounting for variability that can occur across different studies, specifically gene expression data. While it is possible for a difference in variance between the originally reported results and the replication data, this will be reflected in the presentation of the data and a possible reason for obtaining a different effect size estimate.

[Editors’ note: further revisions were requested prior to acceptance, as described below.]

Thank you for resubmitting your work entitled ”Registered report: Transcriptional Amplification in Tumor Cells with Elevated c-Myc” for further consideration at eLife. Your revised article has been favorably evaluated by Sean Morrison (Senior editor), a Reviewing editor, and the original reviewers. As you might expect, there was a mixed response from the reviewers regarding the changes. On balance, we would like to move forward but would ask you to make one additional change. Different serum batches have only been included for the c-Myc off situation. To complete this control, please also include different serum batches for the c-Myc on situation.

We agree and have adjusted the manuscript. We will use two different lots of serum to grow the cells for all the experiments.

https://doi.org/10.7554/eLife.04024.003

Article and author information

Author details

  1. David Blum

    Bioexpression and Fermentation Facility, University of Georgia, Athens, Georgia
    Contribution
    DB, Drafting or revising the article
    Competing interests
    DB: This is a Science Exchange Associated lab.
  2. Haiping Hao

    JHMI Deep Sequencing and Microarray Core Facility, Johns Hopkins University, Baltimore
    Contribution
    HH, Drafting or revising the article
    Competing interests
    HH: This is a Science Exchange Associated lab.
  3. Michael McCarthy

    University of Oxford, Oxford, United Kingdom
    Contribution
    MMC, Drafting or revising the article
    Competing interests
    No competing interests declared.
  4. Reproducibility Project: Cancer Biology

    Contribution
    RP:CB, Conception and design, Drafting or revising the article
    For correspondence
    tim@cos.io
    Competing interests
    RP:CB: EI, FT and JL are employed by and hold shares in Science Exchange Inc.
    1. Elizabeth Iorns, Science Exchange, Palo Alto, California
    2. William Gunn, Mendeley, London, United Kingdom
    3. Fraser Tan, Science Exchange, Palo Alto, California
    4. Joelle Lomax, Science Exchange, Palo Alto, California
    5. Timothy Errington, Center for Open Science, Charlottesville, Virginia

Funding

Laura and John Arnold Foundation

  • Reproducibility Project: Cancer Biology

The Reproducibility Project: Cancer Biology is funded by the Laura and John Arnold Foundation, provided to the Center for Open Science in collaboration with Science Exchange. The funder had no role in study design or the decision to submit the work for publication.

Acknowledgements

The Reproducibility Project: Cancer Biology core team would like to thank the original authors, in particular Charles Lin, for generously sharing critical information as well as the P493-6 modified Burkitt's lymphoma cells to ensure the fidelity and quality of this replication attempt. We thank Courtney Soderberg at the Center for Open Science for assistance with statistical analyses. We would also like to thank the following companies for generously donating reagents to the Reproducbility Project: Cancer Biology; American Type Culture Collection (ATCC), BioLegend, Cell Signaling Technology, Charles River Laboratories, Corning Incorporated, DDC Medical, EMD Millipore, Harlan Laboratories, LI-COR Biosciences, Mirus Bio, Novus Biologicals, Sigma–Aldrich, and System Biosciences (SBI).

Reviewing Editor

  1. Michael R Green, Reviewing Editor, Howard Hughes Medical Institute, University of Massachusetts Medical School, United States

Publication history

  1. Received: July 16, 2014
  2. Accepted: December 14, 2014
  3. Version of Record published: January 26, 2015 (version 1)

Copyright

© 2015, Blum et al.

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 1,637
    Page views
  • 82
    Downloads
  • 2
    Citations

Article citation count generated by polling the highest count across the following sources: Crossref, PubMed Central, Scopus.

Comments

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Download citations (links to download the citations from this article in formats compatible with various reference manager tools)

Open citations (links to open the citations from this article in various online reference manager services)

Further reading

  1. CY Lin et al.
    1
    1. Cancer Biology
    Curated by Roger Davis et al.
    Collection Updated

    Investigating reproducibility in preclinical cancer research.

    1. Biochemistry
    2. Biophysics and Structural Biology
    Lindsay D Clark et al.
    Research Article Updated