Abstract

The Reproducibility Project: Cancer Biology seeks to address growing concerns about reproducibility in scientific research by conducting replications of 50 papers in the field of cancer biology published between 2010 and 2012. This Registered report describes the proposed replication plan of key experiments from ‘Transcriptional amplification in tumor cells with elevated c-Myc’ by Lin et al. (2012), published in Cell in 2012. The experiments that will be replicated are those reported in Figures 3E and 3F. In these experiments, elevated levels of c-Myc in the P493-6 cell model of Burkitt's lymphoma results in an increase of the total level of RNA using UV/VIS spectrophotometry (Figure 3E; Lin et al., 2012) and on the mRNA levels/cell for a large set of genes using digital gene expression technology (Figure 3F; Lin et al., 2012). The Reproducibility Project: Cancer Biology is a collaboration between the Center for Open Science and Science Exchange, and the results of the replications will be published in eLife.

DOI: http://dx.doi.org/10.7554/eLife.04024.001

Original article

  1. Transcriptional amplification in tumor cells with elevated c-Myc

    1. CY Lin
    2. J Loven
    3. PB Rahl
    4. RM Paranal
    5. CB Burge
    6. JE Bradner
    7. TI Lee
    8. RA Young
    Cell 2012;151:56-67

Main text

Introduction

The proto-oncogene MYC is frequently amplified in human cancers and encodes the transcription factor c-Myc, which is associated with a variety of cellular processes such as cell growth and proliferation (Dang, 2013). While overexpressed c-Myc is known to contribute to tumorigenesis, an understanding of how this process occurs is complicated by a number of issues, including the large number of binding sites and the diversity between systems. While this was thought to occur by regulation of a specific subset of genes, Lin et al. (2012) present findings that c-Myc functions by globally amplifying the expression of actively transcribed genes.

The system used is the human P493-6 B cell model of Burkitt's lymphoma, which contains a tetracycline-repressible MYC transgene, allowing for titration of c-Myc protein (Schuhmacher et al., 1999; Pajic et al., 2000). The levels of c-Myc can be reduced and subsequently re-induced in a gradual time-dependent manner, as determined by western blot, which is shown in Figure 1B (Lin et al., 2012). As soon as 1 hr after re-induction, the protein levels of c-Myc increased above the repressed levels, which by 24 hr were similar to the tetracycline-free condition (Lin et al., 2012). This system has been used in other studies with similar results observed (Schuhmacher et al., 1999; Pajic et al., 2000). This experiment is important to replicate because it assesses the level of c-Myc re-induction with this system that will be used in the following experiments. This experiment is replicated in Protocol 1.

In Figure 3E the total levels of RNA, in P493-6 cells before and after re-induction of c-Myc, was determined by UV/VIS spectrophotometry (Lin et al., 2012). Lin et al. (2012) reported an increase in levels of absolute RNA over the timecourse of c-Myc re-induction. This experiment shows that c-Myc increases RNA content per cell, indicating c-Myc functions primarily in transcriptional amplification. Related experiments, using mouse B cells, also observed the same c-Myc-dependent amplification of cellular RNA content (Nie et al., 2012; Sabò et al., 2014). However, similar experiments in 3T9 fibroblasts or U2OS cells expressing an inducible c-Myc did not observe an increase in cellular RNA content (Sabò et al., 2014; Walz et al., 2014). Interestingly, an increase in total cellular RNA content was observed in 3T9 fibroblasts following serum stimulation, which was not observed when c-Myc was deleted in these cells (Sabò et al., 2014). This experiment is replicated in Protocol 2.

In Figure 3F, Lin et al. (2012) examined the transcriptional profile of a large number of genes from multiple functional categories in cells before and after re-induction of c-Myc using digital gene expression. Re-induction of c-Myc increased the number of active genes (defined as greater than 1 transcript/cell), but did not alter silent genes (defined as less than 0.5 transcript/cell) (Lin et al., 2012). This key finding suggests that elevated c-Myc levels lead to an amplification of the existing transcriptional profile. This finding was observed in other experiments using mouse B cells treated with the Myc-Max dimerization inhibitor 10058-F4 and analyzed by ChIP-Seq (Nie et al., 2012). Another report using P493-6 cells also found an increase in transcription of c-Myc target genes when c-Myc re-induction was titrated to different concentrations by microarray analysis (Schuhmacher and Eick, 2013). This experiment is replicated in Protocols 3 and 4. Recently, two papers were published that used primarily RNA-seq and ChIP-seq to focus on assessing if the transcriptional effects of c-Myc are direct or indirect (Alderton, 2014). These studies found that RNA amplification and promoter/enhancer invasion by c-Myc were separable events in 3T9 fibroblasts and U2OS cells, suggesting that c-Myc regulates a distinct subset of genes, which indirectly lead to RNA amplification (Sabò et al., 2014; Walz et al., 2014).

Materials and methods

Protocol 1: Western blot of c-Myc reactivation in tetracyclin-repressible system

This experiment tests the expression of the tetracycline (tet)-repressible Myc transgene after repression by tet and re-induction by removal into tet-free medium. This is a replication of the data presented in Figure 1B and assesses the levels of c-Myc protein. One major variable in this system is serum, which has been reported to stimulate the expression of a majority of genes independently from c-Myc (Schlosser et al., 2005). P493-6 cells will be cultured with two separate lots of serum that will be maintained throughout all the experiments to assess if there is variability between different batches of serum.

Sampling

  • Experiment to be repeated a total of three times.

    1. This replication attempt will be repeated three times to analyze if this effect is detected and to assess the variance of this system, since it will be repeated three times in the other replication protocols.

  • Each experiment has two cohorts:

    1. Cohort 1: P493-6 cells cultured in FBS lot #1.

    2. Cohort 2: P493-6 cells cultured in FBS lot #2.

  • Each cohort has four conditions:

    1. Cells cultured in tet-free media.

    2. Cells cultured 0 hr after tet induction.

    3. Cells cultured 1 hr after tet induction.

    4. Cells cultured 24 hr after tet induction.

Materials and reagents

ReagentTypeManufacturerCatalog #Comments
P493-6 modified Burkitt's lymphoma cellsCell lineOriginal labFrom original lab
RPMI 1640 medium with sodium bicarbonate, without L-glutamineCell cultureSigma–AldrichR0883No catalog # listed in original paper
Tetracycline system approved FBSSerumClontech631105Two separate lots will be used
Ala-Gln (L-glutamine substitute)Cell cultureSigma–AldrichG8541Original lab used GlutaMAX from Invitrogen
T75 flaskLabwareSigma–AldrichZ707503Originally not specified
TetracyclinePharmacological agentSigma–AldrichT7660
C-chip disposable hemocytometerLabwareDigital BioDHC-N01No catalog # listed in original paper
Refrigerated centrifugeEquipmentEppendorf5810ROriginal lab used a Sorvall Legend centrifuge
PBS, without MgCl2 and CaCl2BuffersSigma–AldrichD8537Originally not specified
RIPA lysis bufferBuffersSigma–AldrichR0278
SIGMAFAST Protease Inhibitor TabletsInhibitorsSigma-AldrichS8820Original lab used Halt Protease Inhibitor cocktail from Thermo-Fisher
Phosphatase inhibitor cocktail 2InhibitorsSigma–AldrichP5726
Phosphatase inhibitor cocktail 3InhibitorsSigma–AldrichP0044
Bradford ReagentReporter assaySigma–AldrichB6916Original lab used Bradford Reagent from Bio-Rad
TruPAGE LDS sample buffer (4X)BuffersSigma-AldrichPCG3009Originally not specified
ß-mercaptoethanolChemicalsSigma–AldrichM3148Originally not specified
Nuclease-free waterChemicalsSigma–AldrichW4502Original lab used nuclease-free water from Ambion (AM9938)
Mini-PROTEAN Electrophoresis SystemEquipmentBio-RadOriginally not specified
TruPAGE Precast Gels 4–20%Western materialsSigma-AldrichPCG2012Original lab used 4–12% Bis-Tris Midi gels from Invitrogen
TruPAGE TEA-Tricine SDS Running Buffer (20X)BuffersSigma-AldrichPCG3001Originally not specified
ECL DualVue Western Markers (15–150 kDa)Western materialsSigma–AldrichGERPN810Originally not specified
Hybond P Western blotting membranes; PVDFWestern materialsSigma–AldrichGE10600029No catalog # listed in original paper
TruPAGE Transfer Buffer (20X)BuffersSigma-AldrichPCG3011Originally not specified
MethanolChemicalsSigma–Aldrich494437Originally not specified
Semi-dry TransBlot SD SystemEquipmentBio-Rad170–3940
5% milk powderWestern materialsSigma–AldrichM7409Originally not specified
10X TBS buffered salineBuffersSigma–AldrichT5912Originally not specified
Tween-20ChemicalsSigma–AldrichP1379Originally not specified
Rabbit c-MycAntibodiesEpitomics1472-1Dilute 1:5000; 57 kDa
Mouse ß-actinAntibodiesSigma–AldrichA5441Dilute 1:10,000; 42 kDa
Goat anti-rabbit-HRPAntibodiesSigma–AldrichGERPN2124Originally not specified. Dilute 1:10,000
Goat anti-mouse-HRPAntibodiesSigma–AldrichGERPN2124Originally not specified. Dilute 1:10,000
ECL Prime Chemiluminescent reagentWestern materialsSigma–AldrichGERPN2232Originally not specified
G:BOX iChemi XTEquipmentSyngeneOriginal lab used a Gel Dox XR+ System from Bio-Rad
GeneSnapSoftwareSyngeneVersion 6.00.19Original lab used Image Lab software from Bio-Rad
Procedure
Notes

  • All cells will be sent for mycoplasma testing and STR profiling.

  • Tet-free media: RPMI1640 supplemented with 10% tet system approved FBS and 1% Ala-Gln.

  • Cells maintained at 37°C in a humidified atmosphere at 5% CO2.

  • P493-6 cells are maintained in two separate FBS lots (lot #1 and lot #2).

    1. Prepare stocks in upright T75 flasks of P493-6 modified Burkitt's lymphoma cells cultured in tet-free media.

    2. On day 0, seed Flask A with ∼1 × 107 P493-6 cells in 20 ml tet-free media, and Flask B with ∼2 × 107 P493-6 cells in 40 ml tet-free media with 0.1 µg/ml tet for a concentration of 0.5 × 106 cells/ml.

      • a. Count cells in each flask using a C-chip disposable hemocytometer.

    3. 72 hr later, spin cells down from flasks, and wash cells three times in tet-free media.

    4. From Flask A cells, seed 1 new flask with 1.2 × 107 in fresh tet-free media.

      • a. Count cells using a C-chip disposable hemocytometer.

    5. From Flask B cells, seed three new flasks with 1.2 × 107 cells in 20 ml of fresh tet-free media.

      • a. Count cells using a C-chip disposable hemocytometer.

    6. At time points 0 hr (immediately), 1 hr, and 24 hr, harvest protein lysates from 1 × 107 cells from a Flask B cell suspension.

      • a. Count cells in each flask using a C-chip disposable hemocytometer.

      • b. To harvest lysates:

        • i. Pellet 1 × 107 cells at 4°C at 1200 rpm for 5 min in a Sorvall Legend centrifuge.

        • ii. Wash pellets once with ice-cold 1X PBS.

          • Cell pellets can be snap frozen in liquid nitrogen and stored at −80°C until needed.

        • iii. Resuspend pellets in 100 µl RIPA lysis buffer containing 2× SIGMAFAST Protease inhibitors and 2× Phosphatase inhibitor cocktails 2 and 3.

    7. At 24 hr, harvest protein lysate from tet-free cells (Flask A) as described in step 6.

    8. Quantify total protein concentration in each lysate using the Bradford assay, according to the manufacturer's instructions.

      • a. Use a bovine serum albumin standard curve for the quantification.

    9. Run an electrophoresis gel under denaturing conditions, loading the same volume of 50 µg of total protein lysate, with 1× SDS loading buffer, 2.5% ß-mercaptoethanol, and nuclease-free water for each condition, on a 4–20% TruPAGE precast gel. Run at 200 V until the dye front reaches the reference line following manufacturer's instructions.

      • a. Samples run per gel:

        • i. Protein molecular weight marker.

        • ii. 0 hr from tet release (FBS lot #1).

        • iii. 1 hr from tet release (FBS lot #1).

        • iv. 24 hr from tet release (FBS lot #1).

        • v. tet-free control (FBS lot #1).

        • vi. 0 hr from tet release (FBS lot #2).

        • vii. 1 hr from tet release (FBS lot #2).

        • viii. 24 hr from tet release (FBS lot #2).

        • ix. tet-free control (FBS lot #2).

    10. Transfer gel to a PVDF membrane, using the semi-dry TransBlot SD system according to manufacturer's instructions.

    11. Block with 5% milk powder in TBST buffer.

      • a. TBST buffer: TBS with 0.2% Tween-20.

    12. Probe membrane with the following primary antibodies diluted in 5% milk powder in TBST buffer:

      • a. rabbit c-myc; use at 1:5000; 57 kDa.

      • b. mouse beta-actin; use at 1:10,000; 42 kDa.

    13. Wash membranes in TBST buffer.

    14. Detect primary antibodies with the following appropriate secondary antibodies diluted in 5% milk powder in TBST buffer.

      • a. goat anti-rabbit-HRP; use at 1:10,000.

      • b. goat anti-mouse-HRP; use at 1:10,000.

    15. Detect signal with chemiluminescent reagent.

    16. Image the chemiluminescent signal using a charge-coupled device (CCD) detection system.

    17. Repeat independently two additional times.

Deliverables

  • Data to be collected:

    1. STR profile and result of mycoplasma testing of P493-6 cells.

    2. Images of the chemiluminescence signal from the c-myc and beta-actin probed membrane (full images with ladder). (Compare to Figure 1B).

    3. Raw values and bar graphs of mean signal intensities of the bands normalized for beta-actin levels.

Confirmatory analysis plan

This experiment assesses the relative levels of c-Myc protein after re-induction, which was showed as a representative image in the original paper. The original paper showed an increase in c-Myc as soon as 1 hr after re-induction, which by 24 hr was similar to the tet-free condition. This replication attempt will perform the following statistical analysis listed below for each cohort.

  • Statistical Analysis:

Note: At the time of analysis we will perform the Shapiro–Wilk test and generate a quantile–quantile (q–q) plot to assess the normality of the data and also perform Levene's test to assess homoscedasticity. If the data appear skewed, we will perform an appropriate transformation in order to proceed with the proposed statistical analysis. If this is not possible, we will perform the equivalent non-parametric test.

  1. One-way ANOVA comparing the normalized c-Myc levels in cells cultured 0 hr, 1 hr, and 24 hr from tet release and tet-free control.

    • Planned comparisons with the Bonferroni correction:

      1. Normalized c-Myc levels in cells cultured 0 hr from tet release compared to cells cultured 24 hr from tet release.

      2. Normalized c-Myc levels in cells cultured in tet-free control medium compared to cells cultured 24 hr from tet release.

  • Meta-analysis of effect sizes:

    1. Compare the effect sizes of the two cohorts (separate FBS lots) and use a meta-analytic approach to combine the two cohort effects, which will be presented as a forest plot.

Known differences from the original study

The replication will only include the time points used in further studies instead of the full time-course shown in Figure 1B. A separate lot of serum was included to assess if there is variability between different batches of serum. All known differences are listed in the materials and reagents section above, with the originally used item listed in the comments section. All differences have the same capabilities as the original and are not expected to alter the experimental design.

Provisions for quality control

The cell line used in this experiment will undergo STR profiling to confirm its identity and will be sent for mycoplasma testing to ensure that there is no contamination. All of the raw data, including the image files and quantified bands from the western blot, will be uploaded to the project page on the OSF (https://osf.io/mokeb/) and made publically available. This experiment is also the quality control for the other replication protocols as it assesses the level of c-Myc re-induction with this system.

Protocol 2: total RNA expression during c-Myc re-activation

This experiment tests the effect of elevated levels of c-Myc on the total levels of RNA in P493-6 cells. This is a replication of the data presented in Figure 3E, which assess RNA levels by UV/VIS spectrophotometry.

Sampling

  • Experiment to be repeated a total of three times for a minimum power of 96%.

    1. See Power calculations section for details.

  • Each experiment has two cohorts:

    1. Cohort 1: P493-6 cells cultured in FBS lot #1.

    2. Cohort 2: P493-6 cells cultured in FBS lot #2.

  • Each cohort has three conditions:

    1. Cells cultured 0 hr after tet induction.

    2. Cells cultured 1 hr after tet induction.

    3. Cells cultured 24 hr after tet induction.

Materials and reagents

ReagentTypeManufacturerCatalog #Comments
P493-6 modified Burkitt's lymphoma cellsCell lineOriginal labFrom original lab
RPMI 1640 medium with sodium bicarbonate, without L-glutamineCell cultureSigma–AldrichR0883No catalog # listed in original paper
Tetracycline system approved FBSSerumClontech631105Two separate lots will be used
Ala-Gln (L-glutamine substitute)Cell cultureSigma–AldrichG8541Original lab used GlutaMAX from Invitrogen
T75 flaskLabwareSigma–AldrichZ707503Originally not specified
TetracyclinePharmacological agentSigma–AldrichT7660
C-chip disposable hemocytometerLabwareDigital BioDHC-N01No catalog # listed in original paper
Refrigerated centrifugeEquipmentEppendorf5810ROriginal lab used a Sorvall Legend centrifuge
TRI reagentChemicalSigma–AldrichT9424Included during communication with original authors. Original lab used Trizol
1-Bromo-3-chloropropaneChemicalSigma–AldrichB9673Included during communication with original authors
miRVana miRNA extraction kitBufferAmbionAM1561Original lab used #AM1560 with phenol, which is not used in protocol
Nuclease-free waterChemicalsSigma–AldrichW4502Original lab used nuclease-free water from Ambion (AM9938)
NanoDrop UV/Vis SpectrophotometerEquipmentThermo ScientificND-1000
NanoDrop OperatingSoftwareThermo ScientificVersion 3.3
Procedure

  • All cells will be sent for mycoplasma testing and STR profiling.

  • Tet-free media: RPMI1640 supplemented with 10% tet system approved FBS and 1% Ala-Gln.

  • Cells maintained at 37°C in a humidified atmosphere at 5% CO2.

  • P493-6 cells are maintained in two separate FBS lots (lot #1 and lot #2).

    1. Prepare stocks in upright T75 flask of P493-6 modified Burkitt's lymphoma cells cultured in tet-free media.

    2. On day 0, seed a flask with ∼2 × 107 P493-6 cells in 40 ml tet-free media with 0.1 µg/ml tet.

      • a. Count cells in each flask using a C-chip disposable hemocytometer.

    3. 72 hr later, spin cells down from both flasks, and wash cells three times in tet-free media.

    4. Seed three new flasks with 1.2 × 107 cells in 20 ml of fresh tet-free media.

      • a. Count cells using a C-chip disposable hemocytometer.

    5. At time points 0 hr (immediately), 1 hr, and 24 hr, harvest one flask, and prepare an aliquot of 1 × 107 cells.

      • a. Count cells in each flask using a C-chip disposable hemocytometer.

      • b. Homogenize sample in 1 ml Tri Reagent according to manufacturer's instructions.

      • c. Store at −80°C until further processing.

      • d. Do not exceed 1 × 107 cells per 1 ml Tri Reagent.

    6. For each sample, add 1/10 vol (100 µl per 1 ml Tri Reagent) miRNA Homogenate Additive, vortex, and incubate on ice for 10 min.

    7. Add 100 µl bromochloropropane per 1 ml Tri Reagent, vortex for 15–30 s, incubate the homogenate for 5 min at RT, then centrifuge at 12,000×g for 10 min at 4°C.

      • a. Avoid using chloroform containing isoamyl alcohol or water soluble organic solvents (ethanol, DMSO) as this will reduce DNA contamination downstream.

    8. Recover the aqueous phase and proceed directly to step F.I (Total RNA Isolation Procedure) in the miRVana isolation kit manufacturer's instructions.

      • a. Expect to take 312 µl of aqueous phase, add 390 µl of 100% ethanol, and place the total volume (702 µl) onto the filter cartridge.

      • b. Resuspend RNA in 100 µl of nuclease-free water for a total concentration of 100,000 cells/µl.

    9. Quantify RNA concentrations in each sample using a spectrometer.

      • a. Convert total RNA to ng per 1000 cells.

      • b. Record sample purity (A260/280 and A260/280 ratios).

    10. Repeat independently two additional times.

Deliverables

  • Data to be collected:

    1. STR profile and result of mycoplasma testing of P493-6 cells.

    2. Nanodrop measurements of RNA concentrations in total RNA preparations for each sample. Include A260/280 and A260/230 ratios.

    3. Bar graph of total RNA (ng) per 1000 cells for each condition. (Compare to Figure 3E).

Confirmatory analysis plan

This experiment assesses the relative levels of total RNA before and after re-induction of c-Myc. The original paper reported an increase in levels of absolute RNA over the timecourse of c-Myc re-induction. This replication attempt will perform the following statistical analysis listed below for each cohort.

  • Statistical analysis:

Note: At the time of analysis we will perform the Shapiro–Wilk test and generate a q–q plot to assess the normality of the data and also perform Levene's test to assess homoscedasiticity. If the data appear skewed we will perform an appropriate transformation in order to proceed with the proposed statistical analysis. If this is not possible we will perform the equivalent non-parametric test.

  1. One-way ANOVA comparing total RNA (ng/1000 cells) in cells cultured 0 hr, 1 hr, and 24 hr from tet release.

    • a. Planned comparisons:

      • i. Total RNA in cells cultured 0 hr from tet release compared to cells cultured 24 hr from tet release.

  • Meta-analysis of effect sizes:

    1. Compare the effect sizes of the original data to the two cohorts (separate FBS lots) and use a meta-analytic approach to combine the original and replication effects, which will be presented as a forest plot.

Known differences from the original study

A separate lot of serum was included to assess if there is variability between different batches of serum. All known differences are listed in the materials and reagents section above, with the originally used item listed in the comments section. All differences have the same capabilities as the original and are not expected to alter the experimental design.

Provisions for quality control

The cell line used in this experiment will undergo STR profiling to confirm its identity and will be sent for mycoplasma testing to ensure there is no contamination. The level of c-Myc re-induction in this system will be established in Protocol 1 as a comparison to the original paper. The sample purity (A260/280 and A260/230 ratios) of the isolated RNA from each sample will be reported. All of the raw data will be uploaded to the project page on the OSF (https://osf.io/mokeb/) and made publically available.

Protocol 3: transcript levels during c-Myc re-activation

This experiment, which includes Protocol 3 and 4, tests the effect of elevated levels of c-Myc on the mRNA levels/cell for a large set of genes in P493-6 cells. The data were analyzed to test the effect of c-Myc on transcript levels of genes that were expressed at reliable levels (>1 transcript/cell) or genes that were not expressed (<0.5 transcript/cell). This is a replication of the data presented in Figure 3F and Table S1, which assess the quantified mRNA levels/cells by digital gene expression (NanoString).

Sampling

  • Each experiment has two cohorts:

    1. Cohort 1: P493-6 cells cultured in FBS lot #1.

    2. Cohort 2: P493-6 cells cultured in FBS lot #2.

  • Each cohort has three conditons:

    1. Cells cultured 0 hr after tet induction.

    2. Cells cultured 1 hr after tet induction.

    3. Cells cultured 24 hr after tet induction.

  • Experiment to be repeated a total of three times.

    1. This replication attempt will be repeated three times to obtain averaged values of transcript/cell since it will be repeated three times in the other replication protocols.

Materials and reagents

ReagentTypeManufacturerCatalog #Comments
P493-6 modified Burkitt's lymphoma cellsCell lineOriginal labFrom original lab
RPMI 1640 medium with sodium bicarbonate, without L-glutamineCell cultureSigma–AldrichR0883No catalog # listed in original paper
Tetracycline system approved FBSSerumClontech631105Two separate lots will be used
Ala-Gln (L-glutamine substitute)Cell cultureSigma–AldrichG8541Original lab used GlutaMAX from Invitrogen
T75 flaskLabwareSigma–AldrichZ707503Originally not specified
TetracyclinePharmacological agentSigma–AldrichT7660
C-chip disposable hemocytometerLabwareDigital BioDHC-N01No catalog # listed in original paper
Refrigerated centrifugeEquipmentEppendorf5810ROriginal lab used a Sorvall Legend centrifuge
Buffer RLTBufferQiagen79216Original lab used the Buffer RLT from the RNeasy kit (#74104)
ß-mercaptoethanolChemicalsSigma–AldrichM3148Included during communication with original authors
Nuclease-free waterChemicalsSigma–AldrichW4502Original lab used nuclease-free water from Ambion (AM9938)
NanoDrop UV/Vis SpectrophotometerEquipmentThermo ScientificND-1000
NanoDrop OperatingSoftwareThermo ScientificVersion 3.3
Procedure

  • All cells will be sent for mycoplasma testing and STR profiling.

  • Tet-free media: RPMI1640 supplemented with 10% tet system approved FBS and 1% Ala-Gln.

  • Cells maintained at 37°C in a humidified atmosphere at 5% CO2.

  • P493-6 cells are maintained in two separate FBS lots (lot #1 and lot #2).

    1. Prepare stocks of upright T75 flask of P493-6 modified Burkitt's lymphoma cells cultured in tet-free media.

    2. On day 0, seed a flask with ∼2 × 107 P493-6 cells in 40-ml tet-free media with 0.1 µg/ml tet.

      • a. Count cells in each flask using a C-chip disposable hemocytometer.

      • b. This is performed for cells grown in both lots of FBS.

    3. 72 hr later, spin cells down from both flasks, and wash cells three times in tet-free media.

    4. Seed three new flasks with 1.2 × 107 cells in 20 ml of fresh tet-free media.

      • a. Count cells using a C-chip disposable hemocytometer.

    5. At time points 0 hr (immediately), 1 hr, and 24 hr, harvest one flask, and collect 1 × 106 cells.

      • a. Count cells in each flask using a C-chip disposable hemocytometer.

      • b. Lyse cells directly in 100 µl Buffer RLT supplemented with ß-mercaptoethanol (10 µl ß-ME per 1 ml Buffer RLT) for a concentration of 10,000 cells per µl.

      • c. Make multiple 4 µl aliquots and store RNA at −80°C until shipment. Avoid freeze/thaw cycles. Ship one aliquot/sample and store others for backup.

    6. Repeat independently two additional times.

    7. Ship cell lysate samples on dry ice to lab for NanoString nCounter gene expression assay (Protocol 4).

Deliverables

  • Data to be collected:

    1. STR profile and result of mycoplasma testing of P493-6 cells.

  • Sample delivered for further analysis:

    1. Cell lysate samples for digital gene expression assay (Protocol 4).

Confirmatory analysis plan

This experiment assesses if c-Myc re-induction alters the number of transcripts/cell in genes that were active (>1 transcript/cell) or silent (<0.5 transcript/cell) in cells with lower levels of c-Myc (0 hr after tet induction). The original paper reported an upregulation of active genes upon c-Myc re-induction, while genes not expressed remained silent. This replication attempt will perform the statistical analysis listed in Protocol 4 for each cohort.

Known differences from the original study

A separate lot of serum was included to assess if there is variability between different batches of serum. All known differences are listed in the materials and reagents section above, with the originally used item listed in the comments section. All differences have the same capabilities as the original and are not expected to alter the experimental design.

Provisions for quality control

The cell line used in this experiment will undergo STR profiling to confirm its identity and will be sent for mycoplasma testing to ensure there is no contamination. The level of c-Myc re-induction in this system will be established in Protocol 1 as a comparison to the original paper. All of the raw data will be uploaded to the project page on the OSF (https://osf.io/mokeb/) and made publically available.

Protocol 4: NanoString nCounter digital gene expression assay

This experiment, which includes Protocol 3 and 4, tests the effect of elevated levels of c-Myc on the mRNA levels/cell for a large set of genes in P493-6 cells. The data were analyzed to test the effect of c-Myc on transcript levels of genes that were expressed at reliable levels (>1 transcript/cell) or genes that were not expressed (<0.5 transcript/cell). This is a replication of the data presented in Figure 3F and Table S1, which assess the quantified mRNA levels/cells by digital gene expression (NanoString).

Sampling

  • Each experiment has two cohorts:

    1. Cohort 1: P493-6 cells cultured in FBS lot #1.

    2. Cohort 2: P493-6 cells cultured in FBS lot #2.

  • Each cohort has three conditons:

    1. Cells cultured 0 hr after tet induction.

    2. Cells cultured 1 hr after tet induction.

    3. Cells cultured 24 hr after tet induction.

  • Experiment will analyze transcript levels from 1409 unique genes for a minimum power of 80%.

    1. Based on the estimation of detecting 795 active genes and 541 silent genes, which is 56.5% and 38.4%, respectively, of total unique genes, similar to the original data.

    2. See Power Calculations section for details.

Materials and reagents

ReagentTypeManufacturerCatalog #Comments
nCounter Custom CodeSet (CS-1, CS-2, CS-MYC combined)ProbeNanoStringSee Table S1 of original paper for RefSeq IDs
nCounter GX Human Immunology KitProbeNanoStringGXA-HIM1-12
nCounter GX Human Kinase KitProbeNanoStringGXA-P2K1-12
nCounter Analysis SystemEquipmentNanoStringNCT-PREP-120Includes Prep Station and Digital Analyzer
nSolver AnalysisSoftwareNanoStringVersion 1.1
Procedure

  1. Process samples (from Protocol 3) according to manufacturer's instructions for ‘Cell Lysate Protocol’ (nCounter Gene Expression Assay Manual).

    • a. Thaw samples on ice, mix well, and briefly spin down contents of the tubes.

    • b. Incubate 4 µl of cell lysate overnight at 65°C in nCounter Reporter CodeSet, Capture ProbeSet and hybridization buffer.

    • c. Following hybridization, process samples immediately with the nCounter PrepStation and subsequently analyze on an nCounter Digital Analyzer.

    • d. Approximately 40,000 cells/hybridization assay.

  2. Analyze data for all target genes in all samples/flow cells using the nSolver Analysis Software.

    • a. Normalize the counts for all target genes in all samples/flow cells based on the positive spike-in controls to account for differences in hybridization efficiency and post-hybridization processing.

      • i. Sum the counts for the positive spike-in controls for each sample/flow cell to estimate overall assay efficiency.

      • ii. Calculate a normalization factor for each sample/flow cell based on the relative number of positive control counts.

      • iii. For each sample/flow cell, multiply the counts for each target gene and the control genes by the normalization factor for that sample/flow cell.

    • b. Calculate the average normalized background count for a sample/flow cell from the average of the eight negative control counts. Subtract this value from the normalized count for each gene target in that sample/flow cell to yield normalized, background-subtracted counts for each target gene.

    • c. Average triplicates for each sample.

    • d. For each gene, estimate its transcripts per cell level through linear interpolation of RNA spike-in positive controls using the approximation that the signal detected from a 0.5 femtomolar RNA spike in is equivalent to the signal detected for a transcript expressed at the equivalent of 1 transcript/cell.

Deliverables

  • Data to be collected:

    1. RCC data output files from nCounter Digital Analyzer before and after normalization, background subtraction, and approximation of transcripts/cell for each gene. Include positive and negative controls for each sample/flow cell.

    2. List of all genes with the transcript/cell estimations for each individual sample and the average across the triplicate samples. (Compare to Table S1).

    3. Box-and-whisker plot of transcript/cell estimates for the active (>1 transcript/cell) and silent (<0.5 transcript/cell) genes at 0, 1, and 24 hr. (Compare to Figure 3F).

Confirmatory analysis plan

To determine if the normalized, background-subtracted counts are statistically above background, a t-test or Wilcoxon rank-sum test, will be used to test against the average counts of the negative control genes. Genes are defined as transcriptionally active (if average expression across replicates is >1 transcript/cell at 0 hr) or transcriptionally silent (if average expression across replicates is <0.5 transcript/cell). Some genes are represented on multiple nCounter Reporter CodeSets, so the average expression will be computed from replicates of all CodeSets with the same RefSeq ID. Genes with an average expression between 0.5 and 1 transcript/cell at 0 hr will be excluded from the analysis of silent genes, similar to how the original paper analyzed the data, but will be included in an additional analysis of non-active genes.

This experiment assesses if c-Myc re-induction alters the number of transcripts/cell in genes that were active (>1 transcript/cell) or silent (<0.5 transcript/cell) in cells with lower levels of c-Myc (0 hr after tet induction). The original paper reported an upregulation of active genes upon c-Myc re-induction, while genes not expressed remained silent. This replication attempt will perform the following statistical analysis listed below.

  • Statistical analysis:

    1. Two-tailed Wilcoxon signed rank test of transcript levels of active genes in cells for the following comparisons with the Bonferroni correction:

      • a. Cultured 0 hr from tet release compared to 1 hr from tet release.

      • b. Cultured 0 hr from tet release compared to 24 hr from tet release.

      • c. Cultured 1 hr from tet release compared to 24 hr from tet release.

    2. Two-tailed Wilcoxon signed rank test of transcript levels of silent genes in cells for the following comparisons with the Bonferroni correction:

      • a. Cultured 0 hr from tet release compared to 1 hr from tet release.

      • b. Cultured 0 hr from tet release compared to 24 hr from tet release.

      • c. Cultured 1 hr from tet release compared to 24 hr from tet release.

  • Meta-analysis of effect sizes:

    1. Compare the effect sizes of the original data to the two cohorts (separate FBS lots) and use a meta-analytic approach to combine the original and replication effects, which will be presented as a forest plot.

Known differences from the original study

The original study used three separate custom code sets (CS-1, CS-2, and CS-MYC) and the replication will combine these into one custom code set. The three original custom code sets had a total of 495 genes, of which only 479 were unique, so the replication custom code set will only contain these unique genes. The nCounter GX human immunology kit from NanoString has been updated to a new version. In the second version, there are 106 new genes not present in the first version, of which 101 are unique to all genes examined in this experiment, while 38 genes that were in the first version are excluded in the second version. Thus, 63 unique genes will be added to the total gene set compared with the original study changing the total examined genes from 1346 to 1409. A separate lot of serum was included to assess if there is variability between different batches of serum. The original study analyzed these data using the Wilcoxon sum rank test, however since expressions of the same gene across different conditions are not independent, the Wilcoxon signed rank test, a non-parametric test for comparing two paired samples, will be used instead.

Provisions for quality control

Each of the nCounter Reporter CodeSets contain internal positive and negative controls that are used to normalize/plot a regression line to quantify relative levels of expression and to estimate the non-specific background, respectively. Genes will be tested to determine if the average normalized, background-subtracted counts are statistically above background. All of the raw data, including the data output files from the nCounter Digital Analyzer, will be uploaded to the project page on the OSF (https://osf.io/mokeb/) and made publically available.

Power calculations

Protocol 1

Power calculations are not applicable.

Protocol 2

Summary of original data (shared by original authors).

Data set being analyzedMeanSDN
Total RNA 0 hr from tet release4.2520.16173
Total RNA 1 hr from tet release4.0370.29463
Total RNA 24 hr from tet release5.4710.37523
Test family

  • ANOVA: Fixed effects, omnibus, one-way, alpha error = 0.05.

    1. Power calculations were performed with G*Power software (version 3.1.7) (Faul et al., 2007).

    2. ANOVA F statistic calculated with Graphpad Prism 6.0.

    3. Partial η2 calculated from Lakens (2013).

Power calculations for replication

GroupsF test statisticPartial η2Effect size fA priori powerTotal sample size
0 hr, 1 hr, 24 hrF(2, 6) = 21.21830.87612.6591499.9%9 (3/group)
Test family

  • Two-tailed t-test, difference between two independent means, alpha error = 0.05.

    1. Power calculations were performed with G*Power software (version 3.1.7) (Faul et al., 2007).

Power calculations for replication

Group 1Group 2Effect size dA priori powerGroup 1 sample sizeGroup 2 sample size
0 hr24 hr4.1926894796.3%33

Protocol 3 and 4

Summary of original data (obtained from Table S1)

Data set being analyzedNMeanSD
Active genes 0 hr from tet release76023.66788.104
Active genes 1 hr from tet release76030.418117.36
Active genes 24 hr from tet release76054.959179.67
Silent genes 0 hr from tet release5170.06460.1225
Silent genes 1 hr from tet release5170.08370.2810
Silent genes 24 hr from tet release5170.29811.3787
  • Note: Active genes and silent genes are defined as >1 transcript/cell and <0.5 transcript/cell as in the original paper.

Test family

  • Two-tailed Wilcoxon signed rank test, alpha error = 0.01667.

    1. Power calculations were performed with G*Power software (version 3.1.7) (Faul et al., 2007).

Power calculations for replication (active genes)

Group 1Group 2Effect size dzA priori powerTotal sample size
0 hr from tet release1 hr from tet release0.223189780.2%*224*
0 hr from tet release24 hr from tet release0.303179380.3%*123*
1 hr from tet release24 hr from tet release0.289814780.2%*134*
  • 795 is used based on the estimation of 56.5% of total unique genes being active making the power 99.9%.

Test family

  • Two-tailed Wilcoxon signed rank test, Bonferroni's correction, alpha error = 0.01667.

    1. Power calculations were performed with G*Power software (version 3.1.7) (Faul et al., 2007).

Power calculations for replication (silent genes)

Group 1Group 2Effect size dzA priori powerTotal sample size
0 hr from tet release1 hr from tet release0.1427513*80.0%*541*
0 hr from tet release24 hr from tet release0.174370780.1%364
1 hr from tet release24 hr from tet release0.168616080.0%389
  • This is a sensitivity calculation. The original effect size is 0.0807517.

  • 541 is used based on the estimation of 38.4% of total unique genes being silent making the power 94.0%.

  • 541 is used making the power 92.3%.

References

Acknowledgements

The Reproducibility Project: Cancer Biology core team would like to thank the original authors, in particular Charles Lin, for generously sharing critical information as well as the P493-6 modified Burkitt's lymphoma cells to ensure the fidelity and quality of this replication attempt. We thank Courtney Soderberg at the Center for Open Science for assistance with statistical analyses. We would also like to thank the following companies for generously donating reagents to the Reproducbility Project: Cancer Biology; American Type Culture Collection (ATCC), BioLegend, Cell Signaling Technology, Charles River Laboratories, Corning Incorporated, DDC Medical, EMD Millipore, Harlan Laboratories, LI-COR Biosciences, Mirus Bio, Novus Biologicals, Sigma–Aldrich, and System Biosciences (SBI).

Decision letter

Michael R Green, Reviewing editor, Howard Hughes Medical Institute, University of Massachusetts Medical School, United States

eLife posts the editorial decision letter and author response on a selection of the published articles (subject to the approval of the authors). An edited version of the letter sent to the authors after peer review is shown, indicating the substantive concerns or comments; minor concerns are not usually shown. Reviewers have the opportunity to discuss the decision before the letter is sent (see review process). Similarly, the author response typically shows only responses to the major concerns raised by the reviewers.

Thank you for sending your work entitled “Registered report: Transcriptional Amplification in Tumor Cells with Elevated c-Myc” for consideration at eLife. Your article has been evaluated by Sean Morrison (Senior editor), a Reviewing editor, and 3 reviewers, one of whom is a biostatistician.

The Reviewing editor and the reviewers discussed their comments before we reached this decision, and the Reviewing editor has assembled the following comments to help you prepare a revised submission.

As detailed below, the reviewers raised a number of major concerns that need to be addressed in a revised Registered report.

Major comments:

1) Any replication of the Lin et al. paper needs to include both RNA profiling and ChIP-seq experiments. The Lin et al. paper has been quite controversial and was based in very large part on ChIP-seq data and its interpretation. Many of the conclusions were based on very subtle changes in data profiles. Subsequently, two papers were published in Nature (Walz et al. and Sabo et al.) that challenge the global claims of Lin et al. In both papers, the authors worked hard to accumulate comprehensive ChIP-seq and RNA expression data in carefully designed experimental systems. While some aspects of the Lin et al. paper may be correct, both Nature papers conclude that there is a set of defined target genes that are far more Myc responsive than others. Hence, reproducing only a subset of the Lin et al. experiments is unlikely to add anything new or resolve controversial claims.

The authors do not propose to reproduce the critical ChIP-seq data and they do not propose any analysis of RNAPII profiles that would support or conflict with the conclusions of the Lin et al. paper, namely that Myc promotes genome-wide transcriptional elongation. The proposal only focuses on RNA profiling without integration with binding of Myc and RNAPII.

2) More comprehensive RNA-seq analysis would determine global RNA expression in response to Myc and not be limited to a subset of genes represented by NanoString.

3) The cell line P493-6 has been established 15 years ago. The proliferation of these cells depends on c-Myc expression and the presence of serum. Serum is a major variable in this system and the majority of genes in stimulated cells are regulated by serum and not by c-Myc (Schlosser et al., Oncogene, 2005). The impact of different serum changes on P493-6 cells is highly significant. In some serum batches the cells barely grow after c-Myc activation. Unfortunately, the impact of various serum batches on P493-6 cells has never been systematically analyzed. Moreover, meanwhile many batches of P493-6 cells are distributed worldwide. These cells have been cultured with different types of serum in different laboratories. Exposure to different sera probably has altered the epigenetic state of P493-6 cells further contributing to variation in gene expression.

From the scientific view, it would be more helpful to study the stability of this biological system, e.g. by culturing P493-6 cells over longer periods of time in different batches of serum followed by a subsequent transcriptome analysis +/- Myc. At minimum, the authors should perform their experiments using multiple batches of serum to assess whether this significantly alters their results.

Statistical comments to the authors:

4) For protocol 1 and 2, authors propose to use ANOVA to analyze the data. Please make sure that the data do not violate the assumptions of the ANOVA: normality and homoscedasiticity. If the data do not fit the assumptions well enough, try to find a data transformation that makes them fit. If this doesn't work, you will need to apply a nonparametric counterpart of ANOVA such as Kruskal-Wallis test. In addition, performing contrast within the framework of ANOVA is more powerful than performing a separate t-test if the assumption of ANOVA is valid.

5) Authors used G*Power to calculate the power. I think that power calculation for protocol 3 & 4 is probably based on the test family t-test implemented in G*Power since there is no Wilcoxon sum rank test implemented in the G*Power. I suggest using t-tests as test family and matched pairs as statistical test to recalculate the power for protocol 3 and 4 (see below for justification). You will need to re-compute the effect size by calculating the SD for paired design, although mean difference between two groups will stay the same regardless.

6) Authors propose to use two-tailed Wilcoxon sum rank test, which has been used in the original paper. I suggest use either two-tailed Wilcoxon signed rank test or two-tailed paired t-test. If you prefer use G* Power to calculate power, then you will be left with two-tailed paired t-test option. The reason why paired analysis is needed is that expressions of the same gene across different conditions are not independent.

7) One major conclusion from the original paper (Lin et al., 2012) is that elevated c-Myc in tumor cells leads to amplification of the expression of actively transcribed genes, but has no effect on silent genes. I am wondering whether the authors will perform the same test to the silent genes, as well as the actively transcribed genes, to confirm the results from the original paper.

8) While it is very useful to leverage the previously reported effects to compute minimum power a priori, what you really need is to guarantee a minimum power on your own data. This can be done, a priori, by including some cross-study variation. This will be helpful for you to plan on the number of replicates and so forth. Papers by Giovanni Parmigiani and collaborators at the Dana–Farber provide some estimates about cross-study variation that could be used for this purpose. Worst case, you should budget some additional variability because of cross-study reproducibility, and increase the sample size as appropriate. We also want you to compute and report power post-hoc/on-the-fly on your own data. Some minimum power should be guaranteed using summaries of your own data.

[Editors' note: further revisions were requested prior to acceptance, as described below.]

Thank you for resubmitting your work entitled “Registered report: Transcriptional Amplification in Tumor Cells with Elevated c-Myc” for further consideration at eLife. Your revised article has been favorably evaluated by Sean Morrison (Senior editor), a Reviewing editor, and the original reviewers. As you might expect, there was a mixed response from the reviewers regarding the changes. On balance, we would like to move forward but would ask you to make one additional change. Different serum batches have only been included for the c-Myc off situation. To complete this control, please also include different serum batches for the c-Myc on situation.

DOI: http://dx.doi.org/10.7554/eLife.04024.002

Author response