1. Human Biology and Medicine
Download icon

Registered report: BET bromodomain inhibition as a therapeutic strategy to target c-Myc

  1. Irawati Kandela
  2. Hyun Yong Jin
  3. Katherine Owen
  4. Reproducibility Project: Cancer Biology Is a corresponding author
  1. Northwestern University, Illinois
  2. The Scripps Research Institute, California
  3. University of Virginia, Virgina
Registered Report
Cited
4
Views
1,904
Comments
0
Cite as: eLife 2015;4:e07072 doi: 10.7554/eLife.07072

Abstract

The Reproducibility Project: Cancer Biology seeks to address growing concerns about reproducibility in scientific research by replicating selected results from a substantial number of high-profile papers in the field of cancer biology published between 2010 and 2012. This Registered report describes the proposed replication plan of key experiments from ‘BET bromodomain inhibition as a therapeutic strategy to target c-Myc’ by Delmore and colleagues, published in Cell in 2011 (Delmore et al., 2011). The key experiments that will be replicated are those reported in Figures 3B and 7C-E. Delmore and colleagues demonstrated that treatment with JQ1, a small molecular inhibitor targeting BET bromodomains, resulted in the transcriptional down-regulation of the c-Myc oncogene in vitro (Figure 3B; Delmore et al., 2011). To assess the therapeutic efficacy of JQ1 in vivo, mice bearing multiple myeloma (MM) lesions were treated with JQ1 before evaluation for tumor burden and overall survival. JQ1 treatment significantly reduced disease burden and increased survival time (Figure 7C-E; Delmore et al., 2011). The Reproducibility Project: Cancer Biology is a collaboration between the Center for Open Science and Science Exchange and the results of the replications will be published in eLife.

https://doi.org/10.7554/eLife.07072.001

Introduction

c-Myc is a DNA binding transcription factor involved in the regulation of cell proliferation, differentiation, and apoptosis (McKeown and Bradner, 2014). Abnormal expression of c-Myc is frequently observed in a range of malignancies including breast, colon and cervical cancer, small cell lung carcinoma, osteosarcomas, glioblastomas and myeloid leukemias (Meyer and Penn, 2008; Conacci-Sorrell et al., 2014). While c-Myc expression is required for tumor initiation and maintenance, c-Myc inactivation leads to tumor regression (Felsher and Bishop, 1999; Flores et al., 2004; Soucek et al., 2008; Gabay et al., 2014). Therefore, c-Myc represents an enticing target for pharmacological inhibition.

Multiple myeloma (MM) is an incurable disease characterized by the unrestricted proliferation of terminally differentiated plasma cells (Anderson et al., 2011). The primary tumor-initiating events include genetic translocation and hyperploidy, while secondary events, such as oncogenic c-Myc activation and overexpression, drive MM progression (Bergsagel and Kuehl, 2005; Morgan et al., 2012). Furthermore, studies by Chng and colleagues determined that c-Myc activation was prevalent in more than 60% of patient-derived MM cells (Chng et al., 2011). Despite therapeutic advances and increases in survival, patients eventually succumb to treatment-refractory disease (Anderson, 2011; Kumar et al., 2012).

Therapeutic strategies targeting c-Myc are complicated by the fact that c-Myc lacks a clear ligand-binding domain (Darnell, 2002). However, it is possible that c-Myc could be disrupted by other means, such as through disruption of chromatin-dependent signaling. Bromodomain and extraterminal (BET) proteins are transcriptional regulators that epigenetically control the expression of genes involved in cell cycle, growth and inflammation (Darnell, 2002; Wu and Chiang, 2007; LeRoy et al., 2008; Dey et al., 2009; Nicodeme et al., 2010). BETs therefore provide potential therapeutic targets for modulating gene expression programs associated with various human diseases. Specifically, bromodomain protein 4 (BRD4), a member of the BET subfamily that associates with acetylated chromatin to promote transcription, was reported to interact with the positive transcription elongation factor complex b (P-TEFb) (Dey et al., 2009; Filippakopoulos et al., 2010). Recruitment of P-TEFb by c-Myc was also reported, providing the rationale for Delmore and colleagues to explore targeting BET proteins to inhibit c-Myc transcriptional activity (Bisgrove et al., 2007). Importantly, BRD4 expression was found to positively correlate with MM disease progression (Delmore et al., 2011). To interrogate this relationship, they used JQ1, a small molecule inhibitor of the BET family of bromodomain-containing proteins, which has the highest affinity with BRD4 and competitively inhibits BET proteins from binding to chromatin (Filippakopoulos et al., 2010). While the (+)-JQ1 enantiomer potently inhibits BET proteins, the (−)-JQ1 enantiomer is structurally incapable of inhibiting BET bromodomains supporting an on-target mechanism of action (Filippakopoulos et al., 2010). Further support for the relationship between c-Myc and BET proteins was reported by Mertz and colleagues, who used gene expression profiling of cells treated with the active and inactive forms of the JQ1 inhibitor to identify MYC as a highly down-regulated gene following BET bromodomain inhibition (Mertz et al., 2011).

As an alternative approach to direct c-Myc-targeting, Delmore and colleagues tested whether the BET inhibitor, JQ1, could effect c-Myc-specific gene silencing in MM (Delmore et al., 2011). In Figure 3B, Delmore and colleagues assessed the ability of JQ1 to downregulate MYC transcription in the MM cell line MM.1S. In this experiment, MM.1S cells were treated with JQ1 for up to 8 hours and the relative expression of MYC was compared to untreated control cells. JQ1 treatment resulted in a significant reduction in MYC transcripts as determined by qRT-PCR. This key experiment shows that JQ1 was effective at silencing MYC gene transcription and will be replicated in Protocol 1. Importantly, Loven and colleagues also recently corroborated these results through the demonstration that JQ1 treatment in MM.1S cells significantly decreases MYC mRNA levels (Loven et al., 2013). In addition to MM cell lines, JQ1 has proven to potently inhibit MYC in Merkel cell carcinoma cells (MCC-3 and 5), primary effusion lymphoma cells (PELs) and B cell acute lymphoblastic lymphomas (B-ALL) cells at the transcript level, as well as in diffuse large B cell lymphoma (DLBCL) cells at the protein expression level (Ott et al., 2012; Shao et al., 2014; Tolani et al., 2014; Trabucco et al., 2015). However, JQ1-resistant cells have also been described. Specifically, JQ1 did not alter MYC transcription in embryonic stem cells (ESCs) or in non-small cell lung carcinoma (NSCLC) harboring alteration in KRAS (Shimamura et al., 2013; Horne et al., 2014). In lung adenocarcinoma cells (LACs), JQ1 was found to inhibit cell growth independent of MYC down regulation (Lockwood et al., 2012).

In Figure 7C, 7D and 7E, the efficacy of JQ1 treatment was tested in mice harboring bioluminescent MM lesions. In these experiments, tumor burden was measured by whole-body bioluminescent imaging. Delmore and colleagues showed that JQ1 treatment significantly decreased disease burden and increased survival time compared to vehicle-treated control animals (Delmore et al., 2011). Similar findings recapitulating the suppressive effect of JQ1 on solid tumor growth have been reported in MCC, DLBCL and PEL xenograft models (Ott et al., 2012; Tolani et al., 2014; Trabucco et al., 2015), and reduced leukemic burden in a B-ALL xenograft model with corresponding improvements in survival (Ott et al., 2012). These experiments will be replicated in Protocol 2.

Materials and methods

Protocol 1: evaluation of MYC expression in JQ1-treated MM.1S cells

This experiment analyzes the expression of endogenous MYC during pharmacological inhibition of BET bromodomains with JQ1. This is a replication of the data presented in Figure 3B and assesses the levels of MYC by quantitative RT-PCR.

Sampling

  • ■ Each experiment has 9 conditions:

    • ◯ qRT-PCR of MYC (and GAPDH) 0 hr after (+)-JQ1 treatment.

    • ◯ qRT-PCR of MYC (and GAPDH) 1 hr after (+)-JQ1 treatment.

    • ◯ qRT-PCR of MYC (and GAPDH) 8 hr after (+)-JQ1 treatment.

    • ◯ qRT-PCR of MYC (and GAPDH) 0 hr after (−)-JQ1 treatment [additional].

    • ◯ qRT-PCR of MYC (and GAPDH) 1 hr after (−)-JQ1 treatment [additional].

    • ◯ qRT-PCR of MYC (and GAPDH) 8 hr after (−)-JQ1 treatment [additional].

    • ◯ qRT-PCR of MYC (and GAPDH) 0 hr after vehicle treatment [additional].

    • ◯ qRT-PCR of MYC (and GAPDH) 1 hr after vehicle treatment [additional].

    • ◯ qRT-PCR of MYC (and GAPDH) 8 hr after vehicle treatment [additional].

  • ■ Experiment will be performed five times with each run using three technical replicates, for a total power of ≥91%.

    • ◯ See ‘Power calculations’ section for details.

Materials and reagents

ReagentTypeManufacturerCatalog #Comments
MM.1S-LucNeoCell lineOriginal authorsN/AEngineered to express luciferase
RPMI 1640 mediumCell cultureSigma–AldrichR8758With 2 mM L-glutamine. Original brand not specified
Fetal bovine serum (FBS)Cell cultureSigma–AldrichF0392Original brand not specified
100× Penicillin/streptomycinCell cultureSigma–AldrichP4333Original brand not specified
PBS, without MgCl2 and CaCl2BufferSigma–AldrichD8537Originally not specified
0.05% trypsin/0.48 mM EDTACell cultureSigma–AldrichT3924Originally not specified
35-mm tissue culture dishesLabwareCorning430165Originally not specified
(+)-JQ1 enantiomerChemicalEMD Millipore500586Original made by authors
(−)-JQ1 enantiomerChemical
DMSOChemicalSigma–AldrichD8418Original brand not specified
TRI reagentChemicalSigma–AldrichT9424Replaces TRIzol from Invitrogen (Cat #15596-026)
First-Strand cDNA Synthesis kitNucleic acidGE Healthcare (Sigma–Aldrich)GE27-9261-01
Real-time PCR systemInstrumentApplied Biosystems7900HTReplaces 7500 model
TaqMan Gene Expression Master MixNucleic acidLife Technologies4369016Replaces a real-time PCR kit from Applied Biosystems (Cat #N15597), which is discontinued
Taq-Man probe (MYC)Nucleic acidApplied BiosystemsHs00905030_m1
Taq-Man probe (Gapdh)Nucleic acidApplied BiosystemsHs02758991_g1

Procedure

Notes

  • All cells will be sent for mycoplasma testing and STR profiling.

  • Cells maintained in RPMI 1640 with 2 mM L-glutamine supplemented with 10% FBS, 100 U/ml penicillin, and 50 µg/ml streptomycin at 37°C in a humidified atmosphere at 5% CO2.

  1. Seed 8 × 105 MM.1S-LucNeo cells into three 35-mm tissue culture dishes.

  2. The next day treat the dishes of cells with 2 ml of media with a final concentration of 500 nM (+)-JQ1, 500 nM (−)-JQ1, or an equivalent volume of DMSO.

    • a. Make 10 mM stock of (+)-JQ1 and (−)-JQ1 by diluting in DMSO.

  3. Isolate RNA from dishes at the following time points after treatment using TRI Reagent following manufacturer's instructions.

    • a. 0 hr (immediately).

    • b. 1 hr.

    • c. 8 hr.

  4. Reverse transcribe total RNA to cDNA with reverse transcription kit following manufacturer's instructions.

    • a. Record RNA concentration and purity.

    • b. Use 1 µg of RNA per 50 µl reaction.

    • c. Use random hexamers for first-strand synthesis.

  5. Perform qPCR to assess MYC expression levels using a real-time PCR system with a real-time PCR kit following manufacturer's instructions. Perform triplicate technical replicates for each biological replicate.

    • a. Use 5 µl of undiluted cDNA mixture per 50 µl reaction.

    • b. Use TaqMan probes for MYC (Hs00905030_m1) and Gapdh (Hs02758991_g1).

  6. Analyze and compute ΔΔCT values.

    • a. The first qRT-PCR assay will be analyzed to ensure conditions are appropriate for proper quantitation. If it is determined that conditions need to be adjusted, such as input volume, the conditions will be adjusted and the reaction will be repeated. Once optimized, the conditions will be used for all subsequent reactions.

      • i. All details and data associated with this process will be recorded.

  7. Repeat steps 1–6 independently four additional times.

Deliverables

  • ■ Data to be collected:

    • ◯ Purity (A260/280 ratio) and concentration of isolated total RNA from cells.

    • ◯ Assay conditions used initially and, if necessary, modified, to ensure conditions are appropriate for proper quantitation.

    • ◯ Raw qRT-PCR values, as well as analyzed ΔΔCT values.

    • ◯ Bar graph of MYC mRNA levels normalized to 0 hr after (+)-JQ1 treatment. (Compare to Figure 3B).

Confirmatory analysis plan

This replication attempt will perform the following statistical analysis listed below.

  • ■ Statistical analysis:

    • ◯ Repeated measures ANOVA of normalized MYC mRNA levels in MM.1S cells treated with (+)-JQ1, (−)-JQ1, or DMSO.

      • Paired t-tests with the Bonferroni correction:

        1. MM.1S cells harvested 8 hr after (+)-JQ1 treatment compared to cells 0 hr after (+)-JQ1 treatment.

        2. MM.1S cells harvested 1 hr after (+)-JQ1 treatment compared to cells 0 hr after (+)-JQ1 treatment.

  • ■ Additional exploratory statistical Analysis:

    • ◯ Two-way ANOVA of normalized MYC mRNA levels in MM.1S cells treated with (+)-JQ1, (−)-JQ1, or DMSO.

      • Planned comparisons with the Bonferroni correction:

        1. MM.1S cells harvested 8 hr after (+)-JQ1 treatment compared to cells 0 hr after (+)-JQ1 treatment.

        2. MM.1S cells harvested 1 hr after (+)-JQ1 treatment compared to cells 0 hr after (+)-JQ1 treatment.

  • ■ Meta-analysis of effect sizes:

    • ◯ Compute the effect sizes of each comparison, compare them against the effect size in the original paper and use a random effects meta-analytic approach to combine the original and replication effects, which will be presented as a forest plot.

Known differences from the original study

The replication experiment will not include the full time course treatment with (+)-JQ1, but will only include the 0 hr, 1 hr, and 8 hr treatments. The replication experiment will also include six additional conditions; MM.1S cells harvested 0 hr, 1 hr, and 8 hr after treatment with (−)-JQ1 or DMSO. The original report was unclear if the cells used for this experiment were MM.1S or MM.1S-LucNeo cells. This replication attempt will use MM.1S-LucNeo cells since this is the same cell line used in protocol 2 to assess the efficacy of JQ1 treatment in mice. The original details for the reverse transcription and qRT-PCR reactions were not known, thus the details used here are manufacturer recommended with the use of random hexamers to generate cDNA that are more representative of all regions of the transcripts. Following the first reaction, conditions will be adjusted if necessary to ensure proper quantitation. All known differences of materials and reagents are listed in the ‛Materials and reagents’ section above with the originally used item listed in the comments section. All differences have the same capabilities as the original and are not expected to alter the experimental design.

Provisions for quality control

The cell lines used in this experiment will undergo STR profiling to confirm their identity and will be sent for mycoplasma testing to ensure there is no contamination. The sample purity (A260/280 and A260/230 ratios) of the isolated RNA from each sample will be reported. The first qRT-PCR reaction will be analyzed to determine if the assay conditions are appropriate for proper quantitation. This would be apparent after the first qRT-PCR assay, and the assay would then be repeated adjusting parameters. After conditions that allow for proper quantitation are achieved, then the number of pre-determined replicates will be completed under those same conditions. All assay details, adjustments, and data will be recorded and made available during this optimization process. All data obtained from the experiment—raw data, data analysis, control data, and quality control data—will be made publicly available, either in the published manuscript or as an open access data set available on the Open Science Framework project page for this study (https://osf.io/7zqxp/).

Protocol 2: JQ1 treatment in orthotopic xenograft model

This experiment tests the efficacy of JQ1 treatment in mice harboring bioluminescent MM lesions. This is a replication of the data presented in Figures 7C, 7D, and 7E and assesses tumor burden by whole-body bioluminescent imaging and monitors overall survival with daily treatment of JQ1.

Sampling

  • ■ Each experiment has 3 cohorts:

    • ◯ Cohort 1: Vehicle (D5W) treatment.

    • ◯ Cohort 2: (+)-JQ1 treatment.

    • ◯ Cohort 3: (−)-JQ1 treatment [additional].

  • ■ Experiment will analyze at least eight mice per cohort for a minimum of 80% power.

    • ◯ See ‘Power calculations’ section for details.

  • ■ To account for unexpected euthanasia of mice before the end of the experiment or exclusion of mice before treatment, the sample size was increased by ∼ 30%.

    • ◯ A total of 33 mice will be injected with MM.1S-LucNeo cells.

Materials and reagents

ReagentTypeManufacturerCatalog #Comments
MM.1S-LucNeoCell lineOriginal authorsN/AEngineered to express luciferase
RPMI 1640 mediumCell cultureSigma–AldrichR8758With 2 mM L-glutamine. Original brand not specified
Fetal bovine serum (FBS)Cell cultureSigma–AldrichF0392Original brand not specified
100× penicillin/streptomycinCell cultureSigma–AldrichP4333Original brand not specified
PBS, without MgCl2 and CaCl2BufferSigma–AldrichD8537Originally not specified
0.05% trypsin/0.48 mM EDTACell cultureSigma–AldrichT3924Originally not specified
T150 tissue culture flasksLabwareCorning430825Originally not specified
5 week old female Fox Chase SCID Beige (CB17.Cg-PrkdcscidLystbg-J/Crl)Animal modelCharles River LabsStrain 250
30½G needleLabwareSigma–AldrichZ192341Originally not specified
27½G needleLabwareSigma–AldrichZ192384Originally not specified
1 ml syringeLabwareSigma–AldrichZ192090Originally not specified
VivoGlo luciferinReporter assayPromegaP1042Original catalog # not specified
Xenogen IVIS SpectrumInstrumentCaliper Life SciencesSpectrum
Living ImagesSoftwareCaliper Life SciencesVersion used will be recorded and included in the Replication Study
(+)-JQ1 enantiomerChemicalEMD Millipore500586Original made by authors
(−)-JQ1 enantiomerChemical
Dextrose (D-(+)-glucose)ChemicalSigma–AldrichG8270Original brand not specified

Procedure

Notes

  • All cells will be sent for mycoplasma testing and STR profiling, as well as screened against a Rodent Pathogen Panel.

  • Cells maintained in RPMI 1640 with 2 mM L-glutamine supplemented with 10% FBS, 100 U/ml penicillin, and 50 µg/ml streptomycin at 37°C in a humidified atmosphere at 5% CO2.

  • MM.1S-LucNeo cells stably express a luciferase construct.

  1. After 1 week of acclimation, intravenously inject 2 × 106 MM.1S-LucNeo cells suspended in 200 µl PBS into 6-week old female SCID-beige mice using a 30½G needle via the lateral tail vein.

  2. 8 days later, inject mice intraperitoneally with 75 mg/kg of D-luciferin in 0.1 ml using a 27½G needle. Anesthetize mice and image using a Xenogen IVIS Spectrum using the Living Images software package.

    • a. Record weight of mice.

    • b. Anesthetize with isoflurane.

    • c. Image mice 20 min post injection.

  3. 5 days later, inject mice with 75 mg/kg of D-luciferin as described in step 2 and image using a Xenogen IVIS Spectrum using the Living Images software package.

    • a. Record weight of mice.

    • b. Determine difference between bioluminescence of first imaging and second imaging.

  4. For mice with established disease, randomly divide into three cohorts.

    • a. Established disease is defined as detection of MM.1S-LucNeo lesions diffusely engrafted in the skeleton with an increase in bioluminescence between the first and second images.

    • b. Exclude any mice with no detectable disease or no increase in bioluminescence. If over 30 mice are present with detectable disease, exclude mice with lowest disease burden to obtain 30 mice for randomization and treatment.

      • i. Original report saw 90–95% engraftment.

    • c. Animals are ranked according to disease burden (difference between bioluminescence of first imaging and second imaging), to balance groups for baseline tumor characteristics, and assigned to group 1, group 2, or group 3 using an alternating serpentine method. (rank 1 = group 1, rank 2 = group 2, rank 3 = group 3, rank 4 = group 3, rank 5 = group 2, rank 6 = group 1, rank 7 = group 1, etc).

      • i. Designation of Vehicle, (+)-JQ1, or (−)-JQ1 treatments as group 1, group 2, or group 3 will be determined by randomly assigning the three treatments into one block using www.randomization.com.

      • ii. Record seed number.

  5. After imaging and randomization, treat mice daily with either (+)-JQ1 at 50 mg/kg, (−)-JQ1 at 50 mg/kg, or vehicle (5% dextrose in water) control by intraperitoneal injection with a 27½G needle.

    • a. Inject 10 ml/kg body weight of a 5 mg/ml solution to give a final dose of 50 mg/kg.

      • i. (+)-JQ1 and (−)-JQ1 solutions are prepared in 5% dextrose in water.

    • b. Record weight of mice.

  6. 6, 14, and 21 days after the start of treatment IP injections, assess tumor burden by bioluminescence imaging after IP injection of 75 mg/kg of D-luciferin as described in step 2 above.

    • a. Record weight of mice.

  7. Continue to treat mice with daily injections of (+)-JQ1, (−)-JQ1, or vehicle control until mice are euthanized according to IACUC guidelines or until end of experiment (5 weeks total treatment).

    • a. In this model, mice are euthanized when they develop hind limb paralysis.

Deliverables

  • ■ Data to be collected:

    • ◯ Mouse health records (including number of mice with established disease and reason for euthanasia, weight at time of each injection).

    • ◯ All images of mice in vivo to detect established disease and tumor burden (compare to Figure 7C).

    • ◯ Raw photon flux measurements of each mouse and graph of bioluminescence vs day of treatment of cohorts (compare to Figure 7D).

    • ◯ Raw survival data and Kaplan–Meier curves generated for percent survival (compare to Figure 7E).

Confirmatory analysis plan

This replication attempt will perform the following statistical analyses listed below.

  • ■ Statistical analysis:

    • ◯ Tumor burden:

      • One-way ANOVA test on day 22 data points with the following planned comparisons using Fisher's LSD correction:

        • i. (+)-JQ1 treatment to vehicle treatment.

        • ii. (+)-JQ1 treatment to (−)-JQ1 treatment.

      • One-way ANCOVA test of the area under the curve (AUC) measurements (determined from day 1, 7, 15, and 22 data for each mouse) with the AUC pre-treatment measurements (determined from day −4 and 1 for each mouse) as the covariate, with the following planned comparisons with the Bonferroni correction:

        • i. (+)-JQ1 treatment to vehicle treatment.

        • ii. (+)-JQ1 treatment to (−)-JQ1 treatment.

        • Note: This is an additional test not originally performed, which analyzes all data points opposed to just day 22 data points.

    • ◯ Kaplan–Meier curves:

      • Log-rank Mantel–Cox test on the following comparisons with the Bonferroni correction:

        • i. (+)-JQ1 treatment to vehicle treatment.

        • ii. (+)-JQ1 treatment to (−)-JQ1 treatment.

  • ■ Meta-analysis of effect sizes:

    • ◯ Compute the effect sizes of each comparison, compare them against the effect size in the original paper and use a random effects meta-analytic approach to combine the original and replication effects, which will be presented as a forest plot.

Known differences from the original study

The replication experiment will include an additional cohort receiving treatment with the inactive (−)-JQ1 enantiomer, which was not included in the original study. All known differences of materials and reagents are listed in the ‘Materials and reagents’ section above with the originally used item listed in the comments section. All differences have the same capabilities as the original and are not expected to alter the experimental design.

Provisions for quality control

The cell lines used in this experiment will undergo STR profiling to confirm their identity and will be sent for mycoplasma testing to ensure there is no contamination. Additionally, cells used for xenograft injection will be screened against a Rodent Pathogen Panel to ensure no contamination prior to injection. The bioluminescence images and measurements will be reported for all mice when determining the inclusion based on detection of lesions diffusely engrafted in the skeleton. Mice will be randomly assigned to treatment group with disease burden balanced among groups with the seed number recorded to reproduce the plan. All data obtained from the experiment—raw data, data analysis, control data, and quality control data—will be made publicly available, either in the published manuscript or as an open access data set available on the Open Science Framework project page for this study (https://osf.io/7zqxp/).

Power calculations

For additional details on power calculations, please see analysis scripts and associated files on the Open Science Framework:

Protocol 1

Summary of original data presented in Figure 3B (estimated from graph).

Dataset being analyzedNMeanSD
MM.1S cells treated with (+)-JQ1–0 hr2*1.00.375
MM.1S cells treated with (+)-JQ1–1 hr2*0.068750.00625
MM.1S cells treated with (+)-JQ1–8 hr2*0.08750.05
  1. *

    This is the number of biological replicates reported for this experiment.

We are including the following groups in the replication study: (−)-JQ1 and vehicle treatment. We performed these calculations with the assumption that (−)-JQ1–0 hr, 1 hr, and 8 hr, and vehicle–0 hr, 1 hr, and 8 hr will have similar values as (+)-JQ1–0 hr.

Analysis with samples paired

Test family

  • ■ ANOVA: Repeated measures, within factors, alpha error = 0.05.

    • Power calculations performed with G*Power software, version 3.1.7 (Faul et al., 2007).

      • ◯ The correlation among repeated measures was assumed to be 0 and the nonsphericity correction was assumed to be 1.

TimeGroupsDetectable effect size fA priori powerTotal sample size
0 hr, 1 hr, and 8 hr(+)-JQ1, (−)-JQ1*, vehicle*0.4161180.0%15 (3 groups, 3 measurments)
  1. *

    (−)-JQ1 and vehicle values were the same as (+)-JQ1–0 hr values for this calculation.

  2. This is the effect size detectable with 80% power and the indicated sample size.

  3. A total sample size of 15 will be used based on the paired t-test planned comparison calculations.

Test family

  • 2 tailed t test, difference between two dependent means (matched pairs): Bonferroni's correction: alpha error = 0.025.

    • ■ Power calculations performed with G*Power software, version 3.1.7 (Faul et al., 2007).

      • ◯ Correlation between groups was assumed to be 0.

Group 1Group 2Effect size dzA priori powerTotal sample size
(+)-JQ1–0 hr(+)-JQ1–8 hr2.4119990.8%5
(+)-JQ1–0 hr(+)-JQ1–1 hr2.4829992.3%5

Analysis with samples unpaired

Test family

  • ■ Two-way ANOVA: Fixed effects, special, main effects and interactions, alpha error = 0.05.

TimeGroupsF test statisticPartial η2Effect size fA priori powerTotal sample size
0 hr, 1 hr, and 8 hr(+)-JQ1, (−)-JQ1*, vehicle*F(4,9) = 1.7228 (interaction)0.43360.8750382.4%23 (9 groups)
  1. *

    (−)-JQ1 and vehicle values were the same as (+)-JQ1–0 hr values for this calculation.

  2. A total sample size of 45 will be used based on the paired t-test planned comparison calculations making the power 99.7%.

Test family

  • 2 tailed t test, difference between two independent means: Bonferroni's correction: alpha error = 0.025.

Power calculations performed with G*Power software, version 3.1.7 (Faul et al., 2007).

Group 1Group 2Effect size dA priori powerGroup 1 sample sizeGroup 2 sample size
(+)-JQ1–0 hr(+)-JQ1–8 hr3.4110793.3%*4*4*
(+)-JQ1–0 hr(+)-JQ1–1 hr3.5114895.5%44
  1. *

    5 per group will be used based on the paired t-test planned comparisons making the power 98.7%.

  2. 5 per group will be used based on the paired t-test planned comparisons making the power 99.1%.

Protocol 2

Tumor burden as determined by bioluminescence

Summary of original data presented in Figure 7D (provided by authors).

Dataset being analyzedDayNMeanSD
Vehicle-treated mice−4102.14 × 1069.38 × 105
1101.06 × 1077.63 × 106
7102.37 × 1087.98 × 107
15106.40 × 1093.13 × 109
22101.85 × 10101.01 × 1010
(+)-JQ1-treated mice−491.59 × 1062.84 × 105
198.10 × 1063.31 × 106
795.78 × 1072.84 × 107
1591.10 × 1095.70 × 108
2295.52 × 1092.25 × 109

Area under the curve (AUC) calculations from estimated values from graph in Figure 7D.

Calculations performed with R software 3.1.2 (Team RC, 2014).

Data set being analyzedDaysNMeanSD
Vehicle-treated mice−4 to 1103.02 × 1072.30 × 107
1 to 22101.14 × 10115.08 × 1010
(+)-JQ1-treated mice−4 to 1102.51 × 1079.46 × 106
1 to 22102.80 × 10108.72 × 109

We are including the following group in the replication study: (−)-JQ1-treated mice. We performed these calculations with the assumption that (−)-JQ1-treated mice will have similar values as vehicle-treated mice.

Day 22 values

Test family

  • ■ F test: ANOVA: Fixed effects, omnibus, one-way, alpha error = 0.05.

GroupsDataF test statisticPartial η2Effect size fA priori powerTotal sample size
(+)-JQ1, (−)-JQ1*, vehicle22 daysF(2,26) = 7.21140.356800.7448080.8%21 (3 groups)
  1. *

    (−)-JQ1 values were the same as vehicle values for this calculation.

  2. 8 per group (24 total) will be used based on the planned comparisons making the power 86.8%.

Test family

  • ■ 2 tailed t test: Means: Difference between two independent means: Fisher's LSD correction: alpha error = 0.05.

    • Power calculations performed with G*Power software, version 3.1.7 (Faul et al., 2007).

Group 1Group 2Effect size dA priori powerGroup 1 sample sizeGroup 2 sample size
(+)-JQ1Vehicle1.5243780.9%88
(+)-JQ1(−)-JQ11.5243780.9%88

AUC values

Test family

  • ■ F test: ANCOVA: Fixed effects, main effects and interactions, alpha error = 0.05.

GroupsDataF test statisticPartial η2Effect size fA priori powerTotal sample size
(+)-JQ1, (−)-JQ1*, vehicleAUCF(2,25) = 36.0510.742541.6982698.3%11 (3 groups)
  1. *

    (−)-JQ1 values were the same as vehicle values for this calculation.

  2. One covariate was used (days −4 to 1 AUC) for this calculation.

  3. 8 per group (24 total) will be used based on the day 22 calculations making the power 99.9%.

Test family

  • ■ 2 tailed t test: Means: Difference between two independent means: Bonferroni correction: alpha error = 0.025.

    • Power calculations performed with G*Power software, version 3.1.7 (Faul et al., 2007).

Group 1Group 1 adjusted meanGroup 2Group 2 adjusted meanEffect size dA priori powerGroup 1 sample sizeGroup 2 sample size
(+)-JQ11.12 × 1011Vehicle3.45 × 10103.4352293.6%*4*4*
(+)-JQ11.12 × 1011(−)-JQ13.45 × 10103.4352293.6%*4*4*
  1. *

    8 per group will be used based on the day 22 calculations making the power 99.9%.

Survival data

Summary of original data presented in Figure 7E (provided by authors).

Dataset being analyzedMedian survivalHazard ratio (to vehicle control)N
Vehicle-treated mice22 daysNA10
JQ1-treated mice35 days0.0385659*
  1. *

    Only 9 animals were analyzed in this group.

We are including the following comparisons in the replication study: (+)-JQ1-treated mice to (−)-JQ1-treated mice. We performed these calculations with the assumption that (−)-JQ1-treated mice will have similar values as vehicle-treated mice.

Test family

Group 1Group 2Experiment durationA priori powerTotal events neededGroup 1 sample sizeGroup 2 sample size
(+)-JQ1Vehicle40 days80%4*5*5*
(+)-JQ1(−)-JQ140 days80%4*5*5*
  1. *

    7 per group will be used based on the bioluminescence analysis making the power 94%.

References

  1. 1
    The future of treatment for patients with relapsed/refractory multiple myeloma
    1. KC Anderson
    (2011)
    Oncology 25:2.
  2. 2
  3. 3
  4. 4
  5. 5
  6. 6
  7. 7
  8. 8
  9. 9
  10. 10
  11. 11
  12. 12
  13. 13
  14. 14
  15. 15
  16. 16
  17. 17
  18. 18
  19. 19
  20. 20
  21. 21
  22. 22
  23. 23
  24. 24
  25. 25
  26. 26
  27. 27
  28. 28
  29. 29
  30. 30
  31. 31
  32. 32
    R: a language and environment for statistical computing
    1. Team RC
    (2014)
    R Foundation for Statistical Computing. http://www.R-project.org/.
  33. 33
  34. 34
  35. 35

Decision letter

  1. Christopher Glass
    Reviewing Editor; University of California, San Diego, United States

eLife posts the editorial decision letter and author response on a selection of the published articles (subject to the approval of the authors). An edited version of the letter sent to the authors after peer review is shown, indicating the substantive concerns or comments; minor concerns are not usually shown. Reviewers have the opportunity to discuss the decision before the letter is sent (see review process). Similarly, the author response typically shows only responses to the major concerns raised by the reviewers.

Thank you for sending your work entitled “Registered report: BET bromodomain inhibition as a therapeutic strategy to target c-Myc” for consideration at eLife. Your article has been favorably evaluated by Charles Sawyers (Senior editor), a Reviewing editor, and three reviewers.

The Reviewing editor and the reviewers discussed their comments before we reached this decision, and the Reviewing editor has assembled the following comments to help you prepare a revised submission.

This Registered Report submission proposes to replicate key findings in Delmore et al., 2011, which reported the ability of the BRD inhibitor JQ1 to suppress MYC expression in a myeloma cell line and to extend survival of mice transplanted with this cell line. There was general agreement among the three Referees that the proposed studies will assess the major findings of Delmore et al., and will be highly relevant. The referees made the following recommendations to improve the protocols.

Protocol 1:

One reviewer noted that one of the main strengths of the published study (Delmore et al., 2011) was that JQ1 treatment led to downregulation of Myc was not limited to MM1.S cells, and that the effect was observed in several MM cell lines (Figure 3H and 3I). This is a key observation and we recommend that the qPCR analyses be extended to the MM cell types indicated in Figures 2A and 3I.

Addition of one more time point is recommended, the 1h treatment with 500 nM (+)-JQ1. This will indicate whether the Myc downregulation by JQ1 is as dynamic as reported.

In addition to the qPCR primers for Myc and GAPDH mentioned, the exact qPCR primers used by (Delmore et al., 2011) should be included.

One reviewer requested scripts and a detailed description of the calculation performed with R. For example in Protocol 1, in the subsection headed “Test family”, the following sentence can be added: F test statistic (interaction) has been calculated following Cohen (2002) and the partial η2 has been calculated following Lakens (2013).

This reviewer disagrees with the choice of a simple two-way ANOVA. In the original paper (Figure 3B) paired Student's t-tests were used. Since the replicated experiments are similar to the original ones a repeated measures anova is more appropriate. A carefully chosen repeated measures ANOVA is a natural extension of the paired t-test and can simplify the implementation of the proposed meta-analysis. A drawback with a repeated measures ANOVA is that it can be more difficult to set the parameters to determine the power. In such a case a sensitivity analysis can be performed with the G* power software.

Protocol 2:

Weight of mice should be recorded at day-0/day of injection.

In this protocol the analyses following an ANOVA have been performed using Fisher's LSD correction and alpha error = 0.05. In its basic form (I think the one used in the protocol) the LSD correction is not taking into account that multiple comparisons will be performed and therefore a Bonferroni correction (or other corrections) must be employed. For Protocol 2 this brings alpha to 0.025 and in practice is not dramatically changing the power calculations. As an alternative to Fisher's LSD followed by Bonferroni, the Hayter-Fisher's LSD procedure (Hayter, 1986) controls the MFWER (maximum family wise error rate).

Fisher's LSD correction has been reported also for survival data but it doesn't apply to this kind of data. Also here we need a (Bonferroni) correction. For the survival data, power calculations were performed with the Sample Size Calculator, however I do not have a clear link to the software. The authors should provide all the used parameters and references.

References:

Anthony J. Hayter. The maximum familywise error rate of fisher's least significant difference test. Journal of the American Statistical Association, 81(396):1000–1004, 1986. doi:10.1080/01621459.1986.10478364

https://doi.org/10.7554/eLife.07072.002

Author response

Protocol 1:

One reviewer noted that one of the main strengths of the published study (Delmore et al., 2011) was that JQ1 treatment led to downregulation of Myc was not limited to MM1.S cells, and that the effect was observed in several MM cell lines (Figure 3H and 3I). This is a key observation and we recommend that the qPCR analyses be extended to the MM cell types indicated in Figures 2A and 3I.

We agree that testing additional cell types (KMS11, OPM1, LR5, and INA6) provided additional evidence that the downregulation of Myc was not limited to MM1.S cells, however the Reproducibility Project: Cancer Biology aims to perform direct replications using the same methodology reported in the original paper. The additional cell types would be a conceptual replication, which we agree is a useful approach to test the experiment’s underlying hypothesis, but which is not an aim of the project. Aspects of an experiment not included in the original study are occasionally added to ensure the quality of the research, but by no means is a requirement of this project; rather, it is an extension of the original work. Adding additional aspects not included in the original study can be of scientific interest, and can be included if it is possible to balance them with the main aim of this project: to perform a direct replication of the original experiment(s). As such, we will restrict our analysis to the experiments being replicated and will not include discussion of experiments not being replicated in this study.

Addition of one more time point is recommended, the 1h treatment with 500 nM (+)-JQ1. This will indicate whether the Myc downregulation by JQ1 is as dynamic as reported.

Thank you for the recommendation. We have updated the manuscript to reflect this additional time point.

In addition to the qPCR primers for Myc and GAPDH mentioned, the exact qPCR primers used by (Delmore et al., 2011) should be included.

The qPCR primers included in this experimental design were reported in Delmore et al., 2011 (Supplemental information, Extended Materials and methods, Expression analysis).

One reviewer requested scripts and a detailed description of the calculation performed with R. For example in Protocol 1, in the subsection headed “Test family”, the following sentence can be added: F test statistic (interaction) has been calculated following Cohen (2002) and the partial η2 has been calculated following Lakens (2013).

Thank you for this recommendation. We have included a link to the scripts (https://osf.io/bjrpc/?view_only=737ba0f51c474aa1bc2782a44fba34d5). Additionally, we have added descriptions as suggested to more clearly describe the approach.

This reviewer disagrees with the choice of a simple two-way ANOVA. In the original paper (Figure 3B) paired Student's t-tests were used. Since the replicated experiments are similar to the original ones a repeated measures ANOVA is more appropriate. A carefully chosen repeated measures ANOVA is a natural extension of the paired t-test and can simplify the implementation of the proposed meta-analysis. A drawback with a repeated measures ANOVA is that it can be more difficult to set the parameters to determine the power. In such a case a sensitivity analysis can be performed with the G* power software.

We thank the reviewer for catching the original analysis method. We have adjusted the planned analysis to include this approach. As the reviewer suggested, we conducted a sensitivity analysis for the repeated measures ANOVA using the planned sample size and assuming 0 correlation among repeat measures and a nonsphericity correction of 1 to allow for a conservative estimate. We included the two planned comparisons (paired t-tests) assuming a correlation between groups of 0. This is because we do not have access to the original raw data. Additionally, it is not clear what df was used in the original paper since the reported p values suggest a larger effect size estimate than using the estimated means and standard deviations reported in the figure. This is likely due to the combination of the technical replicates (3) being combined with the biological replicates (2) giving a df of 5, opposed to using only the biological replicates (what is proposed in this manuscript), which gives a df of 2. Thus, we used the point estimated from G*Power using the estimated values from the graph reported in Figure 3B.

We are also proposing to analyze the data as originally planned (as a between subjects design) since the experimental set-up suggests this analysis. The sample comes from multiple random dishes of cells treated with or without drug. As such, it is possible to have a different number of data points in one group (vehicle) than the other (JQ1), thus making matched samples difficult. We think it is reasonable to use the independent test, and both analysis designs are reported in the literature. This will be considered additional exploratory analysis since it was not originally reported.

Protocol 2:

Weight of mice should be recorded at day-0/day of injection.

This is a parameter we have in the manuscript. Protocol 2, Step 5b. Weight of mice will be recorded during each day of injection during the course of the experiment.

In this protocol the analyses following an ANOVA have been performed using Fisher's LSD correction and alpha error = 0.05. In its basic form (I think the one used in the protocol) the LSD correction is not taking into account that multiple comparisons will be performed and therefore a Bonferroni correction (or other corrections) must be employed. For Protocol 2 this brings alpha to 0.025 and in practice is not dramatically changing the power calculations. As an alternative to Fisher's LSD followed by Bonferroni, the Hayter-Fisher's LSD procedure (Hayter, 1986) controls the MFWER (maximum family wise error rate).

We agree with the reviewers comment on the use of a correction, such as Bonferroni or the modification of LSD by Hayter are ways to control for the MFWER, however as Hayter describes in his 1986 paper, this applies in situations where the ANOVA is unbalanced or with a balanced design with four or more populations. Since the proposed analysis is balanced with three population groups, the LSD is sufficiently conservative and powerful to account for the multiple comparisons in this specific situation. This is further explained by Levin et al., 1994 and discussed in Maxwell and Delaney, 2004 (Chapter 5) and Cohen, 2001 (Chapter 12).

Fisher's LSD correction has been reported also for survival data but it doesn't apply to this kind of data. Also here we need a (Bonferroni) correction. For the survival data, power calculations were performed with the Sample Size Calculator, however I do not have a clear link to the software. The authors should provide all the used parameters and references.

Thank you for this correction. We have updated the power calculations to reflect this adjustment. The link to the online calculator used is included as a hyperlink in the manuscript and should direct you to here: http://www.sample-size.net/sample-size-survival-analysis/, which includes the reference (Schoenfeld, 1983) for the formulas used.

Additionally, screen shots of the input and output parameters are included on the project page on the Open Science Framework.

References:

Levin, J.R., Serline, R.C., & Seaman M.A. (1994). A controlled, powerful multiple-comparison strategy for several situations. Psychological Bulletin, 115, 153–159.

Maxwell, S.E. & Delaney, H.D. (2004). Designing experiments and analyzing data: a model comparison perspecitive. Lawrence Erlbaum Associates, Mahwah, N.J., second edition.

Cohen, B.H. (2001). Explaining psychological statistics. John Wiley and Sons, New York, second edition.

https://doi.org/10.7554/eLife.07072.003

Article and author information

Author details

  1. Irawati Kandela

    1. Developmental Therapeutics Core, Northwestern University, Evanston, Illinois
    Contribution
    IK, Drafting or revising the article
    Competing interests
    IK: This is a Science Exchange associated lab.
  2. Hyun Yong Jin

    1. The Scripps Research Institute, La Jolla, California
    Contribution
    HYJ, Drafting or revising the article
    Competing interests
    No competing interests declared.
  3. Katherine Owen

    1. University of Virginia, Charlottesville, Virgina
    Contribution
    KO, Drafting or revising the article
    Competing interests
    No competing interests declared.
  4. Reproducibility Project: Cancer Biology

    Contribution
    RP:CB, Conception and design, Drafting or revising the article
    For correspondence
    1. tim@cos.io
    Competing interests
    RP:CB: EI, FT, JL, and NP: Employed by and hold shares in Science Exchange Inc.
    1. Elizabeth Iorns, Science Exchange, Palo Alto, California
    2. William Gunn, Mendeley, London, United Kingdom
    3. Fraser Tan, Science Exchange, Palo Alto, California
    4. Joelle Lomax, Science Exchange, Palo Alto, California
    5. Nicole Perfito, Science Exchange, Palo Alto, California
    6. Timothy Errington, Center for Open Science, Charlottesville, Virginia

Funding

Laura and John Arnold Foundation

  • Reproducibility Project: Cancer Biology

The Reproducibility Project: Cancer Biology is funded by the Laura and John Arnold Foundation, provided to the Center for Open Science in collaboration with Science Exchange. The funder had no role in study design or the decision to submit the work for publication.

Acknowledgements

The Reproducibility Project: Cancer Biology core team would like to thank the original authors, in particular Dr Andrew Kung, for generously sharing critical information as well as reagents to ensure the fidelity and quality of this replication attempt. We thank Courtney Soderberg at the Center for Open Science for assistance with statistical analyses. We would also like to thank the following companies for generously donating reagents to the Reproducibility Project: Cancer Biology; American Tissue Culture Collection (ATCC), BioLegend, Charles River Laboratories, Corning Incorporated, DDC Medical, EMD Millipore, Harlan Laboratories, LI-COR Biosciences, Mirus Bio, Novus Biologicals, Sigma–Aldrich and System Biosciences (SBI).

Reviewing Editor

  1. Christopher Glass, Reviewing Editor, University of California, San Diego, United States

Publication history

  1. Received: February 17, 2015
  2. Accepted: June 8, 2015
  3. Version of Record published: June 25, 2015 (version 1)

Copyright

© 2015, Kandela et al

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.

Metrics

  • 1,904
    Page views
  • 268
    Downloads
  • 4
    Citations

Article citation count generated by polling the highest count across the following sources: Scopus, PubMed Central, Crossref.

Comments

Download links

A two-part list of links to download the article, or parts of the article, in various formats.

Downloads (link to download the article as PDF)

Download citations (links to download the citations from this article in formats compatible with various reference manager tools)

Open citations (links to open the citations from this article in various online reference manager services)