Abstract

The Reproducibility Project: Cancer Biology seeks to address growing concerns about reproducibility in scientific research by conducting replications of 50 papers in the field of cancer biology published between 2010 and 2012. This Registered report describes the proposed replication plan of key experiments from ‘Intestinal Inflammation Targets Cancer-Inducing Activity of the Microbiota’ by Arthur et al. (2012), published in Science in 2012. Arthur and colleagues identified a genotoxic island in Escherichia coli NC101 that appeared to be responsible for causing neoplastic lesions in inflammation-induced IL10−/− mice treated with azoxymethane. The experiments that will be replicated are those reported in Figure 4 (Arthur et al., 2012). Arthur and colleagues inoculated IL10−/− mice with a mutated strain of E. coli NC101 lacking the genotoxic island, and showed that those mice suffered from fewer neoplastic lesions than mice inoculated with the wild type form of E. coli NC101 (Figure 4). The Reproducibility Project: Cancer Biology is a collaboration between the Center for Open Science and Science Exchange, and the results of the replications will be published by eLife.

DOI: http://dx.doi.org/10.7554/eLife.04186.001

Original article

  1. Intestinal inflammation targets cancer-inducing activity of the microbiota

    1. JC Arthur
    2. E Perez-Chanona
    3. M Muhlbauer
    4. S Tomkovich
    5. JM Uronis
    6. TJ Fan
    7. BJ Campbell
    8. T Abujamel
    9. B Dogan
    10. AB Rogers
    11. et al.
    Science 2012;338:120-123

Main text

Introduction

In their 2012 Science paper, Arthur and colleagues examined the interplay between colitis and colon cancer. They identified a shift in the composition of the microbiota in Il10−/− mice, which develop chronic colitis; amongst other changes noted, Escherichia coli was over 100-fold more represented in Il10−/− mice than wild type. After treatment with azoxymethane (AOM), a carcinogen that induces colon cancer, germ-free Il10−/− mice mono-associated with the colitis-inducing E. coli NC101 strain developed invasive mucinous carcinomas, while mice mono-associated with Enterococcus faecalis, another colitis-inducing bacterial strain, did not. E. coli NC101 harbors a 54 kb polyketide synthases (pks) genotoxic island encoding several enzymes involved in the production of toxin called Colibactin. This island has been previously shown to induce DNA damage, double strand breaks and aneuploidy (Nougayrede, 2006; Cuevas-Ramos et al., 2010) and was not found in E. faecalis or another non-colitic E. coli strain, K12.

In Figure 4, Arthur et al. inoculated germ-free IL10−/− mice treated with or without AOM with either wild-type E. coli NC101, or with a mutant of E. coli NC101 lacking the pks island (NC1010Δpks). Arthur and colleagues first confirmed that loss of the pks island did not impair bacterial growth (Supplemental figure 7, replicated in Protocol 1). Presence or absence of the genotoxic pks island had no effect on colonic inflammation in IL10−/− mice alone or treated with AOM (Figure 4A). However, at 12 and 18 weeks, mice treated with AOM and inoculated with E. coli NC101Δpks had many fewer neoplastic lesions than mice inoculated with wild-type E. coli NC101 (Figure 4B). At 18 weeks, invasion (Figure 4C), tumor burden (Figure 4D) and tumor size (Figure 4E) were all reduced in mice mono-associated with E. coli NC101Δpks as compared to NC101. Taken together, the data indicate that loss of the pks genotoxic island from E. coli NC101 strongly reduces the incidence of colon cancer in mice with chronic colitis. These experiments are replicated in Protocol 3.

Buc et al. (2013) performed an experiment similar to Figure 3B (not included for replication in this study), wherein Arthur et al. (2012) examined if the pks genotoxic island was more prevalent in patients with colorectal cancer. Both the dataset from Arthur et al. (2012) and the dataset presented by Buc et al. (2013) support the hypothesis that the genotoxic pks island is more prevalent in patients with colorectal cancer. Cougnoux et al. (2014), while not performing a direct replication, extended the findings of Arthur and colleagues to explore the mechanism of pks-produced colobactin toxicity effects on colorectal cancer. Finally, Arthur and colleagues have since published further work exploring in greater detail the genetic mechanisms behind the association of colorectal cancer with colitis and colonization by E. coli NC101 with or without the pks island, in which they demonstrate that colon inflammation itself has an influence on the expression of the genotoxic pks island (Arthur et al., 2014).

Materials and methods

Unless otherwise noted, all protocol information was derived from the original paper, references from the original paper, or information obtained directly from the authors. An asterisk (*) indicates data or information provided by the Reproducibility Project: Cancer Biology core team. A hashtag (#) indicates information provided by the replicating lab.

Protocol 1: comparing the growth curves of E. coli NC101 and E. coli NC101Δpks

This protocol describes how to grow both E. coli NC101 and E. coli NC1010Δpks to compare their growth curve, as seen in Supplemental figure 7.

Sampling

  • This experiment will be repeated a total of three times.

    • a. Power calculations were not performed, as no significant difference was reported in the original study.

Materials and reagents

  • Reagents that differ from those used originally are indicated with an asterisk (*).

ReagentTypeManufacturerCatalog #Comments
E. coli NC101 strainCellsOriginal authorsn/a
E. coli NC101Δpks strainCellsOriginal authorsn/a
Luria–Bertani (LB) broth*MaterialsSigmaL3022Original unspecified
Procedure

  1. Streak E. coli strains NC101 and NC101∆pks on agar plates and incubate at 37°C overnight.

  2. Inoculate 5 ml Luria–Bertani broth (LB broth) with a single colony of each E. coli strain picked from the freshly streaked plates and grow for ∼8 hr at 37°C in a shaking incubator.

  3. Inoculate E. coli strains at a 1:500 dilution in 150 ml LB broth and incubate at 37°C overnight (12–16 hr) in a shaking incubator.

  4. Inoculate E. coli strains at 1:500 dilution in 10 ml LB broth and measure the OD600 of the diluted cultures (timepoint 0). Collect the remaining E. coli for genomic DNA extraction in protocol 2.

  5. Incubate at 37°C in a shaking incubator.

    • a. Measure the OD600 of cultures every 20–30 min until saturation phase has been reached for both cultures (∼8 hr).

    • b. Lab will note instrument make, model and RPM.

  6. Repeat independently two additional times.

Deliverables

  • Data to be collected:

    • a. Raw values for OD600 measurements at each time point for NC101 and NC101∆pks.

    • b. Semi-logarithmic graph of average OD600 (log) vs time (linear) for NC101 and NC101∆pks.

  • Sample delivered for further analysis:

    • a. Cultures from Step 3 for use in Protocol 2.

Confirmatory analysis plan

  • Statistical analysis of the replication data:

    • a. Comparison of growth curve fit by nonlinear regression.

Known differences from the original study

  • The 37°C incubator used by the lab may not be the same make and model as used in the original study. The lab will note the make and model of the incubator they use.

Provisions for quality control

All data obtained from the experiment—raw data, data analysis, control data and quality control data—will be made publicly available, either in the published manuscript or as an open access dataset available on the Open Science Framework (https://osf.io/y4tvd/).

  • This protocol will confirm that loss of the pks genotoxic island does not affect the growth curve of E. coli NC1010Δpks as compared to its parental strain, E. coli NC101.

Protocol 2: PCR amplification and sequencing of polyketide synthase (pks) genotoxic island in E. coli NC101 and NC101∆pks

This protocol describes the PCR amplification of the 5′ and 3′ ends of the pks genotoxic island to confirm its presence in E. coli NC101 and its absence in E. coli NC101Δpks. This is a quality control step to confirm the absence of the pks island in the E. coli NC101Δpks strain.

Sampling

  • This experiment will be performed once.

    • a. Power calculations are not necessary for this PCR screen.

  • The experiment consists of two cohorts:

    • a. Genomic DNA from E. coli NC101.

    • b. Genomic DNA from E. coli NC101Δpks.

    • c. Each cohort has four PCR reactions run:

      • ■ L1 + L2: detects the 5′ end of the pks island.

      • ■ R1 + R2: detects the 3′ end of the pks island.

      • ■ 16S F + 16S R: control gene for amplification.

      • ■ ClbB-F + ClbB-R: also detects the colibactin gene.

        • i. Additional reaction recommended by original authors.

Materials and reagents

  • Reagents that differ from those used originally are indicated with an *.

ReagentTypeManufacturerCatalog #Comments
pks L1PrimerAt replicating lab's discretionn/aOriginal synthesis provider unspecified
pks L2PrimerAt replicating lab's discretionn/aOriginal synthesis provider unspecified
pks R1PrimerAt replicating lab's discretionn/aOriginal synthesis provider unspecified
pks R2PrimerAt replicating lab's discretionn/aOriginal synthesis provider unspecified
16S FPrimerAt replicating lab's discretionn/aOriginal synthesis provider unspecified
16S RPrimerAt replicating lab's discretionn/aOriginal synthesis provider unspecified
ClbB-FPrimerAt replicating lab's discretionn/aRecommended by the original authors
ClbB-RPrimerAt replicating lab's discretionn/aRecommended by the original authors
Agarose*ReagentSigmaA9539Original unspecified
Ethidium bromide*ReagentSigmaE1510Original unspecified
Procedure

  1. Extract bacterial genomic DNA from the sample collected in Protocol 1.

    • a. Lab will document their methodology for bacterial genomic DNA extraction.

    • b. Lab will include quality control data such as A260/A280 ratios from quantifying DNA concentration.

  2. Run the following PCR reactions:

    • a. Experimental.

      TemplateForward primerReverse primer
      E. coli NC101 gDNAL1L2
      E. coli NC101Δpks gDNAL1L2
      Water (no DNA control)L1L2
      E. coli NC101 gDNAR1R2
      E. coli NC101Δpks gDNAR1R2
      Water (no DNA control)R1R2
      E. coli NC101 gDNAClbB-FClbB-R
      E. coli NC101Δpks gDNAClbB-FClbB-R
      Water (no DNA control)ClbB-FClbB-R
    • b. Controls (additional control).

      TemplateForward primerReverse primer
      E. coli NC101 gDNA16S F16S R
      E. coli NC101Δpks gDNA16S F16S R
      Water (no DNA control)16S F16S R
    • c. Primers.

      PrimerSequenceExpected band sizeComment
      L15′-AAT CAA CCC AGC TGC AAA TC-3′1824 bpL1 + L2 detect the 3′ end of the pks
      L25′-CAC CCC CAT CAT TAA AAA CG-3′
      R15′-AGC CGT ATC CTG CTC AAA AC-3′1413 bpR1 + R2 detect the 5′ end of the pks
      R25′-TCG GTA TGT CCG GTT AAA GC-3′
      ClbB-F5′-GCG CAT CCT CAA GAG TAA ATA-3′280 bp
      ClbB-R5′-GCG CTC TAT GCT CAT CAA CC-3′
      16S F5′-GTG STG CAY GGY TGT CGT CA-3′
      16S R5′-GTG STG CAY GGY TGT CGT CA-3′
    • d. Reaction set-up.

      10× buffer5 µl
      5 µM dNTPs0.5 µl
      50 mM MgCl21.5 µl
      5 µM F primer0.5 µl
      5 µM R primer0.5 µl
      Invitrogen Taq polymerase0.5 µl
      Bacterial genomic DNA2 µl
      WaterBring up to 50 µl
    • e. Cycling parameters.

      • i. Denature at 95°C for 5 min.

      • ii. 35 (up to 50) cycles of:

        • ■ 95°C for 45 s.

        • ■ 56°C for 45 s.

        • ■ 72°C for 45 s.

      • iii. 72°C for 10 min.

      • iv. Hold at 4°C forever.

  3. Run out PCR amplicons on a 1.5% agarose gel alongside a size marker. Visualize with ethidium bromide.

  4. Sequence pks amplicons by Sanger automated DNA sequencing.

    • a. BLAST sequencing results against the E. coli Colibactin synthesis cluster (AM229678.1).

  5. Sequence 16S amplicons to confirm identity of bacterial strains (additional control).

Deliverables

  • Data to be collected:

    • a. Full gel image of ethidium bromide stained gel showing PCR products for pks-L, pks-R and 16S amplicons.

    • b. Sequencing chromatograms.

    • c. BLAST comparison results.

Confirmatory analysis plan

  • Statistical analysis of the replication data:

    • a. No statistical test required.

    • b. Visually confirm presence of pks bands in E. coli NC101 and absence in E. coli NC101Δpks.

Known differences from the original study

  • Lab will use in-house bacterial genomic DNA extraction protocol.

    • a. Original was unspecified.

Provisions for quality control

All data obtained from the experiment—raw data, data analysis, control data and quality control data—will be made publicly available, either in the published manuscript or as an open access dataset available on the Open Science Framework (https://osf.io/y4tvd/).

  • We will sequence the 16S rRNA of the bacteria to confirm the identity of the strain. The sample purity (A260/A280 ratio) of the extracted DNA will be recorded.

Protocol 3: mono-associate mice with E. coli NC101 or NC101∆pks and analyze intestinal tumorigenesis and inflammation

This protocol describes the inoculation of germ-free Il10−/− mice with E. coli as well as treatment with the carcinogen azoxymethane (AOM), as seen in Figure 4.

Sampling

  • This experiment will use at least 10 mice per group for a final power of 80–96%.

    • a. See ‘Power calculations’ section for details.

  • The experiment consists of the following cohorts:

    • a. Germ-free IL10−/− mice treated with AOM and inoculated with E. coli NC101, harvested at 18 weeks post AOM.

      • n = 14.

        • i. Expected survival is 75% (as seen in Supplemental figure 10), thus 14 mice are required to predict 10 will survive.

    • b. Germ-free IL10−/− mice treated with AOM and inoculated with E. coli NC101Δpks, harvested at 18 weeks post AOM.

      • n = 16.

        • i. Expected survival is 62.5% (as seen in Supplemental figure 10), thus 16 mice are required to predict 10 will survive.

Materials and reagents

  • Reagents that differ from those used originally are indicated with an *.

ReagentTypeManufacturerCatalog #Comments
IL10−/− miceMiceOriginal labn/a
Wild-type miceMiceOriginal labn/a
Azoxymethane (AOM)reagentSigmaA5486
Formalin*ReagentSigmaHT501128original unspecified
0.1 mm zirconium beads and bead beaterEquipmentBioSpec Products1107900-101
DNeasy kitReagentQiagen69504
16S FPrimerAt replicating lab's discretionn/a
16S RPrimerAt replicating lab's discretionn/a
Procedure
Notes

  • Experiment should be conducted by experimenters blinded to genotype and treatment group.

  • Azoxymethane (AOM) can show lot-to-lot variation in potency, and will lose potency and gain toxicity over time. In order to minimize these effects, a single lot of AOM will be used throughout the experiment. AOM will be aliquoted into 25 mg/ml aliquots and stored at −80°C. A fresh aliquot will be used each time AOM is needed to avoid repeated freeze-thaw cycles.

    1. Breed and raise IL10−/− mice in germ-free isolators.

      • a. Use both male and female mice; house sexes separately.

      • b. House mice 2–4 mice per cage.

      • c. 12 hr/12 hr light/dark cycle.

      • d. Diet is Purine Lab Diet 3500.

    2. At age 7–12 weeks initiate program to induce colitis/colorectal cancer.

      • a. Maintain mice in gnotobiotic isolators.

    3. Randomly assign Il10−/− mice to two groups. As each mouse becomes eligible for induction of colitis/colorectal cancer, randomly assign to a treatment group using the adaptive randomization approach with the gender of the mice as the covariate that is assessed as mice are sequentially assigned to a particular treatment group. Assignment will aim for a similar distribution of genders in each cohort while also taking into account the pre-determined size of each treatment group.

      • a. Group 1: E. coli NC101; n = 14.

      • b. Group 2: E. coli NC101∆pks; n = 16.

    4. Colonize mice by oral gavage and rectal swabbing (dip sterile Q-tip in culture and swab anus) with log phase growth bactera:

      • a. Use 200 µl of an overnight bacterial culture with a concentration of 2 × 109 CFU/ml.

        • i. Record OD600 of culture used.

        • ii. Perform serial dilution and plating of the culture used to swab for quantitative culturing. Record the actual CFUs of the culture used to colonize each mouse.

      • b. E. coli NC101 or E. coli NC101∆pks for Il10−/− mice.

        • i. Maintain in separate gnotobiotic isolators that will contain only the bacterium of interest throughout the study.

      • c. After 4 weeks, confirm colonization by stool culture. Note: Steps i through vi are derived from Uronis et al. (2011). Confirm bacterial strain by 16S RT-PCR:

        • i. Collect up to 500 mg of fecal material.

          • ■ Pellets can be stored in an eppendorf tube at −80°C until processing.

        • ii. Resuspend in lysis buffer with 20 mg/ml lysozyme and incubate at 37°C for 30 min.

        • iii. Add 10% SDS and 350 µg/ml Proteinase K for further lysis.

        • iv. Homogenize samples with a bead beater and 0.1 mm zirconium beads.

        • v. Extract DNA with a DNeasy kit.

        • vi. Amplify the bacterial 16S ribosomal RNA gene.

          • ■ Forward primer: 5′-GTG STG CAY GGY TGT CGT CA-3′.

          • ■ Reverse primer: 5′-ACG TCR TCC MCA CCT TCC TC-3′.

          • ■ See Protocol 2, Procedure Step 2d and e for reaction and annealing conditions.

        • vii. *Sequence amplicons and BLAST sequencing results to confirm colonization with E. coli.

      • d. Quantify level of colonization by serial dilution culture:

        • i. Collect at least 200 mg of fecal material.

        • ii. Serially dilute 200 mg fecal material.

        • iii. Plate dilutions on LB agar plates.

          • ■ Dilute enough to resolve single colony forming units (CFUs).

        • iv. Calculate the total number of CFUs per 200 mg fecal matter.

    5. At the same time as first stool culture, intraperitoneally inject with 10 mg/kg azoxymethane (AOM).

    6. Repeat AOM injections every week for 5 more weeks (6 weeks total).

    7. 18 weeks after last AOM injection, sacrifice mice.

      • a. #Mice are anesthetized with isofluorane in a drop jar. Once respiration has ceased, the mice are exsanguinated.

      • b. *Collect stool and quantify colonization as performed in Step 5b.

    8. Macroscopically examine tumor formation:

      • a. Remove colons from the cecum to the rectum, flush with PBS, and splay longitudinally.

      • b. Blindly count tumors per mouse and measure tumor diameter macroscopically. Image colon and tumors.

    9. Prepare tissue for histological analysis:

      • a. Collect distal colon tissue samples.

      • b. Swiss-roll colon tissue samples from the distal to the proximal end and fix overnight in 10% formalin.

    10. Paraffin-embed tissues.

      • a. #The replicating lab uses an automated embedding station:

        • i. Samples are passed through a dehydration series consisting of 70%, 80%, 2 × 95% and 3 × 100% ethanol for 30 min each wash.

        • ii. Samples are washed into xylene; first wash is 30 min, the second is an hour.

        • iii. The samples are washed into 57°C Paraplast; four washes of 30 min each.

        • iv. Samples are mounted in mold and allowed to cool and harden.

    11. Cut 6 µm sections and mount on slides.

    12. Stain with hematoxylin and eosin for histologic analysis.

      • a. Lab will record H&E staining protocol used.

    13. Blindly score for inflammation, dysplasia, and invasion (scoring should by performed by an expert animal histopathologist).

      • a. Score mucosal inflammation (0–4) by the degree of lamina propria mononuclear cell (LPMNC) infiltration, crypt hyperplasia, goblet cell depletion, and architectural distortion.

      • b. Score dysplasia as follows:

        • i. 0 = no dysplasia.

        • ii. 1 = mild dysplasia characterized by aberrant crypt foci (ACF), +0.5 for small gastrointestinal neoplasia (GIN), or multiple ACF.

        • iii. 2 = moderate dysplasia with GIN, +0.5 for multiple occurrences or small adenoma.

        • iv. 3 = severe or high grade dysplasia restricted to the mucosa.

        • v. 3.5 = adenocarcinoma, invasion through the muscularis mucosa.

        • vi. 4 = adenocarcinoma, full invasion through the submucosa and into or through the muscularis propria.

      • c. Score invasion as follows:

        • i. 0 = no invasion.

        • ii. 1 = 5–10% involvement.

        • iii. 2 = 10–15% involvement.

        • iv. 3 = 25–50% involvement.

        • v. 4 = >50% involvement.

          • ■ Multiply invasion score by 1 if invasion is through the muscularis mucosa, and by 2 if invasion is through the muscularis propria and serosa.

Deliverables

  • Data to be collected:

    • a. Mouse records (gender used in each group, type of colonization procedure, health records, condition for early euthanasia, etc).

    • b. OD600 of overnight bacterial cultures used in colonization.

    • c. (Compare to Supplemental figure 10): Raw data and Kaplan–Meier survival curve of mice for all conditions.

    • d. Sequencing chromatograms and gel image of amplicons from stool sample confirming colonization with E. coli.

    • e. (Compare to Figure 4D): Images, raw numbers and dot plot graph of macroscopic tumor number per mouse (multiplicity) at 18 weeks of colon tissue from mice for all conditions.

    • f. (Compare to Figure 4F): Micrographs of H&E histology for each mouse at 18 weeks for all conditions.

    • g. (Compare to Figure 4E): Raw numbers and dot plot graph of mean macroscopic tumor diameter in each mouse at 18 weeks of colon tissue from mice for all conditions.

    • h. (Compare to Figure 4A): Raw numbers and dot plot graph of inflammation scores at 18 weeks of colon tissue from mice for all conditions.

    • i. (Compare to Figure 4B): Raw numbers and dot plot graph of neoplasia scores at 18 weeks of colon tissue from mice for all conditions.

    • j. (Compare to Figure 4C): Raw numbers and dot plot graph of invasion scores at 18 weeks of colon tissue from mice for all conditions.

    • k. Counts of bacterial colonization from stool sample per mouse at 4 week time point and at sacrifice.

Confirmatory analysis plan

  • Statistical analysis of the replication data:

    • a. (As seen in Supplemental figure 10): Compare survival of AOM-treated Il10−/− mice inoculated with E. coli NC101 relative to NC101∆pks.

      • ■ Log-rank test (Mantel Cox).

    • b. (As seen in Figure 4A, right panel): Compare mean inflammation scores of AOM-treated Il10−/− mice mono-associated with E. coli NC101 relative to NC101∆pks.

      • ■ Unpaired two-tailed Student's t-test.

    • c. (As seen in Figure 4B): Compare mean neoplasia score of AOM-treated Il10−/− mice mono-associated with E. coli NC101 relative to NC101∆pks.

      • ■ Unpaired two-tailed Student's t-test.

    • d. (As seen in Figure 4C): Compare mean invasion score of AOM-treated Il10−/− mice mono-associated with E. coli NC101 relative to NC101∆pks.

      • ■ Unpaired two-tailed Student's t-test.

    • e. (As seen in Figure 4D): Compare mean macroscopic tumor number (multiplicity) of AOM-treated Il10−/− mice mono-associated with E. coli NC101 relative to NC101∆pks.

      • ■ Unpaired two-tailed Student's t-test.

    • f. (As seen in Figure 4E): Mean macroscopic tumor diameter per mouse of AOM-treated Il10−/− mice mono-associated with E. coli NC101 relative to NC101∆pks.

      • ■ Unpaired two-tailed Student's t-test.

    • g. Compare mean number of CFUs per 200 mg stool pellet between E. coli NC1010-colonized and E. coli NC1010Δpks-colonized mice at 4 weeks post-inoculation and at sacrifice.

      • ■ Two-way ANOVA.

  • Meta-analysis of original and replication attempt effect sizes:

    • a. This replication attempt will perform the statistical analysis listed above, compute the effects sizes, compare them against the reported effect size in the original paper and use a meta-analytic approach to combine the original and replication effects, which will be presented as a forest plot.

Known differences from the original study

  • The replication attempt will be restricted to the 18 week time point.

  • The microscope used in the original lab was an Olympus CX41; the replicating lab will use an Olympus BX41.

  • We will be using a different lot of azoxymethane that used by the original authors. AOM is known to have lot-to-lot variation in efficacy that may affect the absolute numbers of tumors generated per mouse.

  • The original study collected data on 4 female and 8 male IL10-/- mice inoculated with E. coli NC101 and 8 male IL10-/- mice inoculated with E. coli NC101Δpks. While the gender of the mice in the replication is not currently known, the mice will be randomly assigned when they reach the age for inoculation with the aim of a similar gender distribution in each group. This will likely generate a different gender distribution than the original study.

Provisions for quality control

All data obtained from the experiment—raw data, data analysis, control data and quality control data—will be made publicly available, either in the published manuscript or as an open access dataset available on the Open Science Framework (https://osf.io/y4tvd/).

  • The experiment will be performed by a Science Exchange lab with expertise in germ free mouse studies.

  • Experimenters will be blinded to the genotype and treatment group. Mice will be randomly assigned to treatment groups.

Power calculations

Protocol 1

  • Not applicable.

Protocol 2

  • Not applicable.

Protocol 3

Summary of original data

Figure 4A:

4A: Inflammation score at 18 weeksMeanSDN
IL10−/− mice inoculated with E. coli NC101409
IL10−/− mice inoculated with E. coli NC101deltaPKS3.80.455
Test family

  • Two-tailed t-test, difference between two independent means, alpha error = 0.05.

    • a. Sensitivity calculations were performed using G*power software (Faul et al., 2007).

Power calculations

  • Because the original data shows a non-significant effect, we will not be powering this replication to detect an effect. Based on the sample size of 10 mice per group derived from Figure 4C, with α of 0.05 we will be powered to 80% to detect a Cohen's d of 1.3249474.

Summary of original data

  • Note: Raw data values obtained from scatterplot with confirmation of accuracy from the authors.

    4B: Neoplasia score at 18 weeks with AOM treatmentMeanSEMSDN
    IL10−/− mice inoculated with E. coli NC1014.440.180.539
    IL10−/− mice inoculated with E. coli NC101Δpks3.600.240.555
  • Stdev was calculated using formula SD = SEM*(SQRT n).

Test family

  • Two-tailed t-test, difference between two independent means, alpha error = 0.05.

    • a. Power calculations were performed for statistically significant effects reported in original study using G*power software (Faul et al., 2007).

Power calculations

Group 1 vsGroup 2Pooled SDEffect sizeA priori powerGroup 1 sample sizeGroup 2 sample size
NC101NC101Δpks0.5341.57303483.2%*8*8*
  • Based on the sample size required for Figure 4C, we will use 10 mice per group. This brings the a priori power to 91.3%.

Summary of original data

  • Note: Raw data values obtained from scatterplot with confirmation of accuracy from the authors.

4C: Invasion score at 18 weeks with AOM treatmentMeanSEMSDN
IL10−/− mice inoculated with E. coli NC10130.631.99
IL10−/− mice inoculated with E. coli NC101Δpks0.80.370.845
  • Stdev was calculated using formula SD = SEM*(SQRT n).

Test family

  • Two-tailed t-test, difference between two independent means, alpha error = 0.05.

    • a. Power calculations were performed for statistically significant effects reported in original study using G*power software (Faul et al., 2007).

Power calculations

Group 1 vsGroup 2Pooled SDEffect sizeA priori powerGroup 1 sample sizeGroup 2 sample size
NC101NC101Δpks4.4101.814059*80.1%6*6*
  • Based on the sample size required for 4C, we will use 10 mice per group. This brings the a priori power to 96.9%.

Summary of original data

  • Note: Raw data values obtained from scatterplot with confirmation of accuracy from the authors.

4D: Tumor number per mouse at 18 weeks with AOM treatmentMeanSEMSDN
IL10−/− mice inoculated with E. coli NC10110.61.85.39
IL10−/− mice inoculated with E. coli NC101Δpks2.60.71.55
  • Stdev was calculated using formula SD = SEM*(SQRT n).

Test family

  • Two-tailed t-test, difference between two independent means, alpha error = 0.05.

    • a. Power calculations were performed for statistically significant effects reported in original study using G*power software.

Power calculations

Group 1 vsGroup 2Pooled SDEffect sizeA priori powerGroup 1 sample sizeGroup 2 sample size
NC101NC101Δpks1.6251.35384681.7%1010

Figure 4E:

4E: Mean macroscopic tumor diameter at 18 weeksMeanSDN
IL10−/− mice inoculated with E. coli NC1013.020.749
IL10−/− mice inoculated with E. coli NC101deltaPKS3.661.595
Test family

  • Two-tailed t-test, difference between two independent means, alpha error = 0.05.

    • a. Sensitivity calculations were performed using G*power software (Faul et al., 2007).

Power calculations

  • Because the original data shows a non-significant effect, we will not be powering this replication to detect an effect. Based on the sample size of 10 mice per group derived from Figure 4C, with α of 0.05 we will be powered to 80% to detect a Cohen's d of 1.3249474.

Supplemental figure 10:

Test family

  • Two-tailed t-test, difference between two independent means, alpha error = 0.05.

    • a. Sensitivity calculations were performed using G*power software (Faul et al., 2007).

Power calculations

  • Because the original data shows a non-significant effect, we will not be powering this replication to detect an effect. Based on the sample size of 10 mice per group derived from Figure 4C, with α of 0.05 we will be powered to 80% to detect a Cohen's d of 1.1453705.

References

Acknowledgements

The Reproducibility Project: Cancer Biology core team would like to thank the original authors, in particular Janelle C Arthur and Christian Jobin, for generously sharing critical information as well as reagents to ensure the fidelity and quality of this replication attempt. We would also like to thanks the following companies for generously donating reagents to the Reproducibility Project: Cancer Biology; American Tissue Culture Collection (ATCC), BioLegend, Cell Signaling Technology, Charles River Laboratories, Corning Incorporated, DDC Medical, EMD Millipore, Harlan Laboratories, LI-COR Biosciences, Mirus Bio, Novus Biologicals, Sigma–Aldrich, and System Biosciences (SBI).

Decision letter

Cynthia Sears, Reviewing editor, Johns Hopkins Univ SOMl, United States

eLife posts the editorial decision letter and author response on a selection of the published articles (subject to the approval of the authors). An edited version of the letter sent to the authors after peer review is shown, indicating the substantive concerns or comments; minor concerns are not usually shown. Reviewers have the opportunity to discuss the decision before the letter is sent (see review process). Similarly, the author response typically shows only responses to the major concerns raised by the reviewers.

Thank you for sending your work entitled “Registered report: Intestinal inflammation targets cancer-inducing activity of the microbiota” for consideration at eLife. Your Registered report has been favorably evaluated by Richard Losick (Senior editor), a guest editor, 4 reviewers, and a biostatistician.

The Reviewing editor and the other reviewers discussed their comments before we reached this decision, and the Reviewing editor has assembled the following comments to help you prepare a revised submission.

The proposed experiments are largely appropriately designed although there are important points of clarification required.

1) The authors accurately summarize the literature directly related to the original paper, but should also be aware of two key manuscripts relevant to Colibactin: PMIDs 20534522 (PNAS 107:11537, 2010) and 16902142 (Science 313:848, 2006).

2) There are key concerns regarding the azoxymethane lot and its handling and documentation. First, the authors should note that each lot of azoxymethane may have differing potency, so one lot should be used throughout this replication experiment. Furthermore, this lot should first be tested for its ability to induce tumors in the AOM/DSS model. Using a different lot of azoxymethane from the original study could result in differences in tumorigenesis (i.e. penetrance, size, invasion, etc.) between the original study and the reproduced study. Importantly though, this should not affect differences in tumorigenesis between experimental groups but could lead to uninterpretable results if tumorigenesis is very low in the E. coli NC101, positive control group, associated with the lot of AOM. Second, the replication authors should describe better how they aliquot, store and use AOM, which soon after dilution becomes less stable and less carcinogenic, but more toxic and therefore can cause unexpected mortality that would hamper the experimental design.

3) While this project has been under review, another original paper has been published by the authors of the paper to be replicated: Arthur et al., Nature Communications, Sept 2014 (on-line). The E. coli NC101 tumorigenesis model is further explored in this manuscript. In consultation with the original authors, the replication project should clarify the contribution of AOM to the model. In Uronis et al. (PLOS ONE 4:e6026, 2009), it appears that IL10-/- mice alone were not evaluated for tumorigenesis upon colonization with E. coli NC101. In this replication project, the text of the original Science paper suggests that IL10-/- is insufficient to promote tumorigenesis upon colonization with NC101 whereas in the most recent Arthur publication noted above, IL10-/- mice exhibit similar tumorigenesis, but not invasive extent of neoplasia, compared to the AOM + IL10-/- model (Figure 4, Nature Communications, on-line as above). The authors of the replication project should comment on the impact of these recent data on their replication plan. We query whether the replication authors should consider including IL-10-/- mice colonized with E. coli NC101 without AOM treatment as a group into their experiments.

4) There is no explicit indication that the mice in the gnotobiotic study will be age-matched or sex-matched. It is only stated that cancer/colitis will be induced at 7-12 weeks and that male and female mice will be used. Although not explicitly stated, one assumes that male and female mice will be housed in separate isolators. Will the numbers of male and female mice used approximate that in the original experiments?

5) Because application of histopathologic criteria can be somewhat subjective and pathologist-dependent, the authors should consider whether histologic images should be evaluated independently by two pathologists and, if possible, by the animal histopathologist who evaluated the images in the original publication. The scoring system is specific about lamina propria mononuclear cell infiltration but polymorphonuclear infiltration should also be considered.

6) The method of inoculation allows for more than one approach although the original paper utilized gavage and rectal swabbing. While the authors suggest colonization with oral swab as an alternative, oral gavage is likely to yield more reproducible colonization results. Please clarify.

7) Please specify the typical OD600 or range considered acceptable to be used for the bacterial inoculation cultures. Confirmation of exact CFUs and culture purity by quantitative culturing of the inoculum should be performed. Confirmation that the inoculated strains are ClbB-positive and ClbB-negative should be done. Similarly, cultures to verify colonization with the specific strain inoculated at week 4 and at the end of the experiment, should be completed.

8) Sampling. It is not clear why the expected mortality for E. coli NC101Δpks is higher, when this strain induces less tumorigenesis than the parent strain E. coli NC101.

9) Can the authors clarify the approach/plan if the proposed experiments do not replicate the original data? Is there any back-up plan to adjust the experimental plan to adjust for potential technical concerns such as the potency of the azozymethane lot or if the experimental course differs in a different facility (e.g., more mouse death)?

Statistical comments:

1) Cross-study variation should be taken into account to determine power pre-data collection. This is hard to estimate, but papers by Giovanni Parmigiani and collaborators at the Dana Farber provide some estimates about cross-study variation that could be used for this purpose. The authors should budget some additional variability because of cross-study reproducibility, and increase the sample size on-the-fly, as they deem appropriate.

2) The final report on the replicated study should report the actual power of the tests, based on the standard deviations in the replicated study.

3) Please clarify the comment 'based on the sample size of 13 mice per group derived from Figure 4C'. The sample size for Figure 4C is stated as 10 to achieve 80% power, α = 0.05.

[Editors' note: further revisions were requested prior to acceptance, as described below.]

Thank you for resubmitting your article entitled “Registered report: Intestinal inflammation targets cancer-inducing activity of the microbiota” for further consideration at eLife. Your revised Registered Report has been favorably evaluated by Richard Losick (Senior editor), a guest Reviewing editor, three of the original reviewers, and a biostatistician.

There are two remaining issues that need to be addressed before acceptance, as outlined below:

1) Response to point 7: the reviewers feel strongly that quantitative cultures of the inoculum to prove purity and numbers of organisms inoculated should be performed. Relying on OD600 solely does not account for culture purity, nor potentially nonviable organisms in the culture.

2) Response to point 1 of the statistical review: one approach to incorporate putative extra variation in the proposed study would be to calculate sample size by assuming a range of the anticipated effect size. For example, the effect size for, say, Figure 4C used in the power calculation is 1.353846. So, one may examine sample size for smaller effect size: say, effect size of 1.2 and 1.1 and see how much the sample size would change in these two additional settings. When we do this calculation, we can see that the sample size is approximately 11 mice in each group if the effect size is 1.2, and approximately 13 mice in each group when the effect size is 1.1. The sample size does not increase drastically under these additional effect sizes. So, perhaps, the sample size of 10 is fine. Of course, if the observed effect size at the end of the study is less than 1.353846, then the result is likely going to be non-significant since the study was powered to detect an effect size of 1.353846 or more.

We would not recommend a post-hoc power calculation. But the investigators should consider saying upfront in the Methods section (when they publish their work) that they calculated the power at the beginning of the study using a two-sample t-test framework to detect a certain effect size.

DOI: http://dx.doi.org/10.7554/eLife.04186.002

Author response