The Reproducibility Project: Cancer Biology seeks to address growing concerns about reproducibility in scientific research by conducting replications of selected experiments from a number of high-profile papers in the field of cancer biology. The papers, which were published between 2010 and 2012, were selected on the basis of citations and Altmetric scores (Errington et al., 2014). This Registered Report describes the proposed replication plan of key experiments from 'Fusobacterium nucleatum infection is prevalent in human colorectal carcinoma' by Castellarin and colleagues published in Genome Research in 2012 (Castellarin et al., 2012). The experiment to be replicated is reported in Figure 2. Here, Castellarin and colleagues performed a metagenomic analysis of colorectal carcinoma (CRC) to identify potential associations between inflammatory microorganisms and gastrointestinal cancers. They conducted quantitative real-time PCR on genomic DNA isolated from tumor and matched normal biopsies from a patient cohort and found that the overall abundance of Fusobacterium was 415 times greater in CRC versus adjacent normal tissue. These results confirmed earlier studies and provide evidence for a link between tissue-associated bacteria and tumorigenesis. The Reproducibility Project: Cancer Biology is a collaboration between the Center for Open Science and Science Exchange and the results of the replications will be published in eLife.https://doi.org/10.7554/eLife.10012.001
The human intestine is populated by an estimated 1014 microbes comprising over 1000 bacterial phylotypes (Ley et al., 2006). The overall composition of the intestinal microbiota is determined by a number of factors, including host genetics, environment, diet and hygiene (Arrieta et al., 2014; Keku et al., 2015). These bacteria play important roles in host biology by maintaining intestinal homeostasis, barrier function, immunity and metabolic function (Backhed et al., 2005; Jones et al., 2014). Perturbations or imbalances in the microbiome (microbial dysbiosis) are linked to a number of disease pathologies such as inflammatory bowel disease (Collins, 2014; Hold et al., 2014), obesity (Bajzer and Seeley, 2006; Brown et al., 2012), and colorectal cancers (CRCs; Dulal and Keku, 2014; Keku et al., 2015).
CRC is a complex disease arising from the sequential accumulation of somatic mutations and epigenetic alterations. Activating mutations in the K-ras oncogene, as well as the loss of tumor suppressor genes like p53 (TP53) and adenomatous polyposis coli (APC), contribute to the tumorigenic transformation of normal colonic epithelium (Vogelstein et al., 1988; Fearon, 2011; Mundade et al., 2014). In addition to genetic factors, microbial dysbiosis, such as altered bacterial diversity, is strongly associated with the development of CRC (Keku et al., 2015). However, despite numerous longitudinal studies comparing intestinal microbial communities over time (Rodriguez et al., 2015), and across various cancer stages (Kubota, 1990; Chen et al., 2013; Nugent et al., 2014), there is limited information on the contribution of specific bacteria to CRC development.
To identify potential associations between inflammatory microorganisms and gastrointestinal cancers, Castellarin et al. (2012) first performed RNA sequencing (RNA-seq) on a limited number of tumor and matched normal tissue samples. Initial observations indicated a striking overrepresentation of Fusobacterium nucleatum sequences in carcinoma samples compared to controls. To confirm these findings, Castellarin et al. (2012) assessed the relative abundance of Fusobacterium in a larger cohort of tumor and matched normal biopsy samples. In Figure 2, the authors performed quantitative real-time PCR (qPCR) on genomic DNA (gDNA) isolated from an additional 88 colorectal carcinoma (CRC) specimens and adjacent matched control tissues. Fusobacterium abundance was observed to be significantly higher in the tumor samples compared to matching control samples. This key experiment will be replicated in Protocol 1.
Similar findings confirming the higher relative abundance of Fusobacterium in CRC tumor tissues compared to control biopsies have been reported by other investigators (Kostic et al., 2012; McCoy et al., 2013; Warren et al., 2013; Tahara et al., 2014). In fact, the study by Kostic et al. (2012) is considered a co-discovery of this phenomenon. McCoy et al. (2013) successfully validated the association between Fusobacterium and CRC in a set of matched CRC tumor and normal human colon tissue samples using both pyrosequencing and qPCR analysis of the 16S bacterial rRNA gene. Findings by Mira-Pascual et al. (2015) further confirm this trend, as this group observed a significantly higher presence of F. nucleatum in mucosal samples from the CRC patients compared to the healthy subjects (as opposed to matched tissue biopsies). Recent studies have also reported a higher presence of Fusobacterium species in human colonic adenomas (polyps) and in stool samples from adenoma and tumor carcinoma patients compared to healthy subjects (Kostic et al., 2012; 2013; McCoy et al., 2013). Furthermore, other studies have expanded these findings to identify potential mechanisms of action of F. nucleatum during tumorigenesis (Rubinstein et al., 2013; Gur et al., 2015). Rubenstein et al. (2013) also indirectly confirm a higher abundance of Fusobacterium in CRC patients by measuring higher F. nucleatum FadA mRNA expression relative to healthy controls.
Unless otherwise noted, all protocol information was derived from the original paper, references from the original paper, or information obtained directly from the authors. An asterisk (*) indicates data or information provided by the Reproducibility Project: Cancer Biology core team. A hashtag (#) indicates information provided by the replicating lab.
This protocol utilizes quantitative PCR to test the relative abundance of F. nucleatum DNA in gDNA isolated from matched normal and tumor human colon cancer specimens. It is a replication of Figure 2.
This experiment will include 40 matched samples for a final power of 87.26%.
See power calculations for details.
Each patient sample has two cohorts:
Cohort 1: Colon tumor sample (n = 40)
Cohort 2: Matched normal tissue within the same individual (n = 40)
Cohort 3: Age/ethnicity-matched normal tissue from additional control individuals (n = 40)
Tissue is collected during surgery (either partial colectomy, ileocolectomy, colorectal resection, or proctocolectomy) from tumor tissue, adjacent normal tissue, or from normal controls. Samples are frozen on liquid nitrogen within 30 min after extractions. Diagnosis is confirmed by a pathologist using histological sections from each sample.
Quantitative PCR will be performed for each sample two independent times in technical triplicate for the following:
F. nucleatum DNA
Prostaglandin transporter—reference gene
|Frozen human colon tumor samples |
and matched normal samples
|#iSpecimen||Data include age, gender, ethnicity, |
diagnosis, histopathology report
|Gentra Puregene Genomic |
DNA extraction kit
|Qiagen||158667||Replaces Qiagen 69504|
|PicoGreen Assay||#Life Technologies||P7589|
|384-well optical PCR plate||#Phoenix Research||MPS-3898|
|Fusobacteria forward qPCR primer||Part of a custom-designed |
Taqman primer/probe set
|Fusobacteria reverse qPCR primer||GTTGACTTTACAGAAGGAGA |
|Fusobacteria FAM probe||TCAGCAACTTGTCCTTCTTGA |
|PGT forward qPCR primer||Part of a custom-designed |
Taqman primer/probe set
|PGT reverse qPCR primer||AGAGGCCAAGATAGTCCTG |
|PGT FAM probe||CCATCCATGTCCTCATCTC|
|TaqMan Universal Master Mix||ABI||#4304437|
|qPCR thermal cycling system||ABI||#4351405||7900HT system|
†Note: Probe sequence from original manuscript incorrect. Correct sequence seen here from Flanagan et al., 2014.
Obtain ~40 sets from frozen human CRC tumors with matched normal control, and an additional control group of age/ethnicity-matched tissue from healthy individuals.
Tissue will have been flash-frozen in liquid nitrogen very soon after harvest.
Pathological data showing positive diagnosis for CRC will be included with samples.
Extract gDNA using Gentra Puregene genomic DNA extraction kit according to manufacturer’s instructions.
Quantify gDNA concentration by Nanodrop spectrophotometer.
Assemble 20 μL qPCR reactions in a 384-well optical PCR plate. Each sample is assayed in triplicate for each primer/probe set. Each reaction contains:
5 ng of gDNA
18 μM of each primer
5 μM of probe
1 X final concentration of TaqMan Universal Master Mix
Perform amplification and detection of DNA using the following reaction conditions:
2 min at 50°C
10 min at 95°C
40 cycles of 15 s at 95°C and 1 min at 60°C.
Calculate cycle threshold using the automated settings. Analyze and compute ΔΔCT values by normalizing to prostaglandin transporter reference gene.
The mean ΔΔCT values from the technical replicates from the tumor and normal sample will be used to calculate the ratio of tumor versus normal for each matched biopsy.
Repeat steps 3–5 for each sample a second time.
The mean ratios of ΔΔCT values in tumor versus normal sample from the two independent experimental replicates will be calculated for each matched biopsy.
Data to be collected:
Descriptive data of gDNA samples including: patient sample age/sex, ethnicity, and % area of the tumor involved with necrosis.
Purity (A260/280 and A260/230 ratios) and concentration of isolated total gDNA from tumor biopsies.
Raw qRT-PCR values, as well as analyzed ΔΔCT values for each tumor and matched biopsy sample. Bar graph of mean relative abundance of F. nucleatum in tumor versus normal colorectal samples (compare to Figure 2A).
This replication attempt will perform the statistical analysis listed below:
Statistical analysis of replication data:
Note: At the time of analysis, we will perform the Shapiro–Wilk test and generate a quantile–quantile (q–q) plot to assess the normality of the data. If the data appear skewed, we will perform the appropriate transformation in order to proceed with the proposed statistical analysis. If this is not possible, we will perform the equivalent nonparametric test (e.g., Wilcoxon-signed rank test).
One-sample Student’s t-test using the log of the mean ratios of ΔΔCT values from the two independent experimental replicates, tumor ΔΔCT/matched within individual controls compared to a mean value of zero.
Additional exploratory analysis:
Two Student’s t-tests with Bonferroni correction comparing absolute values from:
Mean tumor Fusobacterium abundance versus within subject matched control (paired)
Mean tumor Fusobacterium abundance versus healthy matched control (unpaired)
Meta-analysis of original and replication attempt effect sizes:
Compute the effect size, compare it against the effect size in the original paper and use a random effects meta-analytic approach to combine the original and replication effects, which will be presented as a forest plot.
All known differences are listed in the 'Materials and reagents' section with the originally used item listed in the comments section. All differences have the same capabilities as the original and are not expected to alter the experimental design. We have added an additional control of matched gDNA from healthy individuals.
The sample purity (A260/280 and A260/230 ratios) of the isolated gDNA from each sample will be reported. All of the raw data, including the analysis files, will be uploaded to the project page on the OSF (https://osf.io/v4se2) and made publically available.
For a detailed breakdown of all power calculations, see spreadsheet at https://osf.io/yadgq/
Note: Data estimated from graph reported in Figure 2.
Ratio one-sample t-test: aerror = 0.05, µ = 0.
Ratio t-test and power calculations were performed with R software, version 3.1.2 (Team RC 2014).
|A priori power||Total sample size|
*Forty total ratios (40 tumor 40 matched controls) will be used.
Paired Student’s t-test (two-tailed): aerror = 0.025.
Sensitivity calculations were performed with G*Power software, version 3.1.7 (Faul et al., 2007).
|Group 1||Group 2||Detectable effect |
|A priori power||Total |
|Tumor sample||Adjacent matched |
Independent Student’s t-test (two-tailed): aerror = 0.025.
Sensitivity calculations were performed with G*Power software, version 3.1.7. (Faul et al., 2007).
|Group 1||Group 2||Detectable effect |
|A priori power||Total |
|Tumor sample||Healthy |
The intestinal microbiome in early life: health and diseaseFrontiers in Immunology 5:427.https://doi.org/10.3389/fimmu.2014.00427
Decreased dietary fiber intake and structural alteration of gut microbiota in patients with advanced colorectal adenomaAmerican Journal of Clinical Nutrition 97:1044–1052.https://doi.org/10.3945/ajcn.112.046607
G*power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciencesBehavior Research Methods 39:175–191.https://doi.org/10.3758/BF03193146
Molecular genetics of colorectal cancerAnnual Review of Pathology: Mechanisms of Disease 6:479–507.https://doi.org/10.1146/annurev-pathol-011110-130235
Fusobacterium nucleatum associates with stages of colorectal neoplasia development, colorectal cancer and disease outcomeEuropean Journal of Clinical Microbiology & Infectious Diseases 33:1381–1390.https://doi.org/10.1007/s10096-014-2081-3
Role of the gut microbiota in inflammatory bowel disease pathogenesis: what have we learnt in the past 10 years?World Journal of Gastroenterology 20:1192–1210.https://doi.org/10.3748/wjg.v20.i5.1192
The gut microbiome, probiotics, bile acids axis, and human healthTrends in Microbiology 22:306–308.https://doi.org/10.1016/j.tim.2014.04.010
The gastrointestinal microbiota and colorectal cancerAmerican Journal of Physiology - Gastrointestinal and Liver Physiology 308:G351–G363.https://doi.org/10.1152/ajpgi.00360.2012
[Fecal intestinal flora in patients with colon adenoma and colon cancer]Nihon Shokakibyo Gakkai Zasshi 87:771–779.
Altered tissue metabolites correlate with microbial dysbiosis in colorectal adenomasJournal of Proteome Research 13:1921–1929.https://doi.org/10.1021/pr4009783
The composition of the gut microbiota throughout life, with an emphasis on early lifeMicrobial Ecology in Health & Disease 26:26050.https://doi.org/10.3402/mehd.v26.26050
R: a language and environment for statistical computingVienna, Austria: R Foundation for Statistical Computing.
Genetic alterations during colorectal-tumor developmentNew England Journal of Medicine 319:525–532.https://doi.org/10.1056/NEJM198809013190901
Cynthia L SearsReviewing Editor; Johns Hopkins University School of Medicine, United States
In the interests of transparency, eLife includes the editorial decision letter and accompanying author responses. A lightly edited version of the letter sent to the authors after peer review is shown, indicating the most substantive concerns; minor comments are not usually included.
Thank you for submitting your work entitled "Registered report: Fusobacterium nucleatum infection is prevalent in human colorectal carcinoma" for consideration by eLife. Your article has been reviewed by four peer reviewers, and the evaluation has been overseen by a guest Reviewing Editor and Sean Morrison as the Senior Editor.
The reviewers have discussed the reviews with one another and the Reviewing Editor has drafted this decision to help you prepare a revised submission.
The project goal is to test the reproducibility of the results in the Castellarin et al., 2012 paper in which Fusobacterium was associated with CRC by qPCR detection of Fusobacterium (versus a reference gene) in DNA isolated from colon tumors compared to matched uninvolved tissue and normal tissue from age-matched controls.
After discussion among all reviewers, the key clarifications requested are to define in more detail the source of the populations to be studied. Information about how subjects and controls will be recruited, how disease will be defined (including histopathology), how samples will be obtained as well as length and type of storage of samples should be explicitly stated. How many healthy individuals will be included in the study? Are tissues from healthy controls obtained through biopsy or colorectal resection? The title page should be corrected to reflect that the project is human subjects research.
[Editors’ note: a previous version of this study was rejected after peer review, but the authors submitted for reconsideration. The previous decision letter after peer review is shown below.]
Thank you for submitting your work entitled "Registered report: Fusobacterium nucleatum infection is prevalent in human colorectal carcinoma" for consideration at eLife. Your Registered Report has been evaluated by Sean Morrison (Senior editor) and a guest Reviewing Editor with appropriate expertise, and the decision was reached after discussions between the two of us. We regret to inform you that this version of the Registered Report will not be considered further for publication in eLife.
Unfortunately the proposed replication misses at least one key issue in this area of work. There are several reports of finding excess Fusobacterium in a subset of colon cancers versus matched normal and we have two major concerns:
1) Normal tissues from cancer hosts are not normal controls for these experiments (the authors would need to identify a bank of appropriate colonoscopy control biopsies or age-match normal tissues from rapid autopsies).
2) The key result that needs to be replicated is whether Fusobacterium nucleatum versus other species induce tumorigenesis in tumor-susceptible murine models. Addressing this key aspect of Fusobacterium biology by the Reproducibility Project is essential before we can move forwards with further in-depth review. Absent this experiment, it will not be possible to conclude whether they key claims in the paper in question are reproducible.https://doi.org/10.7554/eLife.10012.002
- Reproducibility Project: Cancer Biology
The Reproducibility Project: Cancer Biology is funded by the Laura and John Arnold Foundation, provided to the Center for Open Science in collaboration with Science Exchange. The funder had no role in study design or the decision to submit the work for publication.
The Reproducibility Project: Cancer Biology core team thank Courtney Soderberg at the Center for Open Science for assistance with statistical analyses. We also thank the following companies for generously donating reagents to the Reproducibility Project: Cancer Biology; American Type Culture Collection (ATCC), Applied Biological Materials, BioLegend, Charles River Laboratories, Corning, DDC Medical, EMD Millipore, Harlan Laboratories, LI-COR Biosciences, Mirus Bio, Novus Biologicals, Sigma–Aldrich, and System Biosciences (SBI).
- Cynthia L Sears, Reviewing Editor, Johns Hopkins University School of Medicine, United States
- Received: July 13, 2015
- Accepted: November 30, 2015
- Version of Record published: February 11, 2016 (version 1)
© 2016, Repass et al
This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.
Downloads (link to download the article as PDF)
Download citations (links to download the citations from this article in formats compatible with various reference manager tools)
Open citations (links to open the citations from this article in various online reference manager services)
Investigating reproducibility in preclinical cancer research.