Registered report: Interactions between cancer stem cells and their niche govern metastatic colonization

The Reproducibility Project: Cancer Biology seeks to address growing concerns about reproducibility in scientific research by replicating selected results from a substantial number of high-profile papers in the field of cancer biology published between 2010 and 2012. This Registered report describes the proposed replication plan of key experiments from ‘Interactions between cancer stem cells and their niche govern metastatic colonization’ by Malanchi and colleagues, published in Nature in 2012 (Malanchi et al., 2012). The key experiments that will be replicated are those reported in Figures 2H, 3A, 3B, and S13. In these experiments, Malanchi and colleagues analyze messenger RNA levels of periostin (POSTN) in pulmonary fibroblasts, endothelial cells, and immune cells isolated from mice with micrometastases to determine which cell type is producing POSTN in the metastatic niche (Figure 2H; Malanchi et al., 2012). Additionally, they examine MMTV-PyMT control or POSTN null mice to test the effect of POSTN on primary tumor growth and metastasis (Figures 3A, 3B, and S13; Malanchi et al., 2012). The Reproducibility Project: Cancer Biology is a collaboration between the Center for Open Science and Science Exchange, and the results of the replications will be published in eLife. DOI: http://dx.doi.org/10.7554/eLife.06938.001


Introduction
Metastatic colonization is a highly inefficient process that only a small subset of disseminated tumor cells accomplish (Nguyen et al., 2009). A growing body of literature suggests that cancer stem cells (CSC), tumor cells with the ability to self-renew and differentiate, play important roles not only in metastatic colonization but also in establishing the metastatic niche to support metastatic colonization (Visvader and Lindeman, 2012). Using the MMTV-PyMT mouse breast cancer model, which spontaneously metastasizes to the lungs, Malanchi and colleagues reported that only the CSC population, identified as CD24 + CD90 + , were capable of initiating lung metastases and secondary metastases (Guy et al., 1992;Lin et al., 2003;Malanchi et al., 2012). Additionally, only a subset of the injected CSC population resulted in metastatic nodules. Periostin (POSTN) was identified by microarray RNA profiling studies as a stromal factor involved in maintaining the normal stem cell niche and demonstrated to be secreted by stromal fibroblasts, but not by infiltrating tumor cells (Malanchi et al., 2012). POSTN is a secreted protein that is incorporated in the extracellular matrix and has been associated with metastasis in several human cancers (Conway et al., 2014). The functional necessity of POSTN was investigated by observing the pulmonary metastatic potential in POSTN-knockout MMTV-PyMT mice, which showed a statistically significant decrease compared to controls (Malanchi et al., 2012). This was further demonstrated by observing a rescue in metastatic efficiency by injecting POSTN-deficient tumor cells into wild-type recipient mice (Malanchi et al., 2012). Investigating the mechanism of POSTN-induced metastasis, Malanchi and colleagues reported a decrease in colony *For correspondence: tim@cos.io formation in vitro with POSTN-deficient tumor cells or wild-type CSCs co-cultured with POSTNdeficient stromal cells, demonstrating the involvement of POSTN in stem cell maintenance (Malanchi et al., 2012). Furthermore, POSTN was reported to bind to Wnt ligands, leading to an increase in Wnt signaling in CSCs, a known regulator of stem cell maintenance in a variety of tissues (Malanchi et al., 2012). Taken together, these results suggest that CSCs are essential for metastatic colonization and that CSCs induce stromal fibroblasts to secrete POSTN in the metastatic niche to support tumor cell outgrowth by augmenting the Wnt signaling pathway.
Malanchi and colleagues' findings suggest that targeting of POSTN in the metastatic niche could potentially be used to treat metastasis. The key experiments included for replication were selected because they examine the induction of POSTN expression in the pulmonary stromal fibroblasts and test the role of POSTN in primary tumor formation and metastatic efficiency, which are relevant as the role of POSTN as a possible prognostic marker and target for anticancer therapies is explored (Xu et al., 2012;Nuzzo et al., 2014). Indeed, treatment of mice with POSTN specific DNA aptamers, single-stranded DNA oligonucleotides designed to bind and inhibit POSTN, was shown to decrease primary tumor growth and metastasis in a xenograft model of mammary tumorigenesis (Lee et al., 2013). Additionally, treatment of ovarian xenograft models with a neutralizing antibody to POSTN resulted in a reduction of metastatic potential and tumor growth, migration, and invasion (Zhu et al., 2011).
Figures 3A, 3B, and Supplemental Figure S13 examine the role of POSTN in primary tumor formation and metastasis utilizing a MMTV-PyMT mouse model of mammary tumor formation. This mouse model develops primary mammary tumors that spontaneously metastasize to the lung (Guy et al., 1992;Lin et al., 2003). MMTV-PyMT +/tg ; Postn −/− mice and their Postn +/+ counterparts were analyzed for primary tumor formation and metastasis. Primary tumor size was measured by weight, and the number of metastases was determined by counting lung metastatic lesions. Malanchi and colleagues reported in Figures 3A and S13A that POSTN does not affect the size of the primary tumors, but MMTV-PyMT +/tg ; Postn −/− mice develop significantly fewer metastases than their Postn +/+ counterparts (Malanchi et al., 2012). The importance of stromal POSTN was also demonstrated in another study in which orthotopic inoculation of gastric cancer cells into Rag2 −/− ; Postn −/− mice reduced tumor size, decreased invasiveness, and decreased growth compared to Rag2 −/− ; Postn +/+ mice (Kikuchi et al., 2014). Additionally, overexpression of POSTN in human mammary epithelial and breast cancer cells resulted in enhanced tumor growth and metastasis (Wang et al., 2013), which is similar to a colon cancer cell model where overexpression of POSTN resulted in an increase in the number and size of liver metastases (Bao et al., 2004). The experiment reported in Figures 3A, 3B, and Supplemental Figure S13 will be replicated in Protocol 1.
Malanchi and colleagues show that POSTN is expressed primarily in fibroblasts and to a lesser extent in endothelial cells but is not expressed in immune cells at sites of metastasis (Malanchi et al., 2012). This was determined by FACS sorting cells from lungs with macrometastases to isolate CD34 + /CD31 − pulmonary fibroblasts, CD31 + endothelial cells, and CD45 + immune cells. Quantitative PCR of messenger RNA extracted from each of these cell populations was then used to determine the relative expression of POSTN in each cell type as reported in Figure 2H (Malanchi et al., 2012). RNA in situ hybridization and immunostaining showed similar results (Malanchi et al., 2012). Similarly, another study used immunohistochemistry and immunofluorescence to show that POSTN was localized to stromal fibroblasts in human samples of advanced gastric cancer (Kikuchi et al., 2014). Yet, another study utilized tandem mass spectrometry and immunofluorescence to find POSTN concentrated in the extracellular matrix surrounding sites of neovascularization of micrometastases and within the tips of endothelial cells involved in the neovascularization (Ghajar et al., 2013). The experiment reported in Figure 2H will be replicated in Protocol 2.

Materials and methods
Protocol 1: tumor size and metastases of MMTV-PyMT +/tg ; Postn +/+ and MMTV-PyMT +/tg ; Postn −/− mice This experiment examines the requirement of POSTN in metastatic colonization using the MMTV-PyMT mouse model. Female mice carrying the MMTV-PyMT transgene that are either Postn +/+ or Postn −/− will be examined for changes in primary tumor size and the number of spontaneously formed pulmonary macrometastases, which is a replication of the experiment reported in Figures 3A, 3B, and Supplemental Figure 13. This experiment will also generate lung tissue from MMTV-PyMT +/tg ; Postn +/+ female mice that are positive with metastatic disease for use in Protocol 2.
Sampling ■ Experiment has 2 cohorts: ◯ Cohort 1: MMTV-PyMT +/tg ; Postn +/+ female mice. ◯ Cohort 2: MMTV-PyMT +/tg ; Postn −/− female mice. ■ Experiment will use the following number of mice per cohort: ◯ Cohort 1: 14 mice. ◯ Cohort 2: 15 mice. c Note: Derive mice from consecutive litters and analyze development of tumors and metastases until the two cohorts reach the indicated numbers. ■ To account for outlier data, as presented in the original publication, 5% and 10% more mice were added to each cohort, respectively, to ensure at least 13 mice survive each cohort for a minimum power of 80%. ◯ See 'Power calculations' section for details.

Materials and reagents
Procedure 1. Breed MMTV-PyMT +/tg ; Postn +/− or MMTV-PyMT +/tg ; Postn −/− male mice with Postn +/+ and Postn −/− female mice to obtain MMTV-PyMT +/tg ; Postn +/+ control and MMTV-PyMT +/tg ; Postn −/− experimental female mice, respectively. a. Do not use MMTV-PyMT female mice for breeding as they develop mammary tumors. Additionally, MMTV-PyMT +/tg males cannot be crossed with MMTV-PyMT +/tg females as the progeny will have a double dose of the oncogene and develop extremely aggressive tumors that cannot be used in this study. b. For generation of the MMTV-PyMT +/tg ; Postn −/− experimental female mice, male MMTV-PyMT +/tg ; Postn −/− male mice will be crossed with Postn −/− female mice. c. For generation of the MMTV-PyMT +/tg ; Postn +/+ control female mice, the MMTV-PyMT +/tg ; Postn +/− male mouse will be crossed to Postn +/+ female mice to obtain MMTV-PyMT +/tg ; Postn +/+ male mice that will then be crossed with Postn +/+ female mice. d. As mice obtained were found to contain agents, cross-foster rederivation of mice will be performed based on the following procedure (Artwohl et al., 2008). Donor dam will be placed with male mice 5 days a week Monday afternoon to Friday morning and plugs will be checked daily. When dam is found plugged, she should be single housed and date of plug will be recorded as a sign of pregnancy. Once the pregnancy is confirmed, a timed pregnant mouse will be ordered to use as a foster recipient. Foster recipients will ideally have a different fur coat color than the donor dam, so the identification of the fostered pups will be easier. Consequently, the donor and recipient dams will be removed from their cages and placed in separate clean cages. The litter to be fostered will gently be picked up and cleaned with alcohol and passed to a clean tech to be mixed with dirty bedding, nestlet, and other pups from the recipient dam's cage. When mixing the pups, they will be gently arranged in the palm of the hand, in contact with nestlet and bedding from the recipient dam's cage to transfer the recipient dam's scent. All pups will be placed back in the nest and the recipient dam will be returned to the cage. The cage will be monitored visually every 15 min for the first hour; if there is evidence of rejection by the dam (agitation, carrying the pups around), the pups will be removed from the cage and humanely euthanized. The cages will be visually assessed at least twice daily and will not be disturbed for the first 72 hr after fostering in order to avoid any potential cannibalism. All experimental animals will not be treated with Ivermectin or Fenbendazole, as these could change a number of immune parameters affecting tumor growth and take rate. 2. Extract genomic DNA from mouse tail snips using DNA extraction kit following manufacturer instructions. a. From manufacturer's instructions follow 'Solid Tissue' assay protocol.  i. Multiple tumors will form and grow until they reach a significant size. b. Monitor health status of mice. If mice have to be euthanized prior to reaching fully developed primary tumors exclude mice from study and record reason for euthanasia. c. Tumor disease is fully developed when primary tumors have developed in all mammary glands with an average weight around 1 g per tumor.

Reagent
i. Both cohorts of mice will develop tumors approximately within 3-4 months of age and up to 6 months. ii. Record mice with tumors that are large and form close to the neck as these may give metastasis more efficiently. d. Record age of mice when sacrificed and determine time gap between detection and fully developed tumor. i. Method of euthanasia is isoflurane overexposure (2-5% at 1 l/min) followed by cervical dislocation. 6. Dissect primary tumor and lung tissue from mice.
a. Weigh primary tumors. i. Record total weight of all primary tumors together for each mouse.
ii. Record the number of primary tumors for each mouse.
iii. Divide total weight of all primary tumors by number of primary tumors to obtain a reference primary tumor weight for each mouse. 7. Dissect lungs and blindly count the number of macrometastatic nodules on every side of all separated lobes of the lung using a stereomicroscope. a. Do not fix or stain tissues. b. Quickly count large macrometastatic nodules (≥1 mm in diameter) on all sides. 8. Immediately after counting macrometastasis, use the first six lungs identified from MMTV-PyMT +/tg ;Postn +/+ female mice that are positive with metastatic disease for further analysis (Protocol 2). a. Should be approximately 5-6 months of age. i. Exclude lungs from mice that are euthanized before this age. b. Use mice with detectable metastatic disease and record number of macrometastases in each lung used and the total weight of the primary mammary tumors.  Figure S10). ◯ Number of primary tumors formed and total weight of all tumors for each mouse. ◯ Raw numbers and box and whisker plot of weight of primary tumors (total weight divided by number of primary tumors) for each mouse. (Compare to Figure 3A and S13). ◯ Raw numbers of pulmonary macrometastases for each mouse (Compare to Figure 3B and S13). ◯ Box and whisker plot of number of pulmonary macrometastases for each mouse. (Compare to Figure 3B and S13). ■ Sample delivered for further analysis: ◯ Lungs for FACS and qRT-PCR analysis for Protocol 2.

Confirmatory analysis plan
The original paper reported outliers in each cohort of mice. As these appear to have not been included in the original analysis, we will also remove any outliers from the analysis for comparison. But the analysis will also be performed with all data values. Outliers are determined as 1.5 times the interquartile range. This replication attempt will perform the following statistical analysis listed below: ■ Statistical analysis: ◯ Primary tumor weight in MMTV-PyMT +/tg ;Postn +/+ mice relative to MMTV-PyMT +/tg ;Postn −/− mice. c Unpaired two-tailed t-test. ◯ Number of pulmonary macrometastases in MMTV-PyMT +/tg ;Postn +/+ mice relative to MMTV-PyMT +/tg ; Postn −/− mice. c Unpaired two-tailed t-test. ◯ The replication attempt will also perform negative binomial regression analysis of the macrometastases count data. ■ Meta-analysis of effect sizes: ◯ Compute the effect sizes of each comparison, compare them against the reported effect size in the original paper, and use a meta-analytic approach to combine the original and replication effects, which will be presented as a forest plot.

Known differences from the original study
The mice in the original study were from an 8 th generation backcross to the FVB line, thus it was on a mixed background, while the mice used in the replication will be from a pure 10th generation backcross to the FVB line. This may make a difference in the effect size and will be included in the discussion of the results of the replication. Additionally, mice will undergo cross-foster rederivation to attempt to remove agents currently associated with the mice. All known differences of materials and reagents are listed in the materials and reagents section above with the originally used item listed in the comments section. All differences have the same capabilities as the original and are not expected to alter the experimental design.

Provisions for quality control
Mice will undergo cross-foster rederivation to attempt to remove agents currently associated with the mice that could alter immune parameters affecting tumor growth and take rate. All data obtained from the experiment-raw data, data analysis, control data, and quality control data-will be made publicly available, either in the published manuscript or as an open access data set available on the Open Science Framework project page for this study (https://osf.io/vseix).

Protocol 2: POSTN expression in lung stroma with macrometastases
This experiment uses quantitative PCR to detect the expression level of POSTN in CD34 + /CD31 − pulmonary fibroblasts, CD31 + endothelial cells, and CD45 + immune cells isolated from lungs of mice with macrometastases, which is a replication of the experiment reported in Figure 2H.
Sampling ■ Experiment will use six lungs for a minimum power of 82%. ◯ See appendix for detailed power calculations.

Materials and reagents
a. FACS buffer: 3% FBS in PBS. 8. Resuspend up to 5 × 10 7 cells total in FACS buffer at 2 × 10 7 cells/ml in polypropylene (opaque) FACS tubes. 9. Either add antibodies directly, or add antibody dilution mixes, and incubate on ice for 30 min in the dark (if staining high amount of cells put on roller at 4˚C). a. CD45-PE·Cy5.5 (use at 1:300 dilution iii. Rat IgG2a, κ-PE. 10. Pellet cells at 180×g for 5 min, wash once with 4 ml FACS buffer, and pellet cells at 180×g for 5 min (if staining high amount of cells perform another wash). 11. Resuspend cells in 500 μl FACS buffer and filter through filter-membrane into polypropylene (opaque) FACS tubes protected from light. 12. Just before FACS add viability dye.
iii. CD45 + cells. iv. Use negative controls (unstained and isotype control antibodies) to determine gating of populations. 14. Isolate RNA from each collected cell population using TRI reagent following manufacturer's instructions. Deliverables ■ Data to be collected: ◯ All FACS plots in gating scheme (including all controls), leading to final population of viable, CD34 + /CD31 − , CD31 + , and CD45 + cells. ◯ Purity (A 260/280 and A 260/230 ratios) and concentration of isolated total RNA from cells. ◯ Raw C T qRT-PCR values and ΔΔC T (the C T value of POSTN normalized to GAPDH). ◯ Graph of POSTN normalized expression (ΔΔC T ) for each condition. (Compare to Figure 2H).

Confirmatory analysis plan
This replication attempt will perform the following statistical analysis listed below: ■ Statistical analysis: Note: At the time of analysis, we will perform the Shapiro-Wilk test and generate a quantilequantile (q-q) plot to assess the normality of the data and also perform Levene's test to assess homoscedasticity. If the data appear skewed, we will perform the appropriate transformation in order to proceed with the proposed statistical analysis. If this is not possible, we will perform the equivalent non-parametric test.
◯ Planned comparisons with the Bonferroni correction: c CD34 + /CD31 − vs CD45 + . c CD31 + vs CD45 + . ■ Meta-analysis of effect sizes: ◯ Compute the effect sizes of each comparison, compare them against the reported effect size in the original paper and use a meta-analytic approach to combine the original and replication effects, which will be presented as a forest plot.
Known differences from the original study All known differences are listed in the materials and reagents section above with the originally used item listed in the comments section. All differences have the same capabilities as the original and are not expected to alter the experimental design.

Provisions for quality control
Negative staining and isotype controls are included to assess antibody staining relative to background during FACS analysis. The sample purity (A 260/280 and A 260/230 ratios) of the isolated RNA from each sample will be reported. All data obtained from the experiment-raw data, data analysis, control data, and quality control data-will be made publicly available, either in the published manuscript or as an open access data set available on the Open Science Framework project page for this study (https://osf.io/vseix). Summary of original data (estimated from Figure S13).

Size of primary tumor
Test family ■ 2-tailed t-test, difference between two independent means, alpha error = 0.05.

Number of metastases
Test family ■ 2-tailed t-test, difference between two independent means, alpha error = 0.05.
Power Calculations performed with G*Power software, version 3.1.7 (Faul et al., 2007). Power Calculations performed with R software, version 3.1.2 (R Core Development Team, 2014).

Protocol 2
Summary of original data (estimated from Figure 2H).
Power calculations performed with G*Power software, version 3.1.7 (Faul et al., 2007). ANOVA F test statistic and partial η 2 performed with R software, version 3.1.2 (R Core Development Team, 2014).

Acknowledgements
The Reproducibility Project: Cancer Biology core team would like to thank the original authors, in particular Joerg Huelsken and Ilaria Malanchi, for generously sharing critical information as well as reagents to ensure the fidelity and quality of this replication attempt. We thank Courtney Soderberg at the Center for Open Science for assistance with statistical analyses. We would also like to thank the following companies for generously donating reagents to the Reproducibility Project: Cancer Biology;

Groups
Number of simulations A priori power Sample size

Number of metastases in MMTV-PyMT;
Postn +/+ mice and MMTV-PyMT; Postn −/− mice 10,000* 81.2% 12 per group *The original data were randomly sampled from, with replacement, to create simulated data sets. For a given n (the number of observations) 10,000 simulations were run and the Chi-square goodness of fit test and regression coefficient (Genotype (Postn −/− )) was calculated for each simulated data set. Any model fit with p < 0.05 was excluded. The power was then calculated by counting the number of times p ≤ 0.05 and dividing by the number of model fits. *6 tumors will be used per group based on the CD31 + to CD45 + comparison making the power 99.9%.