We sought to independently replicate the impact iRGD has on the penetrance of DOX, in an unconjugated form, in tumor and organ tissues of mice bearing orthotopic 22Rv1 human prostate tumors. This experiment is similar to what was reported in Figure 2B of Sugahara et al. (2010). This experiment utilized a 10 mg/kg dose of DOX and analyzed DOX accumulation after a brief exposure (1 hr). Mice harboring tumors were injected with phosphate buffered saline (PBS), 10 mg/kg DOX and PBS, or a combination of 10 mg/kg DOX and 4 µmol/kg iRGD. Tissues from mice were examined for DOX accumulation using matched tissues from mice injected with PBS as the blank reference samples (Figure 1, Figure 1—figure supplement 1). DOX accumulation in tumors from mice treated with DOX and iRGD was 0.86 times [n=4, SD=0.68] the amount of DOX in tumors from mice treated with DOX and PBS [n=4, M=1.0, SD=0.45]. The comparison of these two groups, which was planned a priori, was not statistically significant (Two-tailed Student’s t-test; t(6) = 0.352, p=0.737, a priori significance threshold = 0.05). This differed from the ~7.15 times increase in DOX accumulation in tumors from mice treated with DOX and iRGD compared to DOX alone reported in Sugahara et al. (2010).

Figure 1 with 1 supplement see all

Download asset Open asset

To test if iRGD enhances the effect of DOX on tumor growth, mice bearing orthotopic 22Rv1 tumors implanted 2 weeks earlier received intravenous injections every other day of either PBS, 1 mg/kg DOX and PBS (DOX + PBS), or 1 mg/kg DOX combined with 4 µmol/kg of iRGD (DOX + iRGD). This experiment is similar to what was reported in Figure 2C of Sugahara et al. (2010). While the original study included two doses of DOX (1 mg/kg or 3 mg/kg), this replication attempt was restricted to the lower dose (1 mg/kg), which in the original study showed the largest effect when DOX + PBS and DOX + iRGD were compared. The tumors were harvested and weighed 24 days after the start of treatment. PBS treated mice achieved an average tumor weight of 1.20 grams [n=7, SD=0.87], which decreased to an average tumor weight of 0.721 grams [n=7, SD=0.239] in mice treated with DOX + PBS (Figure 2). This is similar to the tumor weights reported in Sugahara et al. (2010), which were ~1.2 grams and ~0.82 grams for PBS and DOX + PBS, respectively. However, while the original study reported a final average tumor weight of ~0.35 grams from mice treated with DOX + iRGD, this replication attempt observed a final average tumor weight of 0.668 grams [n=7, SD=0.576]. Analysis of tumor weights did not detect any statistically significant differences (One-way ANOVA F(2, 18) = 1.58, p=0.233). The pairwise comparison of tumor weights from mice treated with DOX + PBS compared to DOX + iRGD was also not statistically significant (Two-tailed Welch’s t-test; t(8.01) = 0.227, p=0.826, a priori significance threshold = 0.05).

Figure 2

Download asset Open asset

During the course of the treatment, mouse body weights were monitored to evaluate whether DOX given alone or combined with iRGD negatively impacted the health of the animals. This is similar to what was reported in Supplementary Figure 9A of Sugahara et al. (2010), except only the 1 mg/kg dose of DOX conditions were tested. Mice treated with PBS, DOX + PBS, or DOX + iRGD were measured every 4 days throughout the 24 days of treatment (Figure 3, Figure 3—figure supplement 1). There was no shift in body weight during the course of the treatments. This is similar to the original study, which only saw a shift in body weight when a higher dose (3 mg/kg) of DOX was administered. Analysis on percent body weight shift of all groups on day 24 did not detect any statistically significant differences (One-way ANOVA F(2, 18) = 1.666, p=0.217), a priori significance threshold = 0.05). While we conducted this analysis because it was prespecified in the Registered Report (Kandela et al., 2015) and provides a direct comparison to the original analysis, there are additional approaches that could be taken to explore these data. An additional exploratory analysis, as described in the Registered Report (Kandela et al., 2015), was performed by calculating the area under the curve (AUC) of body weight for each mouse over the entire treatment period. Similar to the Day 24 end-point analysis, the analysis on AUC of all groups did not detect any statistically significant differences (One-way ANOVA F(2, 18) = 1.072, p=0.363).

Figure 3 with 1 supplement see all

Download asset Open asset

To further assess the effect of iRGD on DOX efficacy and toxicity, terminal deoxynucleotidyl transferase-mediated deoxyuridine triphosphate nick end labeling (TUNEL) staining, an indicator of cellular apoptosis, was performed (Figure 4, Figure 4—figure supplement 1). This experiment is similar to what was reported in Figure 2D of Sugahara et al. (2010) except that this replication attempt was restricted to the 1 mg/kg DOX conditions. Tumors, to evaluate efficacy, and heart tissues, to evaluate cardiotoxicity a known effect of DOX (Arola et al., 2000), were excised after the mice received intravenous injections of PBS, DOX + PBS, or DOX + iRGD every other day over a 24 day period. Negative and positive controls for TUNEL staining were performed in parallel to the study tissues confirming the specificity of the staining (Figure 4—figure supplement 1B,C). While TUNEL staining was detectable in tumors (Figure 4B–D), no detectable TUNEL staining in heart tissues was observed (Figure 4E–G). This is in contrast to the original study, which reported TUNEL positive scores in heart tissue for these conditions, albeit much smaller than the reported scores in heart tissues from the 3 mg/kg DOX conditions. Scoring of TUNEL positive cells in tumors from mice treated with DOX + PBS were 0.79 times [n=6, SD=0.35] the amount of DOX in tumors from mice treated with PBS alone [n=6, M=1.0, SD=0.37] (Figure 4A). Similarly, tumors from mice treated with DOX + iRGD were 0.71 times [n=6, SD=0.20] the amount of DOX in tumors from PBS treated mice. This is in comparison to the original study, which reported ~1.4 times increase for DOX + PBS treated mice and an ~2.6 times increase in TUNEL positive cells for DOX + iRGD relative to untreated mice. To evaluate if there were any differences in TUNEL staining in tumors among the conditions, we performed an ANOVA, which was not statistically significant (One-way ANOVA F(2, 15) = 1.378, p=0.282). The planned pairwise comparison of TUNEL staining in tumors from mice treated with DOX + PBS compared to DOX + iRGD was also not statistically significant (t(15) = 0.435, p=0.670, a priori significance threshold = 0.05). Other studies that utilized a similarly low dose of doxorubicin to reduce side effects, also reported small reductions in tumor weight compared to vehicle control with minimal and not statistically significant changes in TUNEL staining (Hossain et al., 2012; Sugahara et al., 2015; Wang et al., 2010).

Figure 4 with 1 supplement see all

Download asset Open asset

We performed a meta-analysis using a random-effects model to combine each of the effects described above as pre-specified in the confirmatory analysis plan (Kandela et al., 2015). To provide a standardized measure of the effect, a common effect size was calculated for each effect from the original and replication studies. Glass' $\Delta$ is the standardized difference between two means using the standard deviation of only the control group. It is used in this case because of the unequal variance between the control and treatment conditions in the original and replication studies. The effect size r is a standardized measure of the strength and direction of the association between two variables. In this case, the treatment condition and the dependent variable (i.e. body weight). Cohen’s d is the standardized difference between two means using the pooled sample standard deviation.

The comparison of DOX accumulation in tumors from 10 mg/kg DOX and PBS treated mice compared to 10 mg/kg DOX and 4 µmol/kg iRGD treated mice resulted in Glass' ${\displaystyle \mathrm{\Delta }}$ = 7.41, 95% CI [0.89, 14.38] for the data estimated a priori from Sugahara et al. (2010), Figure 2B. This compares to Glass' ${\displaystyle \mathrm{\Delta }}$ = −0.32, 95% CI [−1.70, 1.11] reported in this study. In both calculations the standard deviation of the DOX + PBS control group was used because of unequal variance in the original study. A meta-analysis (Figure 5A) of these two effects resulted in Glass' ${\displaystyle \mathrm{\Delta }}$ = 4.44, 95% CI [−2.92, 11.81], p=0.237. The replication and the original results are in opposite directions when considering the effect, and the point estimate of the replication effect size was not within the confidence interval of the original result, or vice versa. Also, the random effects meta-analysis did not result in a statistically significant effect. Further, the Cochran's Q test for heterogeneity was statistically significant (p=0.0454), which along with a large confidence interval around the weighted average effect size from the meta-analysis suggests heterogeneity between the original and replication studies.

Figure 5

Download asset Open asset

The comparison of prostate tumor weights from 1 mg/kg DOX and PBS treated mice vs. 1 mg/kg and 4 µmol/kg iRGD treated mice resulted in Glass’ ${\displaystyle \mathrm{\Delta }}$ = 1.61 with a 95% CI [0.44, 2.73] for the data estimated a priori from Sugahara et al. (2010), Figure 2C. This compares to Glass' ${\displaystyle \mathrm{\Delta }}$ = 0.22 with a 95% CI [−0.84, 1.27] reported in this study. In both calculations the standard deviation of the DOX + PBS control group was used because of unequal variance in the original and replication studies. A meta-analysis (Figure 5B) of these two effects resulted in Glass' ${\displaystyle \mathrm{\Delta }}$ = 0.90 with a 95% CI [−0.46, 2.26], p=0.195. Both results are consistent when considering the direction of the effect, however the point estimate of the replication effect size was not within the confidence interval of the original result, or vice versa. The random effects meta-analysis did not result in a statistically significant effect.

The comparison of body weight shift at day 24 from mice treated with PBS, DOX + PBS, or DOX + iRGD resulted in r = 0.06, 95% CI [−0.31, 0.41] for the data estimated a priori from Sugahara et al. (2010), Supplemental Figure 9A. This compares to r = 0.40, 95% CI [−0.04, 0.71] reported in this study. A meta-analysis (Figure 5C) of these two effects resulted in r = 0.21 95% CI [−0.13, 0.51], p=0.229. Neither the original nor the replication results were statistically significant. The effect seen in the original and the replication were in the same direction and the point estimate of the original effect size was within the confidence interval of the replication result, and the point estimate of the replication effect size was within the confidence interval of the original result. The random effects meta-analysis did not result in a statistically significant effect suggesting body weight does not change based on treatment in either the original or replication studies.

The comparison of cellular apoptosis, as determined by a TUNEL assay, from 1 mg/kg DOX and PBS treated mice compared to 1 mg/kg and 4 µmol/kg iRGD treated mice resulted in Cohen's d = 2.30, 95% CI [1.13, 3.43] for the data estimation a priori from Sugahara et al. (2010), Figure 2D. This compares to d = 0.25, 95% CI [−0.89, 1.38] reported in this study. A meta-analysis (Figure 5D) of these two effects resulted in d = 1.28, 95% CI [−0.73, 3.29], p=0.211. Both results are consistent when considering the direction of the effect, however the point estimate of the replication effect size was not within the confidence interval of the original result, and vice versa. The random effects meta-analysis did not result in a statistically significant effect. Further, the Cochran's Q test for heterogeneity was statistically significant (p=0.0212), which along with a large confidence interval around the weighted average effect size from the meta-analysis suggests heterogeneity between the original and replication studies.

This direct replication provides an opportunity to understand the present evidence of these effects. Any known differences, including reagents and protocol differences, were identified prior to conducting the experimental work and described in the Registered Report (Kandela et al., 2015). However, this is limited to what was obtainable from the original paper, which means there might be particular features of the original experimental protocol that could be critical, but unidentified. So while some aspects, such as number of cells injected, strain of mice, and drug treatment schedule were maintained, others were unknown or not easily controlled for. These include variables such as the microbiome of recipient mice (Macpherson and McCoy, 2015), housing temperature in mouse facilities (Kokolus et al., 2013), cell line drift (Hughes et al., 2007), circadian biological responses to therapy (Fu and Kettner, 2013), and differing compound potency resulting from different stock solutions or differences in peptide synthesis (Kannt and Wieland, 2016). Whether these or other factors influence the outcomes of this study is open to hypothesizing and further investigation, which is facilitated by direct replications and transparent reporting.

iRGD: H-Cys-Arg-Gly-Asp-Lys-Gly-Pro-Asp-Cys-NH2 disulfide bridge: C1-C9 The iRGD peptide was chemically synthesized using Fmoc (9-fluorenylmethoxy carbonyl) chemistry by LifeTein, LLC (Somerset, New Jersey). The peptide chains were synthesized from the carboxyl terminus to the Cys amino terminus onto H-Cys(Trt)-2Cl resin. This H-Cys(Trt)-2Cl resin was incubated with dichloromethane (DCM) for 30 min and then washed with dimethylformamide (DMF) three times. Fmoc-protecting groups at the amino terminus were deprotected with an alkaline buffer and then washed with DMF three times to remove the deprotection buffer. The second amino acid was Fmoc-Asp(Otbu)-OH coupled to the first amino acid and then DMF cleaned. After each coupling, the peptide was ninhydrin tested and the coupling and washing steps repeated until the crude peptide was fully synthesized. The full synthesized crude peptide was cleaved from the resin and cleaned with Trifluoroacetic acid (TFA). The crude peptides were diethyl ether precipitated, drained and washed. The rest of the Fmoc group was removed. The peptides were isolated and purified by high-performance liquid chromatography (HPLC) (Shimadzu LC-20A) (data available at https://osf.io/m58sv/). Fractions of greater than 95% purity were used for the investigation. The purity and molecular weight of the respective peptides were confirmed by matrix-assisted laser desorption ionization (MALDI)-time of flight (TOF) mass spectrometry (Shimadzu SEC(GPC)-MALDI-TOF-MS) (data available at https://osf.io/mjxkv/). Detailed synthesis protocols available at (https://osf.io/k9zu3/).

22Rv1 prostate cells (ATCC, CRL-2505) were maintained in DMEM with 10% fetal bovine serum and penicillin/streptomycin at 37°C/5% CO₂. Quality control data for the 22Rv1 cell lines are available at (https://osf.io/en6ru/). This includes results confirming the cell lines were free of Mycoplasma contamination and common mouse pathogens. Additionally, STR DNA profiling of the cell lines was performed and all cells were confirmed to be the indicated cell lines when queried against STR profile databases.

All animal procedures were approved by the Northwestern University IACUC# IS00000556 and were in accordance with the Northwestern University’s policies on the care, welfare, and treatment of laboratory animals. Blinding occurred during TUNEL analysis. For all experiments five-week old male Athymic Nude mice (Harlan Laboratories, HSD: Athymic Nude-Foxn1nu, Order code 069(nu)/070(nu/+)) were used with the exception of 8 week old male C57BL/6J mice (Jackson Laboratory, RRID:IMSR_JAX:000664) that were used as a positive controls in the TUNEL assay. Athymic Nude mice were inoculated with 22rv1 prostate cancer cells at a density of 1x10⁶ cells in 10 µl of Dulbecco’s Phosphate Buffered Saline (PBS) (Sigma-Aldrich, cat # D8537) into the ventral prostate glands of mice. Orthotopic tumors were allowed to grow for two weeks before mice were randomized and assigned to experiments. Each cage contained up to five mice and offered Certified Rodent Diet (Harlan Teklad, cat # 7912) and water ad libitum. The animal room was set to maintain between 68–75°F, a relative humidity of 30–70%, a minimum of 15 room air changes per hour, and a 12 hr light/dark cycle, which was interrupted for study-related activities.

Weighed compounds were dissolved in PBS. Doxorubicin-HCl (Sigma-Aldrich, cat # D1515) was prepared at a final concentration of 5.335 mg/mL for dose administration of 10 mg/kg in tumor and organ DOX penetrance analysis or 0.533 mg/mL for dose administration of 1 mg/kg in multiday experimentation. The iRGD Peptide was prepared at a concentration of 1.896 mg/mL for dose administration of 4 µmol/kg for all experiments.

Mice were intravenously injected (IV) by tail vein, based on body weight on injection day. For tumor and organ penetrance analysis, Figure 1, mice were anesthetized, perfused, and sacrificed 1 hr after drug administration. For multi-day experiments mice were injected by IV every other day for 24 days based on body weight on injection day. On the last day, mice were anesthetized, perfused, and sacrificed 1 hr after drug administration. Further details of these methods are available at (https://osf.io/bkhnp/).

Animals were checked twice after drug administration (AM and PM) for mortality, abnormalities, and signs of pain or distress. Detailed observation was conducted two times after injection and 24 hr post dosing.

For tumor and organ DOX penetrance analysis, body weight was recorded once on injection day by balance (Ohaus Corp. USA, model # Scout Pro SP202). For multiday experiments, body weight was recorded every four days, for 24 days, with additional reads taken on injection days to properly administer compound dose. The treatment group was not blinded to the scientist. Weights taken for each mouse are available at (https://osf.io/y82jp/).

Procedure was performed as outlined in the Registered Report (Kandela et al., 2015). Briefly, 1 hr after the last dose administration animals were anesthetized with isoflurane, placed on a heating pad, and perfused through the heart with DMEM + 1% BSA using an 18G x 3/4in needle (Terumo Winged Infusion Set# SV18BLK). Only animals that organs were pale after perfusion were taken for further analysis.

Organs (Prostate Tumor, Liver, Spleen, Pancreas, Heart, Lung, Kidneys, and Brain) were collected 1 hr after treatment according to the perfusion protocol, as described in the Registered Report (Kandela et al., 2015), and weight recorded on scale (Mettler Toledo model # XL-300). 100 mg of each organ was homogenized separately in 0.5 mL solution of 1% sodium dodecyl sulfate and 1mM H2SO4 in water (pH 7.4) using mortar and pestle. A total of 2 mL of chloroform: isopropyl alcohol (1:1, v/v) was added in stepwise manner, 0.5 mL each time for 4 times. Next, the ground tissue was pipetted several times in order to collect the entire sample from the mortar and placed in a 5 mL Eppendorf tube. A freeze thaw protocol was next used. Samples were placed on dry ice for 1 min followed by a 5 min thaw in a 25°C water bath. Samples were spun at 14,000xg for 15 min at RT and the organic phase was collected and stored at 4°C. Absorbance was read on a UV Vis Spectrophotometer at OD₄₉₀ (Perkin Elmer Lambda 650 UV/VIS Spectrometer #L6020031). The control untreated samples were matched appropriately (e.g. liver for liver and tumor for tumor) to create the blank reference for each tissue and the samples were read against these respective blanks and OD₄₉₀ values were recorded.

After 24 days, the last dose was administered and 1 hr later all surviving animals were sacrificed according to the perfusion protocol, as described in the Registered Report (Kandela et al., 2015). Hearts and prostate tumors were collected and whole organ weights (g) were recorded on scale. Tissues were placed in 4% Paraformaldehyde overnight at 4°C and cut in half. One half was submitted for sectioning to the Mouse Histology Phenotyping Laboratory, Northwestern University (MHPL).

Half of each heart and prostate tumor sample, in 4% Paraformaldehyde was paraffin embedded, and sectioned into 7 slices per sample with a 50-micron gap. TUNEL staining was performed on 5 slides and 1 slide was treated as negative control. The positive control was dexamethasone (DEX)-treated C57BL/6J mouse thymus. An 8 week old male C57BL/6J mouse (Jackson Laboratory, RRID: IMSR_JAX:000664) was given an intraperitoneal injection of dexamethasone (0.1 mg/g body weight). After 6 hr the animal was sacrificed and the thymus was removed. The thymus was fixed in 10% buffered neutral formalin solution, dehydrated, and embedded in paraffin.

TUNEL (EMD Millipore, cat # S7100) stained slides were evaluated by Dr. Gennadiy Bondarenko and Dr. Andrey Ugolkov. Images were captured using a Carl Zeiss Axial Lab A1 microscope and a 40x objective, by Dr. Ugolkov, or a Olympus BX45 microscope and a 40x objective, by Dr. Bondarenko (images available at: https://osf.io/3fs27/). Drs. Bondarenko and Ugolkov were blinded to group allocation, only receiving the animal ID with H or P designation to indicate heart or prostate tumor. The frequency of apoptosis was calculated as an apoptotic index, in which the proportion of cells undergoing apoptosis was expressed as a percentage of all cells observed. The apoptotic index of each tissue sample was calculated as the number of TUNEL-positive cells and bodies per 500 cells/microscopic view or 2500 cells/slide (5 slides/tissue), counted in five randomly selected microscopic fields in each tissue sample. Percent apoptotic index was calculated with the following formulation: (i/500) X100%. i = cell undergoing apoptosis. An average was taken of the apoptotic index from all five fields of all five slices (25 fields total) which is considered one biological replicate. Negative and positive control sections were stained in parallel to the tumor and heart samples and are available at (https://osf.io/gmcyt/). Original counts are available at (https://osf.io/pbg7x/).

Statistical analysis was performed with R software (RRID: SCR_001905), version 3.2.3 (R Core Team, 2016). All data csv files and analysis scripts are available at (https://osf.io/xu1g2/). Confirmatory statistical analysis was pre-registered (https://osf.io/9hr2d/) before the experimental work began as outlined in the Registered Report (Kandela et al., 2015). Additional exploratory analysis (area under the curve) was performed using the weights of the mice over the treatment period. Data were checked to ensure assumptions of statistical tests were met. A meta-analysis of a common original and replication effect size was performed using a random effects model and the metafor R package (Viechtbauer, 2010). (available at https://osf.io/ymxaz/). The original study data were extracted a priori from the published figures by determining the mean and upper/lower error values for each data point. The extracted data were published in the Registered Report (Kandela et al., 2015) and were used in the power calculations to determine the sample sizes for this study. In the meta-analyses where Glass' Δ was used, because of unequal variance between the two conditions being compared, the standard deviation of DOX + PBS was used in the calculations.

The source of the TUNEL kit is different than what is listed in the Registered Report, with the used source and catalog number listed above. (note: the original source was not specified). Additional materials and instrumentation not listed in the Registered Report, but needed during experimentation are also listed.

The Registered Report indicated the TUNEL sections would be analyzed with a Scanscope scanner and ImageJ software, while this replication attempt performed the analysis by blinded counting of TUNEL-positive cells and bodies in random fields as described above. All images are available at https://osf.io/3fs27/.

The funder had no role in study design, data collection and interpretation, or the decision to submit the work for publication.

The Reproducibility Project: Cancer Biology would like to thank the following companies for generously donating reagents to the Reproducibility Project: Cancer Biology; American Type and Tissue Collection (ATCC), Applied Biological Materials, BioLegend, Charles River Laboratories, Corning Incorporated, DDC Medical, EMD Millipore, Harlan Laboratories, LI-COR Biosciences, Mirus Bio, Novus Biologicals, Sigma-Aldrich, Mouse Pathology Core of Northwestern University, and System Biosciences (SBI).

Animal experimentation: All animal procedures were approved by the Northwestern University IACUC# IS00000556 and were in accordance with the Northwestern University's policies on the care, welfare, and treatment of laboratory animals.

© 2017, Mantis et al.

This article is distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use and redistribution provided that the original author and source are credited.