The study was supported by grants from the Medical Science &#38; Technology Plan Project of Zhejiang Province (grant numbers 2013KYA212), the general program of Zhejiang Province Natural Science Foundation (grant number Y19H160126), and the key program of the Jinhua Municipal Science &#38; Technology Bureau (grant number 2016-3-005, 2018-3-001d, and 2019-3-013).

The Surveillance, Epidemiology, and End Results (SEER) database is an open-access cancer database that only contains deidentified patient data (SEER ID: 12296-Nov2018). Therefore, this study was exempted from the approval of the review board of the Affiliated Jinhua Hospital, Zhejiang University School of Medicine.
1. Data preparation and R packages preparation
<ol>
	<li>Prepare and import the data. 
	&#62; Dataset &#60;- read.csv(&#34;&#8230;/Breast cancer Data.xlsx&#34;) #Import data. 
	NOTE: The data are uploaded in Supplementary File 1.</li>
	<li>Install and load the R packages. 
	&#62; packages &#60;- c(&#34;rms&#34;,&#34;cmprsk&#34;,&#34;mstate&#34;,&#34;survival&#34;,&#34;riskRegression&#34;,&#34; 
	prodlim&#34;) 
	&#62; req.pcg &#60;- function(pcg){ 
	new &#60;- pcg[!(pcg %in% installed.packages()[, &#34;Package&#34;])] 
	if (length(new)) install.packages(new, dependencies = T) 
	sapply(pcg, require, ch = T) 
	&#160;} 
	&#62; req.pcg(packages) 
	&#8203;NOTE: Perform the following procedures based on R software (version 3.6.2) using the packages rms, cmprsk, mstate, survival, riskRegression, and prodlim (http://www.r-projectrg/).</li>
</ol>
2. Establish competing risk nomograms in two distinct methods
<ol>
	<li>Establish the competing risk nomogram in a direct method. 
	&#62; mod_cph &#60;- cph(Surv(Survivalmonths, status) ~ factor1+ factor2+&#8230;, 
	&#160;x=T, y=T, surv=T, data=Dataset) 
	&#62; nom &#60;- nomogram(mod_cph, fun=list(function(x) 1-surv_cph(36, x)&#8230;), 
	&#160;funlabel=c(&#34;3-year event1 Prob.&#34;&#8230;), lp=F) #Take the 36th month as an example. 
	&#62; mod_crr &#60;- crr(Survivalmonths, fstatus, failcode=1, cov1=cov) 
	&#62; score &#60;- log(log((1-real.3y),(1-cif.min36)))/(maxbeta/100) 
	&#62; plot(nom)</li>
	<li>Establish the competing risk nomogram in a weighted method. 
	&#62; df.w &#60;- crprep(&#34;Survivalmonths&#34;,&#34; fstatus&#34;, 
	&#160;data=Dataset, trans=c(1,2), cens=0, 
	&#160;keep=c(&#34;factor1&#34;,&#34; factor2&#34;&#8230;)) 
	&#62; mod.w &#60;- cph(Surv(Tstart, Tstop, status==1) ~ factor1+factor2+&#8230;, 
	&#160;data=df.w, weight=weight.cens, subset=failcode==1, surv=T) 
	&#8203;&#62; nom.w &#60;- nomogram(mod.w&#8230;)</li>
</ol>
3. Discrimination ability of the competing risk nomogram
<ol>
	<li>C-index for discrimination
	<ol>
		<li>Fit the matrix cov into the competing risk model mod_crr. and get a predicted matrix suv. 
		&#62; suv &#60;- predict.crr(mod_crr, cov)</li>
		<li>Get the cumulative incidences in a certain month from suv and calculate the C-index with the function rcorr.cens. 
		&#62; cif36 &#60;- suv[which(suv[,1]==36),][-1] 
		&#62; rcorr &#60;- rcorr.cens(1-cif36,Surv(Dataset$Survivalmonths,Dataset$tumordeath)) 
		&#62; cindex &#60;- rcorr[1]</li>
	</ol>
	</li>
	<li>AUC for discrimination
	<ol>
		<li>Score the predictive performance of the competing risk model using the function Score (riskRegression package). 
		&#62; fgr.w &#60;- FGR(Hist(Survivalmonths, fstatus) ~ factor1+ factor2+&#8230;, data=Dataset, cause=1) 
		&#62; score &#60;- Score(list(&#34;Fine-Gray&#34; = fgr.w),</li>
		<li>Extract the AUC from the &#34;score&#34;. 
		&#8203;&#62; score$AUC</li>
	</ol>
	</li>
</ol>
4. Calibration ability of competing risk models
<ol>
	<li>Calibration curves with a 95% confidence interval of the competing risk model
	<ol>
		<li>Get a data frame with the cumulative incidences of each individual in a certain failure time. 
		&#62; cif36 &#60;- data.frame(cif36) #Take the 36th month as an example. 
		&#62; colnames(cif36.36_o)&#60;-c(&#34;36m&#34;)</li>
		<li>Divide the cohort according to the estimated cumulative incidences into five subgroups and calculate the average predicted cumulative incidences of each subgroup. 
		&#62; group36 &#60;- cut(cif36$`36m`, 
		quantile(cif36$`36m`, seq(0, 1, 0.2)), 
		include.lowest = TRUE, labels = 1:5) 
		&#62; mean36 &#60;- as.vector(by(cif36 $`36m`, group36, mean))</li>
		<li>Calculate the observed cumulative incidences, that is, the actual cumulative incidences, using the function cuminc, and then get the observed cumulative incidences with a 95% confidence interval in a certain failure time. 
		&#62; cum36 &#60;- cuminc(Dataset$Survivalmonths,Dataset$fstatus,group36) 
		&#62; obs36 &#60;- timepoints(cum36,Dataset$Survivalmonths)$est[c(1:5),36] 
		&#62; obs36var &#60;- timepoints(cum36,Dataset$Survivalmonths)$var[c(1:5),36] 
		&#62; df &#60;- data.frame(mean36, obs36, obs36var)</li>
		<li>Plot the calibration curve with the predicted cumulative incidences as the x-axis and the observed cumulative incidences as the y-axis using the function ggplot. 
		&#62; ggplot(df)+ geom_point(aes(x=mean36,y=obs36),col=&#34;red&#34;)+ 
		&#160;geom_point(aes(x=mean36,y=obs36),col=&#34;red&#34;,pch=4)+ 
		&#160;geom_line(col=&#34;red&#34;,aes(x=mean36,y=obs36))+ 
		&#160;geom_errorbar(col=&#34;red&#34;,aes(x=mean36,y=obs36+1.96 
		*sqrt(obs36var)), 
		&#160;ymin =obs36-1.96*sqrt(obs36var), ymax = obs36+1.96 
		*sqrt(obs36var)) 
		geom_abline(lty=3,lwd=2,col=c(rgb(0,118,192, 
		maxColorValue=255)))</li>
	</ol>
	</li>
	<li>Calibration curve with risk scores of the competing risk model
	<ol>
		<li>Valuate each level of all the variables and obtain the total RS. 
		&#62; Dataset$factor1[Dataset$factor1==1] &#60;- factor1.scale[&#34;Factor1_level1&#34;] 
		&#62; &#8230; #For example, Dataset$histology[Dataset$histology==1]&#60;-histology.scale[&#34;Histology1&#34;] 
		&#62; Dataset$rs &#60;- Dataset$factor1+Dataset$factor2+Dataset$factor3+&#8230; 
		NOTE: Obtain the total RS for each patient by summing the points of each variable.</li>
		<li>Count the frequencies and calculate the observed cumulative incidences of the different total risk scores. 
		&#62; rs.freq &#60;- as.data.frame(table(Dataset$rs)) 
		&#62; obs.36 &#60;- vector(mode=&#34;numeric&#34;, length=nrow(rs.freq)) 
		&#62; for (i in 1: nrow(rs.freq)) { 
		&#160;dataset &#60;- subset(Dataset,Dataset$rs== rs.freq [i,1]) 
		&#160;cif.dataset &#60;- cuminc(dataset$Survivalmonths,dataset$death3) 
		&#160;cif36.dataset &#60;- timepoints(cif.dataset,36) 
		&#160;obs.36[i] &#60;- cif36.dataset$est[1]}</li>
		<li>Set the range of the x-axis and calculate the predicted cumulative incidences of the total risk scores. 
		&#62; RS &#60;- range(nom$total.points) 
		&#62; x.36 &#60;- seq(min(RS),max(RS),0.01) 
		&#62; pre.36 &#60;- 1-(1-cif.min36)^exp(x.36*maxbeta/100)</li>
		<li>Plot the calibration curve with risk scores. 
		&#62; plot(x.36, pre.36, type=&#39;l&#39;&#8230;) 
		&#62; par(new=TRUE) 
		&#8203;&#62; plot(as.vector(rs.freq[,1]), obs.36&#8230; )</li>
	</ol>
	</li>
</ol>
5. Decision curve analysis of competing risk models
<ol>
	<li>Source the stdca function to perform the decision curve analysis. 
	&#62; source(&#34;stdca.R&#34;)</li>
	<li>Extract the polynomial equations from the nomogram to calculate the survival probability. 
	&#62; nomogramEx(nomo = nom) 
	&#62; Dataset$predictors &#60;- A * (Dataset$rs ^3) + B * (Dataset$rs ^2) + C * Dataset$rs + D #predictors are predicted probabilities of cancer-specific death calculated by the established nomogram</li>
	<li>Perform the decision curve analysis. 
	&#62; stdca(data = Dataset, outcome = &#34;status&#34;, ttoutcome = &#34;Survivalmonths&#34;, timepoint = 36, 
	&#160;predictors = &#34;predictors&#34;, cmprsk = TRUE, smooth = FALSE, probability = FALSE) 
	NOTE: For evaluating an outcome in the presence of a competing risk, TRUE should be chosen for cmprsk.</li>
</ol>
6. Internal validation using the bootstrap method
<ol>
	<li>Get the average predicted cumulative incidences using the bootstrap method.
	<ol>
		<li>Resample the original dataset (Dataset) with replace to generate the bootstrap dataset (Dataset_in). Establish a competing risk model (mod.in_crr) with the bootstrap dataset. Use the function predict.crr to predict mod.in_crr and loop b times to generate suvall.in. 
		B=b 
		suvall.in &#60;- list() 
		for(j in 1:B){ 
		&#160;Dataset_in &#60;- Dataset[sample(c(1:nrow(Dataset)),nrow(Dataset), 
		replace = TRUE),] 
		attach(Dataset_ in) 
		&#160;cov. in &#60;- model.matrix(~factor1+ factor2+&#8230;)[,-1] 
		&#160;mod. in _crr &#60;- crr(Survivalmonths, fstatus, failcode=1, cov1=cov.in) 
		detach(Dataset. inner) 
		suv. in &#60;- predict.crr(mod. in _crr, cov) 
		suvall.in[[j]] &#60;- suv.in}</li>
		<li>Get the average predicted cumulative incidences in a certain month. 
		cif36all. inner &#60;- vector(mode=&#34;numeric&#34;, length=nrow(Dataset)) 
		for (k in 1:B) { 
		&#160;cif36all. inner&#60;- cif36all. inner+ suvall. inner[[k]][which(suvall. inner[[k]][,1]==36),][-1] 
		} 
		cif36.in &#60;- cif36all.in/B</li>
	</ol>
	</li>
	<li>Calculate the C-index using internal cross-validation with the function rcorr.cens. 
	rcorr. inner &#60;- rcorr.cens(1-cif36.in,Surv(Dataset$Survivalmonths,Dataset$tumordeath)) 
	cindex. inner &#60;- rcorr. inner[1]</li>
	<li>Calibrate using the cross internal validation. 
	&#8203;NOTE: The codes of the calibration curve of the competing risk model with internal validation are similar to the codes in section 4, while suv was replaced by suv.in.</li>
</ol>
7. External validation of the competing risk model
<ol>
	<li>Get the predicted cumulative incidences using external data. Get the predicted cumulative incidences with the matrix of external data variables (cov.ex). 
	suv.ex &#60;- predict.crr(mod_crr,cov.ex) 
	cif36.ex &#60;- suv.ex [which(suv.ex $time==&#34;36&#34;),][-1]</li>
	<li>Calculate the C-index using external validation. 
	rcorr.ex &#60;- rcorr.cens(1-cif36.ex,Surv(Dataset.ex$Survivalmonths,Dataset.ex$tumordeath)) 
	cindex.ex &#60;- rcorr.ex[1]</li>
	<li>Calibrate using external validation. 
	NOTE: The codes of the calibration curve of the competing risk model with internal validation are similar to the codes in section 4, while suv is replaced by suv.ex.</li>
</ol>

<ol>
	<li>Andersen, P. K., Gill, R. D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Cox's+regression+model+for+counting+processes:+A+large+sample+study.">Cox's regression model for counting processes: A large sample study.</a> The Annals of Statistics. 10 (4), 1100-1120 (1982).</li><li>Lubsen, J., Pool, J., vander Does, E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+practical+device+for+the+application+of+a+diagnostic+or+prognostic+function.">A practical device for the application of a diagnostic or prognostic function.</a> Methods of Information in Medicine. 17 (2), 127-129 (1978).</li><li>Harrell, F. E., Lee, K. L., Mark, D. B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Multivariable+prognostic+models:+Issues+in+developing+models,+evaluating+assumptions+and+adequacy,+and+measuring+and+reducing+errors.">Multivariable prognostic models: Issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors.</a> Statistics In Medicine. 15 (4), 361-387 (1996).</li><li>Hung, H., Chiang, C. -. T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Estimation+methods+for+time-dependent+AUC+models+with+survival+data.">Estimation methods for time-dependent AUC models with survival data.</a> The Canadian Journal of Statistics / La Revue Canadienne de Statistique. 38 (1), 8-26 (2010).</li><li>Moons, K. G. M., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Risk+prediction+models:+I.+Development,+internal+validation,+and+assessing+the+incremental+value+of+a+new+(bio)marker.">Risk prediction models: I. Development, internal validation, and assessing the incremental value of a new (bio)marker.</a> Heart. 98 (9), 683-690 (2012).</li><li>Fu, J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Real-world+impact+of+non-breast+cancer-specific+death+on+overall+survival+in+resectable+breast+cancer.">Real-world impact of non-breast cancer-specific death on overall survival in resectable breast cancer.</a> Cancer. 123 (13), 2432-2443 (2017).</li><li>Fine, J. P., Gray, R. J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+proportional+hazards+model+for+the+subdistribution+of+a+competing+risk.">A proportional hazards model for the subdistribution of a competing risk.</a> Journal of the American Statistical Association. 94 (446), 496-509 (1999).</li><li>Wu, L., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Establishing+a+competing+risk+regression+nomogram+model+for+survival+data.">Establishing a competing risk regression nomogram model for survival data.</a> Journal of Visualized Experiments. (164), e60684 (2020).</li><li>Zhang, Z., Geskus, R. B., Kattan, M. W., Zhang, H., Liu, T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Nomogram+for+survival+analysis+in+the+presence+of+competing+risks.">Nomogram for survival analysis in the presence of competing risks.</a> Annals of Translational Medicine. 5 (20), 403 (2017).</li><li>Zhang, Z. H., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Overview+of+model+validation+for+survival+regression+model+with+competing+risks+using+melanoma+study+data.">Overview of model validation for survival regression model with competing risks using melanoma study data.</a> Annals Of Translational Medicine. 6 (16), 325 (2018).</li><li>Newson, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Confidence+intervals+for+rank+statistics:+Somers'+D+and+extensions.">Confidence intervals for rank statistics: Somers' D and extensions.</a> Stata Journal. 6 (3), 309-334 (2006).</li><li>Davison, A. C., Hinkley, D. V., Schechtman, E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Efficient+bootstrap+simulation.">Efficient bootstrap simulation.</a> Biometrika. 73 (3), 555-566 (1986).</li><li>Roecker, E. B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Prediction+error+and+its+estimation+for+subset-selected+models.">Prediction error and its estimation for subset-selected models.</a> Technometrics. 33 (4), 459-468 (1991).</li><li>Steyerberg, E. W., Harrell, F. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Prediction+models+need+appropriate+internal,+internal-external,+and+external+validation.">Prediction models need appropriate internal, internal-external, and external validation.</a> Journal of Clinical Epidemiology. 69, 245-247 (2016).</li><li>Zhang, Z., Chen, L., Xu, P., Hong, Y. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Predictive+analytics+with+ensemble+modeling+in+laparoscopic+surgery:+A+technical+note.">Predictive analytics with ensemble modeling in laparoscopic surgery: A technical note.</a> Laparoscopic, Endoscopic and Robotic Surgery. 5 (1), 25-34 (2022).</li></ol>

The authors declare that they have no competing interests.

This study compared competing risk nomograms established by two distinct methods and conducted evaluation and validation of the established nomograms. Specifically, this study provided a step-by-step tutorial for establishing the nomogram based on a direct method, as well as calculating the C-index and plotting the calibration curves.
The rms package in R software is widely used for the construction and evaluation of Cox proportional hazard models, but it is not applicable for competi...

In recent years, emerging prognostic factors have been identified with the development of precision medicine, and prognostic models combining molecular and clinicopathological factors are drawing increasing attention in clinical settings. However, non-graphical models, such as the Cox proportional hazard model, with results of coefficient values, are difficult for clinicians to understand1. In comparison, a nomogram is a visualization tool of regression models (including the Cox regression model, competing risk model, etc.), a two-dimensional diagram designed for the approximate graphical computation of a mathematical function2. It enables the valuation of different levels of variables in a clinical model and the calculation of risk scores (RS) to predict prognosis.
Model evaluation is essential in model construction, and two characteristics are generally accepted for evaluation: discrimination and calibration. In clinical models, discrimination refers to the ability of a model to separate individuals who develop events from those who do not, such as patients who die versus those who remain alive, and the concordance index (C-index) or the area under the receiver operating characteristic curve (AUC) are typically used to characterize it3,4. Calibration is a process of comparing the predicted probabilities of a model with the actual probabilities, and calibration curves have been widely used to represent it. In addition, model validation (internal and external validation) is an important step in model construction, and only validated models can be further extrapolated5.
The Cox proportional hazard model is a regression model used in medical research for investigating the associations between prognostic factors and survival status. However, the Cox proportional hazard model only considers two statuses of outcome [Y (0, 1)], while study subjects often face more than two statuses, and competing risks arise [Y (0, 1, 2)]1. Overall survival (OS), which is defined as the time from the date of origin (e.g., treatment) to the date of death due to any cause, is the most important endpoint in survival analysis. However, the OS fails to differentiate cancer-specific death from non-cancer-specific death (e.g., cardiovascular events and other unrelated causes), thus ignoring competing risks6. In these situations, the competing risk model is preferred for the prediction of survival status with the consideration of competing risks7. The methodology of constructing and validating Cox proportional hazard models is well-established, while there have been few reports regarding the validation of competing risk models.
In our previous study, a specific competing risk nomogram, a combination of a nomogram and competing risk model, and a risk score estimation based on a competing risk model were established8. This study aims to present different methods of evaluation and validation of the established competing risk nomogram, which should serve as a useful tool for clinicians to predict prognosis with the consideration of competing risks.

<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td>R software</td>
			<td>None</td>
			<td>Not Applicable</td>
			<td>Version 3.6.2 or higher&#160;</td>
		</tr>
		<tr>
			<td>Computer system</td>
			<td>Microsoft&#160;</td>
			<td>Windows 10</td>
			<td>&#160;Windows 10 or higher</td>
		</tr>
	</tbody>
</table>

an r-based landscape validation of a competing risk model

The Cox proportional hazard model is widely applied for survival analyses in clinical settings, but it is not able to cope with multiple survival outcomes. Different from the traditional Cox proportional hazard model, competing risk models consider the presence of competing events and their combination with a nomogram, a graphical calculating device, which is a useful tool for clinicians to conduct a precise prognostic prediction. In this study, we report a method for establishing the competing risk nomogram, that is, the evaluation of its discrimination (i.e., concordance index and area under the curve) and calibration (i.e., calibration curves) abilities, as well as the net benefit (i.e., decision curve analysis). In addition, internal validation using bootstrap resamples of the original dataset and external validation using an external dataset of the established competing risk nomogram were also performed to demonstrate its extrapolation ability. The competing risk nomogram should serve as a useful tool for clinicians to predict prognosis with the consideration of competing risks.

In this study, data of patients with breast cancer were retrieved from the SEER database and served as example data. The SEER database provides data on cancer representing around 34.6% of the United States population, and permission to access the database was obtained (reference number 12296-Nov2018).
Two nomograms (Figure 1), both including histological type, differentiated grade, T stage, and N stage, were established using the direct method and the weighted met...

Watch this Scientific Journal Video about An R-Based Landscape Validation of a Competing Risk Model at JoVE.com

An R-Based Landscape Validation of a Competing Risk Model

The present protocol describes codes in R for evaluating the discrimination and calibration abilities of a competing risk model, as well as codes for the internal and external validation of it.

The Cox proportional hazard model is widely applied for survival analyses in clinical settings, but it is not able to cope with multiple survival ...

an-r-based-landscape-validation-of-a-competing-risk-model

Research

Education

JoVE Core

JoVE Journal

Statistics

Cancer Research

Unit 1

Statistical Softwares

SCIENTISTS IN ACTION

1.9K Views.  Zhejiang University School of Medicine. The present protocol describes codes in R for evaluating the discrimination and calibration abilities of a competing risk model, as well as codes for the internal and external validation of it.

Video: An R-Based Landscape Validation of a Competing Risk Model

Isolation of Circulating Tumor Cells in an Orthotopic Mouse Model of Colorectal Cancer

Despite the advantages of easy applicability and cost-effectiveness, subcutaneous mouse models have severe limitations and do not accurately simulate tumor biology and tumor cell dissemination. Orthotopic mouse models have been introduced to overcome these limitations; however, such models are technically demanding, especially in hollow organs such as the large bowel. In order to produce uniform tumors which reliably grow and metastasize, standardized techniques of tumor cell preparation and injection are critical. 
We have developed an orthotopic mouse model of colorectal cancer (CRC) which develops highly uniform tumors and can be used for tumor biology studies as well as therapeutic trials. Tumor cells from either primary tumors, 2-dimensional (2D) cell lines or 3-dimensional (3D) organoids are injected into the cecum and, depending on the metastatic potential of the injected tumor cells, form highly metastatic tumors. In addition, CTCs can be found regularly. We here describe the technique of tumor cell preparation from both 2D cell lines and 3D organoids as well as primary tumor tissue, the surgical and injection techniques as well as the isolation of CTCs from the tumor-bearing mice, and present tips for troubleshooting.

We describe the establishment of orthotopic colorectal tumors via injection of tumor cells or organoids into the cecum of mice and the subsequent isolation of circulating tumor cells (CTCs) from this model.

Despite the advantages of easy applicability and cost-effectiveness, subcutaneous mouse models have severe limitations and do not accurately simulate ...

An In Vivo Murine Sciatic Nerve Model of Perineural Invasion

Cancer cells invade nerves through a process termed perineural invasion (PNI), in which cancer cells proliferate and migrate in the nerve microenvironment. This type of invasion is exhibited by a variety of cancer types, and very frequently is found in pancreatic cancer. The microscopic size of nerve fibers within mouse pancreas renders the study of PNI difficult in orthotopic murine models. Here, we describe a heterotopic in vivo model of PNI, where we inject syngeneic pancreatic cancer cell line Panc02-H7 into the murine sciatic nerve. In this model, sciatic nerves of anesthetized mice are exposed and injected with cancer cells. The cancer cells invade in the nerves proximally toward the spinal cord from the point of injection. The invaded sciatic nerves are then extracted and processed with OCT for frozen sectioning. H&amp;E and immunofluorescence staining of these sections allow quantification of both the degree of invasion and changes in protein expression. This model can be applied to a variety of studies on PNI given its versatility. Using mice with different genetic modifications and/or different types of cancer cells allows for investigation of the cellular and molecular mechanisms of PNI and for different cancer types. Furthermore, the effects of therapeutic agents on nerve invasion can be studied by applying treatment to these mice.

An In Vivo Murine Sciatic Nerve Model of Perineural Invasion

We describe an in vivo murine model of perineural invasion by injecting syngeneic pancreatic cancer cells into the sciatic nerve. The model allows for quantification of the extent of nerve invasion, and supports investigation of the cellular and molecular mechanisms of perineural invasion.

Cancer cells invade nerves through a process termed perineural invasion (PNI), in which cancer cells proliferate and migrate in the nerve ...

Immunophenotyping of Orthotopic Homograft (Syngeneic) of Murine Primary KPC Pancreatic Ductal Adenocarcinoma by Flow Cytometry

Homograft (syngeneic) tumors are the workhorse of today&#39;s immuno-oncology (I/O) preclinical research. The tumor microenvironment (TME), particularly its immune-components, is vital to the prognosis and prediction of treatment outcomes, especially those of immunotherapy. TME immune-components are composed of different subsets of tumor-infiltrating immune cells assessable by multi-color FACS. Pancreatic ductal adenocarcinoma (PDAC) is among the deadliest malignances lacking good treatment options, thus an urgent and unmet medical need. One important reason for its non-responsiveness to various therapies (chemo-, targeted, I/O) has been its abundant TME, consisting of fibroblasts and leukocytes that protect tumor cells from these therapies. Orthotopically implanted PDAC is believed to more accurately recapture the TME of human pancreatic cancers than conventional subcutaneous (SC) models.
Homograft tumors (KPC) are transplants of mouse spontaneous PDAC originating from genetically engineered KPC-mice (KrasG12D/+/P53-/-/Pdx1-Cre) (KPC-GEMM). The primary tumor tissue is cut into small fragments (~2 mm3) and transplanted subcutaneously (SC) to the syngeneic recipients (C57BL/6, 7&#8211;9 weeks old). The homografts were then surgically orthotopically transplanted onto the pancreas of new C57BL/6 mice, along with SC-implantation, which reached tumor volumes of 300&#8211;1,000 mm3 by 17 days. Only tumors of 400&#8211;600 mm3 were harvested per approved autopsy procedure and cleaned to remove the adjacent non-tumor tissues. They were dissociated per protocol using a tissue dissociator into single-cell suspensions, followed by staining with designated panels of fluorescently-labeled antibodies for various markers of different immune cells (lymphoid, myeloid and NK, DCs). The stained samples were analyzed using multi-color FACS to determine numbers of immune cells of different lineages, as well as their relative percentage within tumors. The immune profiles of orthotopic tumors were then compared to those of SC tumors. The preliminary data demonstrated significantly elevated infiltrating TILs/TAMs in tumors over the pancreas, and higher B-cell infiltration into orthotopic rather than SC tumors.

Immunophenotyping of Orthotopic Homograft Syngeneic of Murine Primary KPC Pancreatic Ductal Adenocarcinoma by Flow Cytometry

The experimental procedure on the immunophenotyping of murine orthotopic PDAC homografts aims at profiling the tumor immuno-microenvironment. Tumors are orthotopically implanted via surgery. Tumors of 200&#8211;600 mm3 in size were harvested and dissociated to prepare single-cell suspensions, followed by multi-immune marker FACS analysis using different fluorescently-labeled antibodies.

Homograft (syngeneic) tumors are the workhorse of today&#39;s immuno-oncology (I/O) preclinical research. The tumor microenvironment (TME), particularly ...

Morphology-Based Distinction Between Healthy and Pathological Cells Utilizing Fourier Transforms and Self-Organizing Maps

The appearance and the movements of immune cells are driven by their environment. As a reaction to a pathogen invasion, the immune cells are recruited to the site of inflammation and are activated to prevent a further spreading of the invasion. This is also reflected by changes in the behavior and the morphological appearance of the immune cells. In cancerous tissue, similar morphokinetic changes have been observed in the behavior of microglial cells: intra-tumoral microglia have less complex 3-dimensional shapes, having less-branched cellular processes, and move more rapidly than those in healthy tissue. The examination of such morphokinetic properties requires complex 3D microscopy techniques, which can be extremely challenging when executed longitudinally. Therefore, the recording of a static 3D shape of a cell is much simpler, because this does not require intravital measurements and can be performed on excised tissue as well. However, it is essential to possess analysis tools that allow the fast and precise description of the 3D shapes and allows the diagnostic classification of healthy and pathogenic tissue samples based solely on static, shape-related information. Here, we present a toolkit that analyzes the discrete Fourier components of the outline of a set of 2D projections of the 3D cell surfaces via Self-Organizing Maps. The application of artificial intelligence methods allows our framework to learn about various cell shapes as it is applied to more and more tissue samples, whilst the workflow remains simple.

Here, we provide a workflow that allows the identification of healthy and pathological cells based on their 3-dimensional shape. We describe the process of using 2D projection outlines based on the 3D surfaces to train a Self-Organizing Map that will provide objective clustering of the investigated cell populations.

The appearance and the movements of immune cells are driven by their environment. As a reaction to a pathogen invasion, the immune cells are recruited to ...

Evaluation of Colorectal Cancer Risk and Prevalence by Stool DNA Integrity Detection

Nowadays, stool DNA can be isolated and analyzed by several methods. The long fragments of DNA in stool can be detected by a qPCR assay, which provides a reliable probability of the presence of pre-neoplastic or neoplastic colorectal lesions. This method, called fluorescence long DNA (FL-DNA), is a fast, non-invasive procedure that is an improvement upon the primary prevention system. This method is based on evaluation of fecal DNA integrity by quantitative amplification of specific targets of genomic DNA. In particular, the evaluation of DNA fragments longer than 200 bp allows for detection of patients with colorectal lesions with very high specificity. However, this system and all currently available stool DNA tests present some general issues that need to be addressed (e.g., the frequency at which tests should be carried out and optimal number of stool samples collected at each timepoint for each individual). However, the main advantage of FL-DNA is the possibility to use it in association with a test currently used in the CRC screening program, known as the immunochemical-based fecal occult blood test (iFOBT). Indeed, both tests can be performed on the same sample, reducing costs and achieving a better prediction of the eventual presence of colorectal lesions.

The presented diagnostic FL-DNA kit is a time-saving and user-friendly method to determine the reliable probability of the presence of colorectal cancer lesions.

Nowadays, stool DNA can be isolated and analyzed by several methods. The long fragments of DNA in stool can be detected by a qPCR assay, which provides a ...

Spatial and Temporal Control of Murine Melanoma Initiation from Mutant Melanocyte Stem Cells

Cutaneous melanoma is well known as the most aggressive skin cancer. Although the risk factors and major genetic alterations continue to be documented with increasing depth, the incidence rate of cutaneous melanoma has shown a rapid and continuous increase during recent decades. In order to find effective preventative methods, it is important to understand the early steps of melanoma initiation in the skin. Previous data has demonstrated that follicular melanocyte stem cells (MCSCs) in the adult skin tissues can act as melanoma cells of origin when expressing oncogenic mutations and genetic alterations. Tumorigenesis arising from melanoma-prone MCSCs can be induced when MCSCs transition from a quiescent to active state. This transition in melanoma-prone MCSCs can be promoted by the modulation of either hair follicle stem cells&#39; activity state or through extrinsic environmental factors such as ultraviolet-B (UV-B). These factors can be artificially manipulated in the laboratory by chemical depilation, which causes transition of hair follicle stem cells and MCSCs from a quiescent to active state, and by UV-B exposure using a benchtop light. These methods provide successful spatial and temporal control of cutaneous melanoma initiation in the murine dorsal skin. Therefore, these in vivo model systems will be valuable to define the early steps of cutaneous melanoma initiation and could be used to test potential methods for tumor prevention.

The following procedure describes a method for spatial and temporal control of melanocytic tumor initiation in murine dorsal skin, using a genetically engineered mouse model. This protocol describes macroscopic as well as microscopic cutaneous melanoma initiation.

Cutaneous melanoma is well known as the most aggressive skin cancer. Although the risk factors and major genetic alterations continue to be documented ...

In Vitro Assay to Study Tumor-macrophage Interaction

Tumor associated macrophages (TAMs) account for a large percentage of cells in the tumor mass for different types of cancers. Glioblastoma (GBM), a malignant brain tumor with no cure, has up to a half the tumor mass TAMs. TAMs can be pro-tumoral or anti-tumoral, depending on the activation of specific genes in the cells. Genetic mutations in the tumors, through regulating cytokine expression, can affect recruitment of TAMs to the tumor microenvironment. Here, we describe a quantitative cell-based assay to assess macrophage recruitment by the conditioned medium from the tumor cells. This assay uses the human macrophage cell line MV-4-11 to study macrophage attraction by the conditioned medium from glioblastoma, allowing for high reproducibility and low variability. Data generated with this assay can contribute to a better understanding of the interaction between the tumor and the tumor microenvironment. Similar assay can be used to assess interaction between the tumor cells and other immune cells, including T cells and natural killer (NK) cells.

This article represents a useful in vitro assay to evaluate the capability of conditioned medium from tumor cells to attract macrophages.

Tumor associated macrophages (TAMs) account for a large percentage of cells in the tumor mass for different types of cancers. Glioblastoma (GBM), a ...

Establishing a Competing Risk Regression Nomogram Model for Survival Data

The Kaplan&#8211;Meier method and Cox proportional hazards regression model are the most common analyses in the survival framework. These are relatively easy to apply and interpret and can be depicted visually. However, when competing events (e.g., cardiovascular and cerebrovascular accidents, treatment-related deaths, traffic accidents) are present, the standard survival methods should be applied with caution, and real-world data cannot be correctly interpreted. It may be desirable to distinguish different kinds of events that may lead to the failure and treat them differently in the analysis. Here, the methods focus on using the competing regression model to identify significant prognostic factors or risk factors when competing events are present. Additionally, nomograms based on a proportional hazard regression model and a competing regression model are established to help clinicians make individual assessments and risk stratifications in order to explain the impact of controversial factors on prognosis.

Presented here is a protocol to build nomograms based on the Cox proportional hazards regression model and competing risk regression model. The competing method is a more rational method to apply when competing events are present in the survival analysis.

The Kaplan&#8211;Meier method and Cox proportional hazards regression model are the most common analyses in the survival framework. These are relatively ...

An Orthotopic Resectional Mouse Model of Pancreatic Cancer

There is a lack of satisfactory animal models to study adjuvant and/or neoadjuvant therapy in patients being considered for surgery of pancreatic cancer (PC). To address this deficiency, we describe a mouse model involving orthotopic implantation of PC followed by distal pancreatectomy and splenectomy. The model has been demonstrated to be safe and suitably flexible for the study of various therapeutic approaches in adjuvant and neo adjuvant settings.
In this model, a pancreatic tumor is first generated by implanting a mixture of human pancreatic cancer cells (luciferase-tagged AsPC-1) and human cancer associated pancreatic stellate cells into the distal pancreas of Balb/c athymic nude mice. After three weeks, the cancer is resected by re-laparotomy, distal pancreatectomy and splenectomy. In this model, bioluminescence imaging can be used to follow the progress of cancer development and effects of resection/treatments. Following resection, adjuvant therapy can be given. Alternatively, neoadjuvant treatment can be given prior to resection.
Representative data from 45 mice are presented. All mice underwent successful distal pancreatectomy/splenectomy with no issues of hemostasis. A macroscopic proximal pancreatic margin greater than 5 mm was achieved in 43 (96%) mice. The technical success rate of pancreatic resection was 100%, with 0% early mortality and morbidity. None of the animals died during the week after resection.
In summary, we describe a robust and reproducible technique for a surgical resection model of pancreatic cancer in mice which mimics the clinical scenario. The model may be useful for the testing of both adjuvant and neoadjuvant treatments.

In the clinical context, patients with localized pancreatic cancer will undergo pancreatectomy followed by adjuvant treatment. This protocol reported here aims to establish a safe and effective method of modelling this clinical scenario in nude mice, through orthotopic implantation of pancreatic cancer followed by distal pancreatectomy and splenectomy.

There is a lack of satisfactory animal models to study adjuvant and/or neoadjuvant therapy in patients being considered for surgery of pancreatic cancer ...

An Ex Vivo Brain Slice Model to Study and Target Breast Cancer Brain Metastatic Tumor Growth

Brain metastasis is a serious consequence of breast cancer for women as these tumors are difficult to treat and are associated with poor clinical outcomes. Preclinical mouse models of breast cancer brain metastatic (BCBM) growth are useful but are expensive, and it is difficult to track live cells and tumor cell invasion within the brain parenchyma. Presented here is a protocol for ex vivo brain slice cultures from xenografted mice containing intracranially injected breast cancer brain-seeking clonal sublines. MDA-MB-231BR luciferase tagged cells were injected intracranially into the brains of Nu/Nu female mice, and following tumor formation, the brains were isolated, sliced, and cultured ex vivo. The tumor slices were imaged to identify tumor cells expressing luciferase and monitor their proliferation and invasion in the brain parenchyma for up to 10 days. Further, the protocol describes the use of time-lapse microscopy to image the growth and invasive behavior of the tumor cells following treatment with ionizing radiation or chemotherapy. The response of tumor cells to treatments can be visualized by live-imaging microscopy, measuring bioluminescence intensity, and performing histology on the brain slice containing BCBM cells. Thus, this ex vivo slice model may be a useful platform for rapid testing of novel therapeutic agents alone or in combination with radiation to identify drugs personalized to target an individual patient&#39;s breast cancer brain metastatic growth within the brain microenvironment.

An Ex Vivo Brain Slice Model to Study and Target Breast Cancer Brain Metastatic Tumor Growth

We introduce a protocol for measuring real-time drug and radiation response of breast cancer brain metastatic cells in an organotypic brain slice model. The methods provide a quantitative assay to investigate the therapeutic effects of various treatments on brain metastases from breast cancer in an ex vivo manner within the brain microenvironment interface.