The study was supported by grants from the Medical Science &#38; Technology Plan Project of Zhejiang Province (grant numbers 2013KYA212), the general program of Zhejiang Province Natural Science Foundation (grant number Y19H160126), and the key program of the Jinhua Municipal Science &#38; Technology Bureau (grant number 2016-3-005, 2018-3-001d, and 2019-3-013).

The Surveillance, Epidemiology, and End Results (SEER) database is an open-access cancer database that only contains deidentified patient data (SEER ID: 12296-Nov2018). Therefore, this study was exempted from the approval of the review board of the Affiliated Jinhua Hospital, Zhejiang University School of Medicine.
1. Data preparation and R packages preparation
<ol>
	<li>Prepare and import the data. 
	&#62; Dataset &#60;- read.csv(&#34;&#8230;/Breast cancer Data.xlsx&#34;) #Import data. 
	NOTE: The data are uploaded in Supplementary File 1.</li>
	<li>Install and load the R packages. 
	&#62; packages &#60;- c(&#34;rms&#34;,&#34;cmprsk&#34;,&#34;mstate&#34;,&#34;survival&#34;,&#34;riskRegression&#34;,&#34; 
	prodlim&#34;) 
	&#62; req.pcg &#60;- function(pcg){ 
	new &#60;- pcg[!(pcg %in% installed.packages()[, &#34;Package&#34;])] 
	if (length(new)) install.packages(new, dependencies = T) 
	sapply(pcg, require, ch = T) 
	&#160;} 
	&#62; req.pcg(packages) 
	&#8203;NOTE: Perform the following procedures based on R software (version 3.6.2) using the packages rms, cmprsk, mstate, survival, riskRegression, and prodlim (http://www.r-projectrg/).</li>
</ol>
2. Establish competing risk nomograms in two distinct methods
<ol>
	<li>Establish the competing risk nomogram in a direct method. 
	&#62; mod_cph &#60;- cph(Surv(Survivalmonths, status) ~ factor1+ factor2+&#8230;, 
	&#160;x=T, y=T, surv=T, data=Dataset) 
	&#62; nom &#60;- nomogram(mod_cph, fun=list(function(x) 1-surv_cph(36, x)&#8230;), 
	&#160;funlabel=c(&#34;3-year event1 Prob.&#34;&#8230;), lp=F) #Take the 36th month as an example. 
	&#62; mod_crr &#60;- crr(Survivalmonths, fstatus, failcode=1, cov1=cov) 
	&#62; score &#60;- log(log((1-real.3y),(1-cif.min36)))/(maxbeta/100) 
	&#62; plot(nom)</li>
	<li>Establish the competing risk nomogram in a weighted method. 
	&#62; df.w &#60;- crprep(&#34;Survivalmonths&#34;,&#34; fstatus&#34;, 
	&#160;data=Dataset, trans=c(1,2), cens=0, 
	&#160;keep=c(&#34;factor1&#34;,&#34; factor2&#34;&#8230;)) 
	&#62; mod.w &#60;- cph(Surv(Tstart, Tstop, status==1) ~ factor1+factor2+&#8230;, 
	&#160;data=df.w, weight=weight.cens, subset=failcode==1, surv=T) 
	&#8203;&#62; nom.w &#60;- nomogram(mod.w&#8230;)</li>
</ol>
3. Discrimination ability of the competing risk nomogram
<ol>
	<li>C-index for discrimination
	<ol>
		<li>Fit the matrix cov into the competing risk model mod_crr. and get a predicted matrix suv. 
		&#62; suv &#60;- predict.crr(mod_crr, cov)</li>
		<li>Get the cumulative incidences in a certain month from suv and calculate the C-index with the function rcorr.cens. 
		&#62; cif36 &#60;- suv[which(suv[,1]==36),][-1] 
		&#62; rcorr &#60;- rcorr.cens(1-cif36,Surv(Dataset$Survivalmonths,Dataset$tumordeath)) 
		&#62; cindex &#60;- rcorr[1]</li>
	</ol>
	</li>
	<li>AUC for discrimination
	<ol>
		<li>Score the predictive performance of the competing risk model using the function Score (riskRegression package). 
		&#62; fgr.w &#60;- FGR(Hist(Survivalmonths, fstatus) ~ factor1+ factor2+&#8230;, data=Dataset, cause=1) 
		&#62; score &#60;- Score(list(&#34;Fine-Gray&#34; = fgr.w),</li>
		<li>Extract the AUC from the &#34;score&#34;. 
		&#8203;&#62; score$AUC</li>
	</ol>
	</li>
</ol>
4. Calibration ability of competing risk models
<ol>
	<li>Calibration curves with a 95% confidence interval of the competing risk model
	<ol>
		<li>Get a data frame with the cumulative incidences of each individual in a certain failure time. 
		&#62; cif36 &#60;- data.frame(cif36) #Take the 36th month as an example. 
		&#62; colnames(cif36.36_o)&#60;-c(&#34;36m&#34;)</li>
		<li>Divide the cohort according to the estimated cumulative incidences into five subgroups and calculate the average predicted cumulative incidences of each subgroup. 
		&#62; group36 &#60;- cut(cif36$`36m`, 
		quantile(cif36$`36m`, seq(0, 1, 0.2)), 
		include.lowest = TRUE, labels = 1:5) 
		&#62; mean36 &#60;- as.vector(by(cif36 $`36m`, group36, mean))</li>
		<li>Calculate the observed cumulative incidences, that is, the actual cumulative incidences, using the function cuminc, and then get the observed cumulative incidences with a 95% confidence interval in a certain failure time. 
		&#62; cum36 &#60;- cuminc(Dataset$Survivalmonths,Dataset$fstatus,group36) 
		&#62; obs36 &#60;- timepoints(cum36,Dataset$Survivalmonths)$est[c(1:5),36] 
		&#62; obs36var &#60;- timepoints(cum36,Dataset$Survivalmonths)$var[c(1:5),36] 
		&#62; df &#60;- data.frame(mean36, obs36, obs36var)</li>
		<li>Plot the calibration curve with the predicted cumulative incidences as the x-axis and the observed cumulative incidences as the y-axis using the function ggplot. 
		&#62; ggplot(df)+ geom_point(aes(x=mean36,y=obs36),col=&#34;red&#34;)+ 
		&#160;geom_point(aes(x=mean36,y=obs36),col=&#34;red&#34;,pch=4)+ 
		&#160;geom_line(col=&#34;red&#34;,aes(x=mean36,y=obs36))+ 
		&#160;geom_errorbar(col=&#34;red&#34;,aes(x=mean36,y=obs36+1.96 
		*sqrt(obs36var)), 
		&#160;ymin =obs36-1.96*sqrt(obs36var), ymax = obs36+1.96 
		*sqrt(obs36var)) 
		geom_abline(lty=3,lwd=2,col=c(rgb(0,118,192, 
		maxColorValue=255)))</li>
	</ol>
	</li>
	<li>Calibration curve with risk scores of the competing risk model
	<ol>
		<li>Valuate each level of all the variables and obtain the total RS. 
		&#62; Dataset$factor1[Dataset$factor1==1] &#60;- factor1.scale[&#34;Factor1_level1&#34;] 
		&#62; &#8230; #For example, Dataset$histology[Dataset$histology==1]&#60;-histology.scale[&#34;Histology1&#34;] 
		&#62; Dataset$rs &#60;- Dataset$factor1+Dataset$factor2+Dataset$factor3+&#8230; 
		NOTE: Obtain the total RS for each patient by summing the points of each variable.</li>
		<li>Count the frequencies and calculate the observed cumulative incidences of the different total risk scores. 
		&#62; rs.freq &#60;- as.data.frame(table(Dataset$rs)) 
		&#62; obs.36 &#60;- vector(mode=&#34;numeric&#34;, length=nrow(rs.freq)) 
		&#62; for (i in 1: nrow(rs.freq)) { 
		&#160;dataset &#60;- subset(Dataset,Dataset$rs== rs.freq [i,1]) 
		&#160;cif.dataset &#60;- cuminc(dataset$Survivalmonths,dataset$death3) 
		&#160;cif36.dataset &#60;- timepoints(cif.dataset,36) 
		&#160;obs.36[i] &#60;- cif36.dataset$est[1]}</li>
		<li>Set the range of the x-axis and calculate the predicted cumulative incidences of the total risk scores. 
		&#62; RS &#60;- range(nom$total.points) 
		&#62; x.36 &#60;- seq(min(RS),max(RS),0.01) 
		&#62; pre.36 &#60;- 1-(1-cif.min36)^exp(x.36*maxbeta/100)</li>
		<li>Plot the calibration curve with risk scores. 
		&#62; plot(x.36, pre.36, type=&#39;l&#39;&#8230;) 
		&#62; par(new=TRUE) 
		&#8203;&#62; plot(as.vector(rs.freq[,1]), obs.36&#8230; )</li>
	</ol>
	</li>
</ol>
5. Decision curve analysis of competing risk models
<ol>
	<li>Source the stdca function to perform the decision curve analysis. 
	&#62; source(&#34;stdca.R&#34;)</li>
	<li>Extract the polynomial equations from the nomogram to calculate the survival probability. 
	&#62; nomogramEx(nomo = nom) 
	&#62; Dataset$predictors &#60;- A * (Dataset$rs ^3) + B * (Dataset$rs ^2) + C * Dataset$rs + D #predictors are predicted probabilities of cancer-specific death calculated by the established nomogram</li>
	<li>Perform the decision curve analysis. 
	&#62; stdca(data = Dataset, outcome = &#34;status&#34;, ttoutcome = &#34;Survivalmonths&#34;, timepoint = 36, 
	&#160;predictors = &#34;predictors&#34;, cmprsk = TRUE, smooth = FALSE, probability = FALSE) 
	NOTE: For evaluating an outcome in the presence of a competing risk, TRUE should be chosen for cmprsk.</li>
</ol>
6. Internal validation using the bootstrap method
<ol>
	<li>Get the average predicted cumulative incidences using the bootstrap method.
	<ol>
		<li>Resample the original dataset (Dataset) with replace to generate the bootstrap dataset (Dataset_in). Establish a competing risk model (mod.in_crr) with the bootstrap dataset. Use the function predict.crr to predict mod.in_crr and loop b times to generate suvall.in. 
		B=b 
		suvall.in &#60;- list() 
		for(j in 1:B){ 
		&#160;Dataset_in &#60;- Dataset[sample(c(1:nrow(Dataset)),nrow(Dataset), 
		replace = TRUE),] 
		attach(Dataset_ in) 
		&#160;cov. in &#60;- model.matrix(~factor1+ factor2+&#8230;)[,-1] 
		&#160;mod. in _crr &#60;- crr(Survivalmonths, fstatus, failcode=1, cov1=cov.in) 
		detach(Dataset. inner) 
		suv. in &#60;- predict.crr(mod. in _crr, cov) 
		suvall.in[[j]] &#60;- suv.in}</li>
		<li>Get the average predicted cumulative incidences in a certain month. 
		cif36all. inner &#60;- vector(mode=&#34;numeric&#34;, length=nrow(Dataset)) 
		for (k in 1:B) { 
		&#160;cif36all. inner&#60;- cif36all. inner+ suvall. inner[[k]][which(suvall. inner[[k]][,1]==36),][-1] 
		} 
		cif36.in &#60;- cif36all.in/B</li>
	</ol>
	</li>
	<li>Calculate the C-index using internal cross-validation with the function rcorr.cens. 
	rcorr. inner &#60;- rcorr.cens(1-cif36.in,Surv(Dataset$Survivalmonths,Dataset$tumordeath)) 
	cindex. inner &#60;- rcorr. inner[1]</li>
	<li>Calibrate using the cross internal validation. 
	&#8203;NOTE: The codes of the calibration curve of the competing risk model with internal validation are similar to the codes in section 4, while suv was replaced by suv.in.</li>
</ol>
7. External validation of the competing risk model
<ol>
	<li>Get the predicted cumulative incidences using external data. Get the predicted cumulative incidences with the matrix of external data variables (cov.ex). 
	suv.ex &#60;- predict.crr(mod_crr,cov.ex) 
	cif36.ex &#60;- suv.ex [which(suv.ex $time==&#34;36&#34;),][-1]</li>
	<li>Calculate the C-index using external validation. 
	rcorr.ex &#60;- rcorr.cens(1-cif36.ex,Surv(Dataset.ex$Survivalmonths,Dataset.ex$tumordeath)) 
	cindex.ex &#60;- rcorr.ex[1]</li>
	<li>Calibrate using external validation. 
	NOTE: The codes of the calibration curve of the competing risk model with internal validation are similar to the codes in section 4, while suv is replaced by suv.ex.</li>
</ol>

<ol>
	<li>Andersen, P. K., Gill, R. D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Cox's+regression+model+for+counting+processes:+A+large+sample+study.">Cox's regression model for counting processes: A large sample study.</a> The Annals of Statistics. 10 (4), 1100-1120 (1982).</li><li>Lubsen, J., Pool, J., vander Does, E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+practical+device+for+the+application+of+a+diagnostic+or+prognostic+function.">A practical device for the application of a diagnostic or prognostic function.</a> Methods of Information in Medicine. 17 (2), 127-129 (1978).</li><li>Harrell, F. E., Lee, K. L., Mark, D. B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Multivariable+prognostic+models:+Issues+in+developing+models,+evaluating+assumptions+and+adequacy,+and+measuring+and+reducing+errors.">Multivariable prognostic models: Issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors.</a> Statistics In Medicine. 15 (4), 361-387 (1996).</li><li>Hung, H., Chiang, C. -. T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Estimation+methods+for+time-dependent+AUC+models+with+survival+data.">Estimation methods for time-dependent AUC models with survival data.</a> The Canadian Journal of Statistics / La Revue Canadienne de Statistique. 38 (1), 8-26 (2010).</li><li>Moons, K. G. M., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Risk+prediction+models:+I.+Development,+internal+validation,+and+assessing+the+incremental+value+of+a+new+(bio)marker.">Risk prediction models: I. Development, internal validation, and assessing the incremental value of a new (bio)marker.</a> Heart. 98 (9), 683-690 (2012).</li><li>Fu, J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Real-world+impact+of+non-breast+cancer-specific+death+on+overall+survival+in+resectable+breast+cancer.">Real-world impact of non-breast cancer-specific death on overall survival in resectable breast cancer.</a> Cancer. 123 (13), 2432-2443 (2017).</li><li>Fine, J. P., Gray, R. J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+proportional+hazards+model+for+the+subdistribution+of+a+competing+risk.">A proportional hazards model for the subdistribution of a competing risk.</a> Journal of the American Statistical Association. 94 (446), 496-509 (1999).</li><li>Wu, L., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Establishing+a+competing+risk+regression+nomogram+model+for+survival+data.">Establishing a competing risk regression nomogram model for survival data.</a> Journal of Visualized Experiments. (164), e60684 (2020).</li><li>Zhang, Z., Geskus, R. B., Kattan, M. W., Zhang, H., Liu, T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Nomogram+for+survival+analysis+in+the+presence+of+competing+risks.">Nomogram for survival analysis in the presence of competing risks.</a> Annals of Translational Medicine. 5 (20), 403 (2017).</li><li>Zhang, Z. H., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Overview+of+model+validation+for+survival+regression+model+with+competing+risks+using+melanoma+study+data.">Overview of model validation for survival regression model with competing risks using melanoma study data.</a> Annals Of Translational Medicine. 6 (16), 325 (2018).</li><li>Newson, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Confidence+intervals+for+rank+statistics:+Somers'+D+and+extensions.">Confidence intervals for rank statistics: Somers' D and extensions.</a> Stata Journal. 6 (3), 309-334 (2006).</li><li>Davison, A. C., Hinkley, D. V., Schechtman, E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Efficient+bootstrap+simulation.">Efficient bootstrap simulation.</a> Biometrika. 73 (3), 555-566 (1986).</li><li>Roecker, E. B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Prediction+error+and+its+estimation+for+subset-selected+models.">Prediction error and its estimation for subset-selected models.</a> Technometrics. 33 (4), 459-468 (1991).</li><li>Steyerberg, E. W., Harrell, F. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Prediction+models+need+appropriate+internal,+internal-external,+and+external+validation.">Prediction models need appropriate internal, internal-external, and external validation.</a> Journal of Clinical Epidemiology. 69, 245-247 (2016).</li><li>Zhang, Z., Chen, L., Xu, P., Hong, Y. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Predictive+analytics+with+ensemble+modeling+in+laparoscopic+surgery:+A+technical+note.">Predictive analytics with ensemble modeling in laparoscopic surgery: A technical note.</a> Laparoscopic, Endoscopic and Robotic Surgery. 5 (1), 25-34 (2022).</li></ol>

The authors declare that they have no competing interests.

This study compared competing risk nomograms established by two distinct methods and conducted evaluation and validation of the established nomograms. Specifically, this study provided a step-by-step tutorial for establishing the nomogram based on a direct method, as well as calculating the C-index and plotting the calibration curves.
The rms package in R software is widely used for the construction and evaluation of Cox proportional hazard models, but it is not applicable for competi...

In recent years, emerging prognostic factors have been identified with the development of precision medicine, and prognostic models combining molecular and clinicopathological factors are drawing increasing attention in clinical settings. However, non-graphical models, such as the Cox proportional hazard model, with results of coefficient values, are difficult for clinicians to understand1. In comparison, a nomogram is a visualization tool of regression models (including the Cox regression model, competing risk model, etc.), a two-dimensional diagram designed for the approximate graphical computation of a mathematical function2. It enables the valuation of different levels of variables in a clinical model and the calculation of risk scores (RS) to predict prognosis.
Model evaluation is essential in model construction, and two characteristics are generally accepted for evaluation: discrimination and calibration. In clinical models, discrimination refers to the ability of a model to separate individuals who develop events from those who do not, such as patients who die versus those who remain alive, and the concordance index (C-index) or the area under the receiver operating characteristic curve (AUC) are typically used to characterize it3,4. Calibration is a process of comparing the predicted probabilities of a model with the actual probabilities, and calibration curves have been widely used to represent it. In addition, model validation (internal and external validation) is an important step in model construction, and only validated models can be further extrapolated5.
The Cox proportional hazard model is a regression model used in medical research for investigating the associations between prognostic factors and survival status. However, the Cox proportional hazard model only considers two statuses of outcome [Y (0, 1)], while study subjects often face more than two statuses, and competing risks arise [Y (0, 1, 2)]1. Overall survival (OS), which is defined as the time from the date of origin (e.g., treatment) to the date of death due to any cause, is the most important endpoint in survival analysis. However, the OS fails to differentiate cancer-specific death from non-cancer-specific death (e.g., cardiovascular events and other unrelated causes), thus ignoring competing risks6. In these situations, the competing risk model is preferred for the prediction of survival status with the consideration of competing risks7. The methodology of constructing and validating Cox proportional hazard models is well-established, while there have been few reports regarding the validation of competing risk models.
In our previous study, a specific competing risk nomogram, a combination of a nomogram and competing risk model, and a risk score estimation based on a competing risk model were established8. This study aims to present different methods of evaluation and validation of the established competing risk nomogram, which should serve as a useful tool for clinicians to predict prognosis with the consideration of competing risks.

<table><tbody><tr><td>R software</td><td>None</td><td>Not Applicable</td><td>Version 3.6.2 or higher&amp;nbsp;</td></tr><tr><td>Computer system</td><td>Microsoft&amp;nbsp;</td><td>Windows 10</td><td>&amp;nbsp;Windows 10 or higher</td></tr></tbody></table>

an r-based landscape validation of a competing risk model

The Cox proportional hazard model is widely applied for survival analyses in clinical settings, but it is not able to cope with multiple survival outcomes. Different from the traditional Cox proportional hazard model, competing risk models consider the presence of competing events and their combination with a nomogram, a graphical calculating device, which is a useful tool for clinicians to conduct a precise prognostic prediction. In this study, we report a method for establishing the competing risk nomogram, that is, the evaluation of its discrimination (i.e., concordance index and area under the curve) and calibration (i.e., calibration curves) abilities, as well as the net benefit (i.e., decision curve analysis). In addition, internal validation using bootstrap resamples of the original dataset and external validation using an external dataset of the established competing risk nomogram were also performed to demonstrate its extrapolation ability. The competing risk nomogram should serve as a useful tool for clinicians to predict prognosis with the consideration of competing risks.

In this study, data of patients with breast cancer were retrieved from the SEER database and served as example data. The SEER database provides data on cancer representing around 34.6% of the United States population, and permission to access the database was obtained (reference number 12296-Nov2018).
Two nomograms (Figure 1), both including histological type, differentiated grade, T stage, and N stage, were established using the direct method and the weighted met...

Watch this Scientific Journal Video about An R-Based Landscape Validation of a Competing Risk Model at JoVE.com

An R-Based Landscape Validation of a Competing Risk Model

The present protocol describes codes in R for evaluating the discrimination and calibration abilities of a competing risk model, as well as codes for the internal and external validation of it.

The Cox proportional hazard model is widely applied for survival analyses in clinical settings, but it is not able to cope with multiple survival ...

an-r-based-landscape-validation-of-a-competing-risk-model

Research

Education

JoVE Core

JoVE Journal

Statistics

Cancer Research

Unit 1

Statistical Softwares

SCIENTISTS IN ACTION

2.0K Views.  Zhejiang University School of Medicine. The present protocol describes codes in R for evaluating the discrimination and calibration abilities of a competing risk model, as well as codes for the internal and external validation of it.

Video: An R-Based Landscape Validation of a Competing Risk Model

Watershed Planning within a Quantitative Scenario Analysis Framework

There is a critical need for tools and methodologies capable of managing aquatic systems within heavily impacted watersheds. Current efforts often fall short as a result of an inability to quantify and predict complex cumulative effects of current and future land use scenarios at relevant spatial scales. The goal of this manuscript is to provide methods for conducting a targeted watershed assessment that enables resource managers to produce landscape-based cumulative effects models for use within a scenario analysis management framework. Sites are first selected for inclusion within the watershed assessment by identifying sites that fall along independent gradients and combinations of known stressors. Field and laboratory techniques are then used to obtain data on the physical, chemical, and biological effects of multiple land use activities. Multiple linear regression analysis is then used to produce landscape-based cumulative effects models for predicting aquatic conditions. Lastly, methods for incorporating cumulative effects models within a scenario analysis framework for guiding management and regulatory decisions (e.g., permitting and mitigation) within actively developing watersheds are discussed and demonstrated for 2 sub-watersheds within the mountaintop mining region of central Appalachia. The watershed assessment and management approach provided herein enables resource managers to facilitate economic and development activity while protecting aquatic resources and producing opportunity for net ecological benefits through targeted remediation.

There is a critical need for tools and methodologies capable of managing aquatic systems in the face of uncertain future conditions. We provide methods for conducting a targeted watershed assessment that enables resource managers to produce landscape-based cumulative effects models for use within a scenario analysis management framework.

There is a critical need for tools and methodologies capable of managing aquatic systems within heavily impacted watersheds. Current efforts often fall ...

Environment

Use of Principal Components for Scaling Up Topographic Models to Map Soil Redistribution and Soil Organic Carbon

Landscape topography is a critical factor affecting soil formation and plays an important role in determining soil properties on the earth surface, as it regulates the gravity-driven soil movement induced by runoff and tillage activities. The recent application of Light Detection and Ranging (LiDAR) data holds promise for generating high spatial resolution topographic metrics that can be used to investigate soil property variability. In this study, fifteen topographic metrics derived from LiDAR data were used to investigate topographic impacts on redistribution of soil and spatial distribution of soil organic carbon (SOC). Specifically, we explored the use of topographic principal components (TPCs) for characterizing topography metrics and stepwise principal component regression (SPCR) to develop topography-based soil erosion and SOC models at site and watershed scales. Performance of SPCR models was evaluated against stepwise ordinary least square regression (SOLSR) models. Results showed that SPCR models outperformed SOLSR models in predicting soil redistribution rates and SOC density at different spatial scales. Use of TPCs removes potential collinearity between individual input variables, and dimensionality reduction by principal component analysis (PCA) diminishes the risk of overfitting the prediction models. This study proposes a new approach for modeling soil redistribution across various spatial scales. For one application, access to private lands is often limited, and the need to extrapolate findings from representative study sites to larger settings that include private lands can be important.

Landscape processes are critical components of soil formation and play important roles in determining soil properties and spatial structure in landscapes. We propose a new approach using stepwise principal component regression to predict soil redistribution and soil organic carbon across various spatial scales.

Landscape topography is a critical factor affecting soil formation and plays an important role in determining soil properties on the earth surface, as it ...

Establishing a Competing Risk Regression Nomogram Model for Survival Data

The Kaplan&#8211;Meier method and Cox proportional hazards regression model are the most common analyses in the survival framework. These are relatively easy to apply and interpret and can be depicted visually. However, when competing events (e.g., cardiovascular and cerebrovascular accidents, treatment-related deaths, traffic accidents) are present, the standard survival methods should be applied with caution, and real-world data cannot be correctly interpreted. It may be desirable to distinguish different kinds of events that may lead to the failure and treat them differently in the analysis. Here, the methods focus on using the competing regression model to identify significant prognostic factors or risk factors when competing events are present. Additionally, nomograms based on a proportional hazard regression model and a competing regression model are established to help clinicians make individual assessments and risk stratifications in order to explain the impact of controversial factors on prognosis.

Presented here is a protocol to build nomograms based on the Cox proportional hazards regression model and competing risk regression model. The competing method is a more rational method to apply when competing events are present in the survival analysis.

The Kaplan&#8211;Meier method and Cox proportional hazards regression model are the most common analyses in the survival framework. These are relatively ...

A Machine Learning Approach to Design an Efficient Selective Screening of Mild Cognitive Impairment

Mild cognitive impairment (MCI) is the first sign of dementia among elderly populations and its early detection is crucial in our aging societies. Common MCI tests are time-consuming such that indiscriminate massive screening would not be cost-effective. Here, we describe a protocol that uses machine learning techniques to rapidly select candidates for further screening via a question-based MCI test. This minimizes the number of resources required for screening because only patients who are potentially MCI positive are tested further.
This methodology was applied in an initial MCI research study that formed the starting point for the design of a selective screening decision tree. The initial study collected many demographic and lifestyle variables as well as details about patient medications. The Short Portable Mental Status Questionnaire (SPMSQ) and the Mini-Mental State Examination (MMSE) were used to detect possible cases of MCI. Finally, we used this method to design an efficient process for classifying individuals at risk of MCI. This work also provides insights into lifestyle-related factors associated with MCI that could be leveraged in the prevention and early detection of MCI among elderly populations.

This methodology produces decision trees that target population groups more prone to suffering from mild cognitive impairment and are useful for cost-effective selective screening of the disease.

Mild cognitive impairment (MCI) is the first sign of dementia among elderly populations and its early detection is crucial in our aging societies. Common ...

Behavior

Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack

Recent studies have shown that an automated, lifespan-inclusive, transdiagnostic, and clinically based, individualized risk calculator provides a powerful system for supporting the early detection of individuals at-risk of psychosis at a large scale, by leveraging electronic health records (EHRs). This risk calculator has been externally validated twice and is undergoing feasibility testing for clinical implementation. Integration of this risk calculator in clinical routine should be facilitated by prospective feasibility studies, which are required to address pragmatic challenges, such as missing data, and the usability of this risk calculator in a real-world and routine clinical setting. Here, we present an approach for a prospective implementation of a real-time psychosis risk detection and alerting service in a real-world EHR system. This method leverages the CogStack platform, which is an open-source, lightweight, and distributed information retrieval and text extraction system. The CogStack platform incorporates a set of services that allow for full-text search of clinical data, lifespan-inclusive, real-time calculation of psychosis risk, early risk-alerting to clinicians, and the visual monitoring of patients over time. Our method includes: 1) ingestion and synchronization of data from multiple sources into the CogStack platform, 2) implementation of a risk calculator, whose algorithm was previously developed and validated, for timely computation of a patient&#39;s risk of psychosis, 3) creation of interactive visualizations and dashboards to monitor patients&#39; health status over time, and 4) building automated alerting systems to ensure that clinicians are notified of patients at-risk, so that appropriate actions can be pursued. This is the first ever study that has developed and implemented a similar detection and alerting system in clinical routine for early detection of psychosis.

We demonstrate how to deploy a real-time psychosis risk calculation and alerting system based on CogStack, an information retrieval and extraction platform for electronic health records.

Recent studies have shown that an automated, lifespan-inclusive, transdiagnostic, and clinically based, individualized risk calculator provides a powerful ...

Nomogram Development for Survival Prediction in MPCC Patients

Competing-Risk Nomogram for Predicting Cancer-Specific Survival in Multiple Primary Colorectal Cancer Patients after Surgery

Cancer-specific survival (CSS) in multiple primary colorectal cancer (MPCC) patients is competitively affected by death from other causes. This study aimed to investigate CSS and associated risk factors by competing risk analysis in MPCC patients. The data of this study is from the SEER database. Using univariable and multivariable analysis of the competitive risk model to weaken the impact of competitive events, it explores the risk factors of CSS and develops a nomogram model. Then, the performance of the model is verified by the ROC curve, calibration curve, and DCA. The study encompasses a total of 8931 patients, with 6255 in the training cohort and 2676 in the validation cohort. Univariable and multivariable analyses showed that sex, Tumor Node Metastasis (TNM) stage, and tumor grade are independent risk factors for cancer-specific survival in MPCC patients. Based on the risk factors, we developed a diagram model to predict CSS. ROC curve, calibration curve, and DCA also show good results. In conclusion, a nomogram is developed that serves as a valuable tool for predicting CSS in MPCC patients, providing clinicians with crucial insights for personalized treatment planning.

The study found that male gender, poor tumor grade, and advanced Tumor Node Metastasis stage were associated with poorer cancer-specific survival (CSS) in multiple primary colorectal cancer (MPCC) patients after surgery. We developed a nomogram to predict the CSS of MPCC patients and contribute to clinical treatment decision-making.

Cancer-specific survival (CSS) in multiple primary colorectal cancer (MPCC) patients is competitively affected by death from other causes. This study ...

A Method of Trigonometric Modelling of Seasonal Variation Demonstrated with Multiple Sclerosis Relapse Data

This report describes a novel Stata-based application of trigonometric regression modelling to 55 years of multiple sclerosis relapse data from 46 clinical centers across 20 countries located in both hemispheres. Central to the success of this method was the strategic use of plot analysis to guide and corroborate the statistical regression modelling. Initial plot analysis was necessary for establishing realistic hypotheses regarding the presence and structural form of seasonal and latitudinal influences on relapse probability and then testing the performance of the resultant models. Trigonometric regression was then necessary to quantify these relationships, adjust for important confounders and provide a measure of certainty as to how plausible these associations were. Synchronization of graphing techniques with regression modelling permitted a systematic refinement of models until best-fit convergence was achieved, enabling novel inferences to be made regarding the independent influence of both season and latitude in predicting relapse onset timing in MS. These methods have the potential for application across other complex disease and epidemiological phenomena suspected or known to vary systematically with season and/or geographic location.

Combining plot analysis with trigonometric regression is a robust method for exploring complex, cyclical phenomena such as relapse onset timing in multiple sclerosis (MS). This method enabled unbiased characterisation of seasonal trends in relapse onset permitting novel inferences around the influence of seasonal variation, ultraviolet radiation (UVR) and latitude.

This report describes a novel Stata-based application of trigonometric regression modelling to  55 years of multiple sclerosis relapse data from 46 ...

Medicine

Development of an Individual-Tree Basal Area Increment Model using a Linear Mixed-Effects Approach

Here, we developed an individual-tree model of 5-year basal area increments based on a dataset including 21898 Picea asperata trees from 779 sample plots located in Xinjiang Province, northwest China. To prevent high correlations among observations from the same sampling unit, we developed the model using a linear mixed-effects approach with random plot effect to account for stochastic variability. Various tree- and stand-level variables, such as indices for tree size, competition, and site condition, were included as fixed effects to explain the residual variability. In addition, heteroscedasticity and autocorrelation were described by introducing variance functions and autocorrelation structures. The optimal linear mixed-effects model was determined by several fit statistics: Akaike&#8217;s information criterion, Bayesian information criterion, logarithm likelihood, and a likelihood ratio test. The results indicated that significant variables of individual-tree basal area increment were the inverse transformation of diameter at breast height, the basal area of trees larger than the subject tree, the number of trees per hectare, and elevation. Furthermore, errors in variance structure were most successfully modeled by the exponential function, and the autocorrelation was significantly corrected by first-order autoregressive structure (AR(1)). The performance of the linear mixed-effects model was significantly improved relative to the model using ordinary least squares regression.

Mixed-effects models are flexible and useful tools for analyzing data with a hierarchical stochastic structure in forestry and could also be used to significantly improve the performance of forest growth models. Here, a protocol is presented that synthesizes information relating to linear mixed-effects models.

Here, we developed an individual-tree model of 5-year basal area increments based on a dataset including 21898 Picea asperata trees from 779 sample plots ...

Biology

Comparison of Predictive Performance of Three Lymph Node Staging Systems in Colorectal Signet Ring Cell Carcinoma Based on Machine Learning Model

Lymph node status is a critical prognostic predictor for patients; however, the prognosis of colorectal signet-ring cell carcinoma (SRCC) has garnered limited attention. This study investigates the prognostic predictive capacity of the log odds of positive lymph nodes (LODDS), lymph node ratio (LNR), and pN staging in SRCC patients using machine learning models (Random Forest, XGBoost, and Neural Network) alongside competing risk models. Relevant data were extracted from the Surveillance, Epidemiology, and End Results (SEER) database. For the machine learning models, prognostic factors for cancer-specific survival (CSS) were identified through univariate and multivariate Cox regression analyses, followed by the application of three machine learning methods-XGBoost, RF, and NN-to ascertain the optimal lymph node staging system. In the competing risk model, univariate and multivariate competing risk analyses were employed to identify prognostic factors, and a nomogram was constructed to predict the prognosis of SRCC patients. The area under the receiver operating characteristic curve (AUC-ROC) and calibration curves were utilized to assess the model's performance. A total of 2,409 SRCC patients were included in this study. To validate the effectiveness of the model, an additional cohort of 15,122 colorectal cancer patients, excluding SRCC cases, was included for external validation. Both the machine learning models and the competing risk nomogram exhibited strong performance in predicting survival outcomes. Compared to pN staging, the LODDS staging systems demonstrated superior prognostic capability. Upon evaluation, machine learning models and competing risk models achieved excellent predictive performance characterized by good discrimination, calibration, and interpretability. Our findings may assist in informing clinical decision-making for patients.

This study evaluates prognostic systems for colorectal signet-ring cell carcinoma patients using machine learning models and competing risk analyses. It identifies log odds of positive lymph nodes as a superior predictor compared to pN staging, demonstrating strong predictive performance and aiding clinical decision-making through robust survival prediction tools.

Lymph node status is a critical prognostic predictor for patients; however, the prognosis of colorectal signet-ring cell carcinoma (SRCC) has garnered ...

Assumptions of Survival Analysis

Survival models analyze the time until one or more events occur, such as death in biological organisms or failure in mechanical systems. These models are widely used across fields like medicine, biology, engineering, and public health to study time-to-event phenomena. To ensure accurate results, survival analysis relies on key assumptions and careful study design.
<ol>
	<li>Survival Times Are Positively Skewed 
	&#160;Survival times often exhibit positive skewness, unlike the normal distribution assumed in many other analyses. This means events tend to occur more frequently early on, with fewer occurrences as time progresses.</li>
	<li>Censoring of Data 
	&#160;Censoring occurs when the full survival time for an individual is not observed, but it is distinct from missing data. Common causes of censoring include participants withdrawing from a study, the study period ending before the event occurs, or participants experiencing unrelated events (e.g., death from an unrelated cause). For example, in a study on heart disease, a participant who dies in an accident would have their data censored at the time of death.</li>
	<li>Independent Censoring 
	&#160;This assumption posits that the reasons for censoring are unrelated to the likelihood of the event of interest. For instance, if participants with severe symptoms are more likely to drop out of a study, survival estimates may become biased. Ensuring that censoring is independent of the health status of participants is critical for reliable analysis.</li>
	<li>Proportional Hazards (Specific to Cox Models) 
	&#160;The Cox proportional hazards model assumes that the hazard ratio between any two individuals remains constant over time. For example, if one group&#39;s risk of an event is twice that of another at the start of a study, this risk ratio must hold throughout the study period.</li>
	<li>Stationarity 
	&#160;Stationarity assumes that the probability of the event changing over time does so similarly across all groups unless explicitly modeled. For example, when comparing survival times between patients treated with a new drug versus a standard treatment, external factors influencing survival should impact both groups equally unless accounted for.</li>
	<li>Clear and Clinically Important Events 
	&#160;The event of interest should be clinically significant and clearly defined to enable accurate measurement and analysis. Ambiguous or misclassified events (e.g., unclear relapse criteria) can compromise the validity of survival time data.</li>
	<li>Adequate Follow-Up Period 
	&#160;The follow-up duration should be long enough to observe a sufficient number of events for robust statistical power. Short follow-up times may miss critical events and lead to incomplete or biased conclusions. It is also essential to minimize differences in event risk among participants recruited at different times to avoid skewed results.</li>
</ol>
Design Considerations in Survival Analysis
Survival studies must be carefully designed to account for these assumptions. A clear definition of the event, sufficient follow-up time, and strategies to minimize censoring bias are vital. When these factors are well-managed, survival models can provide valuable insights into time-to-event phenomena across a range of disciplines.

Survival analysis, a statistical method, evaluates the time until an event occurs. It is commonly used in medicine to analyze life expectancy.
It is ...