This work was supported by the Strategic Priority Research Program of the Chinese Academy of Sciences (XDB13040400) and the startup grant from Jilin University. Anonymous reviewers and biomedical testing users were appreciated for their constructive comments on improving the usability and functionality of kSolutionVis.

NOTE: The following protocol describes the details of the informatics analytic procedure and pseudo-codes of the major modules. The automatic analysis system was developed using Python version 3.6.0 and the Python modules pandas, abc, numpy, scipy, sklearn, sys, PyQt5, sys, mRMR, math and matplotlib. The materials used in this study are listed in the Table of Materials.
1. Prepare the Data Matrix and Class Labels
<ol>
	<li>Prepare the data matrix file as a TAB- or comma-delimited matrix file, as illustrated in Figure 1A. 
	NOTE: Each row has all the values of a feature, and the first item is the feature name. A feature is a probeset ID for the microarray-based transcriptome dataset or may be another value ID like a cysteine residue with its methylation value in a methylomic dataset. Each column gives the feature values of a given sample, with the first item being the sample name. A row is separated into columns by a TAB (Figure 1B) or a comma (Figure 1C). A TAB-delimited matrix file is recognized by the file extension .tsv, and a comma-delimited matrix file has the extension .csv. This file may be generated by saving a matrix as either the .tsv or .csv format from software such as Microsoft Excel. The data matrix may also be generated by computer coding.</li>
	<li>Prepare the class label file as a TAB- or comma-delimited matrix file (Figure 1D), similar to the data matrix file. 
	NOTE: The first column gives the sample names, and the class label of each sample is given in the column titled Class. Maximal compatibility is considered in the coding process, so that additional columns may be added. The class label file may be formatted as a .tsv or .csv file. The names in the column Class may be any terms, and there may be more than two classes of samples. The user may choose any two of the classes for the following analysis.</li>
</ol>
2. Load the Data Matrix and Class Labels
<ol>
	<li>Load the data matrix and class labels into the software. Click the button Load data matrix to choose the user-specified data matrix file. Click the button Load class labels to choose the corresponding class label file. 
	NOTE: After both files are loaded, kSolutionVis will conduct a routine screen of the compatibility between the two files.</li>
	<li>Summarize the features and samples from the data matrix file. Estimate the size of the data matrix file.</li>
	<li>Summarize the samples and classes from the class label file. Estimate the size of the class label file.</li>
	<li>Test whether each sample from the data matrix has a class label. Summarize the numbers of the samples with the class labels.</li>
</ol>
3. Summarize and Display the Baseline Statistics of the Dataset
<ol>
	<li>Click the button Summarize, without any specified keyword input, and the software will display 20 indexed features and the corresponding features names. 
	NOTE: Users need to specify the feature name they wish to find to see its baseline statistics and corresponding value distribution among all input samples.</li>
	<li>Provide a keyword, e.g. &#8220;1000_at&#8221;, in the textbox Feature to find a specific feature to be summarized. Click the button Summarize to get the baseline statistics for this given feature. 
	NOTE: The keyword may appear anywhere in the target feature names, facilitating the search process for users.</li>
	<li>Click the button Summarize to find more than one feature with the given keyword, and then specify the unique feature ID to proceed with the above step of summarizing one particular feature.</li>
</ol>
4. Determine the Class Labels and the Number of Top-ranked Features
<ol>
	<li>Choose the names of Positive (&#8220;P (33)&#8221;) and Negative (&#8220;N (95)&#8221;) classes in the dropdown boxes Class Positive and Class Negative, as shown in Figure 2 (middle). 
	NOTE: It is suggested to choose a balanced binary classification dataset, i.e., the difference between the numbers of positive and negative samples is minimal. The number of samples is also given in parenthesis after the name of each class label in the two dropdown boxes.</li>
	<li>Choose 10 as the number of top-ranked features (parameter pTopX) in the dropdown box Top_X (?) for a comprehensive screen of the feature-subset. 
	NOTE: The software automatically ranks all the features by the P-value calculated by a t-test of each feature comparing the positive and negative classes. A feature with a smaller P-value has a better discriminating power between the two classes of samples. The comprehensive screening module is computationally intensive. The parameter pTopX is 10 by default. Users can change this parameter in the range of 10 to 50, until they find satisfying feature subsets with good classification performances.</li>
</ol>
5. Tune System Parameters for Different Performances
<ol>
	<li>Choose the performance measurement (pMeasurement) Accuracy (Acc) in the dropdown box Acc/bAcc (?) for the selected classifier Extreme Learning Machine (ELM). Another option of this parameter is the measurement Balanced Accuracy (bAcc). 
	NOTE: Let TP, FN, TN, and FP be the numbers of true positives, false negatives, true negatives and false positives, respectively. The measurement Acc is defined as (TP+TN)/(TP+FN+TN+FP), which works best on a balanced dataset6. But a classifier optimized for Acc tends to assign all the samples to the negative class if the number of negative samples is much larger than that of positive ones. The bAcc is defined as (Sn+Sp)/2, where Sn = TP/(TP+FN) and Sp = TN/(TN+FP) are the correctly predicted rates for positive and negative samples, respectively. Therefore, bAcc normalizes the prediction performances over the two classes, and may lead to a balanced prediction performance over two unbalanced classes. Acc is the default choice of pMeasurement. The software uses the classifier ELM by default to calculate the classification performances. The user may also choose a classifier from SVM (Support Vector Machine), KNN (k Nearest Neighbor), Decision Tree, or Na&#239;ve Bayes.</li>
	<li>Choose the cutoff value 0.70 (parameter pCutoff) for the specified performance measurement in the input box pCutoff:. 
	NOTE: Both Acc and bAcc range between 0 and 1, and the user may specify a value pCutoff<img alt="Equation" src="/files/ftp_upload/57738/57738eq1.jpg" />[0, 1] as the cutoff to display the matched solutions. The software carries out a comprehensive feature-subset screening, and an appropriate choice of pCutoff will make the 3D visualization more intuitive and explicit. The default value for pCutoff is 0.70.</li>
</ol>
6. Run the Pipeline and Produce the INTERACTIVE VISUALIZED RESULTS
<ol>
	<li>Click the button Analyze to run the pipeline and generate the visualization plots, as shown in Figure 2 (bottom). 
	NOTE: The left table gives all the feature subsets and their pMeasurement calculated by the 10-fold cross validation strategy of the classifier ELM, as described previously5. Two 3D scatter plots and two-line plots are generated for the feature-subset screening procedure with the current parameter settings.</li>
	<li>Choose 0.70 as the default value of the pMeasurement cutoff (parameter piCutoff, input box Value), and 10 as the default of the number of best feature subsets (parameter piFSNum). 
	NOTE: The pipeline is executed using the parameters pTopX, pMeasurement, and pCutoff. The detected feature subsets may be further screened using the cutoff piCutoff, however piCutoff cannot be smaller than pCutoff. Therefore, piCutoff is initialized as pCutoff and only the feature subsets with the performance measurement &#8805; piCutoff will be visualized. The default value of piCutoff is pCutoff. Sometimes kSolutionVis detects many solutions, and only the best piFSNum (default: 10) feature subsets will be visualized. If the number of feature subsets detected by the software is smaller than piFSNum, all the feature subsets will be visualized.</li>
	<li>Collect and interpret the features detected by the software, as shown in Figure 3. 
	NOTE: The table in the left box shows the detected feature subsets and their performance measurements. The names of the first three columns are &#8220;F1&#8221;, &#8220;F2&#8221;, and &#8220;F3&#8221;. The three features in each feature subset are given in their ranking order in one row (F1 &#60; F2 &#60; F3). The last column gives the performance measurement (Acc or bAcc) of each feature subset, and its column name (Acc or bAcc) is the value of pMeasurement.</li>
</ol>
7. Interpret the 3D Scatter Plots-Visualize and Interpret the Feature Subsets with Similarly Effective Binary Classification Performances Using 3D Scatter Plots
<ol>
	<li>Click the button Analyze to generate the 3D scatter plot of the top 10 feature subsets with the best classification performances (Acc or bAcc) detected by the software, as shown in Figure 3 (middle box). Sort the three features in a feature subset in ascending order of their ranks and use the ranks of the three features as the F1/F2/F3 axes, i.e., F1 &#60; F2 &#60; F3. 
	NOTE: The color of a dot represents the binary classification performance of the corresponding feature subset. A dataset may have multiple feature subsets with similarly effective performance measurements. Therefore, an interactive and simplified scatter plot is necessary.</li>
	<li>Change the value to 0.70 in the input box pCutoff: and click the button Analyze to generate the 3D scatter plot of the feature subsets with the performance measurement &#8805; piCutoff, as seen in Figure 3 (right box). Click the button 3D tuning to open a new window to manually tune the viewing angles of the 3D scatter plot. 
	NOTE: Each feature subset is represented by a dot in the same way as above. The 3D scatter plot was generated in the default angle. To facilitate the 3D visualization and tuning, a separate window will be opened by clicking the button 3D tuning.</li>
	<li>Click the button Reduce to reduce the redundancy of the detected feature subsets. 
	NOTE: If users wish to further select the feature triplets and minimize the redundancy of the feature subsets, the software also provides this function using the mRMR feature selection algorithm. After clicking the Reduce button, kSolutionVis will remove those redundant features in the feature triplets and regenerate the table and the two scatter plots mentioned above. The removed features of the feature triplets will be replaced by the key word in the table. The values of None in the F1/F2/F3 axis will be denoted as the value of piFSNum (the range of the normal value of F1/F2/F3 is [1, top_x]). Therefore, the dots that include a None value may appear to be &#8220;outlier&#8221; dots in the 3D plots. The manually tunable 3D plots may be found in &#8220;Manual tuning of the 3D dot plots&#8221; in the supplementary material.</li>
</ol>
8. Find Gene Annotations and Their Associations with Human Diseases
NOTE: Steps 8 to 10 will illustrate how to annotate a gene from the sequence level of both DNA and protein. Firstly, the gene symbol of each biomarker ID from the above steps will be retrieved from the database DAVID32, and then two representative web servers will be used to analyze this gene symbol from the levels of DNA and protein, respectively. The server GeneCard provides a comprehensive functional annotation of a given gene symbol, and the Online Mendelian Inheritance in Man database (OMIM) provides the most comprehensive curation of disease-gene associations. The server UniProtKB is one of the most comprehensive protein database, and the server Group-based Prediction System (GPS) predicts the signaling phosphorylation&#8217;s for a very large list of kinases.
<ol>
	<li>Copy and paste the web link of the database DAVID into a web browser and open the web page of this database. Click the link Gene ID Conversion seen in Figure 4A and input the feature IDs 38319_at/38147_at/33238_at of the first biomarker subset of the dataset ALL1 (Figure 4B). Click the link Gene List and click Submit List as shown in Figure 4B. Retrieve the annotations of interest and click Show Gene List (Figure 4C). Get the list of gene symbols (Figure 4D). 
	NOTE: The gene symbols retrieved here will be used for further functional annotations in the next steps.</li>
	<li>Copy and paste the web link of the database Gene Cards into a web browser and open the web page of this database. Search a gene&#8217;s name CD3D in the database query input box and find the annotations of this gene from Gene Cards33,34, as shown in Table 1 and Figure 5A. 
	NOTE: Gene Cards is a comprehensive gene knowledgebase, providing nomenclature, genomics, proteomics, subcellular localization, and involved pathways and other functional modules. It also provides external links to various other biomedical databases like PDB/PDB_REDO35, Entrez Gene36, OMIM37, and UniProtKB38. If the feature name is not a standard gene symbol, use the database ENSEMBL to convert it39. CD3D is the name of the gene T-Cell Receptor T3 Delta Chain.</li>
	<li>Copy and paste the web link of the database OMIM into a web browser and open the web page of this database. Search a gene&#8217;s name CD3D and find the annotations of this gene from the database OMIM37, as shown in Table 1 and Figure 5B. 
	NOTE: OMIM serves now as one of the most comprehensive and authoritative sources of human gene connections with inheritable diseases. OMIM was initiated by Dr. Victor A. McKusick to catalog the disease-associated genetic mutations40. OMIM now covers over 15,000 human genes and over 8,500 phenotypes, as of December 1st 2017.</li>
</ol>
9. Annotate the Encoded Proteins and the Post-Translational Modifications
<ol>
	<li>Copy and paste the web link of the database UniProtKB into a web browser and open the web page of this database. Search a gene&#8217;s name CD3D in the query input box of UniProtKB and find the annotations of this gene from the database38, as shown in Table 1 and Figure 5C. 
	NOTE: UniProtKB collects a rich source of annotations for proteins, including both nomenclature and functional information. This database also provides external links to other widely used databases, including PDB/PDB_REDO35, OMIM37, and Pfam41.</li>
	<li>Copy and paste the web link of the web server GPS into a web browser and open the web page of this web server. Retrieve the protein sequence encoded by the biomarker gene CD3D from the UniProtKB database38 and predict the protein&#8217;s post-translational modification (PTM) residues using the online tool GPS, as shown in Table 1 and Figure 5D. 
	NOTE: A biological system is dynamic and complicated, and the existing databases collect only known information. Therefore, biomedical prediction online tools as well as offline programs may provide useful evidence to complement a hypothesized mechanism. GPS has been developed and improved for over 12 years7,42 and may be used to predict a protein&#8217;s PTM residues in a given peptide sequence43,44. Tools are also available for various research topics, including the prediction of a protein&#8217;s subcellular location45 and transcription factor binding motifs 46 among others .</li>
</ol>
10. Annotate Protein-Protein Interactions and Their Enriched Functional Modules
<ol>
	<li>Copy and paste the web link of the web server String into a web browser and open the web page of this web server. Search the list for the genes CD3D and P53, and find their orchestrated properties using the database String47. The same procedure may be carried out using another web server, DAVID32. 
	NOTE: Besides the aforementioned annotations for individual genes, there are many large-scale informatics tools available to investigate the properties of a group of genes. A recent study demonstrated that individually bad marker genes might constitute a much-improved gene set5. Therefore, it&#8217;s worth the computational cost to screen for more complicated biomarkers. The database String may visualize the known or predicted interaction connections, and the David server may detect the functional modules with significant phenotype-associations in the queried genes47,32. Various other large-scale informatics analysis tools are also available.</li>
</ol>
11. Export the Generated Biomarker Subsets and the Visualization Plots
<ol>
	<li>Export the detected biomarker subsets as a .tsv or .csv text file for further analysis. Click the button Export the Table under the table of all the detected biomarker subsets and choose which text format to save as.</li>
	<li>Export the visualization plots as an image file. Click the button Save under each plot and choose which image format to save as. 
	NOTE: The software supports the pixel format .png and the vector format .svg. The pixel images are good for displaying on the computer screen, while the vector images may be converted to any resolution required for journal publication purposes.</li>
</ol>

We have no conflicts of interest related to this report.

This study presents an easy-to-follow multi-solution biomarker detection and characterization protocol for a user-specified binary classification dataset. The software puts an emphasis on user-friendliness and flexible import/export interfaces for various file formats, allowing a biomedical researcher to investigate their dataset easily using the GUI of the software. This study also highlights the necessity of generating more than one solution with similarly effective modeling performances, previously ignored by many exi...

Binary classification, one of the most commonly investigated and challenging data mining problems in the biomedical area, is used to build a classification model trained on two groups of samples with the most accurate discrimination power1,2,3,4,5,6,7. However, the big data generated in the biomedical field has the inherent &#34;large p small n&#34; paradigm, with the number of features usually much larger than the number of samples6,8,9. Therefore, biomedical researchers have to reduce the feature dimension before utilizing the classification algorithms to avoid the overfitting problem8,9. Diagnosis biomarkers are defined as a subset of detected features separating patients of a given disease from healthy control samples10,11. Patients are usually defined as the positive samples, and the healthy controls are defined as the negative samples12.
Recent studies have suggested that there exists more than one solution with identical or similarly effective classification performances for a biomedical dataset5. Almost all the feature selection algorithms are deterministic algorithms, producing only one solution for the same dataset. Genetic algorithms may simultaneously generate multiple solutions with similar performances, but they still try to select one solution with the best fitness function as the output for a given dataset13,14.
Feature selection algorithms can be roughly grouped as either filters or wrappers12. A filter algorithm chooses the top-k features ranked by their significant individual association with the binary class labels based on the assumption that features are independent of each other15,16,17. Although this assumption does not hold true for almost all real-world datasets, the heuristic filter rule performs well in many cases, for instance, the mRMR (Minimum Redundancy and Maximum Relevance) algorithm, the Wilcoxon test based feature filtering (WRank) algorithm, and the ROC (Receiver operating characteristic) plot based filtering (ROCRank) algorithm. mRMR, is an efficient filter algorithm because it approximates the combinatorial estimation problem with a series of much smaller problems, comparing to the maximum-dependency feature selection algorithm, each of which only involves two variables, and therefore uses pairwise joint probabilities which are more robust18,19. However, mRMR may underestimate the usefulness of some features as it does not measure the interactions between features which can increase relevancy, and thus misses some feature combinations that are individually useless but are useful only when combined. The WRank algorithm calculates a non-parametric score of how discriminative a feature is between two classes of samples, and is known for its robustness for outliers20,21. Furthermore, the ROCRank algorithm evaluates how significant the Area Under the ROC Curve (AUC) of a particular feature is for the investigated binary classification performance22,23.
On the other hand, a wrapper evaluates the pre-defined classifier&#39;s performance of a given feature subset, iteratively generated by a heuristic rule, and creates the feature subset with the best performance measurement24. A wrapper generally outperforms a filter in the classification performance but runs slower25. For example, the Regularized Random Forest (RRF)26,27 algorithm uses a greedy rule, by evaluating the features on a subset of the training data at each random forest node, whose feature importance scores are evaluated by the Gini index. The choice of a new feature will be penalized if its information gain does not improve that of the chosen features. Additionally, the Prediction Analysis for Microarrays (PAM)28,29 algorithm, also a wrapper algorithm, calculates a centroid for each of the class labels, and then selects features to shrink the gene centroids toward the overall class centroid. PAM is robust for outlying features.
Multiple solutions with the top classification performance may be necessary for any given dataset. Firstly, the optimization goal of a deterministic algorithm is defined by a mathematical formula, e.g., minimum error rate30, which is not necessarily ideal for biological samples. Secondly, a dataset may have multiple, significantly different, solutions with similar effective or even identical performances. Almost all existing feature selection algorithms will randomly select one of these solutions as the output31.
This study will introduce an informatics analytic protocol for generating multiple feature selection solutions with similar performances for any given binary classification dataset. Considering that most biomedical researchers are not familiar with informatic techniques or computer coding, a user-friendly graphical user interface (GUI) was developed to facilitate the rapid analysis of biomedical binary classification datasets. The analytic protocol consists of data loading and summarizing, parameter tuning, pipeline execution, and result interpretations. With a simple click, the researcher is able to generate the biomarker subsets and publication-quality visualization plots. The protocol has been tested using the transcriptomes of two binary classification datasets of Acute Lymphoblastic Leukemia (ALL), i.e., ALL1 and ALL212. The datasets of ALL1 and ALL2 were downloaded from the Broad Institute Genome Data Analysis Center, available at http://www.broadinstitute.org/cgi-bin/cancer/datasets.cgi.&#160;ALL1 contains 128 samples with 12,625 features. Of these samples, 95 are B-cell ALL and 33 are T-cell ALL. ALL2 includes 100 samples with 12,625 features as well. Of these samples, there are 65 patients that suffered relapse and 35 patients that did not. ALL1 was an easy binary classification dataset, with a minimum accuracy of four filters and four wrappers being 96.7%, and 6 of the 8 feature selection algorithms achieving 100%12. While ALL2 was a more difficult dataset, with the above 8 feature selection algorithms achieving no better than 83.7% accuracy12. This best accuracy was achieved with 56 features detected by the wrapper algorithm, Correlation-based Feature Selection (CFS).

<table><tbody><tr><td>&lt;strong&gt;Hardware&lt;/strong&gt;</td><td></td><td></td><td></td></tr><tr><td>laptop</td><td>Lenovo</td><td>X1 carbon</td><td>Any computer works. Recommended minimum configuration: 1GB extra hard disk space, 1 GB memory, 2.0MHz CPU</td></tr><tr><td>&lt;strong&gt;Name&lt;/strong&gt;</td><td>&lt;strong&gt;Company&lt;/strong&gt;</td><td>&lt;strong&gt;Catalog Number&lt;/strong&gt;</td><td>&lt;strong&gt;Comments&lt;/strong&gt;</td></tr><tr><td>&lt;strong&gt;Software&lt;/strong&gt;</td><td></td><td></td><td></td></tr><tr><td>Python 3.0</td><td>WingWare</td><td>Wing Personal</td><td>Any python programming and running environments support Python version 3.0 or above</td></tr></tbody></table>

selecting multiple biomarker subsets with similarly effective binary classification performances

Biomarker detection is one of the more important biomedical questions for high-throughput &#39;omics&#39; researchers, and almost all existing biomarker detection algorithms generate one biomarker subset with the optimized performance measurement for a given dataset. However, a recent study demonstrated the existence of multiple biomarker subsets with similarly effective or even identical classification performances. This protocol presents a simple and straightforward methodology for detecting biomarker subsets with binary classification performances, better than a user-defined cutoff. The protocol consists of data preparation and loading, baseline information summarization, parameter tuning, biomarker screening, result visualization and interpretation, biomarker gene annotations, and result and visualization exportation at publication quality. The proposed biomarker screening strategy is intuitive and demonstrates a general rule for developing biomarker detection algorithms. A user-friendly graphical user interface (GUI) was developed using the programming language Python, allowing biomedical researchers to have direct access to their results. The source code and manual of kSolutionVis can be downloaded from http://www.healthinformaticslab.org/supp/resources.php.

The goal of this workflow (Figure 6) is to detect multiple biomarker subsets with similar efficiencies for a binary classification dataset. The whole process is illustrated by two example datasets ALL1 and ALL2 extracted from a recently-published biomarker detection study12,48. A user may install kSolutionVis by following the instructions in the supplementary materials.
<p class="jove_content" fo:...

Watch this Scientific Journal Video about Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances at JoVE.com

Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances

Existing algorithms generate one solution for a biomarker detection dataset. This protocol demonstrates the existence of multiple similarly effective solutions and presents a user-friendly software to help biomedical researchers investigate their datasets for the proposed challenge. Computer scientists may also provide this feature in their biomarker detection algorithms.

Biomarker detection is one of the more important biomedical questions for high-throughput &#39;omics&#39; researchers, and almost all existing biomarker ...

selecting-multiple-biomarker-subsets-with-similarly-effective-binary

Research

JoVE Journal

Cancer Research

7.4K Views.  Jilin University. Deze maatregel kan helpen bij het beantwoorden van belangrijke vragen in het gebied van biomedische detectie over het genereren van meerdere oplossingen. Het belangrijkste voordeel van deze techniek is dat het een gebruiksvriendelijke grafische gebruikersinterface biedt voor het assisteren van biomedische onderzoekers bij de detectie van meerdere functies verdwijnt. Begin met het laden van de gegevens Matrix en klasse labels in de software.Klik op Gegevensmatrix laden om het opgegeven ...

Video: Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances

Basics of Multivariate Analysis in Neuroimaging Data

Multivariate analysis techniques for neuroimaging data have recently received increasing attention as they have many attractive features that cannot be easily realized by the more commonly used univariate, voxel-wise, techniques1,5,6,7,8,9. Multivariate approaches evaluate correlation/covariance of activation across brain regions, rather than proceeding on a voxel-by-voxel basis. Thus, their results can be more easily interpreted as a signature of neural networks. Univariate approaches, on the other hand, cannot directly address interregional correlation in the brain. Multivariate approaches can also result in greater statistical power when compared with univariate techniques, which are forced to employ very stringent corrections for voxel-wise multiple comparisons. Further, multivariate techniques also lend themselves much better to prospective application of results from the analysis of one dataset to entirely new datasets. Multivariate techniques are thus well placed to provide information about mean differences and correlations with behavior, similarly to univariate approaches, with potentially greater statistical power and better reproducibility checks. In contrast to these advantages is the high barrier of entry to the use of multivariate approaches, preventing more widespread application in the community. To the neuroscientist becoming familiar with multivariate analysis techniques, an initial survey of the field might present a bewildering variety of approaches that, although algorithmically similar, are presented with different emphases, typically by people with mathematics backgrounds. We believe that multivariate analysis techniques have sufficient potential to warrant better dissemination. Researchers should be able to employ them in an informed and accessible manner. The current article is an attempt at a didactic introduction of multivariate techniques for the novice. A conceptual introduction is followed with a very simple application to a diagnostic data set from the Alzheimer s Disease Neuroimaging Initiative (ADNI), clearly demonstrating the superior performance of the multivariate approach.

The current article describes the basics of multivariate analysis and contrasts it to the more commonly used voxel-wise univariate analysis. Both types of analysis are applied to a clinical-neuroscience data set. Supplementary split-half simulations show better replication of the multivariate results in independent data sets.

Multivariate analysis techniques for neuroimaging data have recently received increasing attention as they have many attractive features that cannot be ...

Neuroscience

Performing Data Mining And Integrative Analysis Of Biomarker in Breast Cancer Using Multiple Publicly Accessible Databases

In recent years, emerging databases were designed to lower the barriers for approaching the intricate cancer genomic datasets, thereby, facilitating investigators to analyze and interpret genes, samples and clinical data across different types of cancer. Herein, we describe a practical operation procedure, taking ID1 (Inhibitor of DNA binding proteins 1) as an example, to characterize the expression patterns of biomarker and survival predictors of breast cancer based on pooled clinical datasets derived from online accessible databases, including ONCOMINE, bcGenExMiner v4.0 (Breast cancer gene-expression miner v4.0), GOBO (Gene expression-based Outcome for Breast cancer Online), HPA (The human protein atlas), and Kaplan-Meier plotter. The analysis began with querying the expression pattern of the gene of interest (e.g., ID1) in cancerous samples vs. normal samples. Then, the correlation analysis between ID1 and clinicopathological characteristics in breast cancer was performed. Next, the expression profiles of ID1 was stratified according to different subgroups. Finally, the association between ID1 expression and survival outcome was analyzed. The operation procedure simplifies the concept to integrate multidimensional data types at the gene level from different databases and test hypotheses regarding recurrence and genomic context of gene alteration events in breast cancer. This method can improve the credibility and representativeness of the conclusions, thereby, present informative perspective on a gene of interest.

Here, we present a protocol to explore the biomarker and survival predictor of breast cancer based on the comprehensive analysis of pooled clinical datasets derived from a variety of publicly accessible databases, using the strategy of expression, correlation and survival analysis step by step.

In recent years, emerging databases were designed to lower the barriers for approaching the intricate cancer genomic datasets, thereby, facilitating ...

Machine Learning Algorithms for Early Detection of Bone Metastases in an Experimental Rat Model

Machine learning (ML) algorithms permit the integration of different features into a model to perform classification or regression tasks with an accuracy exceeding its constituents. This protocol describes the development of an ML algorithm to predict the growth of breast cancer bone macrometastases in a rat model before any abnormalities are observable with standard imaging methods. Such an algorithm can facilitate the detection of early metastatic disease (i.e., micrometastasis) that is regularly missed during staging examinations.
The applied metastasis model is site-specific, meaning that the rats develop metastases exclusively in their right hind leg. The model&#8217;s tumor-take rate is 60%&#8211;80%, with macrometastases becoming visible in magnetic resonance imaging (MRI) and positron emission tomography/computed tomography (PET/CT) in a subset of animals 30 days after induction, whereas a second subset of animals exhibit no tumor growth.
Starting from image examinations acquired at an earlier time point, this protocol describes the extraction of features that indicate tissue vascularization detected by MRI, glucose metabolism by PET/CT, and the subsequent determination of the most relevant features for the prediction of macrometastatic disease. These features are then fed into a model-averaged neural network (avNNet) to classify the animals into one of two groups: one that will develop metastases and the other that will not develop any tumors. The protocol also describes the calculation of standard diagnostic parameters, such as overall accuracy, sensitivity, specificity, negative/positive predictive values, likelihood ratios, and the development of a receiver operating characteristic. An advantage of the proposed protocol is its flexibility, as it can be easily adapted to train a plethora of different ML algorithms with adjustable combinations of an unlimited number of features. Moreover, it can be used to analyze different problems in oncology, infection, and inflammation.

This protocol was designed to train a machine learning algorithm to use a combination of imaging parameters derived from magnetic resonance imaging (MRI) and positron emission tomography/computed tomography (PET/CT) in a rat model of breast cancer bone metastases to detect early metastatic disease and predict subsequent progression to macrometastases.

Machine learning (ML) algorithms permit the integration of different features into a model to perform classification or regression tasks with an accuracy ...

Identification of Disease-related Spatial Covariance Patterns using Neuroimaging Data

The scaled subprofile model (SSM)1-4 is a multivariate PCA-based algorithm that identifies major sources of variation in patient and control group brain image data while rejecting lesser components (Figure 1). Applied directly to voxel-by-voxel covariance data of steady-state multimodality images, an entire group image set can be reduced to a few significant linearly independent covariance patterns and corresponding subject scores. Each pattern, termed a group invariant subprofile (GIS), is an orthogonal principal component that represents a spatially distributed network of functionally interrelated brain regions. Large global mean scalar effects that can obscure smaller network-specific contributions are removed by the inherent logarithmic conversion and mean centering of the data2,5,6. Subjects express each of these patterns to a variable degree represented by a simple scalar score that can correlate with independent clinical or psychometric descriptors7,8. Using logistic regression analysis of subject scores (i.e. pattern expression values), linear coefficients can be derived to combine multiple principal components into single disease-related spatial covariance patterns, i.e. composite networks with improved discrimination of patients from healthy control subjects5,6. Cross-validation within the derivation set can be performed using bootstrap resampling techniques9. Forward validation is easily confirmed by direct score evaluation of the derived patterns in prospective datasets10. Once validated, disease-related patterns can be used to score individual patients with respect to a fixed reference sample, often the set of healthy subjects that was used (with the disease group) in the original pattern derivation11. These standardized values can in turn be used to assist in differential diagnosis12,13 and to assess disease progression and treatment effects at the network level7,14-16. We present an example of the application of this methodology to FDG PET data of Parkinson's Disease patients and normal controls using our in-house software to derive a characteristic covariance pattern biomarker of disease.

Multivariate techniques including principal component analysis (PCA) have been used to identify signature patterns of regional change in functional brain images. We have developed an algorithm to identify reproducible network biomarkers for the diagnosis of neurodegenerative disorders, assessment of disease progression, and objective evaluation of treatment effects in patient populations.

The scaled subprofile model (SSM)1-4 is a multivariate PCA-based algorithm that identifies major sources of variation in patient and control group brain ...

Medicine

Detection of Architectural Distortion in Prior Mammograms via Analysis of Oriented Patterns

We demonstrate methods for the detection of architectural distortion in prior mammograms of interval-cancer cases based on analysis of the orientation of breast tissue patterns in mammograms. We hypothesize that architectural distortion modifies the normal orientation of breast tissue patterns in mammographic images before the formation of masses or tumors. In the initial steps of our methods, the oriented structures in a given mammogram are analyzed using Gabor filters and phase portraits to detect node-like sites of radiating or intersecting tissue patterns. Each detected site is then characterized using the node value, fractal dimension, and a measure of angular dispersion specifically designed to represent spiculating patterns associated with architectural distortion.
Our methods were tested with a database of 106 prior mammograms of 56 interval-cancer cases and 52 mammograms of 13 normal cases using the features developed for the characterization of architectural distortion, pattern classification via quadratic discriminant analysis, and validation with the leave-one-patient out procedure. According to the results of free-response receiver operating characteristic analysis, our methods have demonstrated the capability to detect architectural distortion in prior mammograms, taken 15 months (on the average) before clinical diagnosis of breast cancer, with a sensitivity of 80% at about five false positives per patient.

Detection of Architectural Distortion in Prior Mammograms via Analysis of Oriented Patterns

We demonstrate methods for the detection of architectural distortion in prior mammograms. Oriented structures are analyzed using Gabor filters and phase portraits to detect sites of radiating tissue patterns. Each site is characterized and classified using measures to represent spiculating patterns. The methods should assist in the detection of breast cancer.

We demonstrate methods for the detection of architectural distortion in prior mammograms of interval-cancer cases based on analysis of the orientation of ...

Assisted Selection of Biomarkers by Linear Discriminant Analysis Effect Size (LEfSe) in Microbiome Data

There is growing attention toward closed biological genomes in the environment and in health. To explore and reveal the intergroup differences among different samples or environments, it is crucial to discover biomarkers with statistical differences among groups. The application of Linear discriminant analysis Effect Size (LEfSe) can help find good biomarkers. Based on the original genome data, quality control, and quantification of different sequences based on taxa or genes are carried out. First, the Kruskal-Wallis rank test was used to distinguish between specific differences among statistical and biological groups. Then, the Wilcoxon rank test was performed between the two groups obtained in the previous step to assess whether the differences were consistent. Finally, a linear discriminant analysis (LDA) was conducted to evaluate the influence of biomarkers on significantly different groups based on LDA scores. To sum up, LEfSe provided the convenience for identifying genomic biomarkers that characterize statistical differences among biological groups.

Assisted Selection of Biomarkers by Linear Discriminant Analysis Effect Size LEfSe in Microbiome Data

LEfSe (LDA Effect Size) is a tool for high-dimensional biomarker mining to identify genomic features (such as genes, pathways, and taxonomies) that significantly characterize two or more groups in microbiome data.

There is growing attention toward closed biological genomes in the environment and in health. To explore and reveal the intergroup differences among ...

Genetics

A Machine Learning Approach to Design an Efficient Selective Screening of Mild Cognitive Impairment

Mild cognitive impairment (MCI) is the first sign of dementia among elderly populations and its early detection is crucial in our aging societies. Common MCI tests are time-consuming such that indiscriminate massive screening would not be cost-effective. Here, we describe a protocol that uses machine learning techniques to rapidly select candidates for further screening via a question-based MCI test. This minimizes the number of resources required for screening because only patients who are potentially MCI positive are tested further.
This methodology was applied in an initial MCI research study that formed the starting point for the design of a selective screening decision tree. The initial study collected many demographic and lifestyle variables as well as details about patient medications. The Short Portable Mental Status Questionnaire (SPMSQ) and the Mini-Mental State Examination (MMSE) were used to detect possible cases of MCI. Finally, we used this method to design an efficient process for classifying individuals at risk of MCI. This work also provides insights into lifestyle-related factors associated with MCI that could be leveraged in the prevention and early detection of MCI among elderly populations.

This methodology produces decision trees that target population groups more prone to suffering from mild cognitive impairment and are useful for cost-effective selective screening of the disease.

Mild cognitive impairment (MCI) is the first sign of dementia among elderly populations and its early detection is crucial in our aging societies. Common ...

Behavior

Dried Blood Spot Collection of Health Biomarkers to Maximize Participation in Population Studies

Biomarkers are directly-measured biological indicators of disease, health, exposures, or other biological information. In population and social sciences, biomarkers need to be easy to obtain, transport, and analyze. Dried Blood Spots meet this need, and can be collected in the field with high response rates. These elements are particularly important in longitudinal study designs including interventions where attrition is critical to avoid, and high response rates improve the interpretation of results. Dried Blood Spot sample collection is simple, quick, relatively painless, less invasive then venipuncture, and requires minimal field storage requirements (i.e.&#160;samples do not need to be immediately frozen and can be stored for a long period of time in a stable freezer environment before assay). The samples can be analyzed for a variety of different analytes, including cholesterol, C-reactive protein, glycosylated hemoglobin, numerous cytokines, and other analytes, as well as provide genetic material. DBS collection is depicted as employed in several recent studies.

Biomarkers are directly-measured biological indicators of disease or health. In population and social sciences, biomarkers need to be easy to obtain, transport, and analyze. Dried Blood Spot (DBS) collection meets this need, can be collected in the field with high response rates&#160;and analyzed for a variety of biomarkers.

Biomarkers are directly-measured biological indicators of disease, health, exposures, or other biological information. In population and social sciences, ...

Predicting Treatment Response to Image-Guided Therapies Using Machine Learning: An Example for Trans-Arterial Treatment of Hepatocellular Carcinoma

Intra-arterial therapies are the standard of care for patients with hepatocellular carcinoma who cannot undergo surgical resection. The objective of this study was to develop a method to predict response to intra-arterial treatment prior to intervention.
The method provides a general framework for predicting outcomes prior to intra-arterial therapy. It involves pooling clinical, demographic and imaging data across a cohort of patients and using these data to train a machine learning model. The trained model is applied to new patients in order to predict their likelihood of response to intra-arterial therapy.
The method entails the acquisition and parsing of clinical, demographic and imaging data from N patients who have already undergone trans-arterial therapies. These data are parsed into discrete features (age, sex, cirrhosis, degree of tumor enhancement, etc.) and binarized into true/false values (e.g., age over 60, male gender, tumor enhancement beyond a set threshold, etc.). Low-variance features and features with low univariate associations with the outcome are removed. Each treated patient is labeled according to whether they responded or did not respond to treatment. Each training patient is thus represented by a set of binary features and an outcome label. Machine learning models are trained using N - 1 patients with testing on the left-out patient. This process is repeated for each of the N patients. The N models are averaged to arrive at a final model.
The technique is extensible and enables inclusion of additional features in the future. It is also a generalizable process that may be applied to clinical research questions outside of interventional radiology. The main limitation is the need to derive features manually from each patient. A popular modern form of machine learning called deep learning does not suffer from this limitation, but requires larger datasets.

Intra-arterial therapies are the standard of care for patients with hepatocellular carcinoma who cannot undergo surgical resection. A method for predicting response to these therapies is proposed. The technique uses pre-procedural clinical, demographic, and imaging information to train machine learning models capable of predicting response prior to treatment.

Intra-arterial therapies are the standard of care for patients with hepatocellular carcinoma who cannot undergo surgical resection. The objective of this ...

Comparison of Predictive Performance of Three Lymph Node Staging Systems in Colorectal Signet Ring Cell Carcinoma Based on Machine Learning Model

Lymph node status is a critical prognostic predictor for patients; however, the prognosis of colorectal signet-ring cell carcinoma (SRCC) has garnered limited attention. This study investigates the prognostic predictive capacity of the log odds of positive lymph nodes (LODDS), lymph node ratio (LNR), and pN staging in SRCC patients using machine learning models (Random Forest, XGBoost, and Neural Network) alongside competing risk models. Relevant data were extracted from the Surveillance, Epidemiology, and End Results (SEER) database. For the machine learning models, prognostic factors for cancer-specific survival (CSS) were identified through univariate and multivariate Cox regression analyses, followed by the application of three machine learning methods-XGBoost, RF, and NN-to ascertain the optimal lymph node staging system. In the competing risk model, univariate and multivariate competing risk analyses were employed to identify prognostic factors, and a nomogram was constructed to predict the prognosis of SRCC patients. The area under the receiver operating characteristic curve (AUC-ROC) and calibration curves were utilized to assess the model's performance. A total of 2,409 SRCC patients were included in this study. To validate the effectiveness of the model, an additional cohort of 15,122 colorectal cancer patients, excluding SRCC cases, was included for external validation. Both the machine learning models and the competing risk nomogram exhibited strong performance in predicting survival outcomes. Compared to pN staging, the LODDS staging systems demonstrated superior prognostic capability. Upon evaluation, machine learning models and competing risk models achieved excellent predictive performance characterized by good discrimination, calibration, and interpretability. Our findings may assist in informing clinical decision-making for patients.

This study evaluates prognostic systems for colorectal signet-ring cell carcinoma patients using machine learning models and competing risk analyses. It identifies log odds of positive lymph nodes as a superior predictor compared to pN staging, demonstrating strong predictive performance and aiding clinical decision-making through robust survival prediction tools.

Lymph node status is a critical prognostic predictor for patients; however, the prognosis of colorectal signet-ring cell carcinoma (SRCC) has garnered ...

Selecting Multiple Biomarker Subsets with Similarly Effective Binary Classification Performances

Samenvatting

Meer video's verkennen

Hoofdstukken in deze video