A.A. received funding support from the Office of Student Research, Yale School of Medicine.
L.J.S. receives grants from the National Institutes of Health (NIH/NCI R01CA206180), Leopoldina Postdoctoral Fellowship, and the Rolf W. Guenther Foundation of Radiological Sciences (Aachen, Germany).
J.C. receives grants from the National Institutes of Health (NIH/NCI R01CA206180), Philips Healthcare, and the German-Israeli Foundation for Scientific Research and Development (Jerusalem, Israel and Neuherberg, Germany); and scholarships from the Rolf W. Guenther Foundation of Radiological Sciences and the Charite Berlin Institute of Health Clinical Scientist Program (Berlin, Germany).
J.S.D. and M.L. receive grants from the National Institutes of Health (NIH/NCI R01CA206180) and Philips Healthcare (Best, The Netherlands).
J.F.G. receives grants from the National Institutes of Health (NIH/NCI R01CA206180), Philips Healthcare, BTG (London, United Kingdom), Boston Scientific (Marlborough, Massachusetts), and Guerbet Healthcare (Villepinte, France)

1 . Workstation Setup for Machine Learning
<ol>
	<li>Use a system with the following: 
	Intel Core 2 Duo or higher CPU at 2.0 GHz 
	4 GB or more system memory 
	POSIX-compliant operating system (Linux or Mac OS) or Microsoft Windows 7 
	User permissions for executing programs and saving files</li>
	<li>Install the following tools: 
	Anaconda Python3: https://www.anaconda.com/download 
	DICOM to NIfTI converter (dcm2niix) - https://github.com/rordenlab/dcm2niix 
	Sublime Text Editor: h...

<ol>
	<li>Benson, A., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=NCCN+clinical+practice+guidelines+in+oncology:+hepatobiliary+cancers.">NCCN clinical practice guidelines in oncology: hepatobiliary cancers.</a> J National Comprehensive Cancer Network. 7 (4), 350-391 (2009).</li><li>Siegel, R., Miller, K., Jemal, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Cancer+statistics,+2016.">Cancer statistics, 2016.</a> CA Cancer J Clin. 66 (1), 7-30 (2016).</li><li>Bruix, J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Clinical+management+of+hepatocellular+carcinoma.+Conclusions+of+the+Barcelona-2000+European+Association+for+the+Study+of+the+Liver+conference.">Clinical management of hepatocellular carcinoma. Conclusions of the Barcelona-2000 European Association for the Study of the Liver conference.</a> Journal of Hepatology. 35 (3), 421-430 (2001).</li><li>Eisenhauer, E., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=New+response+evaluation+criteria+in+solid+tumours:+revised+RECIST+guideline+(version+1.1).">New response evaluation criteria in solid tumours: revised RECIST guideline (version 1.1).</a> European Journal of Cancer. 45 (2), 228-247 (2009).</li><li>Gillmore, R., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=EASL+and+mRECIST+responses+are+independent+prognostic+factors+for+survival+in+hepatocellular+cancer+patients+treated+with+transarterial+embolization.">EASL and mRECIST responses are independent prognostic factors for survival in hepatocellular cancer patients treated with transarterial embolization.</a> Journal of Hepatology. 55 (6), 1309-1316 (2011).</li><li>Lin, M., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Quantitative+and+volumetric+European+Association+for+the+Study+of+the+Liver+and+Response+Evaluation+Criteria+in+Solid+Tumors+measurements:+feasibility+of+a+semiautomated+software+method+to+assess+tumor+response+after+transcatheter+arterial+chemoembolization.">Quantitative and volumetric European Association for the Study of the Liver and Response Evaluation Criteria in Solid Tumors measurements: feasibility of a semiautomated software method to assess tumor response after transcatheter arterial chemoembolization.</a> Journal of Vascular and Interventional Radiology. 23 (12), 1629-1637 (2012).</li><li>Tacher, V., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Comparison+of+Existing+Response+Criteria+in+Patients+with+Hepatocellular+Carcinoma+Treated+with+Transarterial+Chemoembolization+Using+a+3D+Quantitative+Approach.">Comparison of Existing Response Criteria in Patients with Hepatocellular Carcinoma Treated with Transarterial Chemoembolization Using a 3D Quantitative Approach.</a> Radiology. 278 (1), 275-284 (2016).</li><li>Pedregosa, F., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Scikit-learn:+Machine+Learning+in+Python.">Scikit-learn: Machine Learning in Python.</a> Journal of Machine Learning Research. 12, 2825-2830 (2011).</li><li>Bishop, C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Pattern recognition and machine learning. , 738 (2006).</li><li>Alpaydin, E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Introduction to machine learning. Third edition. , 613 (2014).</li><li>Kim, S., Cho, K., Oh, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Development+of+machine+learning+models+for+diagnosis+of+glaucoma.">Development of machine learning models for diagnosis of glaucoma.</a> PLoS One. 12 (5), (2017).</li><li>Son, Y., Kim, H., Kim, E., Choi, S., Lee, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Application+of+support+vector+machine+for+prediction+of+medication+adherence+in+heart+failure+patients.">Application of support vector machine for prediction of medication adherence in heart failure patients.</a> Healthcare Informatics Research. 16 (4), 253-259 (2010).</li><li>Wang, S., Summers, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Machine+learning+and+radiology.">Machine learning and radiology.</a> Medical Image Analysis. 16 (5), 933-951 (2012).</li><li>Abajian, A., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Predicting+Treatment+Response+to+Intra-arterial+Therapies+for+Hepatocellular+Carcinoma+with+the+Use+of+Supervised+Machine+Learning-An+Artificial+Intelligence+Concept.">Predicting Treatment Response to Intra-arterial Therapies for Hepatocellular Carcinoma with the Use of Supervised Machine Learning-An Artificial Intelligence Concept.</a> Journal of Vascular and Interventional Radiology. , (2018).</li></ol>

Patients with hepatocellular carcinoma who are not candidates for surgical resection are offered intra-arterial therapies. Few methods exist to determine if a patient will respond pre-treatment. Post-treatment evaluation techniques rely upon changes in tumor size or tumor contrast uptake. These are called response criteria, with the most accurate being the Quantitative European Association for the Study of the Liver (qEASL) criterion. qEASL relies upon both volumetric and enhancement changes following therapy to...

Patients with hepatocellular carcinoma who are not surgical candidates are offered intra-arterial therapies1,2,3. There is no single metric that determines whether a patient will respond to an intra-arterial therapy before the treatment is administered. The objective of this study was to demonstrate a method that predicts treatment response by applying methods from machine learning. Such models provide guidance to practitioners and patients when choosing whether to proceed with a treatment.
The protocol entails a reproducible process for training and updating a model starting from primary patient data (clinical notes, demographics, laboratory data, and imaging). The data is initially parsed for specific features, with each patient represented by a set of binary features and a binary outcome target label. The outcome label is determined using an established imaging-based response criterion for hepatocellular therapy4,5,6,7. The features and target labels are passed to machine learning software that learns the mapping between features and outcomes under a specific learning model (logistic regression or random forest)8,9,10. Similar techniques have been applied in radiology and other areas of cancer research for diagnosis and treatment prediction11,12,13.
The method adapts techniques from computer science to the field of interventional radiology. Traditional significance studies in interventional radiology, and medicine in general, rely upon mono- or oligo- feature analyses. For example, the Model for End-Stage Liver Disease incorporates five clinical metrics to assess the extent of liver disease. The benefit of the proposed method is the ability to add features liberally; twenty-five features are considered in the example analysis. Additional features may be added as desired.
The technique may be applied to other radiographic interventions where pre- and post- intervention imaging data are available. For example, outcomes following percutaneous treatments could be predicted in a similar manner. The main limitation of the study is the need to manual curate features for inclusion in the model. Data curation and feature extraction is time-consuming for the practitioner and may impede clinical adoption of such machine learning models.

<table border="0" cellpadding="0" cellspacing="0" style="width:851px;" width="850">
	<colgroup>
		<col />
		<col />
		<col />
		<col />
	</colgroup>
	<tbody>
		<tr height="105">
			<td height="105" style="height:105px;width:216px;">Computer workstation</td>
			<td style="width:123px;">N/A</td>
			<td style="width:169px;">N/A</td>
			<td style="width:343px;">Intel Core 2 Duo or higher CPU at 2.0 GHz; 4 GB or more system memory; POSIX-compliant operating system (Linux or Mac OS) or Microsoft Windows 7; User permissions for executing programs and saving files</td>
		</tr>
		<tr height="40">
			<td height="40" style="height:40px;width:216px;">Anaconda Python 3</td>
			<td style="width:123px;">Anaconda, Inc.</td>
			<td style="width:169px;">Version 3.6</td>
			<td style="width:343px;">Python 3 system and libraries packaged for scientists and researchers</td>
		</tr>
		<tr height="40">
			<td height="40" style="height:40px;width:216px;">DICOM to NIfTI</td>
			<td style="width:123px;">NeuroImaging Tools &#38; Resources Collaboratory</td>
			<td style="width:169px;">Version 1.0 (4/4/2018 release)</td>
			<td style="width:343px;">Standalone program for converting DICOM imaging files to NIfTI format</td>
		</tr>
		<tr height="33">
			<td height="33" style="height:33px;width:216px;">Sublime Text Editor</td>
			<td style="width:123px;">Sublime HQ Pty Ltd</td>
			<td style="width:169px;">Version 3 (Build 3143)</td>
			<td style="width:343px;">Text-editor for writing Python code</td>
		</tr>
		<tr height="45">
			<td height="45" style="height:45px;width:216px;">Required Python Libraries</td>
			<td style="width:123px;">N/A</td>
			<td style="width:169px;">Version 3.2.25 (nltk) 
			Version 0.19.1 (scikit-learn)</td>
			<td style="width:343px;">Natural Language Toolkit (nltk) 
			Scikit-learn</td>
		</tr>
		<tr height="40">
			<td height="40" style="height:40px;width:216px;">ITK-SNAP</td>
			<td style="width:123px;">N/A</td>
			<td style="width:169px;">Version 3.6.0</td>
			<td style="width:343px;">Optional toolkit for performing segmentation of organ systems in medical images.</td>
		</tr>
	</tbody>
</table>

predicting treatment response to image-guided therapies using machine learning: an example for trans-arterial treatment of hepatocellular carcinoma

Intra-arterial therapies are the standard of care for patients with hepatocellular carcinoma who cannot undergo surgical resection. The objective of this study was to develop a method to predict response to intra-arterial treatment prior to intervention.
The method provides a general framework for predicting outcomes prior to intra-arterial therapy. It involves pooling clinical, demographic and imaging data across a cohort of patients and using these data to train a machine learning model. The trained model is applied to new patients in order to predict their likelihood of response to intra-arterial therapy.
The method entails the acquisition and parsing of clinical, demographic and imaging data from N patients who have already undergone trans-arterial therapies. These data are parsed into discrete features (age, sex, cirrhosis, degree of tumor enhancement, etc.) and binarized into true/false values (e.g., age over 60, male gender, tumor enhancement beyond a set threshold, etc.). Low-variance features and features with low univariate associations with the outcome are removed. Each treated patient is labeled according to whether they responded or did not respond to treatment. Each training patient is thus represented by a set of binary features and an outcome label. Machine learning models are trained using N - 1 patients with testing on the left-out patient. This process is repeated for each of the N patients. The N models are averaged to arrive at a final model.
The technique is extensible and enables inclusion of additional features in the future. It is also a generalizable process that may be applied to clinical research questions outside of interventional radiology. The main limitation is the need to derive features manually from each patient. A popular modern form of machine learning called deep learning does not suffer from this limitation, but requires larger datasets.

The proposed method was applied to 36 patients who had undergone trans-arterial therapies for hepatocellular carcinoma. Twenty-five features were identified and binarized using steps 1-5. Five features satisfied both the variance and univariate association filters (see steps 5.1 and 5.2) and were used for model training. Each patient was labeled as either a responder or non-responder under the qEASL response criteria. The features matrix was thus a 36 x 5 array while the target labels vec...

Watch this Scientific Journal Video about Predicting Treatment Response to Image-Guided Therapies Using Machine Learning: An Example for Trans-Arterial Treatment of Hepatocellular Carcinoma at JoVE.co

Predicting Treatment Response to Image-Guided Therapies Using Machine Learning: An Example for Trans-Arterial Treatment of Hepatocellular Carcinoma

Intra-arterial therapies are the standard of care for patients with hepatocellular carcinoma who cannot undergo surgical resection. A method for predicting response to these therapies is proposed. The technique uses pre-procedural clinical, demographic, and imaging information to train machine learning models capable of predicting response prior to treatment.

Intra-arterial therapies are the standard of care for patients with hepatocellular carcinoma who cannot undergo surgical resection. The objective of this ...

This method can help predict response to intra arterial therapies. Machine learning applications in interventional oncology will change the way we treat liver cancer and other diseases with image-guided therapies. This method has the potential to improve the way two important decisions are made.Generally, individuals new to this method will struggle because of the need to generate the features and to perform machine learning and code. Visual demonstration of this method is critical as the feature generation steps entail extracting those features using imaging masks. A well-annotated and clinically representative dataset is necessary to train the model.More patient data and large cohorts will be necessary to further improve such a machine learning algorithm. After deciding which clinical features to include in the model, use an appropriate natural language tool kit to parse the plain-text clinic notes into sentences to allow searching for terms of interest. Then store each patient's features in a file with one feature per line.For non-binary features, obtain the median value of each feature across all of the patients, and binarize each feature as a true or false value based on the median value. To calculate a mean liver enhancement feature from a medical image. After isolating the voxels containing the liver, enter the mean liver enhancement command.To determine the liver volume feature, enter the commands as indicated. Then calculate median values for each imaging feature and binarize the features as demonstrated. For aggregation and reduction of the features, first remove the low-variance features from consideration and read the features in the binary matrix.Next, use the variance threshold model to compute the features appearing in at least 20%of both responders and non-responders, similar to that illustrated in the table. Remove any features with a low univariate association with the outcome and filter only those features that pass the original low variant screening retaining n features where n is the number of patients. Then read in the binary matrix for each feature, similar to as illustrated in the table.After the features have been binarized and filtered, it's time to train the model. Model training is a straightforward process that fits the features to the outcome under study and can be used to make predictions about new patients. In this representative analysis of 36 patients who underwent trans arterial therapies for hepatocellular carcinoma, 25 features were identified and binarized as demonstrated with 5 features satisfying both the variants and the univariate association filters.Each patient was labeled as either a responder or non-responder under the quantitative European Association for the study of the liver response criteria. As illustrated, both the logistic regression and random forest models predicted a trans arterial chemoembolization treatment response with an overall accuracy of 78%Following this method, it is possible to make predictions on new patients by applying the training model. This machine learning technique can be universally applied to any therapy that involves pre-procedural, inter-procedural, and post-procedural imaging.

predicting-treatment-response-to-image-guided-therapies-using-machine

Research

JoVE Journal

Medicine

8.1K Aufrufe.  Yale School of Medicine. Diese Methode kann helfen, die Reaktion auf intraarterielle Therapien vorherzusagen. Machine Learning-Anwendungen in der interventionellen Onkologie werden die Art und Weise, wie wir Leberkrebs und andere Krankheiten behandeln, mit bildgesteuerten Therapien verändern. Diese Methode hat das Potenzial, die Art und Weise, wie zwei wichtige Entscheidungen getroffen werden, zu verbessern.Im Allgemeinen werden Personen, die neu in dieser Methode sind, Schwierigkeiten haben, weil die Feature...

Serie: Predicting Treatment Response to Image-Guided Therapies Using Machine Learning: An Example for Trans-Arterial Treatment of Hepatocellular Carcinoma