This study is funded by and is a direct output of the King&#39;s College London Confidence in Concept award from the Medical Research Council (MRC) (MC_PC_16048) to PFP. RD and AR were supported by: (a) the Maudsley Charity; (b) the National Institute for Health Research (NIHR) Biomedical Research Centre at South London and Maudsley NHS Foundation Trust and King&#39;s College London; (c) Health Data Research UK, which is funded by the UK Medical Research Council, Engineering and Physical Sciences Research Council, Economic and Social Research Council, Department of Health and Social Care (England), Chief Scientist Office of the Scottish Government Health and Social Care Directorates, Health and Social Care Research and Development Division (Welsh Government), Public Health Agency (Northern Ireland), British Heart Foundation and Wellcome Trust; (d) The BigData@Heart Consortium, funded by the Innovative Medicines Initiative-2 Joint Undertaking under grant agreement No. 116074. This Joint Undertaking receives support from the European Union&#39;s Horizon 2020 research and innovation programme and EFPIA; it is chaired, by DE Grobbee and SD Anker, partnering with 20 academic and industry partners and ESC; and (e) The National Institute for Health Research University College London Hospitals Biomedical Research Centre. These funding bodies had no role in the design of the study, collection and analyses. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR or the Department of Health.

This study was approved by East of England - Cambridgeshire and Hertfordshire Research Ethics Committee (Reference number: 18/EE/0066).
NOTE: We have developed this protocol based on the CogStack platform and the Python programming language. This system requires Docker (more specifically, Docker Compose <a href="https://docs.docker.com/compose/" target="_blank">https://docs.docker.com/compose/</a>), Anaconda Python (<a href="https://www.anaconda.com/distribution/" target="_blank">https://www.anaconda.com/distribution/</a>) and Git (<a href="https://git-scm.com/downloads" target="_blank">https://git-scm.com/downloads</a>) pre-installed on a device. The commands provided in this protocol are based on the Linux environment. In the following, we provide the details of preparing source data from an EHR database, ingesting the data to CogStack platform, and setting up a real-time risk calculation and alerting system for psychosis based on the CogStack platform. Moreover, an online version of the risk calculator was developed to facilitate numeric calculation of the probability of an individual developing psychosis in secondary mental health care on <a href="http://www.psychosis-risk.net/" target="_blank">http://www.psychosis-risk.net</a>.
1. Source data preparation
NOTE: In most use cases, CogStack ingests source data from a specified database view that can combine data from one or more source database tables, where a view is a searchable object in a database that contains the result set of a stored query on the data. The setup of the ingesting view is tailored by the specific use cases and deployment settings of a health record database system. This protocol is developed based on a psychosis risk calculator developed and externally validated twice by Fusar-Poli et al.14,21 and as part of a pilot implementation feasibility study6. The protocol is based on an EHR database deployed with Microsoft SQL Server 2014.
<ol>
	<li>Create a view object (called &#34;vwPsychosisBase&#34; in this protocol) in an existing EHR database system to join necessary information of patients for psychosis risk calculation and alerting. Make sure that this view includes all patients receiving a first primary diagnosis of non-organic and non-psychotic mental disorder (recorded by the International Statistical Classification of Diseases and Related Health Problems, Tenth Revision [ICD-10]), as defined in the original model14,21.</li>
	<li>Ensure that each record in the view involves three types of patient information: 1) the first primary diagnosis of a patient in the EHR system, including ICD-10 diagnosis index (diagnoses were clustered together into the following ten clusters: acute and transient psychotic disorders, anxiety disorders, bipolar mood disorders, childhood and adolescent onset disorders, developmental disorders, nonbipolar mood disorders, mental retardation, personality disorders, physiological syndromes, substance use disorders) and diagnosis date; 2) a patient&#39;s demographic data, including gender, ethnicity and date of birth; and 3) the most recent contact information of care team for a patient, such as details of general practice (GP), consultants and care coordinators. The first two types of information are vital for the psychosis risk calculator14,21, and the third type of information is to enable timely risk alerting.</li>
	<li>Make sure that each record in the view has a unique identifier (e.g., &#34;patient_id&#34; used in this protocol).</li>
	<li>Select the last update timestamps of all source information related to a record in the view (e.g., the last update times of a patient&#39;s demographic information and the patient&#39;s first primary diagnosis information), and choose the latest timestamp as the last update date and time for the record in the view (denoted as &#34;etl_updated_dttm&#34; in this protocol). The last update date and time of a record allows CogStack to synchronize updates in the database, such as new and updated records.</li>
</ol>
2. Data ingestion
<ol>
	<li>Download or clone the code repository from Github (<a href="https://github.com/cogstack-slam/psychosis" target="_blank">https://github.com/cogstack-slam/psychosis</a>) or by typing &#34;git clone <a href="https://github.com/cogstack-slam/psychosis.git" target="_blank">https://github.com/cogstack-slam/psychosis.git</a>&#34; in a terminal window. The downloaded folder contains the code for psychosis risk calculation and configuration files for deploying a CogStack instance.</li>
	<li>Go to the &#34;cogstack_deploy/cogstack/&#34; directory and modify &#34;psychosis.properties&#34; to configure CogStack Pipeline for data ingestion. Modify the settings of section &#34;SOURCE: DB CONFIGURATIONS&#34; based on the EHR database setup, including specifying the IP address of the database server, database name, database username and password. Modify the view name (i.e. &#34;vwPsychosisBase&#34;) and field names (e.g., &#34;patient_id&#34; and &#34;etl_updated_dttm&#34;) if necessary. In case of error in configuring this file, follow the instructions at <a href="https://cogstack.atlassian.net/wiki/spaces/COGDOC/pages/38043684/Quickstart" target="_blank">https://cogstack.atlassian.net/wiki/spaces/COGDOC/pages/38043684/Quickstart</a>.</li>
	<li>Go to the &#34;cogstack_deploy/common/elasticsearch/config/&#34; directory and modify the section &#34;xpack.notification.email.account&#34; in the &#34;elasticsearch.yml&#34; file to configure an Email address for sending alerts. A detailed instruction for Email configuration can be found on <a href="https://www.elastic.co/guide/en/kibana/6.4/watcher-create-threshold-alert.html" target="_blank">https://www.elastic.co/guide/en/kibana/6.4/watcher-create-threshold-alert.html</a>.</li>
	<li>Go to the &#34;cogstack_deploy/&#34; directory and type &#34;docker-compose up&#34; to run the CogStack platform. Execute this command with root access. If the process is completed successfully, there will be printed status logs of the currently running services, including CogStack Pipeline, Elasticsearch and Kibana, in the terminal. As a result, all data and updates in the source database view will be timely ingested to an Elasticsearch index called &#34;psychosis_base&#34; in the CogStack platform.</li>
	<li>Open a web browser and access Kibana user interface by typing &#34;http://localhost:5601/&#34; (or replacing &#34;localhost&#34; with a specific IP address of the server running the CogStack platform). For the first time accessing Kibana, click the Management tab and Index Patterns tab to specify an Elasticsearch index that one wants to access with Kinaba. Type &#34;psychosis_base&#34; in the &#34;Index pattern&#34; field and click Next step. Select &#34;etl_updated_dttm&#34; for the &#34;Time Filter&#34; field name and click Create index pattern to add the &#34;psychosis_base&#34; index pattern for Kinana.</li>
	<li>Once Kibana is connected to the Elasticsearch index (i.e., &#34;psychosis_base&#34;), search and browse the source data interactively through the &#34;Discover&#34; page. Kibana allows non-technical users to search for both structured metadata and free text. Detailed instructions of using &#34;Discover&#34; are available on <a href="https://www.elastic.co/guide/en/kibana/6.4/discover.html" target="_blank">https://www.elastic.co/guide/en/kibana/6.4/discover.html</a>.</li>
</ol>
3. Risk calculation
<ol>
	<li>Open a new terminal window and go to the &#34;psychosis/&#34; directory. Install all required Python packages (including &#34;elasticsearch&#34;, &#34;elasticsearch_dsl&#34;, &#34;pandas&#34; and &#34;numpy&#34;) used in the risk calculator by typing &#34;conda install package-name&#34; or &#34;pip install package-name&#34; in the terminal.</li>
	<li>Type &#34;python risk_calculator.py&#34; to run the psychosis risk calculator. If the process is completed successfully, logs of the risk calculation will be printed in the terminal and the risk results will be stored in a new Elasticsearch index called &#34;psychosis_risk&#34; within the CogStack platform.</li>
	<li>Check the risk results by using the Kibana interface. Similar to Steps 2.5 and 2.6, add a new index pattern &#34;psychosis_risk&#34; to connect Kinbana with the &#34;psychosis_risk&#34; index, and explore the risk results through the &#34;Discover&#34; page. To facilitate identifying new patients at-risk, use &#34;first_primary_diagnosis_date&#34; as the &#34;Time Filter&#34; field in building the &#34;psychosis_risk&#34; index. When exploring data in the &#34;Discover&#34; page, make sure that the index pattern &#34;psychosis_risk&#34; is selected.</li>
</ol>
4. Data visualization
<ol>
	<li>In addition to searching and accessing individual-level information via the &#34;Discover&#34; page in Kibana, one can build visualizations and dashboards to obtain an overview of characteristics for the whole population of at-risk patients. To do this, click on Visualize in the side navigation of Kibana. Then, click the Create new visualization button and choose a visualization type (e.g., pie and line charts). Select &#34;psychosis_risk&#34; as the index that one wants to visualize through Kibana. By default, visualizations will include all records/patients in the &#34;psychosis_risk&#34; index. Detailed instructions of building Kibana visualizations are available on <a href="https://www.elastic.co/guide/en/kibana/6.4/visualize.html" target="_blank">https://www.elastic.co/guide/en/kibana/6.4/visualize.html</a>.</li>
	<li>To select a specific subset of data for visualization, add a &#34;filter&#34;. For example, selecting a filter filed as &#34;h_2_year&#34;, choosing an operator as &#34;is not between&#34; and setting values from &#34;0.0&#34; to &#34;0.05&#34; will only include patients whose risk of psychosis in 2 years are higher than 0.05.</li>
	<li>Once individual visualizations are built, click on Dashboard in the side navigation of Kibana to create a dashboard that displays a set of related visualizations together. Click Create new dashboard and the Add button to create a new dashboard panel. Click visualizations that one wants to show within the new dashboard panel. Click Save and type a title to save the panel. Instructions on building Kibana dashboards are available at <a href="https://www.elastic.co/guide/en/kibana/6.4/dashboard.html" target="_blank">https://www.elastic.co/guide/en/kibana/6.4/dashboard.html</a>.</li>
</ol>
5. Risk Alerting
<ol>
	<li>Click on Management in the side navigation of Kibana and then click Watcher under Elasticsearch to create alerting for clinicians when patients were at-risk of psychosis. If the Watcher button is not visible, click License Management and click Start trial or Update license.</li>
	<li>Click Create advanced watch to set up a new Watcher. Type an &#34;ID&#34; and &#34;Name&#34;. Delete the content of &#34;Watch JSON&#34; section and copy the content in the &#34;watcher.json&#34; file in the &#34;psychosis&#34; directory to the &#34;Watch JSON&#34; section. This watcher will send alerting Email to &#34;clinician@nhs.uk&#34; (which can be replaced with the Email address where one wants to send alerts) from &#34;username@nhs.uk&#34; (which was set in Step 2.3) if there are one or more patients whose risk of psychosis in 2 years are higher than 0.05 (a tentative threshold for feasibility testing) in every 24 hours.</li>
	<li>Before saving the Watcher, click Simulate to test the Watcher execution. If the Watcher is set successfully, one will see the simulation output printed. In case of error in the settings, follow the instructions on <a href="https://www.elastic.co/guide/en/elastic-stack-overview/6.4/watcher-getting-started.html" target="_blank">https://www.elastic.co/guide/en/elastic-stack-overview/6.4/watcher-getting-started.html</a>.</li>
	<li>To stop a Watcher, permanently delete it or temporarily deactivate it from the &#34;Status&#34; page of the Watcher.</li>
</ol>

<ol>
	<li>Lieberman, J. A., First, M. B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Psychotic+Disorders.">Psychotic Disorders.</a> New England Journal of Medicine. 379 (3), 270-280 (2018).</li><li>Oh, H., Koyanagi, A., Kelleher, I., DeVylder, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Psychotic+experiences+and+disability:+findings+from+the+collaborative+psychiatric+epidemiology+surveys.">Psychotic experiences and disability: findings from the collaborative psychiatric epidemiology surveys.</a> Schizophrenia Research. 193, 343-347 (2018).</li><li>Tsiachristas, A., Thomas, T., Leal, J., Lennox, B. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Economic+impact+of+early+intervention+in+psychosis+services:+results+from+a+longitudinal+retrospective+controlled+study+in+England.">Economic impact of early intervention in psychosis services: results from a longitudinal retrospective controlled study in England.</a> BMJ Open. 6 (10), e012611 (2016).</li><li>Fusar-Poli, P., McGorry, P. D., Kane, J. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Improving+outcomes+of+first-episode+psychosis:+an+overview.">Improving outcomes of first-episode psychosis: an overview.</a> World Psychiatry. 16 (3), 251-265 (2017).</li><li>Fusar-Poli, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+clinical+high-risk+state+for+psychosis+(CHR-P),+version+II.">The clinical high-risk state for psychosis (CHR-P), version II.</a> Schizophrenia Bulletin. 43 (1), 44-47 (2017).</li><li>Fusar-Poli, P., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Real-world+Implementation+of+a+Transdiagnostic+Risk+Calculator+for+the+Automatic+Detection+of+Individuals+at-risk+of+Psychosis+in+Clinical+Routine:+Study+Protocol.">Real-world Implementation of a Transdiagnostic Risk Calculator for the Automatic Detection of Individuals at-risk of Psychosis in Clinical Routine: Study Protocol.</a> Frontiers in Psychiatry. 10, 109 (2019).</li><li>Fusar-Poli, P., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Disorder,+not+just+state+of+risk:+meta-analysis+of+functioning+and+quality+of+life+in+people+at+high+risk+of+psychosis.">Disorder, not just state of risk: meta-analysis of functioning and quality of life in people at high risk of psychosis.</a> British Journal of Psychiatry. 207 (3), 198-206 (2015).</li><li>Fusar-Poli, P., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Heterogeneity+of+psychosis+risk+within+individuals+at+clinical+high+risk:+a+meta-analytical+stratification.">Heterogeneity of psychosis risk within individuals at clinical high risk: a meta-analytical stratification.</a> JAMA Psychiatry. 73 (2), 113-120 (2016).</li><li>Fusar-Poli, P., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Diagnostic+and+prognostic+significance+of+brief+limited+intermittent+psychotic+symptoms+(BLIPS)+in+individuals+at+ultra+high+risk.">Diagnostic and prognostic significance of brief limited intermittent psychotic symptoms (BLIPS) in individuals at ultra high risk.</a> Schizophrenia Bulletin. 43 (1), 48-56 (2016).</li><li>Fusar-Poli, P., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Prognosis+of+brief+psychotic+episodes:+a+meta-analysis.">Prognosis of brief psychotic episodes: a meta-analysis.</a> JAMA Psychiatry. 73 (3), 211-220 (2016).</li><li>Fusar-Poli, P., Sullivan, S., Shah, J., Uhlhaas, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Improving+the+detection+of+individuals+at+clinical+risk+for+psychosis+in+the+community,+primary+and+secondary+care:+an+integrated+evidence-based+approach.">Improving the detection of individuals at clinical risk for psychosis in the community, primary and secondary care: an integrated evidence-based approach.</a> Frontiers in Psychiatry. 10, 774 (2019).</li><li>Fusar-Poli, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Extending+the+benefits+of+indicated+prevention+to+improve+outcomes+of+first-episode+psychosis.">Extending the benefits of indicated prevention to improve outcomes of first-episode psychosis.</a> JAMA Psychiatry. 74 (7), 667-668 (2017).</li><li>Fusar-Poli, P., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Transdiagnostic+psychiatry:+a+systematic+review.">Transdiagnostic psychiatry: a systematic review.</a> World Psychiatry. 18 (2), 192-207 (2019).</li><li>Fusar-Poli, P., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Development+and+validation+of+a+clinically+based+risk+calculator+for+the+transdiagnostic+prediction+of+psychosis.">Development and validation of a clinically based risk calculator for the transdiagnostic prediction of psychosis.</a> JAMA Psychiatry. , (2017).</li><li>Fusar-Poli, P., Hijazi, Z., Stahl, D., Steyerberg, E. W. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+science+of+prognosis+in+psychiatry:+a+review.">The science of prognosis in psychiatry: a review.</a> JAMA Psychiatry. 75 (12), 1289-1297 (2018).</li><li>Radua, J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=What+causes+psychosis?+An+umbrella+review+of+risk+and+protective+factors.">What causes psychosis? An umbrella review of risk and protective factors.</a> World Psychiatry. , (2018).</li><li>Fusar-Poli, P., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Deconstructing+vulnerability+for+psychosis:+Meta-analysis+of+environmental+risk+factors+for+psychosis+in+subjects+at+ultra+high-risk.">Deconstructing vulnerability for psychosis: Meta-analysis of environmental risk factors for psychosis in subjects at ultra high-risk.</a> European Psychiatry. 40, 65-75 (2017).</li><li>Fusar-Poli, P., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Clinical-learning+versus+machine-learning+for+transdiagnostic+prediction+of+psychosis+onset+in+individuals+at-risk.">Clinical-learning versus machine-learning for transdiagnostic prediction of psychosis onset in individuals at-risk.</a> Translational Psychiatry. 9 (1), 1-11 (2019).</li><li>Stewart, R., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+South+London+and+Maudsley+NHS+foundation+trust+biomedical+research+centre+(SLAM+BRC)+case+register:+development+and+descriptive+data.">The South London and Maudsley NHS foundation trust biomedical research centre (SLAM BRC) case register: development and descriptive data.</a> BMC Psychiatry. 9 (1), 51 (2009).</li><li>Jongsma, H. E., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Treated+incidence+of+psychotic+disorders+in+the+multinational+EU-GEI+study.">Treated incidence of psychotic disorders in the multinational EU-GEI study.</a> JAMA Psychiatry. 75 (1), 36-46 (2018).</li><li>Fusar-Poli, P., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Transdiagnostic+risk+calculator+for+the+automatic+detection+of+individuals+at-risk+and+the+prediction+of+psychosis:+Second+replication+in+an+independent+national+health+service+trust.">Transdiagnostic risk calculator for the automatic detection of individuals at-risk and the prediction of psychosis: Second replication in an independent national health service trust.</a> Schizophrenia Bulletin. , (2019).</li><li>Fusar-Poli, P., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Transdiagnostic+individualized+clinically+based+risk+calculator+for+the+detection+of+individuals+at-risk+and+the+prediction+of+psychosis:+Model+refinement+including+nonlinear+effects+of+age.">Transdiagnostic individualized clinically based risk calculator for the detection of individuals at-risk and the prediction of psychosis: Model refinement including nonlinear effects of age.</a> Frontiers in Psychiatry. 10, 313 (2019).</li><li>Colditz, G. A., Wei, E. K. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Risk+prediction+models:+applications+in+cancer+prevention.">Risk prediction models: applications in cancer prevention.</a> Current Epidemiology Reports. 2 (4), 245-250 (2015).</li><li>Jackson, R., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=CogStack+-+Experiences+Of+Deploying+Integrated+Information+Retrieval+And+Extraction+Services+In+A+Large+National+Health+Service+Foundation+Trust+Hospital.">CogStack - Experiences Of Deploying Integrated Information Retrieval And Extraction Services In A Large National Health Service Foundation Trust Hospital.</a> BMC Medical Informatics and Decision Making. 18 (47), (2017).</li><li>McGorrian, C., Leong, T., D'Agostino, R., Graham, I. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Risk+estimation+systems+in+clinical+use:+SCORE,+Heart+Score+and+the+Framingham+system.">Risk estimation systems in clinical use: SCORE, Heart Score and the Framingham system.</a> Hyperlipidaemia (Oxford Cardiology Library). , (2012).</li><li>Studerus, E., Beck, K., Fusar-Poli, P., Riecher-R&#246;ssler, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Development+and+Validation+of+a+Dynamic+Risk+Prediction+Model+to+Forecast+Psychosis+Onset+in+Patients+at+Clinical+High+Risk.">Development and Validation of a Dynamic Risk Prediction Model to Forecast Psychosis Onset in Patients at Clinical High Risk.</a> Schizophrenia Bulletin. , (2019).</li><li>Damschroder, L. J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Fostering+implementation+of+health+services+research+findings+into+practice:+a+consolidated+framework+for+advancing+implementation+science.">Fostering implementation of health services research findings into practice: a consolidated framework for advancing implementation science.</a> Implementation Science. 4, 50 (2009).</li><li>Fusar-Poli, P., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Outreach+and+support+in+south+London+(OASIS),+2001-2011:+ten+years+of+early+diagnosis+and+treatment+for+young+individuals+at+high+clinical+risk+for+psychosis.">Outreach and support in south London (OASIS), 2001-2011: ten years of early diagnosis and treatment for young individuals at high clinical risk for psychosis.</a> European Psychiatry. 28 (5), 315-326 (2013).</li></ol>

The authors have nothing to disclose.

We have demonstrated the first EHR implementation of a real-time psychosis risk detection and alerting system based on CogStack, an open source information retrieval and extraction platform. Following this approach, one can transform and ingest a large set of clinical data in various formats, including structured and unstructured information, into a CogStack instance, so as to enable full-text search, interactive analyses and visualization of data, as well as real-time alerting to clinicians of patients that are at-risk of psychosis. Although the original psychosis risk calculator has been validated in pilot studies across several NHS Trusts, albeit using retrospective patient records6,14,21, this experimental design provides the first evidence base that this risk calculator can be replicated and deployed for use in real time. This approach allows the automatic delivery of prognostic results to clinicians through existing clinical notification channels, such as Email, in real time. This clearly demonstrates the technical feasibility for conducting a large-scale effectiveness trial to evaluate the ultimate clinical utility of this risk calculator in the real world.
This protocol is empirically innovative, as there does not exist a similar risk detection and alerting system for psychosis. Moreover, this protocol has high generalizability in clinical use, particularly because of the unique strengths of our approach. From a theoretical perspective, we used a risk prediction model that was developed based on a large retrospective de-identified cohort from the SLaM NHS Trust. SLaM provides secondary mental health care to a population of 1.36 million individuals in South London and has one of the highest recorded rates of psychosis in the world. This large cohort, which has high diversity in sociodemographic and diagnostic characteristics, allows us to develop a risk prediction model that is unlikely to be biased towards a population with specific characteristics. This is supported by evidence that the prognostic accuracy of this risk calculator has already been replicated twice in two different databases14,21, including one outside of SLaM. Another theoretical strength of this risk model is that basic demographic and clinical diagnosis information were used as predictors. Such information is ubiquitous in electronic clinical data and in fact missing data for these predictors have been shown to be relatively rare in our previous studies14,21. The high availability of information for building predictors makes it possible to run the risk calculator over a large number of patient samples across different secondary mental health care sectors. In addition, the risk calculator is a general algorithm which is suitable for all individuals at-risk of developing psychosis in secondary mental health care, regardless of individuals&#39; ages. That is, this calculator is not only suitable for the 15-35 age range of peak psychosis risk16, but also for those outside of this range, showing a high degree of generalizability.
From a practical perspective, both the risk calculator and the CogStack platform are light-weight and open-source services that do not involve resource-heavy techniques or costly infrastructure. Such a low-cost and easy-to-deploy platform can reduce the barriers to its adoption in real-world clinical settings. Also, our solution overcomes the main implementation barrier: risk estimation systems provide little value unless they are used by clinicians in day-to-day practice25. Specifically, our approach accesses data from the EHR, performs analyses independent of an electronic medical record system and can send analysis results back to clinicians through existing notification channels. This method does not require that the business logic in pre-existing systems be modified and can work as a standalone service to support and extend existing clinical decision support systems. Thus, the protocol has high compatibility with pre-existing clinical systems and can be easily integrated into routine clinical practice. Moreover, the protocol provides user-friendly interfaces for searching, analyzing and visualizing of clinical data, which make it easy for clinicians to interpret and explore the risk results.
This protocol also has its limitations. First, the effectiveness of this protocol has not been evaluated in routine clinical practice. This study focused on technical feasibility tests of implementing a real-time psychosis risk detection and alerting system in a local EHR. To further evaluate the effectiveness of this system in routine clinical practice, future large scale randomized controlled trials are needed6. A second limitation is that the predictions of risk scores in this protocol were made based on the first primary diagnoses, which are static data collected at a single snapshot. However, the CHR-P symptoms are intrinsically evolving over time. A dynamic version of psychosis risk calculator, in which prediction models can be dynamically updated to reflect the changes, has been developed recently26. Future work will focus on integrating this dynamic calculator in the current protocol.
The most critical step in this approach was identifying EHR data that were used for extract predictors in the risk calculator. This may also involve creating data element mappings, when an EHR system used a data model different from that used in this protocol, such as distinct coding systems for patients&#39; ethnic groups. We have open-sourced all the code and mapping definitions online (<a href="https://github.com/cogstack-slam/psychosis" target="_blank">https://github.com/cogstack-slam/psychosis</a>). Based on these materials, one would be able to replicate the deployment or adjust the calculator depending on one&#39;s own circumstance. Another critical step was creating a database view for data ingestion in CogStack. Since relational join operations (i.e., combining columns from one or more database tables) in Elasticsearch can lead to high computational cost, we conducted these join operations in the EHR database by creating a database view. This view combined all information that was needed to extract predictors in the risk calculator, and two vital fields that were used by CogStack pipelines for data partitioning in data ingestion. The first field is a unique primary key for each record in the view (&#34;patient_id&#34; used this protocol) and the second is a timestamp when a record was modified most recently. If these two fields were not set properly, CogStack might not synchronize data updates in an EHR database timely. Detailed instructions for troubleshooting issues on CogStack data ingestion are available on <a href="https://cogstack.atlassian.net/wiki/spaces/COGDOC/overview" target="_blank">https://cogstack.atlassian.net/wiki/spaces/COGDOC/overview</a> and <a href="https://github.com/CogStack/CogStack-Pipeline" target="_blank">https://github.com/CogStack/CogStack-Pipeline</a>.
This protocol is highly transportable and can be easily deployed in NHS Trusts that have a CRIS or CogStack platform. So far, the CRIS platform-including the consenting procedures-has been fully described elsewhere and is under expansion across 12 NHS Trusts in the UK, harnessing over 2 million deidentified patient records (<a href="https://crisnetwork.co/" target="_blank">https://crisnetwork.co/</a>). Similarly, the CogStack platform has been deployed not only in SLaM, but also other NHS Trusts across the UK such as University College London Hospitals (UCLH), King&#39;s College Hospital (KCH), Guy&#39;s and St Thomas&#39; (GSTT), and Mersey Care NHS Trusts. Those Trusts without such as platform can use an online version of risk calculator (<a href="http://psychosis-risk.net/" target="_blank">http://psychosis-risk.net</a>), or build this protocol from scratch based on this manuscript and our online documents. Although this protocol is developed for psychosis risk detection, the architectural design of this protocol is not tied-in to this specific use case. The protocol is flexible enough to allow for reconfiguration and repurposing of the real-time monitoring and alerting components for other risk measurement areas, such as adverse drug reactions, thereby allowing clinicians to timely take action to improve patient care, safety and experience.

Psychotic disorders are serious mental health illnesses that lead to difficulties in distinguishing between the internal experience of the mind and the external reality of the environment1, as well as a higher than average risk of self-harm and suicide2. Under standard care, these disorders result in major public health impact with a significant health and economic burden on individuals, families and societies worldwide3. Early interventions in psychosis can improve outcomes of this mental disorder4. In particular, detection, prognostic assessment and preventive treatment of individuals who are at clinical high risk for developing psychosis (CHR-P)5 provides a unique potential to alter the course of the disorder, thereby improving the quality of life for many people and their families3,6. CHR-P individuals are help-seeking young people presenting with attenuated symptoms and functional impairment7: their risk of developing psychosis is 20% at 2-years8 but it is higher in some specific subgroups9,10. Despite some substantial advancements, the impact of preventive approaches in routine clinical practice is limited by the ability to detect most individuals who are at-risk11. Current detection methods are based on help-seeking behaviors and referrals on suspicion of psychosis risk; these methods are highly inefficient in handling a large number of samples11. Thus, the scalability of current detection methods to the vast majority of the at-risk population is quite limited12. In fact, only 5% (standalone specialized early detection services) to 12% (youth mental health services) of individuals at-risk of developing a first psychotic disorder can be detected at the time of their at-risk stage by the current detection strategies6.
To extend the clinical benefits of the preventive approaches in a larger number of at-risk individuals, we developed an automated, lifespan-inclusive (i.e., across all ages), transdiagnostic (i.e., across different diagnoses)13, clinically-based individualized risk calculator, which can detect individuals at-risk of psychosis in secondary mental health care at scale, beyond those meeting CHR-P criteria14. This risk calculator used a Cox proportional hazard model to predict the risk of developing a psychotic disorder over six years from five routinely collected clinical variables selected a priori, in line with methodological guidelines15: age, gender, ethnicity, age-by-gender and primary index diagnosis. These clinical variables were selected based on a priori knowledge obtained from meta analyses16,17, as recommended by the state-of-the-art methodological guidelines15. The number of predictors is limited to preserve the Event Per Variable ratio and minimize overfitting biases; including too many variables without a priori filter leads to overfitting problems and poor prognostic accuracy18. The method used to develop this model provides similar prognostic accuracy to automatic machine learning methods18. Parameters of the Cox model were estimated based on a retrospective de-identified cohort from the South London and Maudsley National Health Service Foundation Trust (SLaM)19. SLaM is a National Health Service (NHS) mental health trust that provides secondary mental health care to a population of 1.36 million individuals in South London (Lambeth, Southwark, Lewisham and Croydon boroughs), and has one of the highest recorded rates of psychosis in the world20. All data used in the model development were extracted from the Clinical Record Interactive Search (CRIS) platform, a digital case register system, which provides researchers with retrospective access and analysis of anonymized clinical records19. The clinical information in CRIS is extracted from a bespoke Electronic Health Record (EHR) system, at SLaM, called electronic Patient Journey System (ePJS). SLaM is paper-free and ePJS represents the standard data collection platform for clinical routine. Thus, the transdiagnostic risk calculator leverages EHRs and has the potential to automatically screen large EHRs of patients accessing secondary mental healthcare, to detect those who may be at-risk of psychosis. The algorithm of this transdiagnostic risk calculator has been published previously6,14,21. The transdiagnostic risk calculator has been externally validated in two NHS Foundation Trusts14,21 and optimized22, demonstrating its adequate prognostic performance and generalizability across different populations.
According to methodological guidelines on the development of a risk prediction model15,23, the next step after model development and validation is to implement the prediction model in routine clinical practice. Implementation studies are usually preceded by pilot or feasibility studies that address potential pragmatic limitations associated with the use of risk algorithms in clinical practice. For example, required data for running a calculator, such as age, gender and ethnicity, may not be available at the date of diagnosis or updated later. Effective methods for handling missing data and synchronizing frequent updates in real-time data streams should be considered to obtain the most reliable prediction results in an implementation. Furthermore, since the initial development of the risk calculator was based on retrospective cohort data, it is not known whether it can be used in a real-time data stream that is typical of a real-world clinical setting. Another challenge is ensuring that relevant clinicians receive the recommendations generated by the risk calculator within an appropriate time frame and within a shared and accepted communication pathway.
To overcome these limitations, we have completed a feasibility implementation study employing the individualized transdiagnostic risk calculator. The study included two phases: an in vitro phase that was conducted using data from the local EHR, without contacting clinicians or patients, and an in vivo phase, which involved direct contact with clinicians. The in vitro phase had two manifold aims: (i) to address implementation barriers according to the Consolidated Framework for Implementation Research (CFIR)27 and (ii) to integrate the transdiagnostic risk calculator into the local EHR. Implementation barriers included the communication of risk outcomes to clinicians. In SLaM, all patients are invited to register for Consent for Contact (C4C), which indicates their willingness to be contacted for research, without affecting the quality of care. This reduces the ethical issues relating to contacting patients. Further to this, working groups with clinicians aided tailoring of how this information was communicated. During the in vivo phase (May 14th 2018 to April 29th 2019), all individuals (i) older than 14 years (ii) who were accessing any SLaM service (boroughs of Lambeth, Southwark, Lewisham, Croydon), (iii) receiving a first ICD-10 index primary diagnosis of any non-organic, non-psychotic mental disorder (with the exception of Acute and Transient Psychotic Disorders; ATPD), or a CHR-P designation and (iv) with existing contact details were deemed eligible. During the in vivo phase, new patients accessing SLaM each week were automatically screened for their psychosis risk, and those with having a risk greater than a certain threshold were detected. The research team then contacted the patients&#39; responsible clinicians to discuss further recommendations and eventually suggest a further face to face assessment6. If those assessed were considered to meet CHR-P criteria, they were referred to specialist CHR-P services, such as Outreach and Support in South London (OASIS)28. This would result in improved detection of individuals prior to the onset of a psychotic disorder and provide a significant opportunity for altering the course of the disorder. Crucially, this feasibility study involved the full integration of the calculator into the local EHR system, which is the topic of the current article. The full protocol of this feasibility study, including an overview of the plan for evaluating the proposed research, details on managing data security and ethical issues, has been presented in our previous work6. The current article, as a part of the feasibility study6, selectively focuses on presenting the technical implementation of a real-time psychosis risk detection and alerting system based on the local EHR data. More specifically, the aim of this study is to investigate the technical feasibility of this risk calculator in timely detecting at-risk patients as soon as they access a secondary mental healthcare service. The full results of the feasibility study, in terms of clinicians&#39; adherence to the recommendations made by the risk calculator, will be presented separately. A comprehensive evaluation of the effectiveness of the proposed research, which requires randomized designs, is outside the scope of the current research program. To our best knowledge, this is the first method describing the implementation of a risk calculator based on live EHR data for early detection of psychosis.
Our approach to psychosis risk detection and alerting takes advantage of the CogStack platform. The CogStack platform is a lightweight, distributed, and fault-tolerant information retrieval and text-extraction platform24. This platform consists of three key components: 1) the CogStack Pipeline that uses the Java Spring Batch framework to ingest and synchronize data from a pre-defined data source (both structured and unstructured EHR data in multiple formats such as Word, PDF files and images) to a predefined data sink in real time; 2) Elasticsearch, a search engine allowing for storage and querying of the full text of EHR data, as well as providing various application programming interfaces (APIs) to embed advanced analytics into the engine; and 3) Kibana, an interactive, web-based user interface that allows users to query data in Elasticsearch, build visualization dashboards and set alerting on anomalies or other patterns of interest from data. Moreover, CogStack incorporates the ability to alert clinicians to potential problems by Email and SMS (text), allowing clinicians to receive timely notifications about at-risk patients reported by the risk calculator.
We present a model of psychosis risk detection and alerting based on ePJS at SLaM, leveraging the CogStack platform. Compared with the CRIS platform that provides a mechanism for retrospective access to de-identified health records from ePJS on a weekly basis19, the CogStack platform at SLaM enables access to an identifiable EHR in real time, bringing the alerting closer to the point-of-care and the risk prediction in a prospective design, although both the CRIS and CogStack platforms use data sourced from ePJS in SLaM. In the section that follows, we provide details of the key steps in our approach, including preparing source data from the EHR, ingesting the source data into the CogStack platform to enable full-text search via Elasticsearch, running the psychosis risk calculator using a Python daemon thread, and setting interactive visualizations and real-time risk alerting via the Kibana user interface. Any researcher who aims to build a real-time risk detection and alerting system based on EHR data can follow the approach and its reference implementation. As we shall elaborate below, the proposed method exploits open-source, lightweight techniques with high flexibility and portability. This enables the risk calculator to be run in various locations and shows a high applicability to other risk estimation algorithms. Moreover, the method works as a straightforward approach to enhance the risk detection and alerting functionalities of an EHR embedded in a general healthcare system.

<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td>CogStack-Pipeline</td>
			<td>King&#39;s College London</td>
			<td></td>
			<td>Open source software</td>
		</tr>
		<tr>
			<td>Elasticsearch</td>
			<td>Elastic NV</td>
			<td></td>
			<td>Open Source Search &#38; Analytics</td>
		</tr>
		<tr>
			<td>Kibana</td>
			<td>Elastic NV</td>
			<td></td>
			<td>Open source data visualization plugin for Elasticsearch</td>
		</tr>
		<tr>
			<td>Python packages (&#34;elasticsearch&#34;, &#34;elasticsearch_dsl&#34;, &#34;pandas&#34; and &#34;numpy&#34;)</td>
			<td>Open source community</td>
			<td></td>
			<td>Open source packages</td>
		</tr>
	</tbody>
</table>

implementation of a real-time psychosis risk detection and alerting system based on electronic health records using cogstack

Recent studies have shown that an automated, lifespan-inclusive, transdiagnostic, and clinically based, individualized risk calculator provides a powerful system for supporting the early detection of individuals at-risk of psychosis at a large scale, by leveraging electronic health records (EHRs). This risk calculator has been externally validated twice and is undergoing feasibility testing for clinical implementation. Integration of this risk calculator in clinical routine should be facilitated by prospective feasibility studies, which are required to address pragmatic challenges, such as missing data, and the usability of this risk calculator in a real-world and routine clinical setting. Here, we present an approach for a prospective implementation of a real-time psychosis risk detection and alerting service in a real-world EHR system. This method leverages the CogStack platform, which is an open-source, lightweight, and distributed information retrieval and text extraction system. The CogStack platform incorporates a set of services that allow for full-text search of clinical data, lifespan-inclusive, real-time calculation of psychosis risk, early risk-alerting to clinicians, and the visual monitoring of patients over time. Our method includes: 1) ingestion and synchronization of data from multiple sources into the CogStack platform, 2) implementation of a risk calculator, whose algorithm was previously developed and validated, for timely computation of a patient&#39;s risk of psychosis, 3) creation of interactive visualizations and dashboards to monitor patients&#39; health status over time, and 4) building automated alerting systems to ensure that clinicians are notified of patients at-risk, so that appropriate actions can be pursued. This is the first ever study that has developed and implemented a similar detection and alerting system in clinical routine for early detection of psychosis.

In this section, we present implementation results focusing in practicality in handling live clinical data streams elaborated through the risk calculator and facilitating timely delivery of prognostic results to clinicians. Evaluations of the clinical utility of the system, such as the adherence of clinicians to the recommendations made by the risk calculator, will be presented in a separate report when complete.
Ingestion of source data 
We deployed the psychosis risk calculation and alerting system based on a replica database of ePJS in SLaM. This replica database synchronizes the live data from ePJS every 10 minutes. A database view combining patients&#39; information for psychosis risk calculation was built in this replica database, where each record contains information for a patient. All records in this view were ingested into the CogStack platform in real time (approximately 0.6 microsecond per record in a virtual machine with 8-core CPU and 16GB RAM). Until 13 July 2019 when this manuscript was prepared, all the records of 202,289 patients who received a first index diagnosis of non-organic and non-psychotic mental disorder in SLaM were ingested into CogStack for psychosis risk calculation, stored in the &#34;psychosis_base&#34; Elasticsearch index. Figure 1 shows the number of records ingested into CogStack over time, in chronological order based on the last update date of a record. By comparing the numbers and content of records in the database and the Elasticsearch index, no missing and discrepant data were found, which confirms the reliability of CogStack Pipeline in data ingestion and synchronization.
Validation of risk results 
To validate the implementation of the psychosis risk detector in this protocol, we compared at-risk patients detected by CogStack (called &#34;CogStack version&#34;) with those detected by the original risk calculator based on CRIS (called &#34;CRIS version&#34;). Since there were no thresholds developed to screen an at-risk patient6,14,21, we here used a tentative threshold of 5% for the risk of psychosis in two years. Note that this tentative threshold is merely to test whether the system can pragmatically work in the NHS and is susceptible to change with future research. The actual threshold for an optimal detection of at-risk individuals will need to be identified in future large-scale studies. Specifically, we first retrieved all patients who had a risk for psychosis above the threshold in the CRIS version (the number of patients N=169). All these patients received a first index diagnosis of non-organic and non-psychotic mental disorder in SLaM from May 14th 2018 to April 29th 2019. By filtering patients who were diagnosed in the same time period, we then retrieved N=170 patients whose risk for psychosis in 2 years were higher than 0.05 in the CogStack version. Finally, we compared the difference between the two sets of patients, where the total number of unique patients in the two sets are N=173. We found that 161 patients (accounting for 93% of 173 patients) had the same scores in both versions. The high degree of agreement confirms the validity of this CogStack-based protocol in generating risk scores.
There were 12 patients having different risk scores in the two versions. By inspecting patients&#39; EHRs, we found that this difference was because data for these patients were updated after the risk scores were calculated in the CRIS version. Specifically, although predictors used in the risk calculator, such as date of birth, gender and self-assigned ethnicity, were static variables, some patients&#39; health records had a missing or default value for a variable (e.g., an unknown ethnicity) at an earlier stage and these variables were entered or updated at a later stage. This can lead to different risk scores at two different stages. Similarly, the first primary index diagnoses of some patients were invalidated after an initial risk score was calculated based on these diagnoses. In this case, the risk calculator will look for the next valid primary diagnosis for such patient and re-calculate a risk score. The updated risk score can also differ from the initial one. As the original risk calculator was developed based on retrospective data in CRIS for research use, the original calculator pipelines did not synchronize these updates in EHR data and refresh the risk scores in a timely manner. In contrast, a patient&#39;s risk score will be re-calculated in the CogStack version if any source data of the patient is updated, which allows this CogStack-based calculator to provide the most reliable and up-to-date risk scores for patients. These results strongly highlight the reliability of risk scores in this protocol.
Result visualization and risk alerting 
To demonstrate the capabilities of CogStack in data visualization, we built a dashboard for information about patients at-risk of psychosis. As used before for feasibility testing, we selected those who have a risk of psychosis in two years higher than 5% as at-risk patients. Figure 2 shows the visualizations of characteristics for patients at-risk of psychosis, including patients&#39; ethnicities, genders, ages and categories of diagnoses. Apart from visualizing risk results via Web interfaces (e.g., Kibana), this protocol allows risk alerts to be sent to users or clinicians through other notification channels such as Email. Figure 3 shows the interface for setting a risk alerting service by using the Watch component in Kibana. Once this service is configured successfully, users can receive an Email notification if there were one or more patients whose risk of psychosis in two years are higher than 5%. Figure 4 shows an example of these Email notifications, which reports the numbers of patients at-risk and these patients&#39; boroughs. Since more work is needed to tailor how the predicted psychosis risk scores are communicated, we have not sent risk notifications directly to clinicians. For testing the technological feasibility, all notifications in this study were sent from a technical researcher (T.W.) to a clinical researcher (D.O.) via the SLaM&#39;s email system within a secure network. Only an aggregated statistic of patient information was included in a notification; no any personally identifiable information was included.
<img alt="Figure 1" class="xfigimg" src="/files/ftp_upload/60794/60794fig1.jpg" /> 
Figure 1: Source data ingested into CogStack. There are 202,289 records in total ingested into the &#34;psychosis_base&#34; Elasticsearch index until 13 July 2019, and the histogram shows the numbers of records ingested over time, ordered by the last update data time of a record. One can also query both structured and unstructured information, and obtain search hits that match the query in this page. <a href="https://www.jove.com/files/ftp_upload/60794/60794fig1large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Figure 2" class="xfigimg" src="/files/ftp_upload/60794/60794fig2.jpg" /> 
Figure 2: Dashboard of characteristics of patients at-risk of psychosis (i.e., the risk of psychosis in 2 years higher than 0.05). (a) Distribution of ethnicities for patients at-risk, where outer pies are the subcategories of an ethnicity category in inner pies. (b) distribution of patients&#39; gender, (c) distribution of patients&#39; ages at diagnosis and (d) number of patients per diagnosis group. <a href="https://www.jove.com/files/ftp_upload/60794/60794fig2large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Figure 3" class="xfigimg" src="/files/ftp_upload/60794/60794fig3.jpg" /> 
Figure 3: Setting and simulating Watch for risk alerting. <a href="https://www.jove.com/files/ftp_upload/60794/60794fig3large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Figure 4" class="xfigimg" src="/files/ftp_upload/60794/60794fig4.jpg" /> 
Figure 4: An example of risk alerting Email. The numbers of patients at-risk of psychosis in each Clinical Commissioning Groups (CCG) are reported in parentheses. <a href="https://www.jove.com/files/ftp_upload/60794/60794fig4large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>

Watch this Scientific Journal Video about Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack at JoVE.com

Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack

We demonstrate how to deploy a real-time psychosis risk calculation and alerting system based on CogStack, an information retrieval and extraction platform for electronic health records.

Recent studies have shown that an automated, lifespan-inclusive, transdiagnostic, and clinically based, individualized risk calculator provides a powerful ...

implementation-real-time-psychosis-risk-detection-alerting-system

Research

JoVE Journal

Behavior

6.9K Views.  King's College London. We demonstrate how to deploy a real-time psychosis risk calculation and alerting system based on CogStack, an information retrieval and extraction platform for electronic health records.

Video: Implementation of a Real-Time Psychosis Risk Detection and Alerting System Based on Electronic Health Records using CogStack

Handwriting Analysis Indicates Spontaneous Dyskinesias in Neuroleptic Na&#239;ve Adolescents at High Risk for Psychosis

Growing evidence suggests that movement abnormalities are a core feature of psychosis. One marker of movement abnormality, dyskinesia, is a result of impaired neuromodulation of dopamine in fronto-striatal pathways. The traditional methods for identifying movement abnormalities include observer-based reports and force stability gauges. The drawbacks of these methods are long training times for raters, experimenter bias, large site differences in instrumental apparatus, and suboptimal reliability. Taking these drawbacks into account has guided the development of better&#160;standardized and more efficient procedures to examine movement abnormalities through handwriting analysis software and tablet. Individuals at&#160;risk for psychosis showed significantly more dysfluent pen movements (a proximal measure for dyskinesia) in a handwriting task. Handwriting kinematics offers a great advance over previous methods of assessing dyskinesia, which could clearly be beneficial for understanding the etiology of psychosis.

Handwriting Analysis Indicates Spontaneous Dyskinesias in Neuroleptic Na&amp;#239;ve Adolescents at High Risk for Psychosis

Handwriting analysis software significantly improves upon existing instruments measuring movement disorders. Individuals at&#160;risk for psychosis and healthy controls completed handwriting tasks to test for dyskinesia. Results suggest that youth at&#160;risk for psychosis exhibit dyskinesia and that handwriting analysis could significantly contribute to wider dissemination of early identification efforts

Growing evidence suggests that movement abnormalities are a core feature of psychosis. One marker of movement abnormality, dyskinesia, is a result of ...

Contextual and Cued Fear Conditioning Test Using a Video Analyzing System in Mice

The contextual and cued fear conditioning test is one of the behavioral tests that assesses the ability of mice to learn and remember an association between environmental cues and aversive experiences. In this test, mice are placed into a conditioning chamber and are given parings of a conditioned stimulus (an auditory cue) and an aversive unconditioned stimulus (an electric footshock). After a delay time, the mice are exposed to the same conditioning chamber and a differently shaped chamber with presentation of the auditory cue. Freezing behavior during the test is measured as an index of fear memory. To analyze the behavior automatically, we have developed a video analyzing system using the ImageFZ application software program, which is available as a free download at http://www.mouse-phenotype.org/. Here, to show the details of our protocol, we demonstrate our procedure for the contextual and cued fear conditioning test in C57BL/6J mice using the ImageFZ system. In addition, we validated our protocol and the video analyzing system performance by comparing freezing time measured by the ImageFZ system or a photobeam-based computer measurement system with that scored by a human observer. As shown in our representative results, the data obtained by ImageFZ were similar to those analyzed by a human observer, indicating that the behavioral analysis using the ImageFZ system is highly reliable. The present movie article provides detailed information regarding the test procedures and will promote understanding of the experimental situation.

This article presents a protocol for a contextual and cued fear conditioning test using a video analyzing system to assess fear learning and memory in mice.

The contextual and cued fear conditioning test is one of the behavioral tests that assesses the ability of mice to learn and remember an association ...

Assessment of Murine Exercise Endurance Without the Use of a Shock Grid: An Alternative to Forced Exercise

Using laboratory mouse models, the molecular pathways responsible for the metabolic benefits of endurance exercise are beginning to be defined. The most common method for assessing exercise endurance in mice utilizes forced running on a motorized treadmill equipped with a shock grid. Animals who quit running are pushed by the moving treadmill belt onto a grid that delivers an electric foot shock; to escape the negative stimulus, the mice return to running on the belt. However, avoidance behavior and psychological stress due to use of a shock apparatus can interfere with quantitation of running endurance, as well as confound measurements of postexercise serum hormone and cytokine levels. Here, we demonstrate and validate a refined method to measure running endurance in na&#239;ve C57BL/6 laboratory mice on a motorized treadmill without utilizing a shock grid. When mice are preacclimated to the treadmill, they run voluntarily with gait speeds specific to each mouse. Use of the shock grid is replaced by gentle encouragement by a human operator using a tongue depressor, coupled with sensitivity to the voluntary willingness to run on the part of the mouse. Clear endpoints for quantifying running time-to-exhaustion for each mouse are defined and reflected in behavioral signs of exhaustion such as splayed posture and labored breathing. This method is a humane refinement which also decreases the confounding effects of stress on experimental parameters.

A method to assess exercise endurance in laboratory mice without the use of a shock grid is demonstrated. This method is a humane refinement that can decrease the confounding effects of stress on experimental parameters.

Using laboratory mouse models, the molecular pathways responsible for the metabolic benefits of endurance exercise are beginning to be defined. The most ...

Design and Implementation of an fMRI Study Examining Thought Suppression in Young Women with, and At-risk, for Depression

Ruminative brooding is associated with increased vulnerability to major depression. Individuals who regularly ruminate will often try to reduce the frequency of their negative thoughts by actively suppressing them. We aim to identify the neural correlates underlying thought suppression in at-risk and depressed individuals. Three groups of women were studied; a major depressive disorder group, an at-risk group (having a first degree relative with depression) and controls. Participants performed a mixed block-event fMRI paradigm involving thought suppression, free thought and motor control periods. Participants identified the re-emergence of &#8220;to-be-suppressed&#8221; thoughts (&#8220;popping&#8221; back into conscious awareness) with a button press. During thought suppression the control group showed the greatest activation of the dorsolateral prefrontal cortex, followed by the at-risk, then depressed group. During the re-emergence of intrusive thoughts compared to successful re-suppression of those thoughts, the control group showed the greatest activation of the anterior cingulate cortices, followed by the at-risk, then depressed group. At-risk participants displayed anomalies in the neural regulation of thought suppression resembling the dysregulation found in depressed individuals. The predictive value of these changes in the onset of depression remains to be determined.

We aim to identify the neural correlates underlying sustained and transient thought suppression, and thought re-emergence in controls, at-risk and depressed individuals. Activation was greatest for controls compared to the at-risk and the depressed group in the dorsolateral prefrontal cortex during thought suppression and anterior cingulate cortex during thought re-emergence.

Ruminative brooding is associated with increased vulnerability to major depression. Individuals who regularly ruminate will often try to reduce the ...

Feeding Experimentation Device (FED): Construction and Validation of an Open-source Device for Measuring Food Intake in Rodents

Food intake measurements are essential for many research studies. Here, we provide a detailed description of a novel solution for measuring food intake in mice: the Feeding Experimentation Device (FED). FED is an open-source system that was designed to facilitate flexibility in food intake studies. Due to its compact and battery powered design, FED can be placed within standard home cages or other experimental equipment. Food intake measurements can also be synchronized with other equipment in real-time via FED's transistor-transistor logic (TTL) digital output, or in post-acquisition processing as FED timestamps every event with a real-time clock. When in use, a food pellet sits within FED's food well where it is monitored via an infrared beam. When the pellet is removed by the mouse, FED logs the timestamp onto its internal secure digital (SD) card and dispenses another pellet. FED can run for up to 5 days before it is necessary to charge the battery and refill the pellet hopper, minimizing human interference in data collection. Assembly of FED requires minimal engineering background, and off-the-shelf materials and electronics were prioritized in its construction. We also provide scripts for analysis of food intake and meal patterns. Finally, FED is open-source and all design and construction files are online, to facilitate modifications and improvements by other researchers.

Feeding Experimentation Device FED: Construction and Validation of an Open-source Device for Measuring Food Intake in Rodents

Feeding Experimentation Device (FED) is an open-source device for measuring food intake in mice. FED can also synchronize food intake measurements with other techniques via a real-time digital output. Here, we provide a step-by-step tutorial for the construction, validation, and usage of FED.

Food intake measurements are essential for many research studies. Here, we provide a detailed description of a novel solution for measuring food intake in ...

Reducing State Anxiety Using Working Memory Maintenance

The purpose of this protocol is to explain how to examine the relationship between working memory processes and anxiety by combining the Sternberg Working Memory (WM) and the threat of shock paradigms. In the Sternberg WM paradigm, subjects are required to maintain a series of letters in the WM for a brief interval and respond by identifying whether the position of a given letter in the series matches a numerical prompt. In the threat of shock paradigm, subjects are exposed to alternating blocks where they are either at risk of receiving unpredictable presentations of a mild electric shock or are safe from the shock. Anxiety is probed throughout the safe and threat blocks using the acoustic startle reflex, which is potentiated under threat (Anxiety-Potentiated Startle (APS)). By conducting the Sternberg WM paradigm during the threat of shock and probing the startle response during either the WM maintenance interval or the intertrial interval, it is possible to determine the effect of WM maintenance on APS.

This protocol demonstrates how to measure anxiety-potentiated startle during the Sternberg Working Memory paradigm.

The purpose of this protocol is to explain how to examine the relationship between working memory processes and anxiety by combining the Sternberg Working ...

Noninvasive, High-throughput Determination of Sleep Duration in Rodents

Traditionally, sleep is monitored by an electroencephalogram (EEG). EEG studies in rodents require surgical implantation of the electrodes followed by a long recovery period. To perform an EEG recording, the animal is connected to a receiver, creating an unnatural tether to the head-mount. EEG monitoring is time consuming, carries risk to the animal, and is not a completely natural setting for the measurement of sleep. Alternative methods to detect sleep, particularly in a high-throughput fashion, would greatly advance the field of sleep research. Here, we describe a validated method for detecting sleep via activity-based home-cage monitoring. Previous studies have shown that sleep assessed via this method has a high degree of agreement with sleep defined by traditional EEG-based measures. Whereas this method is validated for total sleep time, it is important to note that sleep bout duration should be assessed by an EEG which has better temporal resolution. The EEG can also differentiate rapid eye movement (REM) and non-REM sleep, giving more detail about the exact nature of sleep. Nevertheless, activity-based sleep determination can be used to analyze multiple days of undisturbed sleep and to assess sleep as a response to an acute event (like stress). Here, we show the power of this system to detect the response of mice to daily intraperitoneal injections.

We describe a high-throughput method of measuring sleep by means of activity-based home-cage monitoring. This method offers advantages over traditional EEG-based methods. It is well validated for the determination of total sleep duration and can be a powerful tool to monitor sleep in rodent models of human disease.

Traditionally, sleep is monitored by an electroencephalogram (EEG). EEG studies in rodents require surgical implantation of the electrodes followed by a ...

Using a Classroom-Based Deese Roediger McDermott Paradigm to Assess the Effects of Imagery on False Memories

Associated word list procedures can elicit false memories in predictable ways by inducing associative processing, thus making it harder to monitor the accuracy of memories. The purpose of the method presented here was to induce false memories using lists of either semantically or phonologically related words and to assess the effects of imagery instructions on the recall and recognition of those false memories. To do this, we used a modified version of the Deese Roediger McDermott (DRM) paradigm. We adapted word lists from previous DRM studies to suit imagery procedures and created an automated presentation to present the word lists in classroom settings. We then recruited undergraduate classes and instructed some of the classes to create mental images of the list words as they were being presented, while instructing the other classes to simply remember the words. The automated presentation presented word lists to participants, one word at a time, alternating between phonologically and semantically related lists. Participants used paper-pencil recall packets to immediately recall list items, complete a distractor activity, and take a subsequent final recognition test. Often, participants immediately recalled and later recognized words that were related to the list items but were not actually presented; these are known as critical lures and indicate a false memory. The protocol detailed here describes a four-step procedure - list presentation, immediate recall, distractor phase, and final recognition - that can assess the effects of list type and imagery instruction within the DRM paradigm on memory. The automated nature of the list presentation provides the ability to systematically vary variables of interest, and the paper and pencil method of data collection affords an easily accessible method for collecting data in classroom settings. The protocol also offers options to modify the procedure to a more traditional DRM paradigm without imagery and/or list type manipulations. The use of this protocol can provide results relevant to both classroom learning and cognitive science principles.

The method presented here induced false memories using lists of related words and also assessed the effects of imagery instructions on the recall and recognition of those false memories. This protocol details a modified version of the Deese Roediger McDermott (DRM) paradigm.

Associated word list procedures can elicit false memories in predictable ways by inducing associative processing, thus making it harder to monitor the ...

Using the FishSim Animation Toolchain to Investigate Fish Behavior: A Case Study on Mate-Choice Copying In Sailfin Mollies

Over the last decade, employing computer animations for animal behavior research has increased due to its ability to non-invasively manipulate the appearance and behavior of visual stimuli, compared to manipulating live animals. Here, we present the FishSim Animation Toolchain, a software framework developed to provide researchers with an easy-to-use method for implementing 3D computer animations in behavioral experiments with fish. The toolchain offers templates to create virtual 3D stimuli of five different fish species. Stimuli are customizable in both appearance and size, based on photographs taken of live fish. Multiple stimuli can be animated by recording swimming paths in a virtual environment using a video game controller. To increase standardization of the simulated behavior, the prerecorded swimming path may be replayed with different stimuli. Multiple animations can later be organized into playlists and presented on monitors during experiments with live fish.
In a case study with sailfin mollies (Poecilia latipinna), we provide a protocol on how to conduct a mate-choice copying experiment with FishSim. We utilized this method to create and animate virtual males and virtual model females, and then presented these to live focal females in a binary choice experiment. Our results demonstrate that computer animation may be used to simulate virtual fish in a mate-choice copying experiment to investigate the role of female gravid spots as an indication of quality for a model female in mate-choice copying.
Applying this method is not limited to mate-choice copying experiments but can be used in various experimental designs. Still, its usability depends on the visual capabilities of the study species and first needs validation. Overall, computer animations offer a high degree of control and standardization in experiments and bear the potential to &#39;reduce&#39; and &#39;replace&#39; live stimulus animals as well as to &#39;refine&#39; experimental procedures.

Using the FishSim Animation Toolchain to Investigate Fish Behavior: A Case Study on Mate-Choice Copying In Sailfin Mollies

Using the novel FishSim Animation Toolchain, we present a protocol for non-invasive visual manipulation of public information in the context of mate-choice copying in sailfin mollies. FishSim Animation Toolchain provides an easy-to-use framework for the design, animation and presentation of computer-animated fish stimuli for behavioral experiments with live test fish.

Over the last decade, employing computer animations for animal behavior research has increased due to its ability to non-invasively manipulate the ...

Using a Real-Time Locating System to Measure Walking Activity Associated with Wandering Behaviors Among Institutionalized Older Adults

A real-time locating system (RTLS) can be used to track the walking activity of institutionalized older adults in long-term care who are at risk for wandering behaviors. The benefits of a RTLS are objective and continuous measurements of activity. Self-report methods of activity, especially wandering, by health care staff are vulnerable to floor effects and recall bias, and continuous clinical or research observation over the long-term can be time-consuming and expensive. Health care staff also fail to recognize the onset and/or duration of wandering behaviors, which are associated with a variety of adverse health outcomes in this population but amenable to intervention. RTLS technologies can measure the walking activity of institutionalized residents with cognitive impairment over time with a high degree of accuracy. This is particularly useful for the study of wandering, defined as walking for at least 60 seconds with few (if any) breaks in activity. Wandering is associated with disease progression, hospitalizations, falls and death. Previous work suggests older adults with poor balance ability and high sustained walking activity may be particularly susceptible to poor health outcomes. RTLS&#39;s are used to assess cognitive impairment and factors associated with gait and balance; however, supplemental paper and pencil gait/balance tools may be used to further refine risk profiles. This project discusses the use of a RTLS to measure walking activity and also gait quality and balance ability measures on this population.

This paper discusses the use of a continuous and objective real-time locating system to measure walking activity associated with wandering behaviors, focusing on older adults with cognitive impairment. Walking activity is measured by walking distance, sustained walking distance, and sustained gait speed. Also assessed are gait quality and balance ability.

A real-time locating system (RTLS) can be used to track the walking activity of institutionalized older adults in long-term care who are at risk for ...