This work was facilitated by several open-source libraries, most notably llama-index (https://www.llamaindex.ai/), ChromaDB (https://www.trychroma.com/), and LMQL (https://lmql.ai/).

In the use case demonstrated in this paper, the vector store was generated using published guidelines from the Chicago Consensus Working Group17. This expert group was established to develop guidelines for the management of peritoneal cancers. The subject area was chosen as it is within the investigators&#39; area of clinical expertise. The set of papers was accessed from online journal repositories including Cancer and the Annals of Surgical Oncology. A compact (33.4M parameters) embedding model created by the Beijing Academy for Artificial Intelligence (BAAI, https://www.baai.ac.cn/english.html), bge-small-en, was used to generate embeddings from source documents. The resulting database was then used to augment Llama 2 and Open-AI foundation models7. For the reader&#39;s convenience, the code is made available through GitHub (https://github.com/AnaiLab/AugmentedLLM). To ensure replicability, it is recommended to use the same versions of libraries used in the provided requirements list as well as the same version of Python. Additional details on installation or documentation regarding tools used in the following methods can be located at the official websites of the providers for Python (https://www.python.org), git (https://git-scm.com), Llama-Index (https://llamaindex.ai), and Chroma (https://trychroma.com).
1. Prerequisites: Review code and install required libraries
<ol>
	<li>Verify that git, python, and pip are installed.
	<ol>
		<li>In a terminal, run the following commands to verify the installation: 
		git --version 
		python3 --version 
		pip3 --version</li>
	</ol>
	</li>
	<li>Check the code and install requirements.
	<ol>
		<li>In a terminal, run the following commands: 
		git clone&#160;https://github.com/AnaiLab/AugmentedLLM.git 
		cd ./AugmentedLLM/ 
		pip3 install -r requirements.txt</li>
	</ol>
	</li>
</ol>
2. Creation of a vector database with Llama-Index
<ol>
	<li>Convert RTF file formats (only required if files are in an RTF format).
	<ol>
		<li>Edit the file titled config.py, replacing the file paths in the following code with the location of the RTF articles to be converted and the location to which the plain text files are to be written by typing the following commands. Save the file. 
		rtf_file_dir = &#39;./articles_rtf/&#39; 
		converted_file_dir = &#39;./articles_converted/&#39;</li>
		<li>In a terminal, in the same directory, execute the code to generate plain text versions of RTF files by running the following command: 
		python3 ./convert_rtf.py</li>
	</ol>
	</li>
	<li>Create and save the vector database.
	<ol>
		<li>Edit the config.py file, replacing the value of the following variable with the file path of the folder containing documents with which the LLM is to be augmented; save the file. 
		article_dir = &#39;./articles/&#39;</li>
		<li>In a terminal, in the same directory, execute the code with the following command to create and persist the database. Verify the database is now saved in the vector_db folder. 
		python3 ./build_index.py</li>
	</ol>
	</li>
</ol>
3. Augmentation of a Llama model with vector database generated in section 2
<ol>
	<li>Instantiate custom LLM Locally (optional) 
	NOTE: Performing this step is only required if you would like to use a model other than the default Llama-2-7B. If you wish to use the default model, proceed to step 3.2.
	<ol>
		<li>(Using a custom LLM) Specify an LLM to augment by editing run_augmented_llm.py and passing a llama-index LLM object in the constructor as the llm parameter in the following lines of code rather than None. 
		augmentedLLM = AugmentedLLM(vector_store, llm=None)</li>
	</ol>
	</li>
	<li>Query augmented LLM
	<ol>
		<li>Run the following command in the terminal: 
		python3 ./run_augmented_llm.py</li>
		<li>Run user queries to get a response augmented by data in the set of manuscripts (Figure 3 and Figure 4). Press CTRL + C to exit when finished.</li>
	</ol>
	</li>
</ol>
4. Programmatic comparison of alternative LLMs
<ol>
	<li>Create MCQs.
	<ol>
		<li>Edit the file questions.py, taking note of format of the examples. Add questions following a similar format. Save the file.</li>
	</ol>
	</li>
	<li>Connect to GPT-3.5, GPT-4, OpenChat, or other comparator models via API.
	<ol>
		<li>Edit the config.py file, adding the API key for OpenAI or Huggingface if the objective is to benchmark against models from either provider. Save the file. 
		huggingface_key = &#39;&#39; 
		openai_key = &#39;&#39;</li>
		<li>Edit the compare_llms.py file, and choose the set of models to test against by uncommenting (deleting the &#39;# &#39; characters at the begging of that line) models to compare against. 
		NOTE: Some comparators require an API key as set in step 4.2.1. Additionally, edit the output_dir parameter to change where LLM output is stored if desired; otherwise, a default will be used. 
		output_dir = &#39;./llm_responses/&#39;</li>
		<li>In a terminal, execute the code with the following command. After execution, view the model responses in the folder specified in step 4.2.2 for grading or other review. 
		python3 ./compare_llms.py</li>
	</ol>
	</li>
	<li>(Optional) Experiment with automated grading of MCQ responses. The example code uses the LMQL library to constrain LLM output into an expected format.
	<ol>
		<li>Open the file automated_comparison.py and similarly to step 4.2.2, uncomment models to be included, edit the output_dir variable, or otherwise customize. Note that the model output will still be saved in a similar fashion. Save the file.</li>
		<li>Run the code from step 4.3.1 by running the following command in a terminal: 
		python3 ./automated_comparison.py</li>
	</ol>
	</li>
</ol>

Embedding Methods to Integrate Scientific Manuscripts into Large Language Models

<ol>
	<li>Singhal, K., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Large+language+models+encode+clinical+knowledge.">Large language models encode clinical knowledge.</a> Nature. 620 (7972), 172-180 (2023).</li><li>Gilson, A., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=How+does+ChatGPT+perform+on+the+United+States+medical+licensing+examination?+The+implications+of+large+language+models+for+medical+education+and+knowledge+assessment.">How does ChatGPT perform on the United States medical licensing examination? The implications of large language models for medical education and knowledge assessment.</a> JMIR Med Educ. 9 (1), e45312 (2023).</li><li>Guerra, G. A., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=GPT-4+artificial+intelligence+model+outperforms+ChatGPT,+medical+students,+and+neurosurgery+residents+on+neurosurgery+written+board-like+questions.">GPT-4 artificial intelligence model outperforms ChatGPT, medical students, and neurosurgery residents on neurosurgery written board-like questions.</a> World Neurosurg. 179, e160-e165 (2023).</li><li>. Would Chat GPT3 get a Wharton MBA? A prediction based on its performance in the Operations Management course Available from: <a target="_blank" href="https://mackinstitute.wharton.upenn.edu/wp-content/uploads/2023/01/Christian-Terwiesch-Chat-GTP.pdf">https://mackinstitute.wharton.upenn.edu/wp-content/uploads/2023/01/Christian-Terwiesch-Chat-GTP.pdf</a> (2023)</li><li>Weizenbaum, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=ELIZA-a+computer+program+for+the+study+of+natural+language+communication+between+man+and+machine.">ELIZA-a computer program for the study of natural language communication between man and machine.</a> Communications of the ACM. 9 (1), 36-45 (1966).</li><li>Wu, T., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+brief+overview+of+ChatGPT:+The+history,+status+quo+and+potential+future+development.">A brief overview of ChatGPT: The history, status quo and potential future development.</a> IEEE/CAA Journal of Automatica Sinica. 10 (5), 1122-1136 (2023).</li><li>Huang, L., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+survey+on+hallucination+in+large+language+models:+Principles,+taxonomy,+challenges,+and+open+questions.">A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions.</a> arXiv [cs.CL]. , (2023).</li><li>Thirunavukarasu, A. J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Large+language+models+in+medicine.">Large language models in medicine.</a> Nature Med. 29 (8), 1930-1940 (2023).</li><li>Ram, O., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=In-Context+retrieval-augmented+language+models.">In-Context retrieval-augmented language models.</a> Trans Assoc Comput Linguist. 11, 1316-1331 (2023).</li><li>. Sequence distance embeddings Available from: <a target="_blank" href="https://wrap.warwick.ac.uk/61310/7/WRAP_THESIS_Cormode_2003.pdf">https://wrap.warwick.ac.uk/61310/7/WRAP_THESIS_Cormode_2003.pdf</a> (2003)</li><li>Chicago Consensus Working Group. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+Chicago+Consensus+Guidelines+for+peritoneal+surface+malignancies:+Introduction.">The Chicago Consensus Guidelines for peritoneal surface malignancies: Introduction.</a> Cancer. 126 (11), 2510-2512 (2020).</li><li>Dodge, J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Measuring+the+carbon+intensity+of+AI+in+cloud+instances.">Measuring the carbon intensity of AI in cloud instances.</a> FACCT 2022. , (2022).</li><li>Khene, Z. -. E., Bigot, P., Mathieu, R., Roupr&#234;t, M., Bensalah, K. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Development+of+a+personalized+chat+model+based+on+the+European+association+of+urology+oncology+guidelines:+Harnessing+the+power+of+generative+artificial+intelligence+in+clinical+practice.">Development of a personalized chat model based on the European association of urology oncology guidelines: Harnessing the power of generative artificial intelligence in clinical practice.</a> Eur Urol Oncol. 7 (1), 160-162 (2024).</li><li>Kresevic, S., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Optimization+of+hepatological+clinical+guidelines+interpretation+by+large+language+models:+a+retrieval+augmented+generation-based+framework.">Optimization of hepatological clinical guidelines interpretation by large language models: a retrieval augmented generation-based framework.</a> NPJ Digit Med. 7 (1), 102 (2024).</li><li>Ge, J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Development+of+a+liver+disease-specific+large+language+model+chat+interface+using+retrieval+augmented+generation.">Development of a liver disease-specific large language model chat interface using retrieval augmented generation.</a> medRxiv. , (2023).</li><li>Panagoulias, D. P., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Rule-augmented+artificial+intelligence-empowered+systems+for+medical+diagnosis+using+large+language+models.">Rule-augmented artificial intelligence-empowered systems for medical diagnosis using large language models.</a> , 70-77 (2023).</li><li>Panagoulias, D. P., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Augmenting+large+language+models+with+rules+for+enhanced+domain-specific+interactions:+The+case+of+medical+diagnosis.">Augmenting large language models with rules for enhanced domain-specific interactions: The case of medical diagnosis.</a> Electronics. 13 (2), 320 (2024).</li><li>Bommasani, R., Liang, P., Lee, T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Holistic+evaluation+of+language+models.">Holistic evaluation of language models.</a> Ann N Y Acad Sci. 1525 (1), 140-146 (2023).</li><li>Papineni, K., Roukos, S., Ward, T., Zhu, W. -. J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Bleu.">Bleu.</a> Proceedings of the 40th Annual Meeting on Association for Computational Linguistics - ACL '02. , (2001).</li><li>Johnson, D., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Assessing+the+accuracy+and+reliability+of+AI-generated+medical+responses:+An+evaluation+of+the+chat-GPT+model.">Assessing the accuracy and reliability of AI-generated medical responses: An evaluation of the chat-GPT model.</a> Res Sq. , (2023).</li><li>Lin, C. -. Y. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=ROUGE:+A+package+for+automatic+evaluation+of+summaries.">ROUGE: A package for automatic evaluation of summaries.</a> Text Summarization Branches Out. , 74-81 (2004).</li><li>Chen, S., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Evaluating+the+ChatGPT+family+of+models+for+biomedical+reasoning+and+classification.">Evaluating the ChatGPT family of models for biomedical reasoning and classification.</a> J Am Med Inform Assoc. 31 (4), 940-948 (2024).</li></ol>

The authors have no conflicts of interest to declare.

The methods provided here aim to facilitate the research of domain-specific applications of LLMs without the need for de novo training or extensive fine-tuning. As LLMs are becoming an area of significant research interest, approaches for augmenting knowledge bases and improving the accuracy of responses will become increasingly important18,19,20,21. As demonstrated in the provided res...

Large language models (LLMs) such as OpenAI&#39;s ChatGPT or Meta AI&#39;s Llama have rapidly become a popular resource for generating text relevant to a user prompt. Originally functioning to predict the next lexical items in a sequence, these models have evolved to understand context, encode clinical information, and demonstrate high performance on a variety of tasks1,2,3,4. Though language models predate such capabilities and their current level of popularity by decades5, recent advances in deep learning and computing capabilities have made pretrained, high-quality commercial LLMs broadly available to users via web-based technologies and application program interfaces (APIs)6. However, there are several notable limitations to consuming LLMs in this format.
Challenge 1: Static training corpus 
LLMs are trained on an enormous (e.g., two trillion tokens in the case of Llama 27) but static body of text data. This poses a challenge to producing accurate responses pertaining to fields undergoing rapid development or changing literature. In this static approach, LLMs would require frequent retraining to keep up with the latest data, which is neither practical nor scalable. Moreover, prompts that require responses based on information not present in the training data may prevent useful text generation or lead to hallucinations8. Instances of hallucinations or fact fabrication raise significant concerns about the reliability of LLMs, particularly in settings where the accuracy of information is critical9.
Challenge 2: Lack of domain specificity 
Pretrained models are often created for general use, while users may require a model specifically optimized for performance in a particular domain. Additionally, the computational resources and data required for training a model de novo or performing significant fine-tuning are prohibitive to many users.
Challenge 3: Lack of privacy 
Users pursuing applications involving sensitive data or proprietary information may be unwilling or unable to use certain LLM services as they lack information on how data may be stored or utilized.
Challenge 4: Lack of guaranteed stability 
Services with proprietary LLMs may change available models or alter behavior at any time, making stability a concern for the implementation of applications relying on these services.
Retrieval-augmented generation (RAG) is a technique developed to improve LLM performance, particularly on queries related to information outside the model&#39;s training corpus10,11. These systems augment LLMs by incorporating contextual information to be considered when generating a response to a user query. Various recent works have described applications of RAG systems and their potential advantages12,13,14.
The goal of the method outlined in this work is to demonstrate the construction of such a system and provide a framework for researchers to rapidly experiment on domain-specific, augmented LLMs. This method is applicable for users seeking to augment an LLM with an external text-based data source. Specifically, an overarching aim of this protocol is to provide step-by-step code that is extensible to a variety of practical LLM and RAG experiments without the need for significant technical expertise in the language-modeling domain, though a working knowledge of Python is required to apply this approach without modification. To maximize user control, transparency, portability, and affordability of solutions, open-source, publicly available tools are utilized. The proposed system addresses the previously stated issues in the following ways:
Solutions 1 and 2: Static training corpus and lack of domain specificity 
The provided methodology leverages a RAG approach, utilizing embeddings to supply domain-specific information not included in the original training data. At a high level, embedding models transform text or other data into a representation as a vector or single-dimension array of numbers. This technique is beneficial as it converts semantic information contained in text to a dense, numeric form. By projecting a user query into the same embedding space, various algorithms can be used to calculate the distance15, and therefore, approximate semantic similarity, between the user query and sections of text documents. Thus, creating a database of such vectors from documents broken into discrete sections can facilitate searching over a significant number of documents for text most relevant to a user query (Figure 1). This approach is extensible to any text document. While other approaches, such as online search capabilities, are beginning to be implemented to augment LLMs, this approach allows users to choose sources considered of sufficiently high quality for their use case.
Solution 2: Lack of privacy 
In this implementation, a secure cloud environment was used for hosting, with no user prompts, generated responses, or other data leaving this ecosystem. All code is written in a platform-agnostic manner, however, to ensure that another cloud provider or local hardware may be substituted.
Solution 3: Lack of guaranteed stability 
This approach utilizes open-source libraries and focuses on augmenting LLMs with publicly available weights, allowing a higher degree of transparency, stability, and versioning, if required.
A full schematic of our proposed system is shown in Figure 2, and detailed instructions on replicating this or a similar system is outlined in the protocol section. An additional consideration when altering model behavior through fine-tuning or augmentation is the evaluation of performance. In language-generating models, this presents a unique challenge as many traditional machine learning metrics are not applicable. Though a variety of techniques exist16, in this study, expert-written multiple-choice questions (MCQs) were used to assess accuracy and compare performance pre- and post-augmentation as well as against popular alternative LLMs.

<table><tbody><tr><td>pip3 version 22.0.2&amp;nbsp;</td><td></td><td></td><td></td></tr><tr><td>Python version 3.10.12</td><td></td><td></td><td></td></tr></tbody></table>

augmenting large language models via vector embeddings to improve domain-specific responsiveness

Large language models (LLMs) have emerged as a popular resource for generating information relevant to a user query. Such models are created through a resource-intensive training process utilizing an extensive, static corpus of textual data. This static nature results in limitations for adoption in domains with rapidly changing knowledge, proprietary information, and sensitive data. In this work, methods are outlined for augmenting general-purpose LLMs, known as foundation models, with domain-specific information using an embeddings-based approach for incorporating up-to-date, peer-reviewed scientific manuscripts. This is achieved through open-source tools such as Llama-Index and publicly available models such as Llama-2 to maximize transparency, user privacy and control, and replicability. While scientific manuscripts are used as an example use case, this approach can be extended to any text data source. Additionally, methods for evaluating model performance following this enhancement are discussed. These methods enable the rapid development of LLM systems for highly specialized domains regardless of the comprehensiveness of information in the training corpus.

A set of 22 publications from the Chicago Consensus Working Group management guidelines were used to augment the base Llama-7b model17. The documents were converted into a vector index using the tool Llama-Index to generate Llama-2-7b-CCWG-Embed. Popular OpenAI models such as GPT-3.5 and GPT-4 were also augmented in a similar fashion to produce GPT-XX-CCWG-Embed models. A total of 20 multiple choice questions (MCQ) were developed to assess knowledge related to the management of a variety of perito...

Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

In this protocol, foundation large language model response quality is improved via augmentation with peer-reviewed, domain-specific scientific articles through a vector embedding mechanism. Additionally, code is provided to aid in performance comparison across large language models.

Large language models (LLMs) have emerged as a popular resource for generating information relevant to a user query. Such models are created through a ...

augmenting-large-language-models-via-vector-embeddings-to-improve

Research

JoVE Journal

Medicine

461 Views.  Medical College of Wisconsin. In this protocol, foundation large language model response quality is improved via augmentation with peer-reviewed, domain-specific scientific articles through a vector embedding mechanism. Additionally, code is provided to aid in performance comparison across large language models.

Video: Augmenting Large Language Models via Vector Embeddings to Improve Domain-Specific Responsiveness

Transcranial Direct Current Stimulation (tDCS) of Wernicke's and Broca's Areas in Studies of Language Learning and Word Acquisition

Language is a highly important yet poorly understood function of the human brain. While studies of brain activation patterns during language comprehension are abundant, what is often critically missing is causal evidence of brain areas&#39; involvement in a particular linguistic function, not least due to the unique human nature of this ability and a shortage of neurophysiological tools to study causal relationships in the human brain noninvasively. Recent years have seen a rapid rise in the use of transcranial direct current stimulation (tDCS) of the human brain, an easy, inexpensive and safe noninvasive technique that can modulate the state of the stimulated brain area (putatively by shifting excitation/inhibition thresholds), enabling a study of its particular contribution to specific functions. While mostly focusing on motor control, the use of tDCS is becoming more widespread in both basic and clinical research on higher cognitive functions, language included, but the procedures for its application remain variable. Here, we describe the use of tDCS in a psycholinguistic word-learning experiment. We present the techniques and procedures for application of cathodal and anodal stimulation of core language areas of Broca and Wernicke in the left hemisphere of the human brain, describe the procedures of creating balanced sets of psycholinguistic stimuli, a controlled yet naturalistic learning regime, and a comprehensive set of techniques to assess the learning outcomes and tDCS effects. As an example of tDCS application, we show that cathodal stimulation of Wernicke&#39;s area prior to a learning session can impact word learning efficiency. This impact is both present immediately after learning and, importantly, preserved over longer time after the physical effects of stimulation wear off, suggesting that tDCS can have long-term influence on linguistic storage and representations in the human brain.

Transcranial Direct Current Stimulation tDCS of Wernicke's and Broca's Areas in Studies of Language Learning and Word Acquisition

Here, we describe a protocol for using transcranial direct current stimulation for psycho- and neurolinguistic experiments aimed at studying, in a naturalistic yet fully controlled way, the role of cortical areas of the human brain in word learning, and a comprehensive set of behavioral procedures for assessing the outcomes.

Language is a highly important yet poorly understood function of the human brain. While studies of brain activation patterns during language comprehension ...

Behavior

Lexical Decision Task for Studying Written Word Recognition in Adults with and without Dementia or Mild Cognitive Impairment

Older adults are slower at recognizing visual objects than younger adults. The same is true for recognizing that a letter string is a real word. People with Alzheimer&#39;s disease (AD) or Mild Cognitive Impairment (MCI) demonstrate even longer responses in written word recognition than elderly controls. Despite the general tendency towards slower recognition in aging and neurocognitive disorders, certain characteristics of words influence word recognition speed regardless of age or neuropathology (e.g., a word&#8217;s frequency of use). We present here a protocol for examining the influence of lexical characteristics on word recognition response times in a simple lexical decision experiment administered to younger and older adults and people with MCI or AD. In this experiment, participants are asked to decide as quickly and accurately as possible whether a given letter string is an actual word or not. We also describe mixed-effects models and principal components analysis that can be used to detect the influence of different types of lexical variables or individual characteristics of participants on word recognition speed.

This article describes how to implement a simple lexical decision experiment to assess written word recognition in neurologically healthy participants and in individuals with dementia and cognitive decline. We also provide a detailed description of reaction time analysis using principal components analysis (PCA) and mixed-effects modeling.

Older adults are slower at recognizing visual objects than younger adults. The same is true for recognizing that a letter string is a real word. People ...

EEG Study on Recognizing Familiar Speakers in Expressive Speech

Memorization-Based Training and Testing Paradigm for Robust Vocal Identity Recognition in Expressive Speech Using Event-Related Potentials Analysis

Recognizing familiar speakers from vocal streams is a fundamental aspect of human verbal communication. However, it remains unclear how listeners can still discern the speaker's identity in expressive speech. This study develops a memorization-based individual speaker identity recognition approach and an accompanying electroencephalogram (EEG) data analysis pipeline, which monitors how listeners recognize familiar speakers and tell unfamiliar ones apart. EEG data captures online cognitive processes during new versus old speaker distinction based on voice, offering a real-time measure of brain activity, overcoming limits of reaction times and accuracy measurements. The paradigm comprises three steps: listeners establish associations between three voices and their names (training); listeners indicate the name corresponding to a voice from three candidates (checking); listeners distinguish between three old and three new speaker voices in a two-alternative forced-choice task (testing). The speech prosody in testing was either confident or doubtful. EEG data were collected using a 64-channel EEG system, followed by preprocessing and imported into RStudio for ERP and statistical analysis and MATLAB for brain topography. Results showed an enlarged late positive component (LPC) was elicited in the old-talker compared to the new-talker condition in the 400-850 ms window in the Pz and other wider range of electrodes in both prosodies. Yet, the old/new effect was robust in central and posterior electrodes for doubtful prosody perception, whereas the anterior, central, and posterior electrodes are for confident prosody condition. This study proposes that this experiment design can serve as a reference for investigating speaker-specific cue-binding effects in various scenarios (e.g., anaphoric expression) and pathologies in patients like phonagnosia.

Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception

The study introduces a training-testing paradigm to investigate old/new effects of event-related potentials in confident and doubtful prosodic scenarios. Data reveals an enhanced late positive component between 400-850 ms at Pz and other electrodes. This pipeline can explore factors beyond speech prosody and their influence on cue-binding target identification.

Recognizing familiar speakers from vocal streams is a fundamental aspect of human verbal communication. However, it remains unclear how listeners can ...

Prosodic-Acoustic Features and L2 Speech Production for Forensic Applications

Foreign Accent and Forensic Speaker Identification in Voice Lineups: The Influence of Acoustic Features Based on Prosody

This research aims to examine both the prosodic-acoustic features and the perceptual correlates of foreign-accented English and foreign-accented Brazilian Portuguese and check how the speakers&#39; productions of foreign and native accents are correlated to the listeners&#39; perception. In the Methodology, we conducted a speech production procedure with a group of American speakers of L2 Brazilian Portuguese and a group of Brazilian speakers of L2 English, and a speech perception procedure in which we performed voice lineups for both languages.For the speech production statistical analysis, we ran Generalized Additive Models to evaluate the effect of the language groups on each class (metric or prosodic-acoustic) of features controlled for the smoothing effect of the covariate(s) of the opposite class. For the speech perception statistical analysis, we ran a Kruskal-Wallis test and a post-hoc Dunn&#39;s test to evaluate the effect of the voices of the lineups on the scores judged by the listeners. We nevertheless conducted acoustic (voice) similarity tests based on Cosine and Euclidean distances. Results showed significant acoustic differences between the language groups in terms of variability of the f0, duration, and voice quality. For the lineups, the results indicated that prosodic features of f0, intensity, and voice quality correlated to the listeners&#39; perceived judgments.

This research aimed to make a comparison between L1-L2-English and L1-L2 Portuguese to check how much the effect of a foreign accent accounts for both metrics and prosodic-acoustic parameters, as well as for the choice of the target voice in a voice lineup.

This research aims to examine both the prosodic-acoustic features and the perceptual correlates of foreign-accented English and foreign-accented Brazilian ...

Enhanced Precision in Salient Object Detection Using Deep Neural Networks

End-To-End Deep Neural Network for Salient Object Detection in Complex Environments

Salient object detection has emerged as a burgeoning area of interest within the realm of computer vision. However, prevailing algorithms exhibit diminished precision when tasked with detecting salient objects within intricate and multifaceted environments. In light of this pressing concern, this article presents an end-to-end deep neural network that aims to detect salient objects within complex environments. The study introduces an end-to-end deep neural network that aims to detect salient objects within complex environments. Comprising two interrelated components, namely a pixel-level multiscale full convolutional network and a deep encoder-decoder network, the proposed network integrates contextual semantics to produce visual contrast across multiscale feature maps while employing deep and shallow image features to improve the accuracy of object boundary identification. The integration of a fully connected conditional random field (CRF) model further enhances the spatial coherence and contour delineation of salient maps. The proposed algorithm is extensively evaluated against 10 contemporary algorithms on the SOD and ECSSD databases. The evaluation results demonstrate that the proposed algorithm outperforms other approaches in terms of precision and accuracy, thereby establishing its efficacy in salient object detection within complex environments.

Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications

The present protocol describes a novel end-to-end salient object detection algorithm. It leverages deep neural networks to enhance the precision of salient object detection within intricate environmental contexts.

Salient object detection has emerged as a burgeoning area of interest within the realm of computer vision. However, prevailing algorithms exhibit ...

Engineering

Development of an Alpha-synuclein Based Rat Model for Parkinson's Disease via Stereotactic Injection of a Recombinant Adeno-associated Viral Vector

In order to study the molecular pathways of Parkinson&#39;s disease (PD) and to develop novel therapeutic strategies, scientific investigators rely on animal models. The identification of PD-associated genes has led to the development of genetic PD models. Most transgenic &#945;-SYN mouse models develop gradual &#945;-SYN pathology but fail to display clear dopaminergic cell loss and dopamine-dependent behavioral deficits. This hurdle was overcome by direct targeting of the substantia nigra with viral vectors overexpressing PD-associated genes. Local gene delivery using viral vectors provides an attractive way to express transgenes in the central nervous system. Specific brain regions can be targeted (e.g. the substantia nigra), expression can be induced in the adult setting and high expression levels can be achieved. Further, different vector systems based on various viruses can be used. The protocol outlines all crucial steps to perform a viral vector injection in the substantia nigra of the rat to develop a viral vector-based alpha-synuclein animal model for Parkinson&#39;s disease.

This manuscript describes how viral vector-mediated local gene delivery provides an attractive way to express transgenes in the central nervous system. The protocol outlines all crucial steps to perform a viral vector injection in the substantia nigra of the rat to develop a viral vector-based animal model for Parkinson's disease.

In order to study the molecular pathways of Parkinson&#39;s disease (PD) and to develop novel therapeutic strategies, scientific investigators rely on ...

Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques

Statistical learning, a fundamental skill to extract regularities in the environment, is often considered a core supporting mechanism of the first language development. While many studies of statistical learning are conducted within a single domain or modality, recent evidence suggests that this skill may differ based on the context in which the stimuli are presented. In addition, few studies investigate learning as it unfolds in real-time, rather focusing on the outcome of learning. In this protocol, we describe an approach for identifying the cognitive and neural basis of statistical learning, within an individual, across domains (linguistic vs. non-linguistic) and sensory modalities (visual and auditory). The tasks are designed to cast as little cognitive demand as possible on participants, making it ideal for young school-aged children and special populations. The web-based nature of the behavioral tasks offers a unique opportunity for us to reach more representative populations nationwide, to estimate effect sizes with greater precision, and to contribute to open and reproducible research. The neural measures provided by the functional magnetic resonance imaging (fMRI) task can inform researchers about the neural mechanisms engaged during statistical learning, and how these may differ across individuals on the basis of domain or modality. Finally, both tasks allow for the measurement of real-time learning, as changes in reaction time to a target stimulus is tracked across the exposure period. The main limitation of using this protocol relates to the hour-long duration of the experiment. Children might need to complete all four statistical learning tasks in multiple sittings. Therefore, the web-based platform is designed with this limitation in mind so that tasks may be disseminated individually. This methodology will allow users to investigate how the process of statistical learning unfolds across and within domains and modalities in children from different developmental backgrounds.

Presented here is a protocol introducing a set of child-friendly statistical learning tasks geared towards examining children&#8217;s learning of temporal statistical patterns across domains and sensory modalities. The developed tasks collect behavioral data using the web-based platform and task-based functional magnetic resonance imaging (fMRI) data for examining neural engagement during statistical learning.

Statistical learning, a fundamental skill to extract regularities in the environment, is often considered a core supporting mechanism of the first ...

Neuroscience

Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application

Tongue diagnosis is an essential technique of traditional Chinese medicine (TCM) diagnosis, and the need for objectifying tongue images through image processing technology is growing. The present study provides an overview of the progress made in tongue objectification over the past decade and compares segmentation models. Various deep learning models are constructed to verify and compare algorithms using real tongue image sets. The strengths and weaknesses of each model are analyzed. The findings indicate that the U-Net algorithm outperforms other models regarding precision accuracy (PA), recall, and mean intersection over union (MIoU) metrics. However, despite the significant progress in tongue image acquisition and processing, a uniform standard for objectifying tongue diagnosis has yet to be established. To facilitate the widespread application of tongue images captured using mobile devices in tongue diagnosis objectification, further research could address the challenges posed by tongue images captured in complex environments.

The present study employed U-Net and other deep learning algorithms to segment a tongue image and compared the segmentation results to investigate the objectification of tongue diagnosis.

Tongue diagnosis is an essential technique of traditional Chinese medicine (TCM) diagnosis, and the need for objectifying tongue images through image ...

Analyzing Signaling Modules in Alzheimer's Disease with Deep Learning

DeepOmicsAE: Representing Signaling Modules in Alzheimer's Disease with Deep Learning Analysis of Proteomics, Metabolomics, and Clinical Data

Large omics datasets are becoming increasingly available for research into human health. This paper presents DeepOmicsAE, a workflow optimized for the analysis of multi-omics datasets, including&#160;proteomics, metabolomics, and clinical data. This workflow employs a type of neural network called autoencoder, to extract a concise set of features from the high-dimensional multi-omics input data. Furthermore, the workflow provides a method to optimize the key parameters needed to implement the autoencoder. To showcase this workflow, clinical data were analyzed from a cohort of 142 individuals who were either healthy or diagnosed with Alzheimer&#39;s disease, along with the proteome and metabolome of their postmortem brain samples. The features extracted from the latent layer of the autoencoder retain the biological information that separates healthy and diseased patients. In addition, the individual extracted features represent distinct molecular signaling modules, each of which interacts uniquely with the individuals&#39; clinical features,&#160;providing for a mean to integrate&#160;the proteomics, metabolomics, and clinical data.

Author Spotlight: Advancing Alzheimer's Research &#8211; Exploring Early Detection and Multi-Omics Approaches

DeepOmicsAE is a workflow centered on the application of a deep learning method (i.e., an autoencoder) to reduce the dimensionality of multi-omics data, providing a foundation for predictive models and signaling modules representing multiple layers of omics data.

Large omics datasets are becoming increasingly available for research into human health. This paper presents DeepOmicsAE, a workflow optimized for the ...

Biology

DNA Virus Detection System Based on RPA-CRISPR/Cas12a-SPM and Deep Learning

We report a fast, easy-to-implement, highly sensitive, sequence-specific, and point-of-care (POC) DNA virus detection system, which combines recombinase polymerase amplification (RPA) and CRISPR/Cas12a system for trace detection of DNA viruses. Target DNA is amplified and recognized by RPA and CRISPR/Cas12a separately, which triggers the collateral cleavage activity of Cas12a that cleaves a fluorophore-quencher labeled DNA reporter and generalizes fluorescence. For POC detection, portable smartphone microscopy is built to take fluorescent images. Besides, deep learning models for binary classification of positive or negative samples, achieving high accuracy, are deployed within the system. Frog virus 3 (FV3, genera Ranavirus, family Iridoviridae) was tested as an example for this DNA virus POC detection system, and the limits of detection (LoD) can achieve 10 aM within 40 min. Without skilled operators and bulky instruments, the portable and miniature RPA-CRISPR/Cas12a-SPM with artificial intelligence (AI) assisted classification shows great potential for POC DNA virus detection and can help prevent the spread of such viruses.

We present a protocol that combines recombinase polymerase amplification with a CRISPR/Cas12a system for trace detection of DNA viruses and builds portable smartphone microscopy with an artificial intelligence-assisted classification for point-of-care DNA virus detection.

We report a fast, easy-to-implement, highly sensitive, sequence-specific, and point-of-care (POC) DNA virus detection system, which combines recombinase ...