This manuscript&#160;was supported by funding from the National Science Foundation (DRL#1660878, DRL#1661202, DUE#1761178, DRL#1916417), the Social Sciences and Humanities Research Council of Canada (SSHRC 895-2011-1006), the Ministry of Sciences and Innovation I+D+i (PID2019-107201GB-100), and the European Union through the European Regional Development Funds (ERDF) and the Principality of Asturias (FC-GRUPIN-IDI/2018/000199). Any opinions, findings, conclusions, or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the National Science Foundation or Social Sciences and Humanities Research Council of Canada.&#160;The authors would also like to thank members of the SMART Lab at UCF for their assistance and contributions.

The research ethics committee of the Principality of Asturias and the University of Oviedo approved this protocol.
1. Session 1: diagnosis assessment
NOTE: In this session of the protocol, evaluation tests from different publishers are used, which have their own specific application and interpretation manuals. Since these tests, or other similar ones, are widely known by the scientific community in the field of psychology and education, the procedure to apply them is not detailed step by step (for example, given the aim of this paper, it does not make sense to detail each step of the WAIS-IV35 application).
<ol>
	<li>Informed consent
	<ol>
		<li>Explain to the participants the ethical and confidentiality aspects of the research and ask them to acknowledge and sign the individual informed consent.</li>
	</ol>
	</li>
	<li>Structured interview
	<ol>
		<li>Explain the following instructions to the participant: &#34;Now, I&#180;m going to interview you in order to get important information about your life and academic issues. There are open and closed questions but you can interrupt me whenever you want. Please, let me know if you need me to clarify any point. After this initial interview, I may ask you to do some evaluation tests and questionnaires. I will tell you the specific instructions for each one. Are you ready?&#34;</li>
		<li>Collect the biographical information along with the presence of symptoms related to SLD and exclusion criteria that are referred in the DSM-51 following the interview script (see <a href="https://www.jove.com/files/ftp_upload/60331/Supplemental_documents_A.pdf" target="_blank">Supplemental File A</a>&#160;).</li>
	</ol>
	</li>
	<li>First decision point in relation to the structured interview (exclusion criteria)
	<ol>
		<li>Finish the assessment if the participant meets the initial exclusion criteria, that is, they explain that they have a motor disability (upper segments), sensory disability (visual or auditory), a diagnosis of intellectual disability or a serious mental disorder.</li>
		<li>Continue the assessment if it seems that the participant has or thinks he/she has an SLD and does not meet exclusion criteria.</li>
	</ol>
	</li>
	<li>Intellectual ability
	<ol>
		<li>Apply the WAIS-IV35 test to collect information about participant&#8217;s intellectual ability following the instructions in the manual.</li>
	</ol>
	</li>
	<li>Second decision point in relation to intellectual ability (exclusion criteria)
	<ol>
		<li>Finish the assessment if the participant does not understand the instructions of the test, if cannot be evaluated, or they have an IQ of less than 70.</li>
		<li>Continue the assessment if the person has normal or limited intellectual ability. 
		NOTE: The limit of the IQ accepted in the present study has been set as a score of over 70.</li>
	</ol>
	</li>
	<li>ADHD
	<ol>
		<li>Ask the participant to complete the six items of the Self-reported Screening Questionnaire of the Adult-v1.1. (ASRS39) of the World Health Organization (WHO) International Composed Diagnostic Interview. 
		NOTE: This questionnaire provides information on the presence of symptoms related to ADHD that are referred to in the DSM-IV54.</li>
		<li>Apply the Nesplora Aquarium test40 if the participant scores 12 or more in the previous ASRS36 questionnaire.</li>
	</ol>
	</li>
	<li>Reading difficulties
	<ol>
		<li>Apply the PROLEC SE-R Screening Test of reading difficulties37 follow the instructions in the manual.</li>
	</ol>
	</li>
	<li>Autism spectrum disorder (level 1)
	<ol>
		<li>Ask the participant to complete the 28 items of the Autism Spectrum Quotient (AQ-Short) questionnaire from Hoekstra et al.41 
		NOTE: This questionnaire provides information on the presence of symptoms related to social behavior, social skills, routine, switching, imagination and numbers/patterns.</li>
	</ol>
	</li>
	<li>Analyze the results.
	<ol>
		<li>Analyze each participant&#8217;s interview, questionnaires and test results and decide if they have significant learning difficulties or not or are at risk of having them. 
		NOTE: Two members of the expert committee (the evaluator and another member of the research team) analyze each participant&#8217;s learning profile and decide if they is a student with SLD, ADHD and/or ASD or not or are at risk of having them. No test can substitute the expert&#180;s judgment.</li>
	</ol>
	</li>
	<li>Final decision point
	<ol>
		<li>Finish the assessment if the participant is clearly not a student with learning difficulties.</li>
		<li>Continue the assessment if the participant is a person with LDs (or at risk) and go to Session 2.</li>
	</ol>
	</li>
</ol>
2. Session 2: multimodal assessment
NOTE: Session 2 must be done between 1 and 7 days after Session 1.
<ol>
	<li>Prepare the participant.
	<ol>
		<li>Remind the participants that the session lasts approximately 2 hours, and that they are going to complete some questionnaires and tasks in the MetaTutor learning environment while some devices are recording their performance throughout the session.</li>
		<li>Ask the participants tie back their hair, clear their neck, remove their glasses and remove chewing gum if applicable. 
		NOTE: If the participant is wearing glasses, has long hair or bangs that cover part of their face, the eye tracker will not be able to read their eyes movements.</li>
		<li>Introduce MetaTutor to the participants. Explain that the objective of the session is to autonomously learn about the circulatory system using the tool.</li>
		<li>Make sure the speakers are connected and working. 
		NOTE: The participant can also use headphones if preferred.</li>
	</ol>
	</li>
	<li>Galvanic skin response preparation and calibration 
	NOTE: Remember that there are many types of GSRs manufactured by different companies. Use it according to the supplier&#39;s specifications.
	<ol>
		<li>Clean the GSR and the participant&#180;s fingers with alcohol.</li>
		<li>Put the finger/wristband GSR sensors on the index and ring fingers with the connectors on the fingertip side or according to the manufacturer&#39;s instructions.</li>
		<li>Ask the participant to rest their hand on the table quietly and try to relax for 5 min.</li>
		<li>Open the software in the computer.</li>
		<li>Make sure the registration graph is working. Check the registration graph is registering.</li>
		<li>Click Run experiment &#62; Rate 10 per second &#62; Duration &#62; 10 &#62; Minute. Record the information for ten minutes to establish the baseline. 
		NOTE: Rate 10 per second means the frequency with which measures are taken.</li>
		<li>Minimize the screen.</li>
		<li>Continue with the calibration of other devices, and after 10 minutes save the information in a .csv file.</li>
	</ol>
	</li>
	<li>Eye tracking and webcam preparation and calibration 
	NOTE: Remember that there are many types of eye tracking and webcam manufactured by different companies. Use them according to the supplier&#39;s specifications.
	<ol>
		<li>Open the software in the side laptop and in the computer. 
		NOTE: The eye movements are captured on the PC the participant is working on, but the data is recorded on the side laptop. In addition, in the side laptop, the experimenter can see the movements that the participant is making and correct the participant&#8217;s position if necessary.</li>
		<li>Indicate which session will be recorded (Metatutor in this case) and the participant&#8217;s registration data: File &#62; Recent Experiment &#62; Metatutor &#62; Include Registration data of the participant &#62; OK.</li>
		<li>Check that the two computers are connected to each other and that the eye tracking infrared lights are on and ready to capture the movement of the eyes.</li>
		<li>Adjust the webcam on the computer to the participant&#8217;s position.</li>
		<li>Ask the participant to sit facing forward and be as neutral as possible, although it is expected that their facial expressions will vary during the learning session. 
		NOTE: During the learning session a video of the participant&#180;s face is recorded with the webcam which is later analyzed using a desktop app55.</li>
		<li>Ask the participant to be still and to stare at the different points of the screen with their nose put in line with/slightly over the edge of the desk (at 90&#176;).</li>
		<li>Click Record &#62; Write the registration data of the participant &#62; Ok to start the calibration process.</li>
		<li>Ask the participant to press the space bar and follow the points on the screen with their eyes.</li>
		<li>Make sure that the participant&#8217;s eyes, when looking at the screen, are centered before moving on to the next step, using the side laptop to check this information. 
		NOTE: The participant&#39;s gaze is centered when the movements of their eyes are registered on the side laptop screen with two white circles. When the gaze leaves the registration area, the software warns with yellow arrows (if slightly deviated), with red arrows (if deviated a lot) or without white circles (if not registering). The path of the movement of the eyes is reflected with a yellow light (attentional focus) and the track through the screen with a green line.</li>
		<li>Ask the participants to avoid touching their face or resting their head in their hands as much as possible.</li>
		<li>Minimize the screen.</li>
	</ol>
	</li>
	<li>Multimodal tracking of the learning session
	<ol>
		<li>Maximize the GSR screen and click Run experiment &#62; Rate 10 per second &#62; Duration &#62; 5 &#62; hours &#62; Record and minimize the screen again.</li>
		<li>Maximize the eye tracking and webcam screen, make sure the software is working correctly, click Record on the computer and on the side laptop to register and record the session and minimize the screen again. 
		NOTE: Once the devices have been calibrated, do not forget to start recording the evaluation session in each of them. From this point, the entire participant interaction with the learning tool will be recorded until the end of the session.</li>
	</ol>
	</li>
	<li>Questionnaires and learning session in MetaTutor
	<ol>
		<li>Open the software in the PC and complete the participant&#8217;s registration data. Complete ID &#62; Experimenter &#62; Day &#62; Questionnaires yes &#62; Continue. 
		NOTE: All the logs will be registered during the session in a file-data log.</li>
		<li>Explain to the participant that they must follow the instructions in the tool and that they will only be interacting with the computer during the learning session. Explain that the researcher will be in the next room in case anything happens.
		<ol>
			<li>Ask the participant for sociodemographic and academic information. Complete Name &#62; Gender &#62; Age &#62; Ethnic group &#62; Educational level &#62; University &#62; Degree &#62; GPA &#62; Information about biology courses taken if applicable &#62; Continue. Before clicking Continue, explain to the participants that they must follow all the instructions that the tool will give them. Also, that they will only interact with the computer during the learning session.</li>
			<li>Ask the participant to complete some questionnaires. 
			NOTE: The participant has to complete five metacognitive and self-regulated learning questionnaires: a) The Mini International Personality Item Pool49; b) The Connotative Aspects of Epistemological Beliefs50; c) The Rosenberg Self-esteem Scale51; d) The Emotion Regulation Questionnaire52; e) The Achievement Emotions Questionnaire (AEQ)53 and one questionnaire about general knowledge about the circulatory system.</li>
			<li>Show the participant the interface of MetaTutor and its different parts.
			<ol>
				<li>Explain the participant that the content area is where the learning content is displayed throughout the session in text form.</li>
				<li>Show the participant that they can navigate through a table of contents at the side of the screen to go to different pages.</li>
				<li>Show the participant that the overall learning goal is displayed at the top of the screen during the session.</li>
				<li>Show the participant that the sub-goals learners set are displayed at the top in the middle of the screen, and they can manage sub-goals or prioritize them here.</li>
				<li>Show the participant that there is a timer located at the top left corner of the screen displays the amount of time remaining in the session.</li>
				<li>Show the participant the list of self-regulating processes, which are displayed in a palette on the right hand side of the screen, and the participant can click on them throughout the session to deploy planning, monitoring and learning strategies.</li>
				<li>Show the participant the static images relevant to content pages are displayed beside the text to help learners coordinate information from different sources.</li>
				<li>Show the participant the text entered on the keyboard and how students&#180; interactions with agents are displayed and recorded in this part of the interface.</li>
				<li>Show the participant the four artificial agents who help students in their learning throughout the session. 
				NOTE: These agents are Gavin the Guide, Pam the Planner, Mary the Monitor, and Sam the Strategizer.</li>
			</ol>
			</li>
			<li>Ask the participant to click Start to begin the learning session whenever they are ready. 
			NOTE: The participant interacts with the tool.</li>
			<li>Once the session is finished, ask the participant to complete the knowledge questionnaire again.</li>
		</ol>
		</li>
	</ol>
	</li>
</ol>
3. Logoff
<ol>
	<li>At the end of the session save the recorded data from GSR, eye tracking/webcam and Metatutor along with the registration data of the participant. Extract the data in a .csv file for easier use.</li>
	<li>Remove the GSR sensors from the participant&#39;s hand and clean the galvanic sensors with alcohol again.</li>
	<li>Thank the participants for their collaboration and say goodbye.</li>
</ol>
4. Analysis of learning difficulties
<ol>
	<li>Analyze each participant&#8217;s learning performance based on the different reports produced (see Results section) to obtain a multimodal profile. 
	NOTE: At least two members of the expert committee analyze each participant&#8217;s learning process. Although the evaluation can be done exhaustively using new instruments and tools, no report can replace the expert&#39;s judgment.</li>
</ol>

The authors have nothing to disclose.

The current protocol proposes a multimodal evaluation focused on metacognitive, self-regulation, and emotional processes, which make up the basis of the difficulties in adults with LDs.
Session 1 is essential because it is intended to be a diagnostic assessment of the participant&#8217;s learning disabilities. Note that this session here is carried out by therapists with experience in diagnosing developmental and learning difficulties in the research and clinical context. We use these tools in...

Specific learning disorders (SLDs) encompass disorders of those who have difficulty learning and using academic skills, exhibiting performance below expectations for their chronological age in the areas of reading, writing, and/or mathematics1,2. There are different estimations of prevalence rates depending on the age, language and culture analyzed but they are between 5% and 15%1,3. Within the global category of neurodevelopmental disorders in the Diagnostic and Statistical Manual of Mental Disorders (5th Ed.)1, it is also necessary to focus on the incidence of Attention-Deficit/Hyperactivity Disorder (hereinafter ADHD) as it is a common disorder that has given rise to various controversies about how to approach it in recent years. Based on the DSM-51, it can be defined as a pattern of persistent behaviors of inattention and/or hyperactivity-impulsivity. Likewise, autism spectrum disorder (hereinafter ASD) is a category in the same manual that includes students who present neurodevelopmental disorders as a result of multifactorial dysfunctions of the central nervous system, which result in qualitative dysfunctions in three fundamental areas of the development of the person: social interaction, communication and interests and behaviors1,2.
On these lines, a new concept has emerged moving away from the sense of deficit and offering a more positive approach to these disorders to be consistent with current ideas of neurodevelopmental difficulties as highly coexistent and overlapping4. From these new models, it is understood that the skills involved in high-level cognitive processes, which allow managing and regulating one&#39;s behavior in order to achieve a desired goal, are crucial for self-regulation and, therefore, for activities of daily living, including the academic ones5. In the context of adulthood, neurodiversity has evolved to include various types of difficulties, including ADHD and ASD, as well as dyslexia, dyspraxia, and/or dyscalculia. Accordingly, we are approaching this neurodiversity from a broad conception of learning difficulties (LDs). The increase in students with this diversity enrolled in postsecondary education is well documented and is due, in part, to the increase in high school graduation rates for students with disabilities6, but at the same time, there is less research about the learning process of these students than necessary7.
Each of the disorders approached in isolation involve different deficits and manifestations; however, some commonality can be found within that heterogeneity in terms of LD, such as metacognitive, self-regulatory, and emotional malfunctioning8,9,10,11. Three fundamental foundations in the literature of learning in general, and LDs in particular, that represent the basis of successful learning and play an essential role in these well-known difficulties at the academic level12. As well as this, other approaches understand that there could be a certain commonality between deficits in executive functions, such as problems in automatic processing or working memory, that occur in different disorders such as ADHD and reading disorders13 or ADHD and ASD5. However, there is still work to be done in this field, since not all studies reach the same conclusions about these points in common in relation to executive functions. It could be due to the variations presented by the samples from which the studies are based and the evaluation procedures of the executive functions used in the investigations5,14.
In educational terms, this diverse mix affects not only the quality of learning, due to the fundamental nature of the affected functions, but also phenomena such as school dropout, change of degree, etc., with economic implications for governments and universities15. The dropout rate for students with LDs is higher than for students in the general population16 but also higher than the dropout rates for any other category of psychological disabilities except for those students with emotional disturbances17. In contrast, the number of students with LDs who are accessing post-compulsory education (vocational training, college, etc.) is increasing15, specifically in higher education19,20,21,22. Moreover, one might well assume that there are many more students with LD than those who officially pass through student services and typically make up the prevalence statistics23.
These difficulties are not always detected during childhood, especially in adults born before these disorders were considered in the regular academic system, and the symptoms of these disorders persist throughout people&#8217;s lives and cause difficulties in work, education and personal lives24. Research has shown that although people might overcome some of their difficulties, most continue to exhibit struggles with learning during adulthood and their persistence is still problematic at those higher educational levels25.
Paradoxically, unlike in previous educational levels and earlier ages, there are hardly any evidence-based instruments or evaluation protocols for adults with LDs. Despite the proliferation of diagnostic tools to evaluate LDs during childhood, the availability of valid, reliable instruments and methodologies for the adult population is significantly limited24. A recent literature review about learning disabilities in higher education found that most of the information collected in this regard is done through interviews, and only occasionally are self-report questionnaires used26. Self-report methodology and interviews, although valuable, are not enough to accurately assess metacognitive, self-regulation, and emotional skills processes, in fact, among others, because of the process nature. The importance of scales and interview methodology for measuring those processes is undeniable27,28, but so too are the associated problems of validity29 and incongruence with other innovative methods of assessment30. An additional problem in the detection of LDs is the bias in the diagnosis of the disorder due to the absence of comprehensive assessment protocols. The fact that professionals do not have a reference protocol based on objective variables is frequently causing many false positive and false negative cases of LDs31.
In response to both scarcity of instruments for adults and the need to improve existing methodology, the current study proposes a multimodal evaluation protocol focused on metacognitive, self-regulation, and emotional processes, which make up the basis of the difficulties in adults with LDs. In line with the current literature, we propose a move toward integrative and multichannel measurement32,33. The assessment is carried out through an analysis of the on-line learning process using several methods, techniques, and sensors (e.g., hypermedia learning environment, virtual reality, eye tracking, facial expressions of emotion, physiological responses, log files, screen recordings of human-machine interactions) and off-line methods (e.g., questionnaires, interviews, and self-report measures). This mixed methodology provides evidence of the deployment of target processes before, during, and after learning that can be triangulated to enhance the understanding of how students learn and where the problem lies, if there is one34.
The evaluation protocol is carried out over two sessions. The sessions can be done in one sitting or may need partial applications depending on the person. The first is focused on the detection or confirmation of LDs and what specific kind of disorder we are facing, and the second is designed to go into the metacognitive, self-regulation, and emotional processes of each individual case in depth.
Session 1 is intended to be a diagnostic or confirmation assessment of the participant&#8217;s learning disabilities: SLD, ADHD and/or ASD (high functioning) to determine what type of specific problems the participants have. This assessment is essential for two reasons. 1) Adults with Learning Disabilities rarely have accurate information about their dysfunctional behavior. Some of them suspect that they have a LD but have never been evaluated. Others may have been assessed when they were children but do not have any reports or further information. 2) There may be discrepancies with previous diagnoses (e.g., a previous dyslexia diagnosis as opposed to a current diagnosis of attention deficit and slow processing speed; previous ASD diagnosis in contrast to current limited intellectual ability, etc.). The participant is interviewed, and questionnaires and standardized tests are applied. This session here is carried out by therapists with experience in diagnosing developmental and learning difficulties in the research and clinical context in different offices of a Spanish Psychology Faculty. The session begins with a structured interview that collects biographical information along with the presence of symptoms related to SLDs that are referred to in the DSM-51. Following that, the reference intellectual ability test WAIS-IV35 is used in case of exclusion criterion implementation and because it provides very valuable information for learning difficulties from the scales &#8220;work memory&#8221; and &#8220;processing speed&#8221;36. Additionally, the PROLEC SE-Revised Test37 is extensively used to evaluate reading disabilities (lexical, semantic and/or syntactic processes of reading), one of the most prevalent and disabling difficulties for learning in current academic contexts, which overlaps with other disorders such as ADHD38. This evaluation collects reading accuracy, speed and fluency along with reading disabilities, and more importantly, in which reading process the failure occurs37 (this test has been evaluated with pre-university students. Currently, there are no tests in Spain that are adapted to the general adult population, so this test was selected because it is the closest to the target population). Then, we screen symptoms of ADHD through the World Health Organization Adult ADHD Self-Report Scale (ASRS)39 and refine the evaluation of this disorder, introducing multimodality with a cutting-edge virtual reality continuous performance test for the evaluation of attentional processes and working memory in adults, the Nesplora Aquarium31,40. This test is a very useful tool when diagnosing ADHD in adults and adolescents over 16 years old in an ecological scenario, providing objective, reliable data. It evaluates selective and sustained attention, impulsivity, reaction time, auditory and visual attention, perseverance, quality of attentional focus, motor activity, work memory and cost of change of task. Additionally, along with the WAIS-IV35 as a whole for collecting information about the participant&#8217;s intellectual ability, we pay special attention to the scales &#8220;work memory&#8221; and &#8220;processing speed&#8221; because they are related to learning difficulties and the results of these scales are used in the final decision. Finally, we include the Autism Spectrum Quotient (AQ-Short)41 in the protocol, the short version of the reliable AQ-Adult from Baron-Cohen, Wheelwright, Skinner, Martin and Clubley42.
Session 2 focuses on a multimodal assessment of the participant&#8217;s learning process. The key to understanding complex learning lies in understanding the deployment of students&#8217; cognitive, metacognitive, motivational, and affective processes43. To that end, participants work with MetaTutor, where the use of metacognitive and cognitive strategies deployed are observed while they are learning. MetaTutor is a hypermedia learning environment that is designed to detect, model, trace, and foster students&#8217; self-regulated learning while learning different science topic44. The design of MetaTutor is based on extensive research by Azevedo and colleagues43,45,46,47 and belongs to a new trend in the measurement of SRL, the so called third wave, which is characterized by combined use of measurement and advanced learning technologies33. The use of MetaTutor also provides multimodal trace data, incorporating measures such as, eye tracking, emotional physiological responses (galvanic skin response (GSR) and facial expressions of emotions)48, log-data and questionnaires. All these measures are combined to reach a deeper understanding of the participants SRL and metacognition.
Eye tracking provides an understanding of what attracts immediate attention, which target elements are ignored, in which order elements are noticed, or how elements compare to others; electrodermal activity lets us know how emotional arousal changes in response to the environment; facial-emotion-recognition allows the automatic recognition and analysis of facial expressions; and data logging collects and stores the student&#180;s interaction with the learning environment for further analysis. Concerning the questionnaires, the Mini International Personality Item Pool49 informs about a range of activities and thoughts that people experience in everyday life assessing each of the five major personality traits (extraversion, agreeableness, conscientiousness, neuroticism and openness). The Connotative Aspects of Epistemological Beliefs50 provides information about participants&#8217; beliefs about knowledge. The Rosenberg Self-esteem scale shows how the participants feel about themselves overall51. The Emotion Regulation Questionnaire52 provides information about participants&#8217; emotion regulation. The Achievement Emotions Questionnaire (AEQ)53 informs about emotions typically experienced at university.
In short, assessing LDs during adulthood is particularly difficult. Education and experience allow many adults to compensate for their deficits and later show undifferentiated or masked symptoms, on which scientific knowledge is still scarce. Taking into account the critical research gap that arises, this current work aims to ensure theoretically-driven, empirically-based guidelines for accurate assessment of LDs during adulthood in order to design effective prevention and intervention actions.
To help readers decide whether the method described is appropriate or not, it is necessary to specify that the protocol is not suitable for people with intellectual disabilities because their diagnosis invalidates the diagnosis of learning difficulties. In addition, due to the singularities of the equipment used and the format of showing the learning content, it is still not possible to evaluate people with motor disabilities (upper limbs, neck and/or face), hearing or visual impairment. Nor would it be suitable for participants with severe psychiatric disorders. It would require the use of drugs that could alter information processing or the physiological expression of emotions.

<table><tbody><tr><td>AQUARIUM</td><td>Nesplora</td><td></td><td></td></tr><tr><td>Eye-tracker RED500 Systems</td><td>SensoMotoric Instruments GmbH</td><td></td><td></td></tr><tr><td>Face API</td><td>Microsoft</td><td></td><td></td></tr><tr><td>GSR NUL-217</td><td>NeuLog</td><td></td><td></td></tr></tbody></table>

multimodal protocol for assessing metacognition and self-regulation in adults with learning difficulties

Learning disabilities (LDs) encompass disorders of those who have difficulty learning and using academic skills, exhibiting performance below expectations for their chronological age in the areas of reading, writing, and/or mathematics. Each of the disorders making up the LDs involve different deficits; however, some commonalities can be found within that heterogeneity, such in terms of learning self-regulation and metacognition. Unlike in early ages and later educational levels, there are hardly any evidence-based evaluation protocols for adults with LDs. LDs influence academic performance but also have serious consequences in professional, social, and family contexts. In response to this, the current work proposes a multimodal evaluation protocol focused on metacognitive, self-regulation of learning, and emotional processes, which make up the basis of the difficulties in adults with LDs. The assessment is carried out through analysis of the on-line learning process using a variety methods, techniques, and sensors (e.g., eye tracking, facial expressions of emotion, physiological responses, concurrent verbalizations, log files, screen recordings of human-machine interactions) and off-line methods (e.g., questionnaires, interviews, and self-report measures). This theoretically-driven and empirically-based guideline aims to provide an accurate assessment of LDs in adulthood in order to design effective prevention and intervention proposals.

This section illustrates the representative results obtained from the protocol, including an example of conjoint results of Session 1 and an example of each source of information from Session 2.
The results about disorders are collected in Session 1 through diagnostic tests taking into account the procedures and cut-off points specified for the diagnostic assessment of participants&#8217; learning difficulties (SLD, ADHD, and ASD). The expert committee decides whether the participant has learn...

Watch this Scientific Journal Video about Multimodal Protocol for Assessing Metacognition and Self-Regulation in Adults with Learning Difficulties at JoVE.com

Multimodal Protocol for Assessing Metacognition and Self-Regulation in Adults with Learning Difficulties

The current work proposes a multimodal evaluation protocol focused on metacognitive, self-regulation of learning, and emotional processes, which make up the basis of the difficulties in adults with LDs.

Learning disabilities (LDs) encompass disorders of those who have difficulty learning and using academic skills, exhibiting performance below expectations ...

multimodal-protocol-for-assessing-metacognition-self-regulation

Research

Education

JoVE Journal

JoVE Core

Introduction to Psychology

Behavior

Unit 1

Psychological Disorders

Learning

SCIENTISTS IN ACTION

8.3K Views.  University of Oviedo. The current work proposes a multimodal evaluation protocol focused on metacognitive, self-regulation of learning, and emotional processes, which make up the basis of the difficulties in adults with LDs.

Video: Multimodal Protocol for Assessing Metacognition and Self-Regulation in Adults with Learning Difficulties

Practical Methodology of Cognitive Tasks Within a Navigational Assessment

This paper describes an approach for measuring navigation accuracy relative to cognitive skills. The methodology behind the assessment will thus be clearly outlined in a step-by-step manner. Navigational skills are important when trying to find symbols within a speech-generating device (SGD) that has a dynamic screen and taxonomical organization. The following skills have been found to impact children&#8217;s ability to find symbols when navigating within the levels of an SGD: sustained attention, categorization, cognitive flexibility, and fluid reasoning1,2. According to past studies, working memory was not correlated with navigation1,2.
The materials needed for this method include a computerized tablet, an augmentative and alternative communication application, a booklet of symbols, and the Leiter International Performance Scale-Revised (Leiter-R)3. This method has been used in two previous studies. Robillard, Mayer-Crittenden, Roy-Charland, Minor-Corriveau and B&#233;langer1 assessed typically developing children, while Rondeau, Robillard and Roy-Charland2 assessed children and adolescents with a diagnosis of Autism Spectrum Disorder. The direct observation of this method will facilitate the replication of this study for researchers. It will also help clinicians that work with children who have complex communication needs to determine the children&#8217;s ability to navigate an SGD with taxonomical categorization.

Children who have complex communication needs can benefit from the use of a speech-generating device (SGD). Cognitive skills were identified as having an impact on the ability to navigate between levels of SGDs. This protocol describes the steps needed for Speech-language Pathologists to assess cognitive skills and navigational abilities.

This paper describes an approach for measuring navigation accuracy relative to cognitive skills. The methodology behind the assessment will thus be ...

Measuring the Functional Abilities of Children Aged 3-6 Years Old with Observational Methods and Computer Tools

The analysis of functional abilities and their development in early childhood (0-6 years old) are fundamental aspects among young children with certain types of developmental difficulties that can facilitate prevention, through programmed interventions adapted to the needs of each user (student or patient). There are, however, few investigations to date, that have analyzed the use of automated tools for recording and interpreting the results of the initial assessment. Here, a protocol is presented to examine the functional abilities in early childhood in young children, aged between 3-6 years old, with intellectual disabilities, but the protocol can also be used for ages 0 to 6 years. The protocol makes use of a computer application, eEarlyCare, that facilitates the interpretation of the results of systematic observations, which are recorded in natural environments by professionals trained in early intervention. The software can be used to analyze 11 functional areas (Food Autonomy, Personal Care and Hygiene, Dressing and Undressing Independently, Sphincter Control, Functional Mobility, Communication and Language, Daily Life Routines, Adaptive Behavior and Attention) and a total of 114 different behaviors. Its use facilitates the analysis of the observed abilities and greatly assists early intervention. Compared to other observational methods, it allows a more efficient use of personal and material resources. The use of the computer application facilitates the recording of the observation results, which helps with organization and reflection on the observations. The software displays the observation results on-screen compared to normal developmental parameters. This information can be referred to for decision-making about the most suitable intervention program for each user (student or patient). Likewise, clustering techniques are applied to analyze the relation between the type of intellectual disabilities and functional development identified with the software, a relation that is intended to serve as a guide for early-care professional intervention.

We present a protocol to use a computational tool to record and analyze the functional abilities of children aged 3-6 years old. The protocol facilitates the comparison of these abilities throughout their development and can be used to assess developmental difficulties.

The analysis of functional abilities and their development in early childhood (0-6 years old) are fundamental aspects among young children with certain ...

Problem-Solving Before Instruction (PS-I): A Protocol for Assessment and Intervention in Students with Different Abilities

Nowadays, how to encourage students' reflective thinking is one of the main concerns for teachers at various educational levels. Many students have difficulties when facing tasks that involve high levels of reflection, such as on STEM (Science, Technology, Engineering and Mathematics) courses. Many also have deep-rooted anxiety and demotivation towards such courses. In order to overcome these cognitive and affective challenges, researchers have suggested the use of "Problem-Solving before Instruction" (PS-I) approaches. PS-I consists of giving students the opportunity to generate individual solutions to problems that are later solved in class. These solutions are compared with the canonical solution in the following phase of instruction, together with the presentation of the lesson content. It has been suggested that with this approach students can increase their conceptual understanding, transfer their learning to different tasks and contexts, become more aware of the gaps in their knowledge, and generate a personal construct of previous knowledge that can help maintain their motivation. Despite the advantages, this approach has been criticized, as students might spend a lot of time on aimless trial and error during the initial phase of solution generation or they may even feel frustrated in this process, which might be detrimental to future learning. More importantly, there is little research about how pre-existing student characteristics can help them to benefit (or not) from this approach. The aim of the current study is to present the design and implementation of the PS-I approach applied to statistics learning in undergraduate students, as well as a methodological approach used to evaluate its efficacy considering students' pre-existing differences.

Problem-Solving Before Instruction PS-I: A Protocol for Assessment and Intervention in Students with Different Abilities

This protocol guides researchers and educators through implementation of the Problem-Solving before Instruction approach (PS-I) in an undergraduate statistics class. It also describes an embedded experimental evaluation of this implementation, where the efficacy of PS-I is measured in terms of learning and motivation in students with different cognitive and affective predispositions.

Nowadays, how to encourage students' reflective thinking is one of the main concerns for teachers at various educational levels. Many students have ...

Multimedia Battery for Assessment of Cognitive and Basic Skills in Mathematics (BM-PROMA)

Learning mathematics is a complex process that requires the development of multiple domain-general and domain-specific skills. It is therefore not unexpected that many children struggle to stay at grade level, and this becomes especially difficult when several abilities from both domains are impaired, as in the case of mathematical learning disabilities (MLD). Surprisingly, although MLD is one of the most common neurodevelopmental disorders affecting schoolchildren, most of the diagnostic instruments available do not include assessment of domain-general and domain-specific skills. Furthermore, very few are computerized. To the best of our knowledge, there is no tool with these features for Spanish-speaking children. The purpose of this study was to describe the protocol for the diagnosis of Spanish MLD children using the BM-PROMA multimedia battery. BM-PROMA facilitates the evaluation of both skill domains, and the 12&#160;tasks included for this purpose are empirically evidence-based. The strong internal consistency of BM-PROMA and its multidimensional internal structure are demonstrated. BM-PROMA proves to be an appropriate tool for diagnosing children with MLD during primary education. It provides a broad cognitive profile for the child, which will be relevant not only for diagnosis but also for individualized instructional planning.

Multimedia Battery for Assessment of Cognitive and Basic Skills in Mathematics BM-PROMA

BM-PROMA is a valid and reliable multimedia diagnostic tool that can provide a complete cognitive profile of children with mathematical learning disabilities.

Learning mathematics is a complex process that requires the development of multiple domain-general and domain-specific skills. It is therefore not ...

Assessment and Communication for People with Disorders of Consciousness

In this experiment, we demonstrate a suite of hybrid Brain-Computer Interface (BCI)-based paradigms that are designed for two applications: assessing the level of consciousness of people unable to provide motor response and, in a second stage, establishing a communication channel for these people that enables them to answer questions with either &#39;yes&#39; or &#39;no&#39;. The suite of paradigms is designed to test basic responses in the first step and to continue to more comprehensive tasks if the first tests are successful. The latter tasks require more cognitive functions, but they could provide communication, which is not possible with the basic tests. All assessment tests produce accuracy plots that show whether the algorithms were able to detect the patient&#39;s brain&#39;s response to the given tasks. If the accuracy level is beyond the significance level, we assume that the subject understood the task and was able to follow the sequence of commands presented via earphones to the subject. The tasks require users to concentrate on certain stimuli or to imagine moving either the left or right hand. All tasks are designed around the assumption that the user is unable to use the visual modality, and thus, all stimuli presented to the user (including instructions, cues, and feedback) are auditory or tactile.

With this experiment, one might be able to detect consciousness in people with disorders of consciousness. Furthermore, the approach can create a simple communication channel that enables people to give simple YES/NO answers to questions.

In this experiment, we demonstrate a suite of hybrid Brain-Computer Interface (BCI)-based paradigms that are designed for two applications: assessing the ...

Neuroscience

Using Brain Activation (nir-HEG/Q-EEG) and Execution Measures (CPTs) in a ADHD Assessment Protocol

Attention Deficit Hyperactivity Disorder (ADHD) is a problem that impacts academic performance and has serious consequences that result in difficulties in scholastic, social and familial contexts. One of the most common problems in the identification of this disorder relates to the apparent over diagnosis of the disorder due to the absence of global protocols for assessment. The research group of School Learning, Difficulties and Academic Performance (ADIR) from the University of Oviedo, has developed a complete protocol that suggests the existence of certain patterns of cortical activation and executive control for identifying ADHD more objectively. This protocol takes into consideration some of the hypothetical determinants of ADHD, including the relationship between activation of selected areas of the brain, and differences in performance on various aspects of executive functioning such as omissions, commissions or response times, using innovative tools of Continuous Performance Testing (based on Virtual Reality CPT and Traditional CPT) and brain activation measures (two different tools, based on Hemoencephalography- nirHEG; and Quantified Electroencephalography --Q-EEG, respectively). This model of assessment aims to provide an effective assessment of ADHD symptomatology in order to design an accurate intervention and make appropriate recommendations for parents and teachers.

Using Brain Activation nir-HEG/Q-EEG and Execution Measures CPTs in a ADHD Assessment Protocol

This work presents a new protocol for the assessment of Attention Deficit Hyperactivity Disorder (ADHD) by providing a more objective diagnostic procedure for this developmental disorder based on the use of innovative tools. It also analyzes the relationship between activation measures and executive function measures.

Attention Deficit Hyperactivity Disorder (ADHD) is a problem that impacts academic performance and has serious consequences that result in difficulties in ...

Measuring Statistical Learning Across Modalities and Domains in School-Aged Children Via an Online Platform and Neuroimaging Techniques

Statistical learning, a fundamental skill to extract regularities in the environment, is often considered a core supporting mechanism of the first language development. While many studies of statistical learning are conducted within a single domain or modality, recent evidence suggests that this skill may differ based on the context in which the stimuli are presented. In addition, few studies investigate learning as it unfolds in real-time, rather focusing on the outcome of learning. In this protocol, we describe an approach for identifying the cognitive and neural basis of statistical learning, within an individual, across domains (linguistic vs. non-linguistic) and sensory modalities (visual and auditory). The tasks are designed to cast as little cognitive demand as possible on participants, making it ideal for young school-aged children and special populations. The web-based nature of the behavioral tasks offers a unique opportunity for us to reach more representative populations nationwide, to estimate effect sizes with greater precision, and to contribute to open and reproducible research. The neural measures provided by the functional magnetic resonance imaging (fMRI) task can inform researchers about the neural mechanisms engaged during statistical learning, and how these may differ across individuals on the basis of domain or modality. Finally, both tasks allow for the measurement of real-time learning, as changes in reaction time to a target stimulus is tracked across the exposure period. The main limitation of using this protocol relates to the hour-long duration of the experiment. Children might need to complete all four statistical learning tasks in multiple sittings. Therefore, the web-based platform is designed with this limitation in mind so that tasks may be disseminated individually. This methodology will allow users to investigate how the process of statistical learning unfolds across and within domains and modalities in children from different developmental backgrounds.

Presented here is a protocol introducing a set of child-friendly statistical learning tasks geared towards examining children&#8217;s learning of temporal statistical patterns across domains and sensory modalities. The developed tasks collect behavioral data using the web-based platform and task-based functional magnetic resonance imaging (fMRI) data for examining neural engagement during statistical learning.

Statistical learning, a fundamental skill to extract regularities in the environment, is often considered a core supporting mechanism of the first ...

Involving Individuals with Developmental Language Disorder and Their Parents/Carers in Research Priority Setting

A protocol for involving individuals presenting with developmental language disorder (DLD) (iDLD) and their parents/carers (iDLDPC) in a research priority setting exercise is presented.
iDLD have difficulties with communication skills, such as understanding language, word-finding and discourse. Such difficulties mean existing research priority setting protocols are difficult for iDLD to access, since they require sophisticated communication skills. Thus, a novel protocol for involving iDLD in these exercises is warranted. The same protocol is recommended for use with iDLDPC, to ensure accessibility.
The protocol is presented in 4 steps. Step 1 describes a program of activities delivered by trained, specialist DLD speech and language therapists (SLTs) that prepares iDLD/iDLDPC for involvement. Step 2 outlines an approach to elicit iDLD/iDLDPC&#8217;s opinions on research priorities. Steps 3 and 4 describe methods to analyze and integrate this data at multiple stages of the research priority setting process.
9 trained specialist DLD SLTs delivered steps 1 and 2. 17 iDLDs and 25 iDLDPCs consented to involvement. Opinions from all participants were elicited, and this data was used to influence the process and output of the exercise.
An advantage of this protocol is its accommodation of the heterogeneity in support needs of iDLD/iDLDPC, through a menu of options, whilst also providing a structured framework. Due to the novelty of the protocol, the methods for data integration were developed by the research group. These are potential limitations of the protocol, and may bring the reliability and validity under scrutiny, which are yet to be tested.
This protocol enables meaningful involvement of iDLD/iDLDPC in&#160;research priority setting and could be utilized for people with other kinds of speech, language or communication needs. Further research should evaluate the effectiveness of the protocol and whether it can be adapted for involvement of such populations in other research studies.

A protocol to enable individuals with developmental language disorder (DLD) and their parents/carers to meaningfully participate in a research priority setting exercise is established. The protocol includes a defined program of activities for data collection, and methods to incorporate this data into the broader research priority setting process.

A protocol for involving individuals presenting with developmental language disorder (DLD) (iDLD) and their parents/carers (iDLDPC) in a research priority ...

Medicine

Dual-Task Stroop Paradigm for Detecting Cognitive Deficits in High-Functioning Stroke Patients

General clinical cognitive assessment scales are not sensitive enough to cognitive impairment in high-functioning stroke patients. The dual-task assessment has advantages for identifying cognitive deficits in high-functioning stroke patients and has been gradually applied in clinical assessment and cognitive training. Moreover, the Stroop paradigm has higher sensitivity and specificity for attentional assessment than conventional clinical cognitive assessment scales. Therefore, this study presents the dual-task assessment based on the Stroop paradigm to identify cognitive deficits in high-functioning stroke patients. This study demonstrates a single- and dual-task evaluation based on the Stroop paradigm and confirms its feasibility through case experiments and synchronized functional near-infrared spectroscopy evaluation. The Stroop reaction time and correct rate are used as the main indicators to evaluate the cognitive level of the subjects. This study protocol aims to provide new ideas to figure out the ceiling effect in general clinical assessment failure for high-functioning stroke patients.

Clinical assessment scales are notsensitive enough to cognitive dysfunction in high-functioning stroke patients. The dual-task paradigm presents advantages and potential in the assessment and cognitive training of cognitive dysfunction. The study here proposes a dual-task Stroop paradigm to identify cognitive dysfunction in high-functioning stroke patients.

General clinical cognitive assessment scales are not sensitive enough to cognitive impairment in high-functioning stroke patients. The dual-task ...

Metacognitive Development: How Children Estimate Their Memory

Source: Laboratories of Judith Danovitch and Nicholaus Noles&#8212;University of Louisville
Human memory is fallible, and people often cannot accurately recall what they have seen or heard. Adults are aware of their limited memory capacity, so they use strategies, such as rehearsal and mnemonic devices, to improve their recall of important information. Because adults understand the limits of memory, they know it makes more sense to write down the items on their shopping list rather than to try to remember the items when they get to the store. This ability to think about one&#8217;s own memory is called metamemory. Metamemory is one component of a broader set of cognitive processes that allow humans to think about their own knowledge and thinking, which is called metacognition.
Although young children understand that people have thoughts and a limited amount of knowledge, they often have trouble acknowledging the limits of their own knowledge and cognitive skills. Children&#8217;s ability to accurately estimate their own memory capacity and abilities improves over the elementary school years. One common way of measuring metamemory and its development is by giving children an opportunity to predict how well they can remember something, and then observing how well their prediction matches their actual recall.
This experiment demonstrates how to measure children&#8217;s metamemory based on the methods developed by Shin, Bjorklund, and Beck.1

Source: Laboratories of Judith Danovitch and Nicholaus Noles&#8212;University of Louisville
Human memory is fallible, and people often cannot accurately ...