Advancing Dyslexia Assessment in Children Through Computerized Testing

Summary

The Multimedia Battery for Assessment of General-Domain and Specific-Domain Skills in Reading is a reliable and valid multimedia battery designed to assess cognitive and basic reading skills. It enables the generation of a comprehensive cognitive and reading performance profile, which is particularly beneficial for children with dyslexia.

Abstract

The acquisition of reading skills is an intricate process that demands the cultivation of various domain-general and domain-specific abilities. Consequently, it is unsurprising that many children grapple with maintaining proficiency at the grade level, particularly when confronted with challenges spanning multiple abilities across both domains, as observed in individuals with reading difficulties. Strikingly, despite reading difficulties being among the most prevalent neurodevelopmental disorders affecting school-aged children, the majority of available diagnostic tools lack a comprehensive framework for assessing the full spectrum of cognitive skills linked to dyslexia, with minimal computerized options. Notably, there are currently limited tools with these features available for Spanish-speaking children. The aim of this study was to delineate the protocol for diagnosing Spanish-speaking children with reading difficulties using the Sicole-R multimedia battery. This tool for elementary grades focuses on evaluating cognitive skills that are associated with dyslexia as prescribed by the scientific literature. Specifically, it concentrates on assessing a range of cognitive abilities that studies have demonstrated to be linked to dyslexia. This focus is based on the observation that individuals with dyslexia typically exhibit deficits in several of the cognitive areas evaluated by this digital tool. The robust internal consistency and multidimensional internal structure of the battery were demonstrated. This multimedia battery has proven to be a fitting tool for diagnosing children with reading difficulties in primary education, offering a comprehensive cognitive profile that is valuable not only for diagnostic purposes but also for tailoring individualized instructional plans.

Introduction

Dyslexia is a neurodevelopmental disorder characterized by difficulties in accurate and/or fluent word recognition and poor spelling and decoding abilities and is characterized by unexpected and persistent difficulty in acquiring efficient reading skills despite conventional instruction, adequate intelligence, and sociocultural opportunity¹. This neurobiological disorder often manifests as challenges in reading, spelling, and writing, primarily due to phonological deficits^2,3. “The importance of early identification of dyslexia cannot be overstated, as it allows for timely intervention and support^4,5. When a student does not progress beyond Tier-3 in a response-to-intervention model, it becomes essential to conduct a more comprehensive assessment of both domain-general and domain-specific abilities associated with dyslexia, as highlighted by the scientific literature. The development of the technique presented here is grounded in the necessity of conducting thorough evaluations to ensure that appropriate interventions and support are provided. Moreover, previous studies underscore the utility of technology-based screening tools, such as web applications and computer games, in facilitating effective screening processes^6,7. These studies collectively highlight the multifaceted nature of dyslexia, emphasizing the need for comprehensive assessment and intervention strategies to address the diverse cognitive profiles of individuals with dyslexia. Despite the prevalence of dyslexia among school-aged children, most available diagnostic tools lack a framework that comprehensively assesses both domain-general and domain-specific skills. Moreover, there are minimal computerized options, particularly for Spanish-speaking populations. This multimedia battery addresses these gaps by leveraging technology to facilitate a detailed assessment of cognitive skills linked to dyslexia.

Theoretical perspectives and cognitive deficits in Dyslexia
Various theoretical models, including phonological, rapid auditory processing, visual, magnocellular, and cerebellar theories, aim to explain the causes of dyslexia and inform interventions (see for a review)⁸. The phonological theory attributes dyslexia to difficulties in processing language sounds⁹, while the rapid auditory processing theory links dyslexia to deficits in perceiving rapidly changing sounds¹⁰. Visual theory highlights the visual aspects of reading difficulties, and magnocellular theory points to impairments in visual and auditory processing pathways¹¹. The cerebellar theory suggests that dyslexia arises from cerebellar impairments affecting motor control and cognitive functions¹². Nicolson and Fawcett's Delayed Neural Commitment (DNC) framework posits that slower skill acquisition and delayed neural network development are central to dyslexia. Recent models, such as the multiple deficit model, propose that dyslexia is a complex disorder influenced by genetic, cognitive, and environmental factors^13,14,15. For instance, Ring and Black¹⁴ support the multiple deficit model, showing that both phonological and cognitive processing deficits contribute to the heterogeneity of dyslexia. Soriano-Ferrer et al.¹⁵ conducted a study with Spanish-speaking children with developmental dyslexia (DD) and found significant impairments in naming speed, verbal working memory, and phonological awareness (PA). Similarly, Zygouris et al.¹⁶ and Rauschenberger et al.⁶ underscore the importance of cognitive screening tools in identifying these deficits, with dyslexic individuals consistently scoring lower than typically achieving peers.

Examining technological approaches in Dyslexia screening: Insights from research studies
Research on dyslexia screening has evolved with three main approaches: early detection strategies, multifaceted screening methods combining various assessments, and integrating technology for enhanced efficiency¹⁷. Politi-Georgousi's¹⁸ recent systematic review highlights a shift toward more applications for intervening in dyslexia symptoms rather than screening processes, aligning with technology integration to improve reading skills in dyslexic students. Various tools exist, such as the Dyslexia Early Screening Test (DEST) by Fawcett and Nicolson, which assesses speed, phonological skills, motor skills, cerebellar function, and knowledge¹⁹. “Computer-based tools have advanced, including a web application assessing reading and cognitive skills in Greek children²⁰ and tools by Hautala et al.²¹ and Rauschenberg et al.⁶ that use gaming and machine learning for early identification of dyslexia. Ahmad et al. integrated gaming with neural networks, achieving 95% accuracy in detection²². Studies across different orthographies underscore the importance of phonological awareness and rapid automatized naming in dyslexia identification^23,24.

Insights into Dyslexia among Spanish-speaking children
The study of dyslexia in Spanish-speaking children has been significantly advanced through the use of Sicole-R technology. Jiménez et al. demonstrated its effectiveness in assessing dyslexia across age groups, particularly in distinguishing between dyslexic and typically achieving readers based on phonological and syntactic processing during early elementary years²⁵. Guzmán et al. investigated naming speed deficits in dyslexic children with phonological challenges, highlighting interactions between dyslexia and naming speed measured through tasks such as letter-RAN and number-RAN²⁶. Further studies by Jiménez et al. explored phonological awareness deficits across different syllable structures²⁷, while Ortiz et al. investigated speech perception deficits among Spanish children with dyslexia, revealing impairments in speech perception development regardless of phonetic contrast or linguistic unit^28,29. Jiménez et al. investigated the double-deficit hypothesis of dyslexia³⁰, followed by analyses of cognitive processes and gender-related disparities in dyslexia prevalence^31,32. Rodrigo et al. explored lexical access among Spanish dyslexic children³³, and Jiménez et al. scrutinized syntactic processing deficits³⁴. Finally, Jiménez et al. studied phonological and orthographical processes in dyslexic subtypes, highlighting differences in orthographic route efficiency³⁵. These studies collectively enhance our understanding of the cognitive and linguistic challenges of dyslexia in Spanish-speaking populations.

The conducted studies share several common characteristics in terms of the age and background of the participating children. The children included in these studies ranged in age from 7 to 14 years. Most studies focused on primary school children aged between 7 and 12 years, except those that included children up to 14 years old, providing a sample that spans from early school years to preadolescence^31,32. The participating children were primarily from the Canary Islands in Spain. Additionally, some studies included samples from other regions of Spain and Guatemala^31,32. Participants were recruited from both public and private schools whose backgrounds included urban and suburban areas. The socioeconomic levels represented in these studies range from low-middle to working and middle class.

Together, these inquiries significantly advance our understanding of dyslexia's complexities, contributing to the field of dyslexia research. Adapted for use across multiple Ibero-American countries, including Spain, Guatemala, Chile, and Mexico, the tool facilitates the assessment of diagnostic accuracy and precision in a diverse Spanish-speaking sample for this study.

This study aimed to delineate a protocol for diagnosing Spanish-speaking children with reading difficulties using a specialized multimedia battery. The primary goal is to provide a comprehensive assessment tool that evaluates both domain-general and domain-specific skills associated with dyslexia.

Experimental setup overview
The SICOLE-R was programmed in the Java 2 Platform Standard Edition (J2SE). The HSQL database engine is used as a database. The software includes 6 main modules to be evaluated: 1) perceptual processing, which includes the tasks of voicing, placing, and manner of articulation; 2) phonological processing, which includes phoneme isolation, phoneme deletion, phoneme segmentation, and phoneme blending tasks; 3) naming speed, which includes the tasks of naming speed in numbers, letters, colors and pictures; 4) orthographic processing, which includes tasks of morphological comprehension of lexemes and suffixes and homophone comprehension; 5) syntactic processing, including gender, number, function words, and grammatical structure tasks; and 6) semantic processing, which influences reading comprehension tasks through informative and narrative text. Instructions for each task, accompanied by one or two trials (depending on the task) and a demonstration, are delivered by a pedagogical agent prior to the initiation of the testing phase. The application protocol for each task is illustrated here.

Prior to administering the multimedia battery to the study sample, adaptations were made to the Spanish language modality for each country (i.e., Mexico, Guatemala, Ecuador, and Chile), including adjustments to vocabulary, images, and other relevant content. The administration conditions were the same across all Latin American countries. The administration environment had to be quiet within the school and free from noise, distractions, and interruptions. The duration of the multimedia battery administration ranged from 3-4 sessions of 30 min each, depending on the student's ability and age. Due to its database compatibility with most spreadsheet and statistical data processing systems, the evaluator can analyze the results of each child and each task. Concerning the data collection, two distinct task types were employed: 1) tasks where the examiner records students' oral performance, noting successes and errors using an external mouse, and 2) tasks requiring students to independently select options by clicking on them.

Protocol

This protocol was conducted in accordance with the guidelines provided by the Comité de Ética de la Investigación y Bienestar Animal (Research Ethics and Animal Welfare Committee, CEIBA) at Universidad de La Laguna (ULL). The data were collected at different times according to the curriculum of each country, capturing information exclusively from students whose educational administrations, schools, and parents provided consent. The test battery used in this study is registered as intellectual property and can be accessed through a transfer agreement with the ULL. For more information on how to obtain the test battery, interested parties can contact the Office of Knowledge Transfer (OTRI) at ULL.

1. SICOLE-R installation and preparation

1. Use the following inclusion criterion for applying this tool: students from second to sixth grade. Use the following exclusion criterion: do not include students with special educational needs, referring to those who needed support and targeted educational attention due to sensory impairment or acquired neurological problems, among other factors.
   NOTE: The assessment is conducted with the students individually in a quiet space with access to a computer and good lighting. Headphones with an adapted microphone will be necessary to facilitate the reception of task instructions and increase student performance.
2. To install the software on a computer, execute the file as the administrator and click on the icon to open the tool. Upon opening the application, an initial interface appears, displaying various operational options.
3. Fill in the students' information who will perform the tasks before starting the evaluation. Once their data is entered, record them in a list of students, and their information can be modified later.
4. Each time a new student is added, a test screen will appear before starting the tasks. On this screen, ask the student to tap on the boot that will appear on the screen.
   NOTE: This mini-game's objective is to establish a baseline for assessing students' motor control. It allows for the monitoring of individual differences in motor speed, as the software records responses by pressing keys on the keyboard. This mini game appears only once.
5. Before beginning the assessment, ask the examiner to give the following instructions: Now, you will enter a virtual circus environment with different gates. Each gate leads to fun activities where you will participate. A clown will guide you through these gates and explain what to do in each activity. Please sit next to the examiner and listen carefully to the instructions. Make sure to do your best in each task. Enjoy exploring and completing these activities!
6. Begin the tasks by selecting the student scheduled for assessment and clicking on the Start section. The main menu includes 5 colored doors, each corresponding to one of the modules to be evaluated.
7. Each task starts with an introductory instruction, ask the students to pay attention to this. The students are informed about the nature of the game and instructed on how to play it, facilitated by a pedagogical agent. Ensure the agent delivers verbal instructions for each task, exemplifying the procedure through a model and presenting examples for students to emulate.
8. Following the completion of the examples, ensure the pedagogical agent provides corresponding feedback, allowing the students to redo the example if it was executed incorrectly, and the evaluation begins.
   NOTE: The pedagogical agent is a virtual character designed to guide the student through the tasks. This role requires the agent to have clear and understandable speech and the ability to provide instructional guidance, task modeling, and feedback based on student responses. The pedagogical agent must be programmed to respond consistently and accurately according to the student's inputs during the tasks.

2. SICOLE-R tasks

1. Yellow Door: Perceptual processing module using a speech perception task
   1. Open the program on the computer. Navigate to the yellow door within the program interface. Click on One of the Available Subtasks to initiate the task: (1) voicing, (2) manner of articulation, and (3) placing.
   2. Allow the pedagogical agent to provide instructional guidance and task modeling to the student. The agent should say: Now we will hear pairs of syllables. If the syllables are identical, press the blue button; if they are different, press the red button. Watch how I do it. Now it is your turn with these examples.
   3. Present two item examples to the student:/ba/-/pa/;/ja/-/ka/. Instruct the students again to select the blue circle if the pairs are identical and the red button if they are distinct.
   4. Once the task is completed, ensure that the pedagogical agent provides feedback based on the student's answers in the examples. Once the examples are complete, allow the agent to instruct the student to initiate the task.
   5. The procedure of selecting circles (blue or red) will remain consistent throughout the task, recording hits and misses. If the student needs to hear the pair of syllables again, click on the Speaker Icon to replay it once more. Note that only one additional playback per item is allowed.
2. Pink Door: Phonological awareness module
   1. Phoneme segmentation task
      1. Open the program on the computer. Navigate to the pink door within the program interface. Choose Phoneme Segmentation to select the task.
      2. Allow the pedagogical agent to provide instructional guidance to the student. The agent should say: Now we will focus on phoneme segmentation. Watch as I demonstrate. Then, you will see two example items.
      3. Present two example items to the student. Ask the student to express their answers aloud for each word.
      4. Use the external mouse to click on the blue button for correct responses and the red button for incorrect responses. Ensure the pedagogical agent offers feedback based on the student's responses to the examples.
      5. After the examples, ensure the pedagogical agent instructs the student to commence the task.
   2. Phoneme blending task
      1. Enter the program on the computer. Navigate to the pink door within the program interface. Choose Phoneme Blending to select the task.
      2. Allow the pedagogical agent to provide instructional guidance to the student. The agent says: Now we will focus on phoneme synthesis. Watch as I demonstrate. Then, you will see two example items.
      3. Present two example items to the student. Ask the student to articulate their answers aloud for each word.
      4. Utilize the external mouse to click the blue button for accurate responses and the red button for inaccurate responses. Ensure the pedagogical agent provides feedback based on the student's responses to the examples.
      5. After the examples, allow the pedagogical agent to direct the student to initiate the task. The procedure remains consistent throughout task execution, recording both accurate and inaccurate responses.
      6. If the student needs to listen to a word again, click on the Parrot Icon for a replay. Note that only one replay per item is allowed.
   3. Phoneme isolation task
      1. Open the program on the computer. Navigate to the pink door within the program interface. Click on Phoneme Isolation to select the task.
      2. Allow the pedagogical agent to provide instructional guidance and task modeling to the student. The agent should say: Now we will focus on isolating phonemes. Watch as I demonstrate. Then, you will see two example items. Your task is to click on the images whose name starts with the same sound as the target word.
      3. Present two example items to the student. Observe whether the student clicks on the images that begin with the same sound as the target word.
      4. Ensure the pedagogical agent offers feedback based on the student's responses to the examples. After the examples, ensure the agent guides the student to commence the task.
      5. The procedure remains uniform throughout the task, documenting both hits and misses. If the student requires additional listening for a word, click on the speaker Icon to replay it. Note that only one replay per item is allowed.
   4. Phoneme deletion task
      1. Open the program on the computer. Navigate to the pink door within the program interface. Select Phoneme Deletion to choose the task.
      2. Allow the pedagogical agent to provide instructional guidance and demonstrate the task to the student. The agent should say: Now we will focus on phoneme deletion. Watch as I demonstrate. Then, you will see two example items.
      3. Present two example items to the student. Ask the student to say their answer aloud for each word.
      4. Click with the external mouse on the blue button for correct answers and on the red button for incorrect answers. Ensure the pedagogical agent provides feedback based on the student's responses to the examples.
      5. After the examples, ensure the pedagogical agent instructs the student to initiate the task. The procedure remains consistent throughout task execution, recording both correct and incorrect responses.
      6. If the student needs to listen to a word again, click on the speaker Icon for a replay. Note that only one replay per item is allowed.
3. Orange Door: Naming speed and orthographical modules
   1. Rapid automatized naming (RAN) task
      1. Access the program on the computer. Proceed to the orange door within the program interface. Choose Naming Speed to select the task.
      2. Permit the pedagogical agent to provide instructional guidance and demonstrate the task. Present an example of the letter RAN subtask of the naming speed module.
      3. Ask the students to articulate their answers aloud. If the response in the example is accurate, select the blue button on the screen. If incorrect, click on the red button.
      4. Ensure the pedagogical agent provides feedback based on the student's responses. after the example, ensure that the pedagogical agent initiates the task.
      5. Start the task by clicking the left mouse button to begin timing. During the task, a matrix of elements will appear on the screen, depending on the subtask.
      6. Ask the student to name aloud and, in order, the elements of the matrix. Simultaneously, document any errors made by the student.
      7. For each error, use the right button of the external mouse to register the number of errors. Once the student has finished naming the elements, press the left button again to conclude the timing and complete the task for that subtask. Upon completion of the first subtask (letter RAN), the next subtask (number RAN) automatically appears, following the same procedure. This process continues for the remaining subtasks (color RAN and object RAN).
   2. Lexeme and Suffixes task
      1. Access the program on the computer. Navigate to the orange door within the platform interface and click on Lexemes and Suffixes to select the task.
      2. Enable the pedagogical agent to offer instructional guidance and task modeling to the student, including an example item where the student uses an external mouse to select an image corresponding to the target word.
      3. Ensure the pedagogical agent provides feedback based on the student's response to the example and then instructs the student to begin the task.
      4. Maintain procedural consistency throughout the task execution, prompting the student to select an image corresponding to the target word for each item presented and recording both correct hits and missed responses.
   3. Homophone comprehension task
      1. Access the program on the computer. Navigate to the orange door within the platform interface. Click on Homophone Comprehension to select the Task.
      2. Enable the pedagogical agent to offer instructional guidance and task modeling to the student. Present an example item, prompting the student to use an external mouse to select an image corresponding to the target word.
      3. Ensure the pedagogical agent provides feedback based on the student's response to the example and instruct the student to begin the task following the example presentation.
      4. Maintain consistency in the procedural steps throughout the task execution. Prompt the student to select an image corresponding to the target word for each item presented.
      5. Record both correct hits and missed responses during task execution.
4. Green Door: Syntactic processing module
   1. Gender task
      1. Open the program on the computer. Navigate to the green door within the program interface. Click on Gender to select the task.
      2. Allow the pedagogical agent to provide the student with instructions and task modeling. Present two examples for the student to complete, requiring them to click on the corresponding words in each sentence according to their gender agreement using an external mouse.
      3. Ensure the pedagogical agent provides feedback based on the student's response to the examples. Following the example presentation, ensure the pedagogical agent guides the student to begin the task.
      4. Throughout the task execution, maintain consistency in the procedure. Prompt the student to click on the corresponding words in each sentence according to their gender agreement.
      5. Collect both correct and incorrect responses during task execution.
         NOTE: Unlike Spanish, English does not distinguish gender in its grammar.
   2. Number task
      1. Open the program on the computer. Navigate to the green door within the program interface. Click on Number to select the task.
      2. Allow the pedagogical agent to provide the student with instructions and task modeling. Present two examples for the student to complete, requiring them to click on the corresponding words in each sentence according to their number agreement using an external mouse.
      3. Ensure the pedagogical agent provides feedback based on the student's response to the examples. Following the example presentation, ensure the pedagogical agent guides the student to begin the task.
      4. Throughout the task execution, maintain consistency in the procedure. Prompt the student to click on the corresponding words in each sentence according to their number agreement.
      5. Collect both hits and misses during task execution.
         NOTE: Unlike Spanish, English does not distinguish numbers in its grammar.
   3. Functional words task
      1. Open the program on the computer. Navigate to the green door within the program interface. Click on Functional Words to select the task.
      2. Allow the pedagogical agent to provide the student with instructions and task modeling. Present two examples for the student to complete, requiring them to click on the corresponding function words in each sentence according to the sentence's context using an external mouse.
      3. Ensure the pedagogical agent provides feedback based on the student's response to the examples. Following the example presentation, ensure the pedagogical agent guides the student to begin the task.
      4. Throughout the task execution, maintain consistency in the procedure. Prompt the student to click on the corresponding function words in each sentence according to the sentence's context.
      5. Collect both correct and incorrect responses during task execution.
   4. Grammatical structure task
      1. Open the program on the computer. Navigate to the green door within the program interface. Select Grammatical Structure to choose the task.
      2. Allow the pedagogical agent to provide the student with instructions and task demonstrations. Present two examples for the student to complete, requiring them to click on the appropriate sentence for each picture using an external mouse.
      3. Ensure the pedagogical agent provides feedback based on the student's response to the examples. Following the example presentation, ensure the pedagogical agent guides the student to commence the task.
      4. Throughout the task execution, maintain consistency in the procedure. Prompt the student to click on the appropriate sentence for each picture using an external mouse.
      5. Capture both correct and incorrect responses during task execution.
5. Blue Door: Semantic processing module using reading comprehension task
   1. Open the program on the computer. Proceed to the blue door within the program interface. Choose either The Fruits for the informative text or Tino's Getaway for the narrative text.
   2. Allow the pedagogical agent to provide the student with task instructions. Once the type of text has been chosen, display the text on the screen.
   3. The student must read the text and memorize the most relevant information. After finishing reading, ask the student to click on the arrow on the screen using the external mouse to indicate completion of the reading and proceed to the next section.
   4. Allow the pedagogical agent to instruct the student to read the questions and select the correct answer using the external mouse. The student reads the questions and selects the correct answer accordingly.

3. Data analysis

1. To assess the construct validity and diagnostic accuracy of the multimedia battery in a Spanish-speaking population, use confirmatory factor analysis (CFA) and receiver operating characteristic (ROC) curve analysis.
2. Perform CFA to validate the underlying factor structure of the multimedia battery. This analysis allows to test the hypothesis that the data fit a predefined structure based on theoretical expectations.
3. Evaluate the model fit using several fit indices, including the comparative fit index (CFI), Tucker-Lewis index (TLI), root mean square error of approximation (RMSEA), and standardized root mean square residual (SRMR). A good model fit is indicated by CFI and TLI values greater than 0.90, RMSEA values less than 0.08, and SRMR values less than 0.08.
4. To determine the diagnostic accuracy of the multimedia battery, perform ROC curve analysis. This method allows to evaluate the ability of the test to correctly classify individuals with and without reading difficulties. The area under the ROC curve (AUC) provides a measure of the test's overall accuracy. An AUC of 0.5 indicates no diagnostic ability, whereas an AUC of 1.0 indicates perfect diagnostic ability.
5. Identify optimal cutoff points for the battery by analyzing the sensitivity and specificity at various threshold levels.
6. By employing both CFA and ROC curve analysis, perform a comprehensive evaluation of the multimedia battery, confirming its construct validity and diagnostic accuracy in a Spanish-speaking population.

Results

Sample study
The sample included 881 participants from Spain (N = 325), Mexico (N = 169), Guatemala (N = 227), and Chile (N = 160), all of whom were native Spanish speakers. The sample was divided into two groups: 451 in the reading disability (RD) group and 430 in the normally achieving readers (NAR) group. Children with special educational needs-those requiring support and specific educational attention due to sensory impairments, neurological issues, or other conditions-were excluded because the...

Discussion

In this study, confirmatory factor analysis (CFA) was employed to evaluate the factor structure of the Sicole-R battery, comprising one second-order factor and six latent variables representing different modules. The results indicated good model fit, convergent validity, and reliability, confirming the efficacy of the battery in assessing a comprehensive set of cognitive and reading skills that are critical for individuals with dyslexia. Importantly, the consistent performance of the digital tool across diverse demograph...

Materials

Name	Company	Catalog Number	Comments
Sicole-R	Universidad de La Laguna	TF-263- 07