The data used for representative results were collected from the work supported by the National Institute of Child Health and Human Development (NICHD), the National Institutes of Health (NIH) under Grant R21HD054697, and the National Institute on Disability and Rehabilitation Research (NIDRR) in the Department of Education under Grant H133G090005 and Award Number H133P090008. The rest of the work was funded in part by the National Science Foundation (NSF) under award #1910526. Findings and opinions within this work do not necessarily reflect the positions of NICHD, NIH, NIDRR or NSF.

The &#34;CBLE Performance Estimation&#34; GUI was applied to two datasets: &#34;BrainInvaders&#34; dataset and Michigan dataset. For the &#34;BrainInvaders&#34; dataset, the data collection was approved by the Ethical Committee of the University of Grenoble Alpes20. Michigan data were collected under the University of Michigan Institutional Review Board approval19. Data were analyzed under Kansas State University exempt protocol 7516. If collecting new data, follow the user&#39;s IRB-approved process for collecting informed consent. Here, the proposed protocol is evaluated using offline analysis of previously-recorded, de-identified data and therefore did not require additional informed consent.
The graphical user interface (GUI) included in this article is proficient in managing two distinct dataset formats. The first format is associated with the BCI2000 software, while the second format is referred to as the &#34;BrainInvaders&#34; dataset. In order to utilize the &#34;Brain Invaders&#34; format, data must be pre-processed as described in step 1 of the protocol section. However, when dealing with the &#34;BCI2000&#34; dataset format, step 1 can be omitted.
1. Data preparation
<ol>
	<li>BrainInvaders only: Generate the input data file in &#34;.mat&#34; file format that can be used with the &#34;CBLE performance Estimation&#34; graphical user interface (GUI). For a sample script, refer to Supplementary Coding File 2. 
	NOTE: Each data file consists of a two-dimensional matrix comprised of rows that represent observations recorded at distinct time samples. The matrix columns numbered 2 to 17 are recordings derived from 16 EEG electrodes. The first column of the matrix denotes the timestamp of each observation, while column 18 encompasses information related to experimental events. In column 19, there are mostly zeros, but when a non-Target (or Target) flash starts, the numbers change to one (or two) at that specific time. A detailed description may be found in the reference20.</li>
</ol>
2. Downloading and installing the GUI package
<ol>
	<li>Download and install the &#34;CBLE Performance Estimation&#34; GUI.</li>
</ol>
3. Storing the dataset in a subfolder of the GUI location
<ol>
	<li>Ensure that the dataset folder remains within the same directory as the GUI.</li>
	<li>For instance, create a new folder and place the &#34;CBLE Performance Estimation&#34; GUI inside it. Keep all the datasets in a subfolder within &#34;CBLE GUI&#34; named &#34;Dataset.&#34;</li>
</ol>
4. Opening the installed GUI
<ol>
	<li>Open MATLAB (see Table of Materials), change the current directory to the folder where the GUI is placed, click on APPS tab, and select MY APPS.</li>
	<li>Under the &#34;MY APPS&#34; tab, select CBLE Performance Estimation.</li>
</ol>
5. Choosing the dataset format
<ol>
	<li>Select a dataset format from the dropdown Select dataset format.</li>
</ol>
6. Loading the EEG data file
<ol>
	<li>Click on the Select input folder button to choose the directory where the dataset is located.</li>
	<li>Observe the count of data files present in that selected folder. 
	NOTE: In the &#34;Brain Invaders&#34; format, each participant is represented by a single data file. Therefore, the total number of data files indicates the number of participants in the study. However, this is not the case for the &#34;BCI2000&#34; format, as each participant may have multiple train and test files.</li>
</ol>
7. Setting the parameters
<ol>
	<li>Type the number of participants that the user intends to use for the estimation process in the &#34;No. of participants&#34; text box.</li>
	<li>BrainInvaders only: Specify the sampling rate of the dataset. 
	NOTE: BCI2000 files include the sample rate.</li>
	<li>Choose a decimation value to downsample the dataset to approximately 20 Hz in order to improve classification performance21. For example, if the sampling frequency is 256 Hz, then select a decimation value of 13.</li>
	<li>Specify the time window for the classification in milliseconds. 
	NOTE: The recommended initial window size is specified, allowing the starting point to vary from 0 to 100 ms and the ending point from 700 to 800 ms. However, it is important to avoid making the window size excessively large to prevent overlapping with another P300 event.</li>
	<li>Define the shift window for CBLE in milliseconds. 
	NOTE: The &#39;shift window&#39; refers to the expanded time range that the CBLE&#8239;method searches to find the P3 response.&#8239; The classifier will be applied to the epoch starting at the first element of the shift window.&#8239; The classifier is then sequentially applied to epochs starting one sample later, until the epoch extends outside the shift window.&#8239; Thus, the shift window must be larger than the original window; empirically, values less than 100 ms from each side perform well.&#8239; In any case, the margin should be held to less than half the ISI.</li>
	<li>BC2000 only: Enter the length of the subject ID indicated in the dataset files within the &#34;ID length&#34; field. 
	NOTE: The GUI expects the first sub_len characters of the filenames to encode the subject ID.</li>
	<li>BCI2000 only: In the &#34;Channel ID&#34; field, indicate either the total number of channels or specify the specific channel numbers to be used for the analysis.</li>
	<li>Click on the Set parameters button to set all the parameters required for the analysis.</li>
</ol>
8. BrainInvaders only: Splitting the dataset into training and test set
<ol>
	<li>Select a number of targets that represents the size of the training set. The remaining portion of the dataset will be considered as the test dataset. 
	NOTE: To ensure proper training of the model, it is essential to have a sufficiently large training sample. The recommended minimum training sample size is 20, though this may vary depending on the overall dataset size. If regression errors occur during training sessions, it is advisable to increase the training sample size.</li>
	<li>Press the &#34;Split the dataset&#34; button to divide the dataset into the training and test sets. 
	&#8203;NOTE: Each participant will have an equal amount of training data. However, the number of test data may not be equal for all participants due to the possibility of multiple attempts during the task. Consequently, the total number of targets or flashes presented may vary from person to person.</li>
</ol>
9. Training a model with the training dataset
NOTE: Step 9.1 is applicable for &#34;Brain Invaders&#34; format, and step 9.2 is applicable for &#34;BCI2000&#34; format.
<ol>
	<li>BrainInvaders only: Click on the Train a model button to apply linear regression on the training dataset using Equation 2 for training a classifier model.</li>
	<li>BCI2000 only: Indicate training and testing filenames along with their data format (.dat) to distinguish the training and testing files from all files. Then, click on the Train a model button to apply linear regression on the training dataset.</li>
</ol>
10. Predicting the accuracy of the test set
<ol>
	<li>Click on Predict accuracy to apply the trained classifier model to the test feature set and predict&#160;accuracy&#160;using Equation 1.</li>
</ol>
11. Getting X-target accuracies
<ol>
	<li>Select a maximum target number, X, to consider in the test set.</li>
	<li>BCI2000 only: Select a test file number if the user has multiple test files.</li>
	<li>Press Find X target accuracy.</li>
</ol>
12. Calculating vCBLE
<ol>
	<li>Click on the Find vCBLE button to get the vCBLE for all targets.</li>
</ol>
13. Calculating the Root mean square error (RMSE) of BCI accuracy and vCBLE
<ol>
	<li>Click on the Calculate RMSE button to calculate the RMSE between both predictions based on vCBLE with BCI accuracy, and X-target accuracy with BCI accuracy.</li>
</ol>
14. Visualizing the analysis results
<ol>
	<li>Click on the Accuracy vs vCBLE button to observe the relation between total accuracy and total vCBLE for all participants.</li>
	<li>Press on the RMSE of BCI &#38; vCBLE button to show the RMSE curve of BCI accuracy and vCBLE.</li>
</ol>
15. Predicting the performance of an individual participant
<ol>
	<li>To predict the accuracy of an individual participant, place the subject ID in Sub ID. 
	NOTE: Here, the dataset of all participants, excluding the test participant, will be used to train a linear regression model. The vCBLE scores of all other participants and their corresponding test accuracies will be utilized as predictors and labels, respectively, for the classifier.</li>
	<li>Select a target number, n. The prediction will be made based on the accuracy of n-testing characters.</li>
	<li>Click on the Predict button to get the predicted accuracy of the test participant.</li>
</ol>

<ol><li>Rezeika, A., Benda, M., Stawicki, P., Gembler, F., Saboor, A., Volosyak, I. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Brain-Computer+Interface+spellers%3A+A+review%5BTitle%5D)+AND+%22Brain+Science%22%5BJournal%5D)">Brain-Computer Interface spellers: A review.</a> Brain Science. 8 (4), 57(2018).</li>
<li>Gannouni, S., Aledaily, A., Belwafi, K., Aboalsamh, H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Emotion+detection+using+electroencephalography+signals+and+a+zero-time+windowing-based+epoch+estimation+and+relevant+electrode+identification%5BTitle%5D)+AND+%22Scientific+Reports%22%5BJournal%5D)">Emotion detection using electroencephalography signals and a zero-time windowing-based epoch estimation and relevant electrode identification.</a> Scientific Reports. 11 (1), 7071(2021).</li>
<li>Daly, J. J., Wolpaw, J. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Brain-computer+interfaces+in+neurological+rehabilitation%5BTitle%5D)+AND+%22Lancet+Neurology%22%5BJournal%5D)">Brain-computer interfaces in neurological rehabilitation.</a> Lancet Neurology. 7 (11), 1032-1043 (2008).</li>
<li>Birbaumer, N. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Breaking+the+silence%3A+brain-computer+interfaces+%28BCI%29+for+communication+and+motor+control%5BTitle%5D)+AND+%22Psychophysiology%22%5BJournal%5D)">Breaking the silence: brain-computer interfaces (BCI) for communication and motor control.</a> Psychophysiology. 43 (6), 517-532 (2006).</li>
<li>Riccio, A., Simione, L., Schettini, F., Pizzimenti, A., Inghilleri, M., Belardinelli, M. O. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Attention+and+P300-based+BCI+performance+in+people+with+amyotrophic+lateral+sclerosis%5BTitle%5D)+AND+%22Frontiers+in+Human+Neuroscience%22%5BJournal%5D)">Attention and P300-based BCI performance in people with amyotrophic lateral sclerosis.</a> Frontiers in Human Neuroscience. 7, 732(2013).</li>
<li>Finke, A., Lenhardt, A., Ritter, H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((The+MindGame%3A+a+P300-based+brain-computer+interface+game%5BTitle%5D)+AND+%22Neural+Network%22%5BJournal%5D)">The MindGame: a P300-based brain-computer interface game.</a> Neural Network. 22 (9), 1329-1333 (2009).</li>
<li>Farwell, L. A., Donchin, E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Talking+off+the+top+of+your+head%3A+toward+a+mental+prosthesis+utilizing+event-related+brain+potentials.+Electroencephalogr%5BTitle%5D)+AND+%22Clinical+Neurophysiology%22%5BJournal%5D)">Talking off the top of your head: toward a mental prosthesis utilizing event-related brain potentials. Electroencephalogr.</a> Clinical Neurophysiology. 70 (6), 510-523 (1988).</li>
<li>Li, Q., Lu, Z., Gao, N., Yang, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Optimizing+the+performance+of+the+visual+P300-speller+through+active+mental+tasks+based+on+color+distinction+and+modulation+of+task+difficulty%5BTitle%5D)+AND+%22Frontiers+in+Human+Neuroscience%22%5BJournal%5D)">Optimizing the performance of the visual P300-speller through active mental tasks based on color distinction and modulation of task difficulty.</a> Frontiers in Human Neuroscience. 13, 130(2019).</li>
<li>McFarland, D. J., Sarnacki, W. A., Townsend, G., Vaughan, T., Wolpaw, J. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((The+P300-based+brain-computer+interface+%28BCI%29%3A+effects+of+stimulus+rate%5BTitle%5D)+AND+%22Clinical+Neurophysiology%22%5BJournal%5D)">The P300-based brain-computer interface (BCI): effects of stimulus rate.</a> Clinical Neurophysiology. 122 (4), 731-737 (2011).</li>
<li>Krusienski, D. J., Sellers, E. W., Cabestaing, F., Bayoudh, S., McFarland, D. J., Vaughan, T. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((A+comparison+of+classification+techniques+for+the+P300+Speller%5BTitle%5D)+AND+%22Journal+of+Neural+Engineering%22%5BJournal%5D)">A comparison of classification techniques for the P300 Speller.</a> Journal of Neural Engineering. 3 (4), 299-305 (2006).</li>
<li>Sellers, E. W., Donchin, E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((A+P300-based+brain-computer+interface%3A+initial+tests+by+ALS+patients%5BTitle%5D)+AND+%22Clinical+Neurophysiology%22%5BJournal%5D)">A P300-based brain-computer interface: initial tests by ALS patients.</a> Clinical Neurophysiology. 117 (3), 538-548 (2006).</li>
<li>Donchin, E., Spencer, K. M., Wijesinghe, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((The+mental+prosthesis%3A+assessing+the+speed+of+a+P300-based+brain-computer+interface%5BTitle%5D)+AND+%22IEEE+Transactions+on+Rehabilitation+Engineering%22%5BJournal%5D)">The mental prosthesis: assessing the speed of a P300-based brain-computer interface.</a> IEEE Transactions on Rehabilitation Engineering. 8 (2), 174-179 (2000).</li>
<li>Höhne, J., Schreuder, M., Blankertz, B., Tangermann, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((A+novel+9-class+auditory+ERP+paradigm+driving+a+predictive+text+entry+system%5BTitle%5D)+AND+%22Frontiers+in+Neuroscience%22%5BJournal%5D)">A novel 9-class auditory ERP paradigm driving a predictive text entry system.</a> Frontiers in Neuroscience. 5, 99(2011).</li>
<li>Acqualagna, L., Treder, M. S., Blankertz, B. Chroma Speller: Isotropic visual stimuli for truly gaze-independent spelling. 2013 6th International IEEE/EMBS Conference on Neural Engineering (NER), , (2013).</li>
<li>Townsend, G., LaPallo, B. K., Boulay, C. B., Krusienski, D. J., Frye, G. E., Hauser, C. K. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((A+novel+P300-based+brain-computer+interface+stimulus+presentation+paradigm%3A+moving+beyond+rows+and+columns%5BTitle%5D)+AND+%22Clinical+Neurophysiology%22%5BJournal%5D)">A novel P300-based brain-computer interface stimulus presentation paradigm: moving beyond rows and columns.</a> Clinical Neurophysiology. 121 (7), 1109-1120 (2010).</li>
<li>Guger, C., Daban, S., Sellers, E., Holzner, C., Krausz, G., Carabalona, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((How+many+people+are+able+to+control+a+P300-based+brain-computer+interface+%28BCI%29%5BTitle%5D)+AND+%22Neuroscience+Letters%22%5BJournal%5D)">How many people are able to control a P300-based brain-computer interface (BCI).</a> Neuroscience Letters. 462 (1), 94-98 (2009).</li>
<li>Mowla, M. R., Gonzalez-Morales, J. D., Rico-Martinez, J., Ulichnie, D. A., Thompson, D. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((A+comparison+of+classification+techniques+to+predict+Brain-computer+interfaces+accuracy+using+classifier-based+latency+estimation%5BTitle%5D)+AND+%22Brain+Science%22%5BJournal%5D)">A comparison of classification techniques to predict Brain-computer interfaces accuracy using classifier-based latency estimation.</a> Brain Science. 10 (10), 734(2020).</li>
<li>Thompson, D. E., Warschausky, S., Huggins, J. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Classifier-based+latency+estimation%3A+a+novel+way+to+estimate+and+predict+BCI+accuracy%5BTitle%5D)+AND+%22Journal+of+Neural+Engineering%22%5BJournal%5D)">Classifier-based latency estimation: a novel way to estimate and predict BCI accuracy.</a> Journal of Neural Engineering. 10 (1), 016006(2012).</li>
<li>Thompson, D. E., Gruis, K. L., Huggins, J. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((A+plug-and-play+brain-computer+interface+to+operate+commercial+assistive+technology%5BTitle%5D)+AND+%22Disability+and+Rehabilitation%3A+Assistive+Technology%22%5BJournal%5D)">A plug-and-play brain-computer interface to operate commercial assistive technology.</a> Disability and Rehabilitation: Assistive Technology. 9 (2), 144-150 (2014).</li>
<li>Korczowski, L., Ostaschenko, E., Andreev, A., Cattan, G., Coelho Rodrigues, P. L., Gautheret, V., Congedo, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((Brain+Invaders+calibration-less+P300-based+BCI+using+dry+EEG+electrodes+Dataset+%28bi2014a%29+%5BData+set%5D%5BTitle%5D)+AND+%22Zenodo%22%5BJournal%5D)">Brain Invaders calibration-less P300-based BCI using dry EEG electrodes Dataset (bi2014a) [Data set].</a> Zenodo. , (2019).</li>
<li>Krusienski, D. J., Sellers, E. W., Cabestaing, F., Bayoudh, S., McFarland, D. J., Vaughan, T. M., Wolpaw, J. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/pubmed?term=((A+comparison+of+classification+techniques+for+the+P300+Speller%5BTitle%5D)+AND+%22Journal+of+Neural+Engineering%22%5BJournal%5D)">A comparison of classification techniques for the P300 Speller.</a> Journal of Neural Engineering. 3 (4), 299-305 (2006).</li>
</ol>

All authors declare they do not have any conflicts of interest.

This article outlined a method for estimating BCI accuracy using a small P300 dataset. Here, the current protocol was developed based on the &#34;bi2014a&#34; dataset, although the efficacy of the protocol was confirmed on two different datasets. To successfully implement this technique, it is crucial to establish certain variables, such as the epoch window for the original data, the window for time shifting, the down-sampling ratio, and the size of both the training and testing datasets. These variables are determined b...

Brain-computer interfaces (BCIs) are a noninvasive technology that allows individuals to communicate through machines directly without regard for physical limitations imposed by the body. BCI can be utilized as an assistive device operated directly by the brain. BCI uses the brain activity of a user to determine if the user intends to choose a certain key (letter, number, or symbol) displayed on the screen1. In a typical computer system, a user physically presses the intended key on a keyboard. However, in a BCI system with a visual display, the user needs to focus on the desired key. Then, BCI will select the intended key by analyzing the measured brain signals1. The activity of the brain can be measured using various techniques. Though there are competing BCIs technologies, electroencephalogram (EEG) is considered a leading technique due to its noninvasive nature, high temporal resolution, reliability, and relatively low cost2.
Applications of BCI include communication, device control, and also entertainment3,4,5,6. One of the most active BCI application areas is the P300 speller, which was introduced by Farwell and Donchin7. The P300 is an event-related potential (ERP) produced in response to the recognition of a rare but relevant stimulus8. When a person recognizes their target stimulus, they automatically produce a P300. The P300 is an effective signal for a BCI because it conveys the participant&#39;s recognition of the target event without requiring an outward response9.
The P300 BCI has attracted researchers from computer science, electrical engineering, psychology, human factors, and various other disciplines. Advances have been made in signal processing, classification algorithms, user interfaces, stimulation schemes, and many other areas10,11,12,13,14,15. However, regardless of the research area, the common thread in all of this research is the necessity of measuring the BCI system performance. This task typically requires the generation of a test data set. This necessity is not limited to research; eventual clinical application as an assistive technology will likely require individual validation sets for each end user to ensure the system can generate reliable communication.
Despite the considerable research applied toward the P300 BCI, the systems are still quite slow. While the majority of people are able to use a P300 BCI16, most P300 Spellers produce text on the order of 1-5 characters per minute. Unfortunately, this slow speed means that generating test data sets requires substantial time and effort for participants, experimenters, and eventual end users. Measuring BCI system accuracy is a binomial parameter estimation problem, and many characters of data are necessary for a good estimate.
To estimate the presence or absence of the P300 ERP, most classifiers use a binary classification model, which involves assigning a binary label (e.g., &#34;presence&#34; or &#34;absence&#34;) to each trial or epoch of EEG data. The general equation used by most classifiers can be expressed as:
<img alt="Equation 1" src="/files/ftp_upload/64959/64959eq01.jpg" style="margin: auto;" />
where <img alt="Equation 2" src="/files/ftp_upload/64959/64959eq02.jpg" style="margin: auto;" /> is called the classifier&#39;s score, which represents the probability of the P300 response being present, x is the feature vector extracted from the EEG signal, and b is a bias term17. The function f is a decision function that maps the input data to the output label, and is learned from a set of labeled training data using a supervised learning algorithm17. During training, the classifier is trained on a labeled dataset of EEG signals, where each signal is labeled as either having a P300 response or not. The weight vector and bias term are optimized to minimize the error between the predicted output of the classifier and the true label of the EEG signal. Once the classifier is trained, it can be used to predict the presence of the P300 response in new EEG signals.
Different classifiers can use different decision functions, such as linear discriminant analysis (LDA), stepwise linear discriminant analysis (SWLDA), least squares (LS), logistic regression, support vector machines (SVM), or neural networks (NNs). The least squares classifier is a linear classifier that minimizes the sum of squared errors between the predicted class labels and the true class labels. This classifier predicts the class label of a new test sample using the following equation:
<img alt="Equation 3" src="/files/ftp_upload/64959/64959eq03.jpg" style="margin: auto;" />&#160; &#160; (1)
where the sign function returns +1 if the product is positive and -1 if it is negative, and the weight vector <img alt="Equation 4" src="/files/ftp_upload/64959/64959eq04.jpg" style="margin: auto;" /> is obtained from the feature set of the training data, (x) and class labels (y) using the below equation:
<img alt="Equation 5" src="/files/ftp_upload/64959/64959eq05.jpg" style="margin: auto;" />&#160; &#160; (2)
In earlier research, we argued that Classifier-Based Latency Estimation (CBLE) can be used to estimate BCI accuracy17,18,19. CBLE is a strategy for evaluating latency variation by exploiting the classifier&#39;s temporal sensitivity18. While the conventional approach to P300 classification involves using a single time window that is synchronized with each stimulus presentation, the CBLE method involves creating multiple time-shifted copies of the post-stimulus epochs. Then it detects the time shift that results in the maximum score in order to estimate the latency of the P300 response17,18. Here, this work presents a protocol that estimates BCI performance from a small dataset using CBLE. As a representative analysis, the number of characters is varied to make predictions of the overall performance of an individual. For both example datasets, the root mean square error (RMSE) for vCBLE and actual BCI accuracy were computed. The results indicate that the RMSE from vCBLE predictions, using its fitted data, was consistently lower than the accuracy derived from 1 to 7 tested characters.
We developed a Graphical User Interface (GUI) called &#34;CBLE Performance Estimation&#34; for the implementation of the proposed methodology. The example code is also provided (Supplementary Coding File 1) that operates on the MATLAB platform. The example code performs all of the steps applied in the GUI, but the steps are provided to assist the reader with adapting to a new dataset. This code employs a publicly available dataset &#34;Brain Invaders calibration-less P300-based BCI using dry EEG electrodes Dataset (bi2014a)&#34; to evaluate the proposed method20. Participants played up to three game sessions of Brain Invaders, each session having 9 levels of the game. The data collection continued until all levels were completed or the participant lost all control over the BCI system. The Brain Invaders interface included 36 symbols that flashed in 12 groups of six aliens. According to the Brain Invaders P300 paradigm, a repetition was created by 12 flashes, one for each group. Out of these 12 flashes, two flashes contained the Target symbol (known as Target flashes), while the remaining 10 flashes did not contain the Target symbol (known as non-Target flashes). More information on this paradigm can be found in the original reference20.
The CBLE approach was also implemented on a Michigan dataset, which contained data from 40 participants18,19. Here, the data of eight participants had to be discarded because their tasks were incomplete. The whole study required three visits from each participant. On the first day, each participant typed a 19-character training sentence, followed by three 23-character testing sentences on Days 1, 2, and 3. In this example, the keyboard included 36 characters which were grouped into six rows and six columns. Each row or column was flashed for 31.25 milliseconds with an interval of 125 milliseconds between flashes. Between characters, a 3.5 s pause was provided.
Figure 1 shows the block diagram of the proposed method. The detailed procedure is described in the protocol section.

<table><tbody><tr><td>MATLAB 2021</td><td>Matlab</td><td>N/A</td><td>Any recent MATLAB version can be used.</td></tr></tbody></table>

p300-based brain-computer interface speller performance estimation with classifier-based latency estimation

Performance estimation is a necessary step in the development and validation of Brain-Computer Interface (BCI) systems. Unfortunately, even modern BCI systems are slow, making collecting sufficient data for validation a time-consuming task for end users and experimenters alike. Yet without sufficient data, the random variation in performance can lead to false inferences about how well a BCI is working for a particular user. For example, P300 spellers commonly operate around 1-5 characters per minute. To estimate accuracy with a 5% resolution requires 20 characters (4-20 min). Despite this time investment, the confidence bounds for accuracy from 20 characters can be as much as &plusmn;23% depending on observed accuracy. A previously published method, Classifier-Based Latency Estimation (CBLE), was shown to be highly correlated with BCI accuracy. This work presents a protocol for using CBLE to predict a user's P300 speller accuracy from relatively few characters (~3-8) of typing data. The resulting confidence bounds are tighter than those produced by traditional methods. The method can thus be used to estimate BCI performance more quickly and/or more accurately.

The proposed protocol has been tested on two different datasets: &#34;BrainInvaders&#34; and the Michigan dataset. These datasets are already introduced briefly in the Introduction section. The parameters used for this two datasets are mentioned in Table 1. Figures 2-4 depict the findings obtained using the &#34;BrainInvaders&#34; dataset, whereas Figures 5-7 demonstrate the res...

Watch this Scientific Journal Video about P300-Based Brain-Computer Interface Speller Performance Estimation with Classifier-Based Latency Estimation at JoVE.com

P300-Based Brain-Computer Interface Speller Performance Estimation with Classifier-Based Latency Estimation

This article presents a method for estimating same-day P300 speller Brain-Computer Interface (BCI) accuracy using a small testing dataset.

Performance estimation is a necessary step in the development and validation of Brain-Computer Interface (BCI) systems. Unfortunately, even modern BCI ...

p300-based-brain-computer-interface-speller-performance-estimation

Research

JoVE Journal

Neuroscience

503 Views.  Kansas State University. Het meten van prestaties is cruciaal voor elk onderzoek of klinische toepassing waarbij hersencomputerinterfaces betrokken zijn. CBLE helpt bij het evalueren van de effectiviteit van een systeem voor een bepaalde gebruiker. CBLE kan worden gebruikt om de nauwkeurigheid van de P300 Speller van de gebruiker te voorspellen op basis van slechts drie tot acht tekens aan gegevens.Installeer om te beginnen de grafische gebruikersinterface voor het schatten van CBLE-prestaties. Open MATLAB en ...

Video: P300-Based Brain-Computer Interface Speller Performance Estimation with Classifier-Based Latency Estimation

Modulating Cognition Using Transcranial Direct Current Stimulation of the Cerebellum

Numerous studies have emerged recently that demonstrate the possibility of modulating, and in some cases enhancing, cognitive processes by exciting brain regions involved in working memory and attention using transcranial electrical brain stimulation. Some researchers now believe the cerebellum supports cognition, possibly via a remote neuromodulatory effect on the prefrontal cortex. This paper describes a procedure for investigating a role for the cerebellum in cognition using transcranial direct current stimulation (tDCS), and a selection of information-processing tasks of varying task difficulty, which have previously been shown to involve working memory, attention and cerebellar functioning. One task is called the Paced Auditory Serial Addition Task (PASAT) and the other a novel variant of this task called the Paced Auditory Serial Subtraction Task (PASST). A verb generation task and its two controls (noun and verb reading) were also investigated. All five tasks were performed by three separate groups of participants, before and after the modulation of cortico-cerebellar connectivity using anodal, cathodal or sham tDCS over the right cerebellar cortex. The procedure demonstrates how performance (accuracy, verbal response latency and variability) could be selectively improved after cathodal stimulation, but only during tasks that the participants rated as difficult, and not easy. Performance was unchanged by anodal or sham stimulation. These findings demonstrate a role for the cerebellum in cognition, whereby activity in the left prefrontal cortex is likely dis-inhibited by cathodal tDCS over the right cerebellar cortex. Transcranial brain stimulation is growing in popularity in various labs and clinics. However, the after-effects of tDCS are inconsistent between individuals and not always polarity-specific, and may even be task- or load-specific, all of which requires further study. Future efforts might also be guided towards neuro-enhancement in cerebellar patients presenting with cognitive impairment once a better understanding of brain stimulation mechanisms has emerged.

Transcranial direct current stimulation (tDCS) over the cerebellum exerts a remote effect on the prefrontal cortex, which can modulate cognition and performance. This was demonstrated using two information-processing tasks of varying complexity, whereby only cathodal tDCS improved performance when the task was difficult, but not easy.

Numerous studies have emerged recently that demonstrate the possibility of modulating, and in some cases enhancing, cognitive processes by exciting brain ...

Behavior

Interaction between Phonological and Semantic Processes in Visual Word Recognition using Electrophysiology

Controversies have always existed in research related to reading abilities; on whether printed words are perceived in a feedforward manner based on orthographic information after which, other representations, such as phonology and semantics are activated, or whether these are fully interactive and high-level semantic information affects early processing. An interference paradigm was implemented in the presented protocol of phonological and semantic judgment tasks that utilized the same precede-target pairs to explore the relative order of phonological and semantic activation. The high- and low-frequency target words were preceded with three conditions: semantically related, phonological-related (homophones), or unrelated. The results showed that the induced P200 component of low-frequency word pairs was significantly greater than high-frequency words in both the semantic and phonological tasks. In addition, both the homophones in the semantic task and the semantically related pairs in the phonological task caused reduction in N400 when compared to the the control condition, word frequency-independently. It is worth noting that for the low-frequency pairs in the phonological judgment task, the P200 released by the semantically related word pairs was significantly larger than that in the control condition. Overall, semantic processing in phonological tasks and phonological processing in semantic tasks were found in both high- and low-frequency words, suggesting that the interaction between semantics and phonology may operate in a task-independent manner. However, the specific time this interaction occurred may have been affected by the task and frequency.

We present a protocol to explore the relative activation sequence of phonology and semantics in visual word recognition. The results show that consistent with interactive accounts, semantic and phonological representations may be processed interactively, and higher-level linguistic representations may affect early processing.

Controversies have always existed in research related to reading abilities; on whether printed words are perceived in a feedforward manner based on ...

EEG Study on Recognizing Familiar Speakers in Expressive Speech

Memorization-Based Training and Testing Paradigm for Robust Vocal Identity Recognition in Expressive Speech Using Event-Related Potentials Analysis

Recognizing familiar speakers from vocal streams is a fundamental aspect of human verbal communication. However, it remains unclear how listeners can still discern the speaker's identity in expressive speech. This study develops a memorization-based individual speaker identity recognition approach and an accompanying electroencephalogram (EEG) data analysis pipeline, which monitors how listeners recognize familiar speakers and tell unfamiliar ones apart. EEG data captures online cognitive processes during new versus old speaker distinction based on voice, offering a real-time measure of brain activity, overcoming limits of reaction times and accuracy measurements. The paradigm comprises three steps: listeners establish associations between three voices and their names (training); listeners indicate the name corresponding to a voice from three candidates (checking); listeners distinguish between three old and three new speaker voices in a two-alternative forced-choice task (testing). The speech prosody in testing was either confident or doubtful. EEG data were collected using a 64-channel EEG system, followed by preprocessing and imported into RStudio for ERP and statistical analysis and MATLAB for brain topography. Results showed an enlarged late positive component (LPC) was elicited in the old-talker compared to the new-talker condition in the 400-850 ms window in the Pz and other wider range of electrodes in both prosodies. Yet, the old/new effect was robust in central and posterior electrodes for doubtful prosody perception, whereas the anterior, central, and posterior electrodes are for confident prosody condition. This study proposes that this experiment design can serve as a reference for investigating speaker-specific cue-binding effects in various scenarios (e.g., anaphoric expression) and pathologies in patients like phonagnosia.

Author Spotlight: Investigating the Impact of Emotional Prosodies on Voice Recognition and Perception

The study introduces a training-testing paradigm to investigate old/new effects of event-related potentials in confident and doubtful prosodic scenarios. Data reveals an enhanced late positive component between 400-850 ms at Pz and other electrodes. This pipeline can explore factors beyond speech prosody and their influence on cue-binding target identification.

Recognizing familiar speakers from vocal streams is a fundamental aspect of human verbal communication. However, it remains unclear how listeners can ...

A Single-Channel and Non-Invasive Wearable Brain-Computer Interface for Industry and Healthcare

The present work focuses on how to build a wearable brain-computer interface (BCI). BCIs are a novel means of human-computer interaction that relies on direct measurements of brain signals to assist both people with disabilities and those who are able-bodied. Application examples include robotic control, industrial inspection, and neurorehabilitation. Notably, recent studies have shown that steady-state visually evoked potentials (SSVEPs) are particularly suited for communication and control applications, and efforts are currently being made to bring BCI technology into daily life. To achieve this aim, the final system must rely on wearable, portable, and low-cost instrumentation. In exploiting SSVEPs, a flickering visual stimulus with fixed frequencies is required. Thus, in considering daily-life constraints, the possibility to provide visual stimuli by means of smart glasses was explored in this study. Moreover, to detect the elicited potentials, a commercial device for electroencephalography (EEG) was considered. This consists of a single differential channel with dry electrodes (no conductive gel), thus achieving the utmost wearability and portability. In such a BCI, the user can interact with the smart glasses by merely staring at icons appearing on the display. Upon this simple principle, a user-friendly, low-cost BCI was built by integrating extended reality (XR) glasses with a commercially available EEG device. The functionality of this wearable XR-BCI was examined with an experimental campaign involving 20 subjects. The classification accuracy was between 80%-95% on average depending on the stimulation time. Given these results, the system can be used as a human-machine interface for industrial inspection but also for rehabilitation in ADHD and autism.

This paper discusses how to build a brain-computer interface by relying on consumer-grade equipment and steady-state visually evoked potentials. For this, a single-channel electroencephalograph exploiting dry electrodes was integrated with augmented reality glasses for stimuli presentation and output data visualization. The final system was non-invasive, wearable, and portable.

The present work focuses on how to build a wearable brain-computer interface (BCI). BCIs are a novel means of human-computer interaction that relies on ...

Engineering

Assessment and Communication for People with Disorders of Consciousness

In this experiment, we demonstrate a suite of hybrid Brain-Computer Interface (BCI)-based paradigms that are designed for two applications: assessing the level of consciousness of people unable to provide motor response and, in a second stage, establishing a communication channel for these people that enables them to answer questions with either &#39;yes&#39; or &#39;no&#39;. The suite of paradigms is designed to test basic responses in the first step and to continue to more comprehensive tasks if the first tests are successful. The latter tasks require more cognitive functions, but they could provide communication, which is not possible with the basic tests. All assessment tests produce accuracy plots that show whether the algorithms were able to detect the patient&#39;s brain&#39;s response to the given tasks. If the accuracy level is beyond the significance level, we assume that the subject understood the task and was able to follow the sequence of commands presented via earphones to the subject. The tasks require users to concentrate on certain stimuli or to imagine moving either the left or right hand. All tasks are designed around the assumption that the user is unable to use the visual modality, and thus, all stimuli presented to the user (including instructions, cues, and feedback) are auditory or tactile.

With this experiment, one might be able to detect consciousness in people with disorders of consciousness. Furthermore, the approach can create a simple communication channel that enables people to give simple YES/NO answers to questions.

In this experiment, we demonstrate a suite of hybrid Brain-Computer Interface (BCI)-based paradigms that are designed for two applications: assessing the ...

Motor Imagery Using VR Digital Avatars in a Brain-Computer Interface System

Motor Imagery Performance Through Embodied Digital Twins in a Virtual Reality-Enabled Brain-Computer Interface Environment

This study introduces an innovative framework for neurological rehabilitation by integrating brain-computer interfaces (BCI) and virtual reality (VR) technologies with the customization of three-dimensional (3D) avatars. Traditional approaches to rehabilitation often fail to fully engage patients, primarily due to their inability to provide a deeply immersive and interactive experience. This research endeavors to fill this gap by utilizing motor imagery (MI) techniques, where participants visualize physical movements without actual execution. This method capitalizes on the brain's neural mechanisms, activating areas involved in movement execution when imagining movements, thereby facilitating the recovery process. The integration of VR's immersive capabilities with the precision of electroencephalography (EEG) to capture and interpret brain activity associated with imagined movements forms the core of this system. Digital Twins in the form of personalized 3D avatars are employed to significantly enhance the sense of immersion within the virtual environment. This heightened sense of embodiment is crucial for effective rehabilitation, aiming to bolster the connection between the patient and their virtual counterpart. By doing so, the system not only aims to improve motor imagery performance but also seeks to provide a more engaging and efficacious rehabilitation experience. Through the real-time application of BCI, the system allows for the direct translation of imagined movements into virtual actions performed by the 3D avatar, offering immediate feedback to the user. This feedback loop is essential for reinforcing the neural pathways involved in motor control and recovery. The ultimate goal of the developed system is to significantly enhance the effectiveness of motor imagery exercises by making them more interactive and responsive to the user's cognitive processes, thereby paving a new path in the field of neurological rehabilitation.

Author Spotlight: Enhancing Neurorehabilitation Through EEG, Motor Imagery, and Virtual Reality

Motor imagery in a virtual reality environment has wide applications in brain-computer interface systems. This manuscript outlines the use of personalized digital avatars that resemble the participants performing movements imagined by the participant in a virtual reality environment to enhance immersion and a sense of body ownership.

This study introduces an innovative framework for neurological rehabilitation by integrating brain-computer interfaces (BCI) and virtual reality (VR) ...

Bioengineering

Assessment of Audio-Tactile Sensory Substitution Training in Participants with Profound Deafness Using the Event-Related Potential Technique

This paper examines the application of electroencephalogram-based methods to assess the effects of audio-tactile substitution training in young, profoundly deaf (PD) participants, with the aim of analyzing the neural mechanisms associated with vibrotactile complex sound discrimination. Electrical brain activity reflects dynamic neural changes, and the temporal precision of event-related potentials (ERPs) has proven to be key in studying time-locked processes while performing behavioral tasks that involve attention and working memory. 
The current protocol was designed to study electrophysiological activity in PD subjects while they performed a continuous performance task (CPT) using complex-sound stimuli, consisting of five different animal sounds delivered through a portable stimulator system worn on the right index finger. As a repeated-measures design, electroencephalogram (EEG) recordings in standard conditions were performed before and after a brief training program (five 1 h sessions over 15 days), followed by offline artifact correction and epoch averaging, to obtain individual and grand-mean waveforms. Behavioral results show a significant improvement in discrimination and a more robust P3-like centroparietal positive waveform for the target stimuli after training. In this protocol, ERPs contribute to the further understanding of learning-related neural changes in PD subjects associated with audio-tactile discrimination of complex sounds.

This protocol is designed to explore underlying learning-related electrophysiological changes in subjects with profound deafness after a short training period in audio-tactile sensory substitution by applying the event-related potential technique.

This paper examines the application of electroencephalogram-based methods to assess the effects of audio-tactile substitution training in young, ...

A Visuospatial Planning Task Coupled with Eye-Tracker and Electroencephalogram Systems

The planning process, characterized by the capability to formulate an organized plan to reach a goal, is essential for human goal-directed behavior. Since planning is compromised in several neuropsychiatric disorders, the implementation of proper clinical and experimental tests to examine planning is critical. Due to the nature of the deployment of planning, in which several cognitive domains participate, the assessment of planning and the design of behavioral paradigms coupled with neuroimaging methods are current challenges in cognitive neuroscience. A planning task was evaluated in combination with an electroencephalogram (EEG) system and eye movement recordings in 27 healthy adult participants. Planning can be separated into two stages: a mental planning stage in which a sequence of steps is internally represented and an execution stage in which motor action is used to achieve a previously planned goal. Our protocol included a planning task and a control task. The planning task involved solving 36 maze trials, each representing a zoo map. The task had four periods: i) planning, where the subjects were instructed to plan a path to visit the locations of four animals according to a set of rules; ii) maintenance, where the subjects had to retain the planned path in their working memory; iii) execution, where the subjects used eye movements to trace the previously planned path as indicated by the eye-tracker system; and iv) response, where the subjects reported the order of the visited animals. The control task had a similar structure, but the cognitive planning component was removed by modifying the task goal. The spatial and temporal patterns of the EEG revealed that planning induces a gradual and lasting rise in frontal-midline theta activity (FM&#952;) over time. The source of this activity was identified within the prefrontal cortex by source analyses. Our results suggested that the experimental paradigm combining EEG and eye-tracker systems was optimal for evaluating cognitive planning.

The study of cognitive planning combining EEG and eye-tracking systems provides a multimodal approach to investigate the neural mechanisms that mediate cognitive control and goal-directed behavior in humans. Here, we describe a protocol for investigating the role of brain oscillations and eye movements in planning performance.

The planning process, characterized by the capability to formulate an organized plan to reach a goal, is essential for human goal-directed behavior. Since ...

Electroencephalography Network Indices as Biomarkers of Upper Limb Impairment in Chronic Stroke

Alteration of electroencephalography (EEG) signals during task-specific movement of the impaired limb has been reported as a potential biomarker for the severity of motor impairment and for the prediction of motor recovery in individuals with stroke. When implementing EEG experiments, detailed paradigms and well-organized experiment protocols are required to obtain robust and interpretable results. In this protocol, we illustrate a task-specific paradigm with upper limb movement and methods and techniques needed for the acquisition and analysis of EEG data. The paradigm consists of 1 min of rest followed by 10 trials comprising alternating 5 s and 3 s of resting and task (hand extension)-states, respectively, over 4 sessions. EEG signals were acquired using 32 Ag/AgCl scalp electrodes at a sampling rate of 1,000 Hz. Event-related spectral perturbation analysis associated with limb movement and functional network analyses at the global level in the low-beta (12-20 Hz) frequency band were performed. Representative results showed an alteration of the functional network of low-beta EEG frequency bands during movement of the impaired upper limb, and the altered functional network was associated with the degree of motor impairment in chronic stroke patients. The results demonstrate the feasibility of the experimental paradigm in EEG measurements during upper limb movement in individuals with stroke. Further research using this paradigm is needed to determine the potential value of EEG signals as biomarkers of motor impairment and recovery.

The experimental protocol demonstrates the paradigm for acquiring and analyzing electroencephalography (EEG) signals during upper limb movement in individuals with stroke. The alteration of the functional network of low-beta EEG frequency bands was observed during the movement of the impaired upper limb and was associated with the degree of motor impairment.

Alteration of electroencephalography (EEG) signals during task-specific movement of the impaired limb has been reported as a potential biomarker for the ...

Language: The N400 in Semantic Incongruity

Source: Laboratories of Sarah I. Gimbel and Jonas T. Kaplan&#8212; University of Southern California
Understanding language is one of the most complex cognitive tasks that humans are capable of. Given the incredible amount of possible choices when combining individual words to form meaning in sentences, it is crucial that the brain is able to identify when words form coherent combinations and when an anomaly appears that undermines meaning. Extensive research has shown that certain scalp-recorded electrical events are sensitive to deviations in this kind of expectation. Importantly, these electrical signatures of incongruity are specific to unexpected meanings, and are therefore different from the brain's general responses to other kinds of anomalies. 
The neurophysiological correlates of semantic incongruity have been experimentally examined through the use of paradigms that present semantically congruent and incongruent ends to sentences. Originally introduced in 1980, the semantic incongruity task presents the participant with a series of sentences that end with either a congruent or incongruent word. To test that the response is from semantic incongruity and not more generally due to surprise, some sentences included words presented in a different size.1 The semantically incongruent end to a sentence has been shown to elicit specific electrical events that are recordable at the scalp known as event-related potentials (ERPs). An ERP is the measured brain response resulting from a specific sensory, cognitive, or motor event. ERPs are measured using electroencephalography (EEG), a noninvasive means of evaluating brain function in patients with disease and normally functioning individuals. A specific ERP component found across the scalp, known as the N400, shows greater amplitude in response to semantically incongruent events. The N400 is a negative-going deflection in the EEG signal that occurs about between 250 and 400 ms after stimulus onset. In general, early potentials reflect sensory-motor processing, while later potentials like the N400 reflect cognitive processing. 
In this video, we show how to administer a semantic incongruity task using EEG. The video will cover the setup and administration of EEG, and analysis of ERPs related to both control and target stimuli in the semantic incongruity. In this task, participants are set up with the EEG electrodes, then brain activity is recorded while they view control sentences and semantically incongruent sentences. The EEG procedure is similar to that of Habibi et al.,1 and the task is modeled after Kutas and Hillyard.2 When the ERPs are averaged across the congruous and incongruous sentences, the neural correlates of each event can be compared in a selected time window.

Studying Language Processing: the N400 in Semantic Incongruity

Source: Laboratories of Sarah I. Gimbel and Jonas T. Kaplan&#8212; University of Southern California
Understanding language is one of the most complex ...

P300-Based Brain-Computer Interface Speller Performance Estimation with Classifier-Based Latency Estimation

Samenvatting

Meer video's verkennen