Measuring performance is crucial for any research or clinical application involving brain computer interfaces. CBLE helps evaluate the effectiveness of a system for any particular user. CBLE can be used to predict the user's P300 Speller accuracy from just three to eight characters of data.
To begin, install the CBLE performance estimation graphical user interface. Open MATLAB and change the current directory to the graphical user interface folder. Click on the Apps tab, select My apps, and choose CBLE Performance Estimation.
In the dropdown menu, click Select dataset format and choose the desired option. Next, click the Select input folder button to choose the directory for the EEG dataset. In the number of participants text box, enter the number of participants for the estimation.
If using Brain Invader data, specify the sampling rate of the dataset. Choose a decimation value to downs sample the data set to approximately 20 hertz. Specify the time window for the classification in milliseconds.
Then define the shift window for CBLE in milliseconds. Once done, click the Set parameters button to set analysis parameters. To split the data set, select the number of targets for the training set size.
Click the Split the dataset button to divide the dataset into training and test sets. For Braininvaders, click the train a model button to apply linear regression using equation two on the training dataset. Next, click Predict Accuracy to apply the trained classifier model to the test feature set and predict the accuracies using equation one.
Select the maximum target number X for test set consideration and press find X target accuracy. Then click the find vCBLE button to get the vCBLE for all targets. Click the Calculate RMSE button to calculate the RMSE between both predictions based on vCBLE with BCI accuracy and X target accuracy with BCI accuracy.
Now click accuracy versus vCBLE to observe the relationship between total accuracy and total vCBLE for all participants. Click RMSE of BCI and vCBLE to display the RMSE curve of BCI accuracy and vCBLE. For predicting the accuracy of an individual participant in Sub ID, enter the subject ID.Then select a target number N and click Predict to get the predicted accuracy of the test participant.
Open the CBLE Performance Estimation graphical user interface. In the dropdown menu, click Select dataset format and choose the BCI2000 option. Click the Select input folder button to choose the directory for the EEG dataset.
Enter the number of participants for the estimation. Choose a decimation value and specify the original in CBLE window. Then choose character number X.Next, in the ID length field, enter the subject ID length from the dataset files.
In the channel ID field, indicate the total number of channels or specific channel numbers for analysis. After checking the data format, specify training and testing file names. From the BCI2000 dataset, check the test file.
Then in the test file number fields, enter the test file number. Now click Run, and wait until all the parameters from the checklist are ticked. Then click Accuracy versus vCBLE to observe the relationship between total accuracy and total vCBLE for all participants.
Finally, click RMSE of BCI and vCBLE to display the RMSE curve of BCI accuracy and vCBLE. A strong negative correlation was observed between BCI accuracy plotted against vCBLE for the braininvaders dataset. The RMSE of vCBLE plotted against different test dataset sizes showed that vCBLE performs better than BCI accuracy.
vCBLE is capable of predicting BCI accuracy using only seven characters. vCBLE prediction models indicated that 10 individuals were required to build the regression model for the relationship between vCBLE and accuracy for a particular experimental paradigm. The vCBLE model showed better performance for the Michigan dataset with same day training and testing datasets.
The mean RMSE calculated over three days for vCBLE and accuracy models using Michigan data showed that the vCBLE fit had a lower RMSE when the test included less than six characters. The RMSE of vCBLE accuracy drops by only 0.025 from three characters to the optimal number, suggesting little gain beyond three characters for the small test set. In this work, we use simple linear regression, But CBLE can be computed with any time dependent combination of feature extraction, feature selection, and classifier.
We have used CBLE to reduce the data required for our test sets. Other labs have used it to investigate the variable latency between stimuli and the related brain responses.
