Name: using electroencephalography measurements and high-quality video recording for analyzing visual perception of media content
Uploaded: 2018-05-26T14:00:00
Description: Watch this Scientific Journal Video about Using Electroencephalography Measurements and High-quality Video Recording for Analyzing Visual Perception of Media Content at JoVE.com

The present study has been supported by a Spanish Ministry of Economy and Competitiveness (BFU2014-56692-R and BFU2017-82375-R) grants.

All methods described here have been approved by the Ethics Commission for Research with Animals and Humans (CEEAH) of the Universitat Aut&#242;noma de Barcelona.
1. Creation and Presentation of the Visual Stimuli
<ol>
	<li>Create the video stimuli according to the desired goals.
	<ol>
		<li>Decide which variables are of interest for the video stimuli. 
		NOTE: For example, in the present investigation, the main variable of interest is the editing style.
		<ol>
			<li>Decide the duratio...

<ol>
	<li>Shapiro, K. L., Raymond, J. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+attentional+blink:+temporal+constraints+on+consciousness.">The attentional blink: temporal constraints on consciousness.</a> Attention and Time. , 35-48 (2008).</li><li>VanderWerf, F., Brassinga, P., Reits, D., Aramideh, M., Ongerboer de Visser, B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Eyelid+movements:+behavioral+studies+of+blinking+in+humans+under+different+stimulus+conditions.">Eyelid movements: behavioral studies of blinking in humans under different stimulus conditions.</a> Journal of Neurophysiology. 89 (5), 2784-2796 (2003).</li><li>Andreu-S&#225;nchez, C., Mart&#237;n-Pascual, M. &. #. 1. 9. 3. ;., Gruart, A., Delgado-Garc&#237;a, J. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Eyeblink+rate+watching+classical+Hollywood+and+post-classical+MTV+editing+styles,+in+media+and+non-media+professionals.">Eyeblink rate watching classical Hollywood and post-classical MTV editing styles, in media and non-media professionals.</a> Scientific Reports. 7, 43267 (2017).</li><li>Bour, L. J., Aramideh, M., de Visser, B. W. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Neurophysiological+aspects+of+eye+and+eyelid+movements+during+blinking+in+humans.">Neurophysiological aspects of eye and eyelid movements during blinking in humans.</a> Journal of Neurophysiology. 83 (1), 166-176 (2000).</li><li>Delgado-Garc&#237;a, J. M., Gruart, A., M&#250;nera, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Neural+organization+of+eyelid+responses.">Neural organization of eyelid responses.</a> Movement disorders: official journal of the Movement Disorder Society. 17 Suppl 2, S33-S36 (2002).</li><li>Wiseman, R., Nakano, T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Blink+and+you'll+miss+it:+the+role+of+blinking+in+the+perception+of+magic+tricks.">Blink and you'll miss it: the role of blinking in the perception of magic tricks.</a> PeerJ. (4), e1873 (2016).</li><li>Fogarty, C., Stern, J. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Eye+movements+and+blinks:+their+relationship+to+higher+cognitive+processes.">Eye movements and blinks: their relationship to higher cognitive processes.</a> International Journal of Psychophysiology official journal of the International Organization of Psychophysiology. 8, 35-42 (1989).</li><li>Andreu-S&#225;nchez, C., Mart&#237;n-Pascual, M. &. #. 1. 9. 3. ;., Gruart, A., Delgado-Garc&#237;a, J. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Looking+at+reality+versus+watching+screens:+Media+professionalization+effects+on+the+spontaneous+eyeblink+rate.">Looking at reality versus watching screens: Media professionalization effects on the spontaneous eyeblink rate.</a> PLoS ONE. 12 (5), (2017).</li><li>Nakano, T., Yamamoto, Y., Kitajo, K., Takahashi, T., Kitazawa, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Synchronization+of+spontaneous+eyeblinks+while+viewing+video+stories.">Synchronization of spontaneous eyeblinks while viewing video stories.</a> Proceedings. Biological sciences / The Royal Society. 276, 3635-3644 (2009).</li><li>Tadel, F., Bock, E., Mosher, J. C., Baillet, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Detect and remove artifacts. , (2015).</li><li>Jiang, X., Tien, G., Huang, D., Zheng, B., Atkins, M. S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Capturing+and+evaluating+blinks+from+video-based+eyetrackers.">Capturing and evaluating blinks from video-based eyetrackers.</a> Behavior Research Methods. 45 (3), 656-663 (2013).</li><li>Pedrotti, M., Lei, S., Dzaack, J., Rotting, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+data-driven+algorithm+for+offline+pupil+signal+preprocessing+and+eyeblink+detection+in+low-speed+eye-tracking+protocols.">A data-driven algorithm for offline pupil signal preprocessing and eyeblink detection in low-speed eye-tracking protocols.</a> Behavior Research Methods. 43 (2), 372-383 (2011).</li><li>Adam, A., Ibrahim, Z., Mokhtar, N., Shapiai, M. I., Mubin, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Evaluation+of+different+peak+models+of+eye+blink+EEG+for+signal+peak+detection+using+artificial+neural+network.">Evaluation of different peak models of eye blink EEG for signal peak detection using artificial neural network.</a> Neural Network World. 26 (1), 67-89 (2016).</li><li>Zhu, Z., Ji, Q. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Robust+real-time+eye+detection+and+tracking+under+variable+lighting+conditions+and+various+face+orientations.">Robust real-time eye detection and tracking under variable lighting conditions and various face orientations.</a> Computer Vision and Image Understanding. 98, 124-154 (2005).</li><li>Lalonde, M., Byrns, D., Gagnon, L., Teasdale, N., Laurendeau, D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Real-time+eye+blink+detection+with+GPU-based+SIFT+tracking.">Real-time eye blink detection with GPU-based SIFT tracking.</a> Proceedings - Fourth Canadian Conference on Computer and Robot Vision, CRV 2007. , 481-487 (2007).</li><li>Chang, W., Cha, H., Kim, K., Im, C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Detection+of+eye+blink+artifacts+from+single+prefrontal+channel+electroencephalogram.">Detection of eye blink artifacts from single prefrontal channel electroencephalogram.</a> Computer Methods and Programs in Biomedicine. 124, 19-30 (2016).</li><li>Tadel, F., Bock, E., Mosher, J. C., Baillet, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Brainstorm3: Function process_evt_detect.m. , (2018).</li><li>Hall, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+origin+and+purposes+of+blinking.">The origin and purposes of blinking.</a> The British Journal of Ophthalmology. 29 (9), 445-467 (1945).</li><li>Nakano, T., Kitazawa, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Eyeblink+entrainment+at+breakpoints+of+speech.">Eyeblink entrainment at breakpoints of speech.</a> Experimental Brain Research. 250, 577-581 (2010).</li><li>Siegle, G. J., Ichikawa, N., Steinhauer, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Blink+before+and+after+you+think:+Blinks+occur+prior+to+and+following+cognitive+load+indexed+by+pupillary+responses.">Blink before and after you think: Blinks occur prior to and following cognitive load indexed by pupillary responses.</a> Psychophysiology. 45, 679-687 (2008).</li><li>Shultz, S., Klin, A., Jones, W. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Inhibition+of+eye+blinking+reveals+subjective+perceptions+of+stimulus+salience.">Inhibition of eye blinking reveals subjective perceptions of stimulus salience.</a> Proceedings of the National Academy of Sciences of the United States of America. 108 (52), 21270-21275 (2011).</li><li>Wong, K. K. W., Wan, W. Y., Kaye, S. B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Blinking+and+operating+cognition+versus+vision.">Blinking and operating cognition versus vision.</a> BritishJournal of Ophthalmology. 86, 479 (2002).</li><li>Leal, S., Vrij, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Blinking+during+and+after+lying.">Blinking during and after lying.</a> Journal of Nonverbal Behavior. 32 (2008), 187-194 (2008).</li><li>Murch, W. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> In the blink of an eye: A perspective on film editing. , (1995).</li><li>Holland, M. K., Tarlow, G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Blinking+and+thinking.">Blinking and thinking.</a> Perceptual and Motor Skills. 41 (2), 503-506 (1975).</li><li>Orchard, L. N., Stern, J. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Blinks+as+an+index+of+cognitive+activity+during+reading.">Blinks as an index of cognitive activity during reading.</a> Integrative Physiological and Behavioral Science. 2, 108-116 (1991).</li><li>Faubert, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Professional+athletes+have+extraordinary+skills+for+rapidly+learning+complex+and+neutral+dynamic+visual+scenes.">Professional athletes have extraordinary skills for rapidly learning complex and neutral dynamic visual scenes.</a> Scientific Reports. 3 (1154), 1-3 (2013).</li><li>Kirk, U., Skov, M., Schram Christensen, M., Nygaard, N. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Brain+correlates+of+aesthetic+expertise:+A+parametric+fMRI+study.">Brain correlates of aesthetic expertise: A parametric fMRI study.</a> Brain and Cognition. 69, 306-315 (2008).</li><li>Lotze, M., Scheler, G., Tan, H. R., Braun, C., Birbaumer, N. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+musician's+brain:+functional+imaging+of+amateurs+and+professionals+during+performance+and+imagery.">The musician's brain: functional imaging of amateurs and professionals during performance and imagery.</a> Neuroimage. 20 (3), 1817-1829 (2003).</li><li>Calvo-Merino, B., Glaser, D. E., Gr&#232;zes, J., Passingham, R. E., Haggard, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Action+observation+and+acquired+motor+skills:+an+FMRI+study+with+expert+dancers.">Action observation and acquired motor skills: an FMRI study with expert dancers.</a> Cerebral cortex. 15 (8), 1243-1249 (2005).</li></ol>

A method for analyzing visual perception of media content with purpose-made video creation is described here. Many other studies attempt to analyze the perception of media content in narrative contexts with pre-existing films. The present method proposes to create visual content with a narrative construction following the criteria of interest, and is based on the suggestion that eyeblink rate is connected to the viewer's attention9. For that reason, the study detects participants' eyeblinks while ...

The Purpose of This Method
This method proposes a dual protocol for detecting eyeblinks. The aim is to analyze viewers&#39; visual perception of media content, specifically created for this investigation, by using EEG recordings and HD video recording systems.
The Rationale Behind the Development and/or Use of This Method
Each eyeblink hides visual flow for 150-400 ms1,2. Blinking has physiological3,4,5 and psychological6,7 functions. The connection between attention and eyeblink rate has been studied and proven in different studies8. A higher level of attention decreases eyeblink rate and according to previous studies, humans share a mechanism for controlling the timing of blinks that searches for the best moment to avoid loss of important visual information9. Thus, analyzing eyeblink behavior of viewers when watching screens could provide information regarding the level of attention given to media contents.
One method for detecting spontaneous eyeblink rate is by using EEG electrodes to record the electrical activity. Eyeblinks can easily be detected by prefrontal and electrooculogram electrodes connected to an EEG recording system. In most EEG analyses, eyeblinks are considered artifacts. For this reason, many software packages designed for analyzing EEG data have eyeblink detectors10. The advantage of using EEG for detecting eyeblinks is the high temporal resolution (in the order of milliseconds) and the possibility of registering brain effects of different narratives and cuts in movies synchronized with those eyeblinks - a matter open to further study. Recording participants&#39; faces with an HD camera can also be useful for matching/contrast purposes9.
The Advantages over Alternative Methods with References to Relevant Studies
There are multiple methods for counting eye blinks. Some dedicated instruments for detecting blinks are magnetic coils, infrared (IR) light beams, optoelectronic motion detectors with eye movement analysis such as eye-tracking techniques, and several techniques based on bioelectrical signals, e.g., electrooculography (EOG), electromyography (EMG), and EEG. Another more accurate, but time-consuming option is manually counting blinks from a frame-by-frame video recording11. The technologies today can be classified broadly into two groups: a) contact-free recording which includes two modalities, the direct blink detection using computer vision and offline blink detection using eye-tracking, and b) contact-based recording using biological signals through EOG and EEG devices12,13.
The eye-tracking system is a widely used technology, ranging from traditional image-based passive designs to the active near-infrared-based approaches mainly used today with a high-resolution camera. The latter exploits the reflective properties of the pupil under IR illumination14. The concept underlying modern eye-tracking methods is Pupil Center Corneal Reflection (PCCR), which involves a camera tracking the center of the pupil, where light reflects from the cornea. However, there is a lack of blink detection algorithms published for eye-tracking protocols. Moreover, although the different models of eye-tracking on the market provide integrated software with blink detection, the source code is not always provided by manufacturers, making it difficult to modify or know how the algorithms work12. Also, during experiments with eye-tracking there are events that cause data loss, such as tracking delays and significant head or gaze movements. The eye area is very small in video captures, which is a problem for calculating the duration of the blink, and which sometimes introduces various types of artifacts15.
In this experiment, EEG and EOG methods are used. EEG is not usually used alone to detect eyeblinks. However, analyzing eyeblinks recorded with EEG electrodes is a standard procedure for the study of eyelid displacements. This procedure enables researchers to have information of exactly when eyeblinks take place. The most common signal pattern for detecting blinks is that of peak points, representing vertical movement responses. There are several peak detection algorithms applicable to raw EEG, time-domain, or frequency-domain signals. Processes involved in peak identification are peak detection, feature extraction, and classification. Eyeblinks have a considerable effect on frontal channels of the EEG signal. Typically, eyeblinks are detected in EEG by using a pre-determined amplitude threshold16. The algorithms in the analysis software used in this experiment are based on the signals&#39; standard deviation and the root mean square (RMS) of the pre-filtered EEG signal; they are open source and available to the scientific community17. However, some eye movements not involving eyeblinks can provoke electrical activity that may be confusing. For that reason, a second method - recording viewers&#39; faces with an HD video camera - allows researchers to match eyeblinks by manually counting them. With such a double method, the investigator achieves a matrix of eyeblinks that can be easily analyzed with statistical tools.
Therefore, the proposed method performs a data triangulation with two different sources to validate the detected eyeblinks. This method is based on Nakano et al. indications9 for confirmation. At the same time, it also enables researchers to collect brain-activity and frequency-band information for further analysis. The experiment described here is part of a wider future investigation into the effects of editing-style cuts on occipital and prefrontal brain areas.
Determine Whether the Method is Appropriate for an Investigation
This experimental protocol enables viewers&#39; eyeblinks while watching video content to be studied under three experimental conditions. First, eyeblink rate is detected by using two complementary techniques: EEG and recorded HD videos. Here, we use a wireless EEG with 20 channels. Second, specific stimuli adapted to the experiment are created, so that the researcher can manage all the variables of the visual content. Here, three videos with the same narrative but different video-editing style were created. The narrative consisted of a man who entered a room, sat at a desk, juggled with three balls, opened a laptop, looked up information in some books, typed something on the laptop, closed it, ate an apple, looked directly into camera, and left the room. The three video stimuli last 198 s each. The first was a one-shot movie; the second was edited according to classical Hollywood-style rules with 33 different shots; and the third was edited following MTV-style rules with 79 shots. A fourth stimulus was also presented in which the narrative was identical, but the format was a real representation with an actor instead of a video. This fourth non-video stimulus was not used in an initial study of editing-style differences but was used in a different investigation to compare eyeblink-rate differences between real representation and screened media8. Third, different groups of participants are selected depending on their previous expertise in the visual analysis of videos. The purpose is to determine differences in eyeblink rates of subject groups watching the same visual stimuli. In this case, 40 subjects took part in the investigation. Half of them were media professionals (16 males and 4 females; age 30-56 years, with an average age of 44.15 &#177;7.15 years) and the rest were non-media professionals (15 males, and 5 females; age 28-56 years, with an average age of 43.25 &#177; 8.59 years). Media professionals were chosen with the criterion of more than 6 years of experience in making decisions related to media editing in their everyday work.

<table border="0" cellpadding="0" cellspacing="0" width="1126">
	<tbody>
		<tr height="40">
			<td height="40" style="height:40px;width:276px;">EEG Device</td>
			<td style="width:279px;">Neurolectrics</td>
			<td style="width:344px;">Enobio 20 EEG/EMG system&#160;</td>
			<td style="width:227px;"></td>
		</tr>
		<tr height="40">
			<td height="40" style="height:40px;">Ag/AgCl Electrodes</td>
			<td>Neuroelectrics</td>
			<td>[NE022b] GelTrode</td>
			<td style="width:227px;"></td>
		</tr>
		<tr height="40">
			<td height="40" style="height:40px;">Recording EEG software</td>
			<td>Neuroelectrics</td>
			<td>NicOffline software</td>
			<td style="width:227px;"></td>
		</tr>
		<tr height="40">
			<td height="40" style="height:40px;">HD-video camera</td>
			<td>Sony Corporation&#160;</td>
			<td>Sony HDR-GW55VE</td>
			<td style="width:227px;"></td>
		</tr>
		<tr height="40">
			<td height="40" style="height:40px;">Syringe</td>
			<td>Monoject</td>
			<td>Monoject 412, curved tip syringe, 50/box</td>
			<td style="width:227px;"></td>
		</tr>
		<tr height="40">
			<td height="40" style="height:40px;width:276px;">Saline electrode EMG gel</td>
			<td>Signa-Gel</td>
			<td>X32-204: Signa Gel</td>
			<td style="width:227px;"></td>
		</tr>
		<tr height="40">
			<td height="40" style="height:40px;">Visual Stimuli Presentation Software</td>
			<td>Paradigm Stimulus Presentation</td>
			<td>Perception Research System Incorporated</td>
			<td style="width:227px;"></td>
		</tr>
		<tr height="50">
			<td height="50" style="height:51px;">EEG software analysis</td>
			<td style="width:279px;">Centre National de la Recherche Scientifique and Montreal Neurological Institute</td>
			<td>Brainstorm3</td>
			<td style="width:227px;"></td>
		</tr>
		<tr height="40">
			<td height="40" style="height:40px;">EEG software analysis</td>
			<td>The MathWorks Inc.</td>
			<td>MATLAB 2013b</td>
			<td style="width:227px;"></td>
		</tr>
		<tr height="40">
			<td height="40" style="height:40px;">TV for video presentation</td>
			<td>Panasonic Corporation</td>
			<td>PanasonicTH- 42PZ70EA&#160; - 50&#34;</td>
			<td style="width:227px;"></td>
		</tr>
		<tr height="40">
			<td height="40" style="height:40px;">PC for presenting stimuli</td>
			<td>MacBook Air</td>
			<td>Year 2013, running Mac OSX Mountain Lion</td>
			<td style="width:227px;"></td>
		</tr>
		<tr height="40">
			<td height="40" style="height:40px;">PC for recording stimuli</td>
			<td>MacBook&#160;</td>
			<td>Year 2009 running Windows 7</td>
			<td style="width:227px;">With a bootcamp partition of the disk for providing Windows OS</td>
		</tr>
		<tr height="40">
			<td height="40" style="height:40px;">Statistical Analysis</td>
			<td>Systat Software Inc.</td>
			<td>Sigmaplot 11.0&#160;</td>
			<td style="width:227px;"></td>
		</tr>
	</tbody>
</table>

using electroencephalography measurements and high-quality video recording for analyzing visual perception of media content

This article explores a method to detect differences in visual perception in humans. The method used is based on the psychological (or "cognitive") function of eyeblinks. Participants' eyeblinks are detected and acquired while watching videos specifically created for the investigation. The detection and acquisition of eyeblinks are carried out with the help of a 20-channel electroencephalographic (EEG) wireless device. The international 10-20 system for electrode placement is followed. A high-definition (HD) video camera is used to record participants' facial expressions, for contrast purposes. Instead of using pre-existing media content, purpose-made video content has been created following specific criteria of interest for this investigation, with stimuli enabling researchers to manage the precise parameters of interest. Otherwise, results could be contaminated with uncontrolled variables. The synchronization of the presentation of video stimuli with EEG recordings needs to be done in milliseconds. Analysis of collected data is performed with robust software for working with big matrices. Statistically significant differences in eyeblink rate related to media professionalization and editing style are found with the reported experimental procedures.

Using the procedure presented here, the eyeblink rate of 40 participants while watching three different video movies was analyzed. For a comparative analysis on media professionalization, half of the participants were media professionals. They were selected based on the criterion of a minimum 6 years of job experience in making decisions related to media editing and audio-visual cuts. For a comparative analysis on editing styles, the three video stimuli were created with exactly the same ...

Watch this Scientific Journal Video about Using Electroencephalography Measurements and High-quality Video Recording for Analyzing Visual Perception of Media Content at JoVE.com

Using Electroencephalography Measurements and High-quality Video Recording for Analyzing Visual Perception of Media Content

We present detection, acquisition, and analysis of eyeblink rates while watching media content.

This article explores a method to detect differences in visual perception in humans. The method used is based on the psychological (or "cognitive") ...

The overall goal of this experimental procedure is to analyze visual perception of media content by the quantification of spontaneous eye blink rates with the help of electroencelographic and electromyographic recordings. Here we are going to pay attention to eye blink rates of participants while watching movies in a screen. The main functions of the eye blinks is to protect the cornea from mechanical and chemical insults and also to maintain it wet and protect it from foreign debris.This type of eye blinks can be produced by different type of eyelid movements. For example, we have the corneal reflex that is produced by the activation of the mechanical or chemical or painful receptors and they can be found in the cornea and also in the skin. The stimulation of these receptors is stimulating in the same way the trigeminal neurons:first the trigeminal nerve and then the trigeminal neurons and these are activating the motor neurons that they are placed in the facial nucleus.But there is another not so well-known function of the eye blink which is related to the cognition process of visual input. And this is usually for what, in fact, is happening every time along the visual period of the day because of those spontaneous eye blinks, the visual perception is refreshed during the blink. So the blink on one side is modifying slightly our visual perception of reality.We are not conscious of that. In fact, we are not conscious that during those spontaneous blinks that last about 100-200 milliseconds, we have the certainty perception that we still see, but this is impossible because the pupil is covered by the lid, the upper lid, for this period of time. The advantage of using EG/MG measurements for analyzing visual perception of media content is the high temporary solution of the data over other thirty cases such as FMRI.With the use of EG/MG we have access to brain and eyelid activities at different frequencies. The advantage of this procedure is to have a high synchronized response of electrical brain activity to the presentation of the selected visual stimuli. It's important to control all variables and parameters of media content and visual perception.For that reason is desirable to create your own video movies. In this regard, own content can be created with a video camera by the lab or by a media center for this purpose. Participant selection is also very important.Multiple investigations have proven different perceptive patterns between subjects during or looking to a visual task. For that reason, it is needed to pay attention to participant selection. For instance, in this experiment, the spontaneous eye blink rates performed by media versus non-media professionals are compared.In this case, media professionals were selected with some concrete criteria. They had to have a minimum of six years of experience making decisions related to media editing or audio cuts in their everyday work. The specific criteria for the selection of the subjects should be written in the methods part of the publication.Create a proper scenario for the presentation of your video movie. Any distracting element can annoy your experiment participant. In our case, two black structures were created to define a flexible space.One black structure as a backdrop for the live play stimulus. And the second black structure with the same dimensions and a hole in the middle to feed a screen for the presentation of the video stimuli. It is important to create a comfortable space for performing your experimental procedure.Here we are presenting a fragment of the three movies with the same content and duration presented to participants. One was a one-shot movie with an open shot and no camera movements. Another was a Hollywood-style movie with 33 shots and an average shot length of 5.9 seconds and continuity throughout shots, with relationships of content, rhythm, space, and time.Finally, the third video was an MTV-style movie with 79 shots and an average shot length of 2.4 seconds. Here there was discontinuity of content, rhythm, space, and time between the shots. To begin an EEG/EMG session you have to prepare the following materials:The cap, the control box device, the electrodes, the conductive gel, a syringe, a meter for making the measurements of participants'head, and other basic products.Now begin by setting the EEG cap to the subject's head. Clean the surface of the skin to remove any sign of dirt, of hair products. Measure the head of the participant.Measure the distance between the nasion and the inion and mark the half-way point as the vertex. Prepare the electrodes in the EEG cap using the 10-20 system. Put the cap to the subject following the mark, so that the vertex matches the central zero point of the cap.Apply the conductive gel with the help of a syringe to electrodes, and make sure the cap is properly adjusted. Next, place the reference electrode and the electooculography in the subject's head. Ask the experiment subject whether he or she is comfortable and prepared for the session.Check that the control box is properly connected to the computer that registers the data. Check that the signal of every electrode is correct, otherwise check the electrode. While recording the EEG/EMG activity, it is desirable that you check that these electrical signals are properly being recorded with the software provided by the manufacturer or any other developed for the experiment.Confirm the appropriate conductants of EEG/EMG signals and change any parameter of the session if necessary. Once everything is checked, you can start the recording session. Store EEG/EMG data in your files and import them for software analysis.Eye blinks are usually considered as artifacts in EEG recordings. Thus any software protocol should have a good detector of eye blinks. As used here, it is a good procedure to use specific EEG and EMG electrodes, and to filter the respective bands at the most appropriate frequencies.Select and filter your data, and apply an appropriate eye blink filter to determine how many and when those eye blinks took place. A good way to triangle your results is to record your participants'eyes with an HD video camera. This will allow you to check eye movements whenever you have any doubts of an eyelid response detected by the software.In this participant example, we identified several eye blinks while watching at one of the presented stimuli. Those were detected in FP1, FP2, and the EOG electrodes. This figure shows the comparison of the spontaneous eye blink rate in non-media and media professionals while watching different editing styles.According to our results, media professionalization and media edition affect viewers'eye blink rate. Using EEG or EMG recordings for analyzing subjects'visual perception, the researchers can access to different brain activity in multiple frequency bands. In this experiment, we pay attention to the eye blink rate, but multiple analysis can be now done to the EEG data acquired during the sessions.This is the first study to report different brain and muscular activities related to media vision styles and to media professionalization. In comparison with other media reception techniques, EG/MG recordings can help to determine brain and muscle activities during the perception of media content.

using-electroencephalography-measurements-high-quality-video

Research

JoVE Journal

Behavior

Best Current Practice for Obtaining High Quality EEG Data During Simultaneous fMRI

Simultaneous EEG-fMRI allows the excellent temporal resolution of EEG to be combined with the high spatial accuracy of fMRI. The data from these two modalities can be combined in a number of ways, but all rely on the acquisition of high quality EEG and fMRI data. EEG data acquired during simultaneous fMRI are affected by several artifacts, including the gradient artefact (due to the changing magnetic field gradients required for fMRI), the pulse artefact (linked to the cardiac cycle) and movement artifacts (resulting from movements in the strong magnetic field of the scanner, and muscle activity). Post-processing methods for successfully correcting the gradient and pulse artifacts require a number of criteria to be satisfied during data acquisition. Minimizing head motion during EEG-fMRI is also imperative for limiting the generation of artifacts.
Interactions between the radio frequency (RF) pulses required for MRI and the EEG hardware may occur and can cause heating. This is only a significant risk if safety guidelines are not satisfied. Hardware design and set-up, as well as careful selection of which MR sequences are run with the EEG hardware present must therefore be considered.
The above issues highlight the importance of the choice of the experimental protocol employed when performing a simultaneous EEG-fMRI experiment. Based on previous research we describe an optimal experimental set-up. This provides high quality EEG data during simultaneous fMRI when using commercial EEG and fMRI systems, with safety risks to the subject minimized. We demonstrate this set-up in an EEG-fMRI experiment using a simple visual stimulus. However, much more complex stimuli can be used. Here we show the EEG-fMRI set-up using a Brain Products GmbH (Gilching, Germany) MRplus, 32 channel EEG system in conjunction with a Philips Achieva (Best, Netherlands) 3T MR scanner, although many of the techniques are transferable to other systems.

Simultaneous electroencephalography (EEG) and functional Magnetic Resonance imaging (fMRI) is a powerful neuroimaging tool. However, the inside of an MRI scanner forms a difficult environment for EEG data recording and safety must be considered whenever operating EEG equipment inside a scanner. Here, we present an optimised EEG-fMRI data acquisition protocol.

Simultaneous  EEG-fMRI allows the excellent temporal resolution of EEG to be combined with the  high spatial accuracy of fMRI. The data  from these two ...

Simultaneous Scalp Electroencephalography (EEG), Electromyography (EMG), and Whole-body Segmental Inertial Recording for Multi-modal Neural Decoding

Recent studies support the involvement of supraspinal networks in control of bipedal human walking. Part of this evidence encompasses studies, including our previous work, demonstrating that gait kinematics and limb coordination during treadmill walking can be inferred from the scalp electroencephalogram (EEG) with reasonably high decoding accuracies. These results provide impetus for development of non-invasive brain-machine-interface (BMI) systems for use in restoration and/or augmentation of gait- a primary goal of rehabilitation research. To date, studies examining EEG decoding of activity during gait have been limited to treadmill walking in a controlled environment. However, to be practically viable a BMI system must be applicable for use in everyday locomotor tasks such as over ground walking and turning. Here, we present a novel protocol for non-invasive collection of brain activity (EEG), muscle activity (electromyography (EMG)), and whole-body kinematic data (head, torso, and limb trajectories) during both treadmill and over ground walking tasks. By collecting these data in the uncontrolled environment insight can be gained regarding the feasibility of decoding unconstrained gait and surface EMG from scalp EEG.

Simultaneous Scalp Electroencephalography EEG, Electromyography EMG, and Whole-body Segmental Inertial Recording for Multi-modal Neural Decoding

Development of an effective brain-machine-interface (BMI) system for restoration and rehabilitation of bipedal locomotion requires accurate decoding of user's intent. Here we present a novel experimental protocol and data collection technique for simultaneous non-invasive acquisition of neural activity, muscle activity, and whole-body kinematics during various locomotion tasks and conditions.

Recent  studies support the involvement of supraspinal networks in control of bipedal  human walking. Part of this evidence  encompasses studies, ...

Contextual and Cued Fear Conditioning Test Using a Video Analyzing System in Mice

The contextual and cued fear conditioning test is one of the behavioral tests that assesses the ability of mice to learn and remember an association between environmental cues and aversive experiences. In this test, mice are placed into a conditioning chamber and are given parings of a conditioned stimulus (an auditory cue) and an aversive unconditioned stimulus (an electric footshock). After a delay time, the mice are exposed to the same conditioning chamber and a differently shaped chamber with presentation of the auditory cue. Freezing behavior during the test is measured as an index of fear memory. To analyze the behavior automatically, we have developed a video analyzing system using the ImageFZ application software program, which is available as a free download at http://www.mouse-phenotype.org/. Here, to show the details of our protocol, we demonstrate our procedure for the contextual and cued fear conditioning test in C57BL/6J mice using the ImageFZ system. In addition, we validated our protocol and the video analyzing system performance by comparing freezing time measured by the ImageFZ system or a photobeam-based computer measurement system with that scored by a human observer. As shown in our representative results, the data obtained by ImageFZ were similar to those analyzed by a human observer, indicating that the behavioral analysis using the ImageFZ system is highly reliable. The present movie article provides detailed information regarding the test procedures and will promote understanding of the experimental situation.

This article presents a protocol for a contextual and cued fear conditioning test using a video analyzing system to assess fear learning and memory in mice.

The contextual and cued fear conditioning test is one of the behavioral tests that assesses the ability of mice to learn and remember an association ...

Automated Visual Cognitive Tasks for Recording Neural Activity Using a Floor Projection Maze

Neuropsychological tasks used in primates to investigate mechanisms of learning and memory are typically visually guided cognitive tasks. We have developed visual cognitive tasks for rats using the Floor Projection Maze1,2&#160;that are optimized for visual abilities of rats permitting stronger comparisons of experimental findings with other species.
In order to investigate neural correlates of learning and memory, we have integrated electrophysiological recordings into fully automated cognitive tasks on the Floor Projection Maze1,2. Behavioral software interfaced with an animal tracking system allows monitoring of the animal's behavior with precise control of image presentation and reward contingencies for better trained animals. Integration with an in vivo electrophysiological recording system enables examination of behavioral correlates of neural activity at selected epochs of a given cognitive task.
We describe protocols for a model system that combines automated visual presentation of information to rodents and intracranial reward with electrophysiological approaches. Our model system offers a sophisticated set of tools as a framework for other cognitive tasks to better isolate and identify specific mechanisms contributing to particular cognitive processes.

We describe protocols for training rats for chronic electrophysiological recordings in fully automated cognitive tasks on a Floor Projection Maze.

Neuropsychological tasks used in primates to investigate mechanisms of learning and memory are typically visually guided cognitive tasks. We have ...

Combined Shuttle-Box Training with Electrophysiological Cortex Recording and Stimulation as a Tool to Study Perception and Learning

Shuttle-box avoidance learning is a well-established method in behavioral neuroscience and experimental setups were traditionally custom-made; the necessary equipment is now available by several commercial companies. This protocol provides a detailed description of a two-way shuttle-box avoidance learning paradigm in rodents (here Mongolian gerbils; Meriones unguiculatus) in combination with site-specific electrical intracortical microstimulation (ICMS) and simultaneous chronical electrophysiological in vivo recordings. The detailed protocol is applicable to study multiple aspects of learning behavior and perception in different rodent species.
Site-specific ICMS of auditory cortical circuits as conditioned stimuli here is used as a tool to test the perceptual relevance of specific afferent, efferent and intracortical connections. Distinct activation patterns can be evoked by using different stimulation electrode arrays for local, layer-dependent ICMS or distant ICMS sites. Utilizing behavioral signal detection analysis it can be determined which stimulation strategy is most effective for eliciting a behaviorally detectable and salient signal. Further, parallel multichannel-recordings using different electrode designs (surface electrodes, depth electrodes, etc.) allow for investigating neuronal observables over the time course of such learning processes. It will be discussed how changes of the behavioral design can increase the cognitive complexity (e.g. detection, discrimination, reversal learning).

Shuttle-box avoidance learning is well-established in behavioral neuroscience. This protocol describes how shuttle-box learning in rodents can be combined with site-specific electrical intracortical microstimulation (ICMS) and simultaneous chronical in vivo recordings as a tool to study multiple aspects of learning and perception.

Shuttle-box avoidance learning is a well-established method in behavioral neuroscience and experimental setups were traditionally custom-made; the ...

Utilizing Electroencephalography Measurements for Comparison of Task-Specific Neural Efficiencies: Spatial Intelligence Tasks

Spatial intelligence is often linked to success in engineering education and engineering professions. The use of electroencephalography enables comparative calculation of individuals&#39; neural efficiency as they perform successive tasks requiring spatial ability to derive solutions. Neural efficiency here is defined as having less beta activation, and therefore expending fewer neural resources, to perform a task in comparison to other groups or other tasks. For inter-task comparisons of tasks with similar durations, these measurements may enable a comparison of task type difficulty. For intra-participant and inter-participant comparisons, these measurements provide potential insight into the participant&#39;s level of spatial ability and different engineering problem solving tasks. Performance on the selected tasks can be analyzed and correlated with beta activities. This work presents a detailed research protocol studying the neural efficiency of students engaged in the solving of typical spatial ability and Statics problems. Students completed problems specific to the Mental Cutting Test (MCT), Purdue Spatial Visualization test of Rotations (PSVT:R), and Statics. While engaged in solving these problems, participants&#39; brain waves were measured with EEG allowing data to be collected regarding alpha and beta brain wave activation and use. The work looks to correlate functional performance on pure spatial tasks with spatially intensive engineering tasks to identify the pathways to successful performance in engineering and the resulting improvements in engineering education that may follow.

This manuscript describes an approach to measure neural activity of humans while solving spatially focused engineering problems. The electroencephalogram methodology helps interpret beta brain wave measurements in terms of neural efficiency, with the aim of ultimately enabling comparisons of task performance both between problem types and between participants.

Spatial intelligence is often linked to success in engineering education and engineering professions. The use of electroencephalography enables ...

Concurrent Electroencephalography Recording During Transcranial Alternating Current Stimulation (tACS)

Oscillatory brain activities are considered to reflect the basis of rhythmic changes in transmission efficacy across brain networks and are assumed to integrate cognitive neural processes. Transcranial alternating current stimulation (tACS) holds the promise to elucidate the causal link between specific frequencies of oscillatory brain activity and cognitive processes. Simultaneous electroencephalography (EEG) recording during tACS would offer an opportunity to directly explore immediate neurophysiological effects of tACS. However, it is not trivial to measure EEG signals during tACS, as tACS creates a huge artifact in EEG data. Here we explain how to set up concurrent tACS-EEG experiments. Two necessary considerations for successful EEG recording while applying tACS are highlighted. First, bridging of the tACS and EEG electrodes via leaking EEG gel immediately saturates the EEG amplifier. To avoid bridging via gel, the viscosity of the EEG gel is the most important parameter. The EEG gel must be viscous to avoid bridging, but at the same time sufficiently fluid to create contact between the tACS electrode and the scalp. Second, due to the large amplitude of the tACS artifact, it is important to consider using an EEG system with a high resolution analog-to-digital (A/D) converter. In particular, the magnitude of the tACS artifact can exceed 100 mV at the vicinity of a stimulation electrode when 1 mA tACS is applied. The resolution of the A/D converter is of importance to measure good quality EEG data from the vicinity of the stimulation site. By following these guidelines for the procedures and technical considerations, successful concurrent EEG recording during tACS will be realized.

Concurrent Electroencephalography Recording During Transcranial Alternating Current Stimulation tACS

In this article we explain how to set up a concurrent transcranial alternating current stimulation and EEG experiment.

Oscillatory brain activities are considered to reflect the basis of rhythmic changes in transmission efficacy across brain networks and are assumed to ...

Implantation and Recording of Wireless Electroretinogram and Visual Evoked Potential in Conscious Rats

The full-field electroretinogram (ERG) and visual evoked potential (VEP) are useful tools to assess retinal and visual pathway integrity in both laboratory and clinical settings. Currently, preclinical ERG and VEP measurements are performed with anesthesia to ensure stable electrode placements. However, the very presence of anesthesia has been shown to contaminate normal physiological responses. To overcome these anesthesia confounds, we develop a novel platform to assay ERG and VEP in conscious rats. Electrodes are surgically implanted sub-conjunctivally on the eye to assay the ERG and epidurally over the visual cortex to measure the VEP. A range of amplitude and sensitivity/timing parameters are assayed for both the ERG and VEP at increasing luminous energies. The ERG and VEP signals are shown to be stable and repeatable for at least 4 weeks post surgical implantation. This ability to record ERG and VEP signals without anesthesia confounds in the preclinical setting should provide superior translation to clinical data.

We show surgical implantation and Recording procedures to measure visual electrophysiological signals from the eye (electroretinogram) and brain (visual evoked potential) in conscious rats, which is more analogous to the human condition where recordings are conducted without anesthesia confounds.

The full-field electroretinogram (ERG) and visual evoked potential (VEP) are useful tools to assess retinal and visual pathway integrity in both ...

A Methodology for Capturing Joint Visual Attention Using Mobile Eye-Trackers

With the advent of new technological advances, it is possible to study social interactions at a microlevel with unprecedented accuracy. High frequency sensors, such as eye-trackers, electrodermal activity wristbands, EEG bands, and motion sensors provide observations at the millisecond level. This level of precision allows researchers to collect large datasets on social interactions. In this paper, I discuss how multiple eye-trackers can capture a fundamental construct in social interactions, joint visual attention (JVA). JVA has been studied by developmental psychologists to understand how children acquire language, learning scientists to understand how small groups of learners work together, and social scientists to understand interactions in small teams. This paper describes a methodology for capturing JVA in colocated settings using mobile eye-trackers. It presents some empirical results and discusses implications of capturing microobservations to understand social interactions.

Using multimodal sensors is a promising way to understand the role of social interactions in educational settings. This paper describes a methodology for capturing joint visual attention from colocated dyads using mobile eye-trackers.

With the advent of new technological advances, it is possible to study social interactions at a microlevel with unprecedented accuracy. High frequency ...

Interaction between Phonological and Semantic Processes in Visual Word Recognition using Electrophysiology

Controversies have always existed in research related to reading abilities; on whether printed words are perceived in a feedforward manner based on orthographic information after which, other representations, such as phonology and semantics are activated, or whether these are fully interactive and high-level semantic information affects early processing. An interference paradigm was implemented in the presented protocol of phonological and semantic judgment tasks that utilized the same precede-target pairs to explore the relative order of phonological and semantic activation. The high- and low-frequency target words were preceded with three conditions: semantically related, phonological-related (homophones), or unrelated. The results showed that the induced P200 component of low-frequency word pairs was significantly greater than high-frequency words in both the semantic and phonological tasks. In addition, both the homophones in the semantic task and the semantically related pairs in the phonological task caused reduction in N400 when compared to the the control condition, word frequency-independently. It is worth noting that for the low-frequency pairs in the phonological judgment task, the P200 released by the semantically related word pairs was significantly larger than that in the control condition. Overall, semantic processing in phonological tasks and phonological processing in semantic tasks were found in both high- and low-frequency words, suggesting that the interaction between semantics and phonology may operate in a task-independent manner. However, the specific time this interaction occurred may have been affected by the task and frequency.

We present a protocol to explore the relative activation sequence of phonology and semantics in visual word recognition. The results show that consistent with interactive accounts, semantic and phonological representations may be processed interactively, and higher-level linguistic representations may affect early processing.

Controversies have always existed in research related to reading abilities; on whether printed words are perceived in a feedforward manner based on ...