Name: protocol for data collection and analysis applied to automated facial expression analysis technology and temporal analysis for sensory evaluation
Uploaded: 2016-08-26T13:00:00.000+00:00
Duration: 7 min 12 s
Description: Watch this Scientific Journal Video about Protocol for Data Collection and Analysis Applied to Automated Facial Expression Analysis Technology and Temporal Analysis for Sensory Evaluation at JoVE.com

This project was funded, in part, by ConAgra Foods (Omaha, NE, USA), the Virginia Agricultural Experiment Station, the Hatch Program of the National Institute of Food and Agriculture, U.S. Department of Agriculture, and the Virginia Tech Water INTERface Interdisciplinary Graduate Education Program.

伦理陈述:在项目开始之前，这项研究是预先核准弗吉尼亚理工大学的机构审查委员会（IRB）（IRB 14-229）。 注意:人体试验之前，参与需要知情同意书。除了IRB批准，同意使用静态或视频图像，还需要之前释放所有图像打印，视频或图文影像。此外，食物过敏原测试之前公开。参加者前面板开始问他们是否有任何不耐受，过敏或其他问题。 注:排除标准:自动面部表情分析是粗框眼镜，胡子拉碴的沉重面孔和肤色敏感。谁具有这些条件的参与者与软件分析不相容由于失败视频的风险增加。这归因于该软件的无法找到的面。 1.样品制备和受试者招募<ol><li>准备饮料或软食品样品。 <ol><li>准备从科斯特洛和克拉克（2009）22加强用2％牛奶乳制品的解决方案和建议香料以及其它香精。准备以下解决方案:（1）牛奶无味（2％减脂牛奶）; （2）未调味的水（饮用水）; （3）在乳香草精香料（0.02微克/毫升）（仿清楚香草香料）;和（4）咸鲜味牛奶（0.004克/ ml的碘化盐）。 注意:这些解决方案仅用于演示目的。 </li><li>倾各溶液的一半盎司等份（约15 G）成2盎司透明塑料样品杯和顶盖采用彩色编码盖。 注:建议使用透明的杯子;然而，它是由研究人员的判断。 </li></ol></li><li>从校园或当地社区招募学员参加学习。 注:需要研究参与者的样本量高达研究者的自由裁量权。我们建议范围10到50人参加。 </li&gt; <li>获得在参与研究前人类受试者同意。 </li></ol> 2.准备工作小组室视频拍摄注:该协议是在感官评价实验室数据采集。该协议是为了使一个感官展位设置AFEA数据采集有用的。 <ol><li>使用单独的展位与他们（面层）的正面触摸屏显示器，以保持他们的焦点向前，以防止往下看。 </li><li>使用与背部支撑可调节高度的椅子。 注意:这些是用于使被垂直地调整，并放置在适当的范围内的视频捕获参与者是必不可少的。所以参与者的运动是减少使用固定座椅（无滚动功能部件），可调节靠背高度支持。 </li><li>设置为"100％日光"头顶上的灯光最佳面部情感视频捕捉（6504K光源; R = 206，G = 242，B = 255）。 注意:为了避免激烈的阴影，弥漫正面照明IDE人而光强度或颜色不为相关20。最终，它是由研究人员，个别协议/方法，和环境来控制照明捕获的判断。 </li><li>贴上一个可调相机触摸屏监视器记录之上。 <ol><li>使用照相机与至少640×480像素（或更高）20的分辨率。讨论与购买和安装前20的软件供应商所需要的拍照能力。注意:长宽比不重要20。 </li><li>将相机拍摄速度为每秒（或其它标准速度）30帧的一致性。 </li><li>连接并确保媒体录制软件设置相机记录和保存参与者的视频。 </li></ol></li></ol> 3.参与者调整和口头指示<ol><li>有一次只有一个参与者评价在感官展位样本。 注:测试超过同时一个参与者可以与测试环境干扰和破坏参与者的浓度或创建偏压。 </li><li>抵达后，让参与者了解流程和标准作业程序的口头指示。 <ol><li>有与会者直坐起来，靠在椅背上。 </li><li>调整座椅高度，座椅（与相机的距离）的位置，摄像头角度，使参与者的脸在视频录制的中心拍摄的，对下巴和眼睛周围没有阴影。 注:在展台感觉，参与者的头部大约20 - 从相机和摄像机的视频饲料中居中面对监视器24英寸的距离。 </li><li>作为定位，重点突出对显示器的显示面对指示参与者保持坐姿。此外，指导学员在每次采样的30秒评估期间从任何突然的动作后的样品消耗避免。 </li&gt; <li>指导参与者要消耗整个饮料或液化食品样品和吞咽。 </li><li>指示参与者快速立即移动到表中的样品杯的下巴下方上下后的样品是在口中。这是为了消除面部遮挡。提醒他们保持对显示器外观。 注:样品载体提供所述样品是至研究者的判断。可以使用吸管或杯子。无论如何，最初的面部遮挡是不可避免的，因为脸会被遮挡或由于消费扭曲。 </li></ol></li><li>指导参与者，因为他们出现在触摸屏显示器上按照指示。注:烧写到自动化的感官软件指令自动测序。 </li></ol>视频采集4.个别参加流程<ol><li>确认视频摄像机捕捉最佳参与者的脸，而参与者就座舒适的展台（样品演示前）通过查看其上显示的视频拍摄电脑显示器。通过点击电脑显示器上的录制按钮开始录制。 </li><li>指导参与者抿水来清洁自己的口味。 </li><li>同时提供的治疗方法之一，从基线或对照（无味的水）。通过放置在涉及对视频内的样品处理识别样品颜色代码每个样品的顶部独特的彩色索引卡标识每个样本。 注:触摸屏显示器上编程指导指导参与者。说明书通过一系列每个处理样品标准化步骤引导参与者。 </li><li>通过触摸屏显示器，直接参与者: <ol><li>托起相关的颜色索引卡片预消耗在视频样品标识。 注:色卡是研究人员的方式可以在视频中找出治疗方法第二标记样品评估适当的时间框架（时间零）。 </li><li>短暂地保持卡后，将卡重新放回托盘上。 </li><li>完全消费样本，等待大约30秒，通过监视器上的编程指导执行，而对相机面对的问题。 注:30秒受控取样周期包括一个时间跨度足够用于整个采样评估期（ 即，示出了索引卡，打开一个样品（除去盖），消费，和情感捕获）。 </li><li>进入他们的享乐可接受比分触摸屏显示器上（1 =厌恶至极，2 =很不喜欢，3 =不喜欢中度，4 =不喜欢咯，5 =既不喜欢也不厌恶，6 =就好咯，7 =像适度，8 =非常喜欢，9 =极喜欢）。 </li><li>下一个样品前处理饮用水漱口。 </li></ol></li></ol> 5.自动评估面部表情分析选项注:很多面部表情分析软件程序存在。软件命令和功能可以变化。重要的是要遵循制造商的用户指南和参考手册20是很重要的。 <ol><li>保存记录在媒体格式和传送到自动面部表情分析软件。 </li><li>使用自动面部分析软件分析参与者的视频。 <ol><li>双击电脑桌面上的软件图标。 </li><li>一旦程序打开后，点击"文件"，选择"新建..."，然后选择"项目..." </li><li>在弹出的窗口中，将项目命名并保存项目。 </li><li>通过单击"添加参与者"图标（人用（+）号）的参与者加入到项目中。更多的参与者可以通过重复此步骤中加入。 </li><li>与会者的视频添加到用于分析各自的参与者。 <ol><li>在屏幕的左侧单击电影卷轴无线网络的图标第一个加号（+）号以添加一个视频来分析。 </li><li>点击感兴趣的参与者在"放大镜"来浏览视频的补充。 </li></ol></li></ol></li><li>下的软件连续校准分析设置分析视频帧接一帧。 <ol><li>点击铅笔图标，窗口底部调整设置，每个参与者视频"设置"选项卡下。 <ol><li>将"人脸模型"将军。将"平滑化分类"为是。设置"采样率"，以每一帧。 </li><li>设置"图像旋转"，以编号设置"连续校准"，是的。设置"选择的校正"来无。 </li></ol></li><li>保存项目设置。 </li><li>按下批处理分析图标（相同的红色和黑色的靶样的符号）来分析项目的影片。 </li><li>结果保存一次分析完成。 注:在软件如果researc存在其他视频设置她喜欢另一个值得分析的方法。 </li><li>考虑视频故障如果严重的面部闭塞或无法面对映射指定的消费后窗口（ 图1）过程中仍然存在。此外，如果模型没有数据会说在导出输出文件"FIT_FAILED"或"FIND_FAILED"（ 如图2）。这表示丢失的数据，因为该软件不能分类或分析参与者的情绪。 注:转换AFEA面部肌肉运动中立，快乐，厌恶，悲伤，愤怒，惊讶和害怕从0分（不表达）到1（完全表示）为每情感。 </li></ol></li><li>导出AFEA数据输出日志文件（.txt）作进一步的分析。 <ol><li>一旦分析完成，导出整个项目。 <ol><li>点击"文件"，"导出"，"导出项目成果"。 </li><li>当打开一个窗口，选择的出口寿所在的位置LD保存和日志文件（.txt）保存到一个文件夹。 </li><li>转换每个参与者记录生活到数据电子表格（.csv或.xlsx）中提取相关数据。 <ol><li>开放式数据电子表格软件，并选择"数据"选项卡。 </li><li>在"数据"选项卡，在"获取外部数据"组中，单击"从文本"。 </li><li>在"地址栏"，找到，双击参与者的文本文件导入，然后按照屏幕上的向导说明。 </li><li>继续对所有相关参与者的文件导出过程。 </li></ol></li></ol></li></ol></li></ol> 6.时间戳参与者视频数据分析<ol><li>使用AFEA软件，手动检查每个参与者的视频，并确定消费后的零时间每个样品。记录在一个数据的电子表格的时间戳。当样品杯是参与者的下巴下面，不再OC后期消费被定义cludes脸。 注意:时间戳的位置是为评价是至关重要的。其中杯不再闭塞面点是最佳的建议和时间戳需要是所有参与者一致。 </li><li>保存时间戳数据电子表格（.csv），作为从视频中提取相关数据的引用。 注:参加者的视频，也可以在软件中，"事件标志"内部编码。 </li></ol> 7.时间序列分析的情感注意:考虑"基线"是控制（ 即，在本实施例未调味的水）。研究人员已经制造出不同的"基准治疗刺激"或"无刺激的基准时间"取决于调查的利益配对比较的能力。通过配对统计测试提出了一个"默认"状态帐号的方法。换句话说，该程序使用统计阻塞（ 即配对检验）来调节每个参与者的默认外观，因此减少了穿过参与者的可变性。 <ol><li>摘自导出的文件（.csv或.xlsx）格式的相关数据。 <ol><li>确定相关的学习评价（秒）的时间框架。 </li><li>手动提取导出的文件，参加咨询参与者时间戳（时间零）相应的数据（时间段）。 </li><li>在新的数据电子表格中选择时间框架（秒）编译每个参与者的治疗数据（参与者数量，治疗，原创视频的时间，和情感反应）每情感（高兴，中性，悲伤，愤怒，惊讶，恐惧和厌恶）对于未来的分析（ 图3）。 </li><li>继续此过程为所有的参与者。 </li></ol></li><li>从每个参与者处理对时间戳文件标识相应的时间零，并调整视频时间到真正的时间为"0"的直接比较（ <str翁&gt;图4， 图5）。 注:参与者数据被收集在一个连续的视频因此每个处理"的时间零"是不同的（ 即，无味水视频时间零点是02:13.5和未调味乳视频时间零点是03:15.4）。在图4中由于不同的治疗"时间零"，需要的视频时间进行调整和重新调整，在启动"0:00.0"或其它标准的开始时间，以便处理情绪反应数据进行直接比较的时间。 </li><li>对于每一个参与者，情感和调整后的时间点，提取配对处理（ 如 ，无味牛奶）和控制处理（ 如，无味的水）定量情绪得分。换句话说，对准参与者的处理和控制的时间序列的每个情绪（ 图5）的反应。 </li><li>编译所有参与者的信息（参加者，调整后的时间，和配对治疗（ 例如，无味水和未调味乳）在每个时间点（ 图6）。 注意:以下步骤演示了手工配对威尔科克斯测试的步骤。大多数数据分析软件程序会自动完成。建议用统计学家讨论的统计分析过程。 </li><li>一旦样品被复位，并与新的调整后的视频倍对齐，直接的相应样品的情感结果，并使用在整个参与者顺序成对非参数的Wilcoxon检验与对照（未调味的水）（ 图7）之间进行比较。 注:样品的新时间对准将允许5秒消费后的时间框架内的直接比较。如果一个配对观测中不存在的处理，丢弃从该时间点比较的参与者。 <ol><li>使用数据spreadshe计算控制和各样品的每个配对比较的区别等的管理软件。 注:比较将取决于选择用于在软件情绪分析的帧速率。该协议证明了每秒30个人比较5秒（选定的时间帧）。 注意:使用图7为列和步骤的参考。 <ol><li>从控制（ 例如 ，无味水），以确定的差的值中减去的乳（ 例如，无味牛奶）的值。在标题为"治疗差异"的新栏，输入"=（C2） - （D2）"数据电子表格管理软件，其中的"C2"是控制情感价值和"D2"是选定治疗情感价值。继续为所有时间点的这个过程。 </li><li>计算治疗差异的绝对值。在新列中的数据电子表格管理软件，输入"= ABS（E2）"，其中"E2"是治疗差异。继续此过程所有时间点。 </li><li>确定治疗差异的排名顺序。在新列中的数据电子表格管理软件，输入"= RANK（G2，$ G $ 2:$ G $ 25 1）"，其中"G2"是绝对差和"1"是"上升"。继续为所有时间点的这个过程。 </li><li>确定电子表格上的排名次序的符号秩。改征阴性，如果治疗差异为负（I栏）。 </li><li>计算正和（= SUMIF（I2:等级值I25）:I25，"&gt; 0"，I2:I25）和负总和= SUMIF（I2:I25，"&lt;0"，I2。 </li><li>确定检验统计量。检验统计量是绝对值低总和。 </li><li>协商使用数字的观察包括在特定的时间和所选择的α值来确定临界值Wilcoxon符号排名检验统计统计表。 </li><li>如果测试统计量小于临界值拒绝吨他零假设。如果是较大的，接受零假设。 </li></ol></li></ol></li><li>图表上相关的治疗图形（ 即，无味奶相比无味水）为次的结果，当零假设被拒绝。使用差的符号来确定哪个治疗具有较大的情绪（ 图8）。 <ol><li>在数据表格管理软件，创建一个使用存在或不存在的意义的值的曲线图。 <ol><li>点击"插入"选项卡。 </li><li>选择"线" </li><li>右键单击图框。 </li><li>点击"选择数据"，并按照屏幕上的提示选择与图形相关的数据（ 图8）。 注意:图表将描绘情感结果，其中样品或控制较高和显著。图表依赖，情感是在那个特定的时间允许更高的辨别如何参与者的情绪的能力演变在两个样本之间5秒的时间段。 注:与统计学家统计支持，强烈建议来提取相关数据。需要统计编码的发展来分析情感的结果。 </li></ol></li></ol></li></ol>

<ol>
	<li>De Wijk, R. A., Kooijman, V., Verhoeven, R. H. G., Holthuysen, N. T. E., De Graaf, C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Autonomic+nervous+system+responses+on+and+facial+expressions+to+the+sight,+smell,+and+taste+of+liked+and+disliked+foods.">Autonomic nervous system responses on and facial expressions to the sight, smell, and taste of liked and disliked foods.</a> Food Qual Prefer. 26 (2), 196-203 (2012).</li><li>De Wijk, R. A., He, W., Mensink, M. G. J., Verhoeven, R. H. G., De Graaf, C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=ANS+responses+and+facial+expression+differentiate+between+the+taste+of+commercial+breakfast+drinks.">ANS responses and facial expression differentiate between the taste of commercial breakfast drinks.</a> PLoS ONE. 9 (4), 1-9 (2014).</li><li>He, W., Boesveldt, S., De Graaf, C., De Wijk, R. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Behavioural+and+physiological+responses+to+two+food+odours.">Behavioural and physiological responses to two food odours.</a> Appetite. 59 (2), 628 (2012).</li><li>He, W., Boesveldt, S., De Graaf, C., De Wijk, R. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Dynamics+of+autonomic+nervous+system+responses+and+facial+expressions+to+odors.">Dynamics of autonomic nervous system responses and facial expressions to odors.</a> Front Psychol. 5 (110), 1-8 (2014).</li><li>Danner, L., Sidorkina, L., Joechl, M., Duerrschmid, K. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Make+a+face!+Implicit+and+explicit+measurement+of+facial+expressions+elicited+by+orange+juices+using+face+reading+technology.">Make a face! Implicit and explicit measurement of facial expressions elicited by orange juices using face reading technology.</a> Food Qual Prefer. 32 (2014), 167-172 (2013).</li><li>Danner, L., Haindl, S., Joechl, M., Duerrschmid, K. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Facial+expression+and+autonomous+nervous+system+responses+elicited+by+tasting+different+juices.">Facial expression and autonomous nervous system responses elicited by tasting different juices.</a> Food Res Int. 64 (2014), 81-90 (2014).</li><li>Arnade, E. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Measuring consumer emotional response to tastes and foods through facial expression analysis [thesis]. , 1-187 (2013).</li><li>Leitch, K. A., Duncan, S. E., O'Keefe, S., Rudd, R., Gallagher, D. L. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Characterizing+consumer+emotional+response+to+sweeteners+using+an+emotion+terminology+questionnaire+and+facial+expression+analysis.">Characterizing consumer emotional response to sweeteners using an emotion terminology questionnaire and facial expression analysis.</a> Food Res Int. 76, 283-292 (2015).</li><li>Crist, C. A., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Application+of+emotional+facial+analysis+technology+to+consumer+acceptability+using+a+basic+tastes+model.">Application of emotional facial analysis technology to consumer acceptability using a basic tastes model.</a> , (2014).</li><li>Garcia-Burgos, D., Zamora, M. C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Facial+affective+reactions+to+bitter-tasting+foods+and+body+mass+index+in+adults.">Facial affective reactions to bitter-tasting foods and body mass index in adults.</a> Appetite. 71 (2013), 178-186 (2013).</li><li>Garcia-Burgos, D., Zamora, M. C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Exploring+the+hedonic+and+incentive+properties+in+preferences+for+bitter+foods+via+self-reports,+facial+expressions+and+instrumental+behaviours.">Exploring the hedonic and incentive properties in preferences for bitter foods via self-reports, facial expressions and instrumental behaviours.</a> Food Qual Prefer. 39 (2015), 73-81 (2015).</li><li>Lewinski, P., Fransen, M. L., Tan, E. S. H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Predicting+advertising+effectiveness+by+facial+expressions+in+response+to+amusing+persuasive+stimuli.">Predicting advertising effectiveness by facial expressions in response to amusing persuasive stimuli.</a> J. Neurosci. Psychol. Econ. 7 (1), 1-14 (2014).</li><li>Ekman, P., Friesen, W. V. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Facial+action+coding+system:+A+technique+for+the+measurement+of+facial+movement.">Facial action coding system: A technique for the measurement of facial movement.</a> , (1978).</li><li>Viola, P., Jones, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Rapid+object+detection+using+a+boosted+cascade+of+simple+features.">Rapid object detection using a boosted cascade of simple features.</a> Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern. 1, 511-518 (2001).</li><li>Sung, K. K., Poggio, T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Example-based+learning+for+view-based+human+face+detection.">Example-based learning for view-based human face detection.</a> IEEE Trans. Pattern Anal. Mach. Intell. 20 (1), 39-51 (1998).</li><li>. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> FaceReader 5&#8482; Technical Specifications. , (2014).</li><li>Cootes, T., Taylor, C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Statistical models of appearance for computer vision: Technical report. , (2000).</li><li>Bishop, C. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Neural networks for pattern recognition. , (1995).</li><li>Lewinski, P., den Uyl, T. M., Butler, C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Automated+facial+coding:+validation+of+basic+emotions+and+FACS+AUs+in+FaceReader.">Automated facial coding: validation of basic emotions and FACS AUs in FaceReader.</a> J. Neurosci. Psychol. Econ. 7 (4), 227-236 (2014).</li><li>Noldus Information Technology. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> FaceReader Reference Manual Version 6. , (2014).</li><li>Alves, N. T., Fukusima, S. S., Aznar-Casanova, J. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Models+of+brain+asymmetry+in+emotional+processing.">Models of brain asymmetry in emotional processing.</a> Psychol Neurosci. 1 (1), 63-66 (2008).</li><li>Costello, M., Clark, S., Clark, S., Costello, M., Drake, M., Bodyfelt, F. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Preparation+of+samples+for+instructing+students+and+staff+in+dairy+products+evaluation+(Appendix+F).">Preparation of samples for instructing students and staff in dairy products evaluation (Appendix F).</a> The sensory evaluation of dairy foods. , 551-560 (2009).</li><li>Porcherot, C., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=How+do+you+feel+when+you+smell+this?+Optimization+of+a+verbal+measurement+of+odor-elicited+emotions.">How do you feel when you smell this? Optimization of a verbal measurement of odor-elicited emotions.</a> Food Qual Prefer. 21, 938-947 (2010).</li><li>Warrenburg, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Effects+of+fragrance+on+emotions:+Moods+and+physiology.">Effects of fragrance on emotions: Moods and physiology.</a> Chem. Senses. 30, i248-i249 (2005).</li><li>Bredie, W. L. P., Tan, H. S. G., Wendin, K. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+comparative+study+on+facially+expressed+emotions+in+response+to+basic+tastes.">A comparative study on facially expressed emotions in response to basic tastes.</a> Chem. Percept. 7 (1), 1-9 (2014).</li><li>Wendin, K., Allesen-Holm, B. H., Bredie, L. P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Do+facial+reactions+add+new+dimensions+to+measuring+sensory+responses+to+basic+tastes?.">Do facial reactions add new dimensions to measuring sensory responses to basic tastes?.</a> Food Qual Prefer. 22, 346-354 (2011).</li><li>Rosenstein, D., Oster, H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Differential+facial+responses+to+four+basic+tastes+in+newborns.">Differential facial responses to four basic tastes in newborns.</a> Child Dev. 59 (6), 1555-1568 (1988).</li><li>Rosenstein, D., Oster, H., P, E. k. m. a. n., E, R. o. s. e. n. b. e. r. g. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Differential+facial+responses+to+four+basic+tastes+in+newborns.">Differential facial responses to four basic tastes in newborns.</a> What the face reveals: Basic and applied studies of spontaneous expression using the facial action coding system (FACS). , 302-327 (1997).</li><li>Rozin, P., Fallon, A. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+perspective+on+disgust.">A perspective on disgust.</a> Psychol. Rev. 94 (1), 23-41 (1987).</li><li>Delarue, J., Blumenthal, D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Temporal+aspects+of+consumer+preferences.">Temporal aspects of consumer preferences.</a> Curr. Opin. Food Sci. 3, 41-46 (2015).</li><li>Sudre, J., Pineau, N., Loret, C., Marin, N. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Comparison+of+methods+to+monitor+liking+of+food+during+consumption.">Comparison of methods to monitor liking of food during consumption.</a> Food Qual Prefer. 24 (1), 179-189 (2012).</li><li>Weiland, R., Ellgring, H., Macht, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Gustofacial+and+olfactofacial+responses+in+human+adults.">Gustofacial and olfactofacial responses in human adults.</a> Chem. Senses. 35 (9), 841-853 (2010).</li><li>Ekman, P., Cole, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Universal+and+cultural+differences+in+facial+expressions+of+emotion.">Universal and cultural differences in facial expressions of emotion.</a> Nebraska symposium on motivation. , 207-283 (1971).</li><li>Griemel, E., Macht, M., Krumhuber, E., Ellgring, H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Facial+and+affective+reactions+to+tastes+and+their+modulation+by+sadness+and+joy.">Facial and affective reactions to tastes and their modulation by sadness and joy.</a> Physiol Behav. 89 (2), 261-269 (2006).</li></ol>

The authors have nothing to disclose.

在有关食品和饮料文学AFEA应用是非常有限的1-11。食品中的应用是新的，从而为建立方法和数据解释的机会。 Arnade（2013年）7间发现个人的情绪反应巧克力牛奶和曲线分析和方差分析下，使用面积奶白色高个体变异。然而，即使参与者可变性，产生参加一个幸福的响应，而较长的悲伤和厌恶有更短的响应时间7。在另一项研究中使用的高和低浓度基本的味道，Arnade（2013年）7，发现基本味道之间的情感反应，以及水平基本味觉强度（高，低强度）两者之间的差别，并没有想象中显著正如所料，从而质疑当前AFEA方法和数据分析的准确性。食品和饮料的感官评价是一个复杂和DYNAMIC响应流程30。时间变化可以发生在整个口腔处理和吞咽从而潜在影响刺激的可接受性随时间30。出于这个原因，它可能有利于测量整个饮食经验评估器响应。具体口服处理时间已建议31（舌，咀嚼，吞咽等初步接触），但没有一个是标准化的，时间在很大程度上取决于项目和研究人员的自由裁量权30。 所提出的情绪时间序列分析能够检测的情绪变化和控制（无味的水）和相应的处理之间的统计学差异。此外，与可接受性有关的情绪轮廓可以在预测相关的食品和饮料的行为帮助。结果表明，区分时间序列趋势与牛奶相关的口味AFEA存在（ 图10 </st荣&gt;，11，和12）。在整合情感表征整个人口的区分食品可接受的时间序列分析助攻（ 图10，11和12），以及支撑享乐可接受的趋势（ 图9）。的Leitch 等 8观察利用时间序列分析（5秒）增甜剂和水的基线之间的差异，并且还发现，对于数据和结果的更好的解释提供时间序列图的利用率。此外，情绪变化可以在可在不同的时间点或时间间隔来确定时间和情绪响应治疗差异进行观察。例如，利奇等 8观察，观察人造甜味剂-水之间的比较的办法情绪（愤怒，高兴和惊讶），但在5秒观察窗口不同的时间进行观察。然而，的Leitch 等 8没有建立表达的方向性，使得难以了解控制（水），并使用他们的图形解释和介绍的处理（不加糖茶）之间的情感差异。在我们的研究中提出的修改和改进的时间序列分析方法允许统计学差异方向性。方向性和结果绘制使研究人员能够可视化，其中发生在选定的时间框架统计相关的情绪变化。 降低视频分析失败是实现有效的数据，并有效地利用时间和人力资源是必不可少的。在协议中的关键步骤和故障排除步骤，包括优化参与者的感官环境（灯光，摄像机角度，桌椅的高度，深入参与引导指令等 ）。此外，参加者应进行筛选和排除，如果他们陷入一个软件公司ompatibility类别（ 即粗框眼镜，胡子拉碴的沉重面孔和肤色）（ 图2）。这些因素将影响AFEA拟合模型，情绪分类，和数据输出。如果一个参加者的数据输出的一个显著部分由"FIT_FAILED"和"FIND_FAILED"的，数据应重新评估列入时间序列分析（ 图1）。如果数据输出日志文件主要含有"FIT_FAILED"和"FIND_FAILED"，因为这是不好的数据（ 图1）的时间序列分析，不能利用。面部阴影，由于灯光设置可能会严重抑制的视频拍摄质量，导致视频采集差。为了避免激烈的阴影，弥漫正面照明是理想而光强度或色彩不相关的20。激烈的头顶照明应降低，因为它可以在脸上20促进阴影。后面的一个黑暗的背景与会者建议20。它是从AFEA软件制造商建议放置在设置在一窗口的前面有漫射日光照明20。此外，如果使用计算机监视器，两个灯可以被放置在照明和阴影减少了20到用户的脸的两侧。此外，专业摄影灯可用于抵消不良的环境照明20。最终，它是由研究人员，个别协议/方法，和环境来控制照明捕获的判断。建议讨论数据采集环境，并与购买和安装之前，软件供应商的工具。此外，桌椅的高度及拍摄角度都是单独调节每个参与者重要的。参赛者应舒适，但在高度相机进行直在脸上。以减少对面上的摄像机角度尝试鼓励用于优化自动对焦EA的视频拍摄。最后，当务之急是之前取样给参与者口头指示。在视频拍摄参与者的行为可能数据收集限制，由于面部遮挡，运动和相机回避。 对于需要研究的参与者样本大小，作者推荐范围10到50人参加。虽然数量不多将提供几乎没有统计力量，需要在一般的至少2个参与者时间序列分析。参与者可变性高，并在本研究的早期阶段，没有指导提供与样本大小。样本大小将取决于香料，风味强度变化，并且预期治疗的可接受性。较小的味道差异样本将需要更多的参与者。 30秒的控制采样周期涵盖的时间跨度足以为整个取样评估期（ 即 ，显示索引卡，打开一个样品（除去盖子），消费记录化和情绪捕获）。整个30秒未在数据分析中使用。这由30第二捕获时间的好处是，研究者可以决定在数据分析中使用的相关评估时间。 30秒的时间窗口可以帮助视频样本中选择感兴趣的时限，而编码或时间戳的视频。最终，时间窗口是高达研究者的决定。在我们的例子中，我们使用了5秒取样窗口后食用。此外，本方法定义了当样品杯不再闭塞面（杯在下巴）的时间为零。它以减少消费，在出现短暂的样品杯面部闭塞和情绪变化之间的时间是非常重要的。由于样品杯面部闭塞的初始时间，其中所述样品与舌头接触是不可靠的数据（ 见图1）。因此，这里的杯不再闭塞面的点是最优recommendatioñ。时间戳必须为​​所有与会者一致。色卡是用于研究人员确定处理中的视频和标记为样品评估适当的时间帧（时间零）的简便方法。色卡，如果治疗是随机顺序，并作为样品标识在连续的视频额外的验证特别有用。 这种技术的局限性存在作为参与者可能不遵循关于参与者的面部方向或不可避免的遮蔽可能导致面部拟合模型故障（ 图2）。但是，建议的关键步骤提供方法来减轻和降低这些干扰。此外，时间序列分析将不读取导出日志文件与主要含有"FIT_FAILED"和"FIND_FAILED"（ 图1）的文件。这些文件不能被打捞并且将不能够被包括在时间序列分析。另外，食品和贝韦拉河的消耗GES仍可以改变以这样的方式扭曲的情感归类的面部结构。硬或耐嚼的食物需要大量的颚运动。吸管和相关吸吮用，也会导致面部遮挡（秸秆），并扭曲了的脸（吮吸）。这种观察是基于从我们的实验室研究的初步数据。该软件的面部模型不能辨别咀嚼（或抽吸）和情绪分类相关联的马达的表达之间的差异。随着食品和饮料样品，面部遮挡的机会比的观看视频和图片更高。参赛者必须带上样品到脸上，并从这样面对中断的软件模型，并潜在地减少有价值的情感信息（ 见图1）取出容器。如前面提到的，情绪快速在短时间内发生。它减少在努力捕获情绪面部闭塞是重要的。建议methodol术使治疗比较在第二的三十分之一找到跨越时间的情感模式的变化以及情绪的持续时间。搭配建议方法，情感长寿的模式是非常重要的。遗憾的是，可能会出现情绪问题的分类。最值得注意的是有分类的快乐和厌恶6，9，32，33，34个问题。通常情况下，这是由于参与者掩盖自己的厌恶或惊讶的感觉由微笑6，32，33，34，可能是由于"社会显示规则"32。此外，该软件AFEA限于七情感类（中性，高兴，悲伤，恐惧，惊讶，愤怒和厌恶）。到食品和饮料的情绪反应可能是更复杂的比普遍的情绪和分类的当前AFEA分类可以是响应于食品或饮料的刺激不同。手动编码使用FACS已应用于gustofacial和基本口味olfactofacial反应和气味各种各样，似乎是非常敏感的问候检测处理分歧AU的32。 FACS是繁琐和非常耗时，然而，不存在或AU的存在的时间应用可以协助的复杂响应该AFEA可能无法正确​​或者如果情感的结果是意想不到的分类是有用的。虽然时间序列数据允许同时与显著表达出现面部分类，应谨慎使用因情绪复杂性成果转化成一个单一的情感中。 所提出的方法和数据分析技术可以被应用到其他饮料和软的食物。 AFEA软件能够识别情绪味无味和样品。所提出的方法和时间分析可以表征隐含的响应，从而在情绪反应，并与食品人口的行为提供了新的发展援助。个未来应用是技术可以扩展到其他的饮料类别或软的食物。我们已经证明的方法来实现视频捕捉的情绪反应和数据分析方法。我们的目标是创造兼具感性AFEA捕获和情感的时间序列分析的标准方法。该方法的做法已经表明我们的研究成功。我们希望扩大并应用此方法用于评价情绪反应的食物和饮料，并选择和行为的关系。

自动面部表情分析（AFEA）是表征情绪反应饮料和食品的前瞻性分析工具。情感分析，可以添加一个额外的维度现有感官科学方法，食品评价做法，和享乐规模等级通常在研究和工业环境中使用的两​​种。情感分析可以提供显露对食品和饮料更准确的响应的附加指标。享乐得分可能包括参与者的偏见引起的故障记录反应1。 AFEA的研究已经在许多研究中的应用，包括电脑游戏，用户行为，教育/教学，并在同情和欺骗心理学研究中使用。大多数食品相关的研究集中在表征情绪反应的食品质量和与食品的人的行为。随着越来越见解食品行为的近期走势来看，文献报道越来越多使用的AFEA用于表征与食品，饮料，和增味剂1-12相关联的人的情绪反应。 AFEA从面部动作编码系统（FACS）的。面部动作编码系统（FACS）歧视在5点强度等级13特点是动作单元（AU）的面部动作。流式细胞仪方法需要训练有素的审评专家，手工编码，显著评估时间，并提供有限的数据分析选项。 AFEA是作为一个快速评价方法来确定的情绪。 AFEA软件依赖于面部肌肉运动，面部数据库和算法来刻画情绪反应14-18。在这项研究中所使用的AFEA软件平均同时在华沙集情感的面部表情图片（WSEFEP）和阿姆斯特丹动态面部表情集（ADFES），这是接近0.70的标准协议达成的"0.67协议FACS指数为手动编码"19 </sup&gt;。包括在分析通用的情绪是快乐（正），悲（负），恶心（负），惊讶（正或负），愤怒（负），害怕（负）和中性每对0一单独的刻度为1（ 0 =不表达; 1 =完全表达）20。此外，心理学文献包括高兴，惊讶，和愤怒"办法"情绪（对刺激）和悲伤，恐惧，厌恶和作为"撤军"的情绪（远离厌恶刺激）21。 目前AFEA软件表征与食物有关的情绪的一个限制是从咀嚼和吞咽以及其他严重电机的动作，比如极端的头部运动相关的面部动作的干扰。该软件的目标更小的面部肌肉运动，有关位置和运动的程度，基于关于面部16,17超过500肌肉分。咀嚼运动干扰表情分类。此限制通货膨胀可以利用液化食品来解决。然而，其他的方法的挑战也能降低视频灵敏度和AFEA分析包括数据收集环境，技术，研究人员的指令，参与者的行为，和参加者的属性。 一个标准的方法学还没有得到开发和验证为最佳的视频捕获和使用AFEA情绪响应于食物和饮料在感官评价实验室环境数据分析。许多方面可以影响视频采集环境，包括照明，照明，参加者的方向，与会者行为，参加者的高度，以及，照相机高度，照相机钓鱼和设备设置遮蔽所致。此外，数据分析方法是不一致的，缺乏评估的情绪反应的标准方法。在这里，我们将展示我们的标准作业程序，用于捕获情绪数据和处理数据到使用饮料有意义的结果（调味乳，进行评价无味牛奶和未调味的水）。据我们所知，只有一个同行审查的出版物，从我们的实验室群，已用于对情感分析的8个数据解释的时间序列;然而，该方法已被更新为我们提出的方法。我们的目标是建立一个完善和一致的方法，以帮助重复性感官评价实验室环境。为了演示，该研究模型的目的是评估如果AFEA可以补充的调味奶，无味牛奶和水无味传统享乐接受评估。这个视频协议的目的是帮助建立AFEA方法，规范的感官评定实验室视频拍摄标准（感觉展位设置），并说明了人口的时空情绪数据分析的方法。

<table><tbody><tr><td>2% Reduced Fat Milk</td><td>Kroger Brand, Cincinnati, OH or DZA Brands, LLC, Salisbury, NC</td><td>na</td><td>for solutions</td></tr><tr><td>Drinking Water</td><td>Kroger Brand, Cincinnati, OH</td><td>na</td><td>for solutions</td></tr><tr><td>Imitation Clear Vanilla Flavor</td><td>Kroger Brand, Cincinnati, OH</td><td>na</td><td>for solutions</td></tr><tr><td>Iodized Salt</td><td>Kroger Brand, Cincinnati, OH</td><td>na</td><td>for solutions</td></tr><tr><td>FaceReader 6</td><td>Noldus Information Technology, Wageningen, The Netherlands</td><td>na</td><td>For Facial Analysis</td></tr><tr><td>Sensory Information Management System (SIMS) 2000</td><td>Sensory Computer Systems, Berkeley Heights, NJ</td><td>Version 6</td><td>For Sensory Data Capture</td></tr><tr><td>Rhapsody</td><td>Acuity Brands Lighting, Inc., Conyers, GA</td><td></td><td>For Environment Illumination</td></tr><tr><td>R Version&amp;nbsp;</td><td>R Core Team 2015</td><td>3.1.1</td><td>For Statistical Analysis</td></tr><tr><td>Microsoft Office</td><td>Microsoft</td><td>na</td><td>For Statistical Analysis</td></tr><tr><td>JMP</td><td>Statistical Analysis Software (SAS) Version 9.2, SAS Institute, Cary, NC</td><td>na</td><td>For Statistical Analysis</td></tr><tr><td>Media Recorder 2.5</td><td>Noldus Information Technology, Wageningen, The Netherlands</td><td>na</td><td>For capturing participants sensory evaluation</td></tr><tr><td>Axis M1054 Camera</td><td>Axis Communications, Lund, Sweden</td><td>na</td><td></td></tr><tr><td>Beverage</td><td></td><td>na</td><td>Beverage or soft food for evaluation</td></tr></tbody></table>

protocol for data collection and analysis applied to automated facial expression analysis technology and temporal analysis for sensory evaluation

We demonstrate a method for capturing emotional response to beverages and liquefied foods in a sensory evaluation laboratory using automated facial expression analysis (AFEA) software. Additionally, we demonstrate a method for extracting relevant emotional data output and plotting the emotional response of a population over a specified time frame. By time pairing each participant's treatment response to a control stimulus (baseline), the overall emotional response over time and across multiple participants can be quantified. AFEA is a prospective analytical tool for assessing unbiased response to food and beverages. At present, most research has mainly focused on beverages. Methodologies and analyses have not yet been standardized for the application of AFEA to beverages and foods; however, a consistent standard methodology is needed. Optimizing video capture procedures and resulting video quality aids in a successful collection of emotional response to foods. Furthermore, the methodology of data analysis is novel for extracting the pertinent data relevant to the emotional response. The combinations of video capture optimization and data analysis will aid in standardizing the protocol for automated facial expression analysis and interpretation of emotional response data.

该方法提出了AFEA数据收集的标准协议。如果建议的方案步骤之后，不可用的情绪数据输出（ 图1），从数据收集差导致（ 图2:A;左图）可能会受到限制。如果日志文件（.txt）主要含有"FIT_FAILED"和"FIND_FAILED"，因为这是不好的数据（ 图1）的时间序列分析，不能利用。此外，该方法包括用于在一时间帧的情感数据输出的两种治疗之间的直接统计比较建立情感信息的协议。时间序列分析可以提供随时间的情感的趋势，并可以提供一个附加值维度享乐可接受的结果。此外，时间序列分析可以显示随着时间的推移，它的饮食体验过程是有价值的情绪水平的变化。 图9）。享乐结果推断，无不是无味牛奶，无味的水和香草精的味道牛奶之间的区别接受。然而，AFEA时间序列分析表明无味牛奶产生反感少（P &lt;0.025; 0秒），惊奇（P &lt;0.025; 0-2.0秒），少伤心（P &lt;0.025; 2.0-2.5秒），少的中性（P &lt;0.025;〜3.0-3.5秒）的反应比没有未调味的水（ 图10）。此外，牛奶中香草精的味道之前5.0秒（P &lt;0.025），少悲推出更快乐的表达（P &lt;0.025; 2.0-3.0和5.0秒），比无味的水（ 图11）。香草，作为一种气味，已与相关方面"轻松"，"宁静"，"放心"，"幸福"，＆＃34;的福祉"，"惊喜"23和"牛奶愉快"24咸味香精有降低（P &lt;0.05），意味着享乐接受评分（中度不喜欢）（ 图9）和乳咸鲜味产生更多的厌恶（p &lt;0.025）后（3.0-5.0秒），比无味的水（ 图12）。强烈咸已与厌恶相关的惊讶和25，26，然而，一些研究表示，味咸不引起面部响应7，27 -29。 <img alt="图1" src="/files/ftp_upload/54046/54046fig1.jpg" /> 图1次最佳数据采集的实施例由于与导致在导出输出文件[FIT_FAILED的原始情绪数据响应点损失AFEA软件参与者不兼容; FIND_FAILED]视频故障发生时面部严重阻塞或者无法映射脸上PERSI<html>指定后的消费窗口期间STS。 <a href="https://www.jove.com/files/ftp_upload/54046/54046fig1large.jpg" target="_blank">请点击此处查看该图的放大版本。</a> <img alt="图2" src="/files/ftp_upload/54046/54046fig2.jpg" /> 图2次最佳数据采集的实施例由于参与者软件建模。图显示次最佳数据捕捉由于参与者软件建模不相容性和面映射的故障判断的情绪反应（A）中。成功适合建模和捕捉参与者的情绪反应（B）的能力的一个例子。 <a href="https://www.jove.com/files/ftp_upload/54046/54046fig2large.jpg" target="_blank">请点击此处查看该图的放大版本。</a> 6fig3.jpg"/&gt; 图3.在一个新的电子表格的数据提取的编制参与者数据的例子。参与者的数据（参与人数，治疗，原创视频的时间，和情感反应）每情感（高兴，中性，悲伤，愤怒，惊讶，恐惧，厌恶和标识）的选择时间框架（秒）。此电子表格被用于后续分析。 <a href="https://www.jove.com/files/ftp_upload/54046/54046fig3large.jpg" target="_blank">请点击此处查看该图的放大版本。</a> <img alt="图4" src="/files/ftp_upload/54046/54046fig4.jpg" /> 图4.编译用于随后的分析中提取参与者数据的实施例。所提取的参与者数据（A1和B1）被编译（A2和B2），作图（A3和B3）和对准（A4和B4），为视觉为直接比较。有pective零时间控制（A4:惊奇无味的水 ）和治疗（B4:惊奇无味牛奶 ）显示比较惊讶的情绪结果。该示例代表，并从每个参与者处理对时间戳文件确定的相应时间为零。 <a href="https://www.jove.com/files/ftp_upload/54046/54046fig4large.jpg" target="_blank">请点击此处查看该图的放大版本。</a> <img alt="图5" src="/files/ftp_upload/54046/54046fig5.jpg" /> 。图5.调整后的时间帧中提取参与者数据的示例提取参与者数据呈现一个真正的"零时刻"（A1和B1）调整后的时间框架的时间调整允许控制（A之间的直接比较:惊奇无味的水 ）和治疗（B2:惊奇Unfla<html> vored牛奶）（A2和B2）。该示例代表，并确定从每个参与者处理对时间戳文件中相应的真正的"零时刻"（调整后）。 <a href="https://www.jove.com/files/ftp_upload/54046/54046fig5large.jpg" target="_blank">请点击此处查看该图的放大版本。</a> <img alt="图6" src="/files/ftp_upload/54046/54046fig6.jpg" /> 图6.编译所有参与者的数据流程的例子 ，参与者，调整后的时间，和配对处理（ 如，无味的水和无味牛奶）在每个时间点被编译成进行统计分析做准备。 <a href="https://www.jove.com/files/ftp_upload/54046/54046fig6large.jpg" target="_blank">请点击这里查看更大的版本这个数字。</a> ES / ftp_upload / 54046 / 54046fig7.jpg"/&gt; 图7.数据表格例子进行比较的控制（未调味的水），并用跨参与者的Wilcoxon测试在特定的时间点的处理（未调味奶）。该图表示相应的样品和对照的情感结果之间的直接比较（未调味的水）使用顺序配对跨越参与者非参数检验的Wilcoxon。 <a href="https://www.jove.com/files/ftp_upload/54046/54046fig7large.jpg" target="_blank">请点击此处查看该图的放大版本。</a> <img alt="图8" src="/files/ftp_upload/54046/54046fig8.jpg" /> 图8.数据电子表格的例子以图形的相关处理图形的结果，如果（P &lt;0.025）（即无味牛奶相比，无味的水）。连续的结果在整个配对PARTICIPA非参数检验的WilcoxonNTS作图对于其中零假设被拒绝的次数。 <a href="https://www.jove.com/files/ftp_upload/54046/54046fig8large.jpg" target="_blank">请点击此处查看该图的放大版本。</a> <img alt="图9" src="/files/ftp_upload/54046/54046fig9.jpg" /> 图9. 平均可接受的无味水（享乐）分数，无味牛奶，牛奶和咸鲜味的乳饮料的解决方案。基于9点享乐规模可接受香草精的味道 （1 =极其厌恶，5 =既不像也不不喜欢，9 =极喜欢;平均值+/- SD）1。治疗装置具有不同上标在喜欢（P &lt;0.05）显​​著不同。无味牛奶，牛奶中的水无味和香草精的味道是平均分数接受没有什么不同（P&gt; 0.05），被评为"稍微喜欢"。牛奶味咸有降低（P &lt;0.05）平均得分接受（中度不喜欢）。 <a href="https://www.jove.com/files/ftp_upload/54046/54046fig9large.jpg" target="_blank">请点击此处查看该图的放大版本。</a> <img alt="图10" src="/files/ftp_upload/54046/54046fig10.jpg" /> 图大于5.0秒自动面部表情分析数据分类情绪10.时序图比较无味牛奶和未调味的水，基于未调味奶和未调味的水（基线）之间的顺序成对非参数的Wilcoxon测试，结果示于各治疗图表如果治疗中位数较高，为每一个情感更大的意义（p &lt;0.025）。一行存在指示显著差异（p &lt;在特定时间点的中位数越高0.025），而缺少一条线表示在特定时间点（第无差别&gt; 0.025）。缺席相比于在5.0秒无味的水（P &lt;0.025）的无味乳线（A）显示没有情感分类。在无味的水（B），情感结果在0秒相比，原味牛奶揭示反感（深红色线），惊奇（橙色线）出现0之间- 1.5秒，悲伤（绿线）出现约2.5秒，和中性（红线）出现约3 - 3.5秒（p &lt;0.025） <a href="https://www.jove.com/files/ftp_upload/54046/54046fig10large.jpg" target="_blank">点击此处查看该图的放大版本。</a> <img alt="图11" src="/files/ftp_upload/54046/54046fig11.jpg" /> 分类情感基础上的自动面部表情分析数据大于5.0秒，牛奶和水无味（基线），根据顺序比较香草精的味道图11.时间序列图之间的配对非参数检验的Wilcoxon在牛奶和水无味香草精的味道，结果在各自的处理图形绘制如果治疗中位数高和每个情感更大的意义（P &lt;0.025）。一行存在指示显著差异（p &lt;在特定时间点的中位数越高0.025），而缺少一条线表示在特定时间点（第无差别&gt; 0.025）。牛奶（A）香草精的味道展示快乐只有5秒（蓝线），而无味的水（B）显示更难过围绕前2 - 2.5和5秒（绿线）（P &lt;0.025）。 <a href="https://www.jove.com/files/ftp_upload/54046/54046fig11large.jpg" target="_blank">请点击这里查看更大的版本这个数字。</a> <img alt="图12" src="/files/ftp_upload/54046/54046fig12.jpg" /> 分类情感基础上的自动面部EXPRES图12.时间序列图锡安分析数据超过5.0秒比较牛奶和无味的水。基于咸味香精的牛奶和原味水（基线）顺序配对非参数秩检验咸味香精，结果在各自的处理图形绘制如果治疗中位数较高，更大意义（p &lt;0.025）为每情感。一行存在指示显著差异（p &lt;在特定时间点的中位数越高0.025），而缺少一条线表示在特定时间点（第无差别&gt; 0.025）。牛奶（A）咸味香精有3显著厌恶-第5秒（深红色线），而无味的水（B）的开头（深红色线）的厌恶和2个中性-第5秒（红线）（P &lt;0.025 ）。 <a href="https://www.jove.com/files/ftp_upload/54046/54046fig12large.jpg" target="_blank">请点击此处查看该图的放大版本。</a>

Watch this Scientific Journal Video about Protocol for Data Collection and Analysis Applied to Automated Facial Expression Analysis Technology and Temporal Analysis for Sensory Evaluation at JoVE.com

Protocol for Data Collection and Analysis Applied to Automated Facial Expression Analysis Technology and Temporal Analysis for Sensory Evaluation

一种用于捕获和统计分析人口的情绪反应饮料和流质食物中使用自动面部表情分析软件感官评价实验室协议描述。

协议进行数据采集和分析应用到自动人脸表情分析技术和时序分析感官评价

We demonstrate a method for capturing emotional response to beverages and liquefied foods in a sensory evaluation laboratory using automated facial ...

protocol-for-data-collection-analysis-applied-to-automated-facial

Nanyang Technological University

Research

JoVE Journal

Behavior

9.4K 次观看.  Virginia Tech. 一种用于捕获和统计分析人口的情绪反应饮料和流质食物中使用自动面部表情分析软件感官评价实验室协议描述。 

视频: Protocol for Data Collection and Analysis Applied to Automated Facial Expression Analysis Technology and Temporal Analysis for Sensory Evaluation

A Protocol for Comprehensive Assessment of Bulbar Dysfunction in Amyotrophic Lateral Sclerosis (ALS)

Improved methods for assessing bulbar impairment are necessary for expediting diagnosis of bulbar dysfunction in ALS, for predicting disease progression across speech subsystems, and for addressing the critical need for sensitive outcome measures for ongoing experimental treatment trials. To address this need, we are obtaining longitudinal profiles of bulbar impairment in 100 individuals based on a comprehensive instrumentation-based assessment that yield objective measures. Using instrumental approaches to quantify speech-related behaviors is very important in a field that has primarily relied on subjective, auditory-perceptual forms of speech assessment1. Our assessment protocol measures performance across all of the speech subsystems, which include respiratory, phonatory (laryngeal), resonatory (velopharyngeal), and articulatory. The articulatory subsystem is divided into the facial components (jaw and lip), and the tongue. Prior research has suggested that each speech subsystem responds differently to neurological diseases such as ALS. The current protocol is designed to test the performance of each speech subsystem as independently from other subsystems as possible. The speech subsystems are evaluated in the context of more global changes to speech performance. These speech system level variables include speaking rate and intelligibility of speech.
The protocol requires specialized instrumentation, and commercial and custom software. The respiratory, phonatory, and resonatory subsystems are evaluated using pressure-flow (aerodynamic) and acoustic methods. The articulatory subsystem is assessed using 3D motion tracking techniques. The objective measures that are used to quantify bulbar impairment have been well established in the speech literature and show sensitivity to changes in bulbar function with disease progression. The result of the assessment is a comprehensive, across-subsystem performance profile for each participant. The profile, when compared to the same measures obtained from healthy controls, is used for diagnostic purposes. Currently, we are testing the sensitivity and specificity of these measures for diagnosis of ALS and for predicting the rate of disease progression. In the long term, the more refined endophenotype of bulbar ALS derived from this work is expected to strengthen future efforts to identify the genetic loci of ALS and improve diagnostic and treatment specificity of the disease as a whole. The objective assessment that is demonstrated in this video may be used to assess a broad range of speech motor impairments, including those related to stroke, traumatic brain injury, multiple sclerosis, and Parkinson disease.

A Protocol for Comprehensive Assessment of Bulbar Dysfunction in Amyotrophic Lateral Sclerosis ALS

Objective assessments of the physiological mechanisms that support speech are needed to monitor disease onset and progression in persons with ALS and to quantify treatment effects in clinical trials. In this video, we present a comprehensive, instrumentation-based protocol for quantifying speech motor performance in clinical populations.

延髓功能障碍肌萎缩性脊髓侧索硬化症（ALS）的综合评估议定书

Improved methods for assessing bulbar impairment are necessary for expediting diagnosis of bulbar dysfunction in ALS, for predicting disease progression ...

科研

医学

Experimental Assessment of Mouse Sociability Using an Automated Image Processing Approach

Mouse is the preferred model organism for testing drugs designed to increase sociability. We present a method to quantify mouse sociability in which the test mouse is placed in a standardized apparatus and relevant behaviors are assessed in three different sessions (called session I, II, and III).
The apparatus has three compartments (see Figure 1), the left and right compartments contain an inverted cup which can house a mouse (called &#34;stimulus mouse&#34;).
In session I, the test mouse is placed in the cage and its mobility is characterized by the number of transitions made between compartments. In session II, a stimulus mouse is placed under one of the inverted cups and the sociability of the test mouse is quantified by the amounts of time it spends near the cup containing the enclosed stimulus mouse vs. the empty inverted cup. In session III, the inverted cups are removed and both mice interact freely. The sociability of the test mouse in session III is quantified by the number of social approaches it makes toward the stimulus mouse and by the number of times it avoids a social approach by the stimulus mouse.
The automated evaluation of the movie detects the nose of the test mouse, which allows the determination of all described sociability measures in session I and II (in session III, approaches are identified automatically but classified manually). To find the nose, the image of an empty cage is digitally subtracted from each frame of the movie and the resulting image is binarized to identify the mouse pixels. The mouse tail is automatically removed and the two most distant points of the remaining mouse are determined; these are close to nose and base of tail. By analyzing the motion of the mouse and using continuity arguments, the nose is identified.
<img alt="Figure 1" src="/files/ftp_upload/52508/52508fig1.jpg" /> 
Figure 1. Assessment of Sociability During 3 sessions. Session I (top): Acclimation of test mouse to the cage. Session II (middle): Test mouse moving freely in the cage while the stimulus mouse is enclosed in an inverted cup. Session III (bottom): Both test mouse and stimulus mouse are allowed to move freely and interact with each other.

This protocol describes a method to quantify mouse sociability. Mice are videotaped as they move and interact in a special cage. Movie processing allows for the automated quantification of sociability with excellent accuracy and reliability.

使用自动图像处理方法鼠标社交性的实验评估

Mouse is the preferred model organism for testing drugs designed to increase sociability. We present a method to quantify mouse sociability in which the ...

行为学

Assessing the Multiple Dimensions of Engagement to Characterize Learning: A Neurophysiological Perspective

In a recent theoretical synthesis on the concept of engagement, Fredricks, Blumenfeld and Paris1 defined engagement by its multiple dimensions: behavioral, emotional and cognitive. They observed that individual types of engagement had not been studied in conjunction, and little information was available about interactions or synergy between the dimensions; consequently, more studies would contribute to creating finely tuned teaching interventions. Benefiting from the recent technological advances in neurosciences, this paper presents a recently developed methodology to gather and synchronize data on multidimensional engagement during learning tasks. The technique involves the collection of (a) electroencephalography, (b) electrodermal, (c) eye-tracking, and (d) facial emotion recognition data on four different computers. This led to synchronization issues for data collected from multiple sources. Post synchronization in specialized integration software gives researchers a better understanding of the dynamics between the multiple dimensions of engagement. For curriculum developers, these data could provide informed guidelines for achieving better instruction/learning efficiency. This technique also opens up possibilities in the field of brain-computer interactions, where adaptive learning or assessment environments could be developed.

This paper aims to describe the techniques involved in the collection and synchronization of the multiple dimensions (behavioral, affective and cognitive) of learners&#8217; engagement during a task.

评估参与的多维度来表征学习:神经生理学角度

In a recent theoretical synthesis on the concept of engagement, Fredricks, Blumenfeld and Paris1 defined engagement by its multiple dimensions: ...

Exploring the Use of Isolated Expressions and Film Clips to Evaluate Emotion Recognition by People with Traumatic Brain Injury

The current study presented 60 people with traumatic brain injury (TBI) and 60 controls with isolated facial emotion expressions, isolated vocal emotion expressions, and multimodal (i.e., film clips) stimuli that included contextual cues. All stimuli were presented via computer. Participants were required to indicate how the person in each stimulus was feeling using a forced-choice format. Additionally, for the film clips, participants had to indicate how they felt in response to the stimulus, and the level of intensity with which they experienced that emotion.

This paper describes how to implement a battery of behavioral tasks to examine emotion recognition of isolated facial and vocal emotion expressions, and a novel task using commercial television and film clips to assess multimodal emotion recognition that includes contextual cues.

探索孤立的表达式和电影剪辑的使用与创伤性脑损伤的人评价情感识别

The current study presented 60 people with traumatic brain injury (TBI) and 60 controls with isolated facial emotion expressions, isolated vocal emotion ...

Conscious and Non-conscious Representations of Emotional Faces in Asperger's Syndrome

Several neuroimaging studies have suggested that the low spatial frequency content in an emotional face mainly activates the amygdala, pulvinar, and superior colliculus especially with fearful faces1-3. These regions constitute the limbic structure in non-conscious perception of emotions and modulate cortical activity either directly or indirectly2. In contrast, the conscious representation of emotions is more pronounced in the anterior cingulate, prefrontal cortex, and somatosensory cortex for directing voluntary attention to details in faces3,4. Asperger&#39;s syndrome (AS)5,6 represents an atypical mental disturbance that affects sensory, affective and communicative abilities, without interfering with normal linguistic skills and intellectual ability. Several studies have found that functional deficits in the neural circuitry important for facial emotion recognition can partly explain social communication failure in patients with AS7-9. In order to clarify the interplay between conscious and non-conscious representations of emotional faces in AS, an EEG experimental protocol is designed with two tasks involving emotionality evaluation of either photograph or line-drawing faces. A pilot study is introduced for selecting face stimuli that minimize the differences in reaction times and scores assigned to facial emotions between the pretested patients with AS and IQ/gender-matched healthy controls. Information from the pretested patients was used to develop the scoring system used for the emotionality evaluation. Research into facial emotions and visual stimuli with different spatial frequency contents has reached discrepant findings depending on the demographic characteristics of participants and task demands2. The experimental protocol is intended to clarify deficits in patients with AS in processing emotional faces when compared with healthy controls by controlling for factors unrelated to recognition of facial emotions, such as task difficulty, IQ and gender.

An EEG experimental protocol is designed to clarify the interplay between conscious and non-conscious representations of emotional faces in patients with Asperger&#39;s syndrome. The technique suggests that patients with Asperger&#39;s syndrome have deficits in non-conscious representation of emotional faces, but have comparable performance in conscious representation with healthy controls.

在阿斯伯格综合症情绪面孔的自觉和不自觉的交涉

Several neuroimaging studies have suggested that the low spatial frequency content in an emotional face mainly activates the amygdala, pulvinar, and ...

Using Facial Electromyography to Assess Facial Muscle Reactions to Experienced and Observed Affective Touch in Humans

"Affective" touch is believed to be processed in a manner distinct from discriminatory touch and to involve activation of C-tactile (CT) afferent fibers. Touch that optimally activates CT fibers is consistently rated as hedonically pleasant. Patient groups with impaired social-emotional functioning also show disordered affective touch ratings. However, relying on self-reported ratings of touch has many limitations, including recall bias and communication barriers. Here, we describe a methodological approach to study affective responses to touch via facial electromyography (EMG) that circumvents the reliance on self-report ratings. Facial EMG is an objective, quantitative, and non-invasive method to measure facial muscle activity indicative of affective responses. Responses can be assessed across healthy and patient populations without the need for verbal communication. Here, we provide two separate datasets demonstrating that CT-optimal and non-optimal touch elicit distinct facial muscle reactions. Moreover, facial EMG responses are consistent across stimulus modalities, e.g. tactile (experienced touch) and visual (observed touch). Finally, the temporal resolution of facial EMG can detect responses on timescales that supersede that of verbal reporting. Together, our data suggest that facial EMG is a suitable methodology for use in affective tactile research that can be used to supplement, or in some cases, supplant, existing measures.

We describe a protocol to assess facial muscle activity in response to experienced and observed tactile stimulation using facial electromyography.

面部肌电图对人体有经验和观察到的情感触觉的影响评估

"Affective" touch is believed to be processed in a manner distinct from discriminatory touch and to involve activation of C-tactile (CT) afferent fibers. ...

Dynamic Digital Biomarkers of Motor and Cognitive Function in Parkinson's Disease

As Parkinson&#8217;s disease (PD) is a heterogeneous disorder, personalized medicine is truly required to optimize care. In their current form, standard scores from paper and pencil symptom- measures traditionally used to track disease progression are too coarse (discrete) to capture the granularity of the clinical phenomena under consideration, in the face of tremendous symptom diversity. For this reason, sensors, wearables, and mobile devices are increasingly incorporated into PD research and routine care. These digital measures, while more precise, yield data that are less standardized and interpretable than traditional measures, and consequently, the two types of data remain largely siloed. Both of these issues present barriers to the broad clinical application of the field&#8217;s most precise assessment tools. This protocol addresses both problems. Using traditional tasks to measure cognition and motor control, we test the participant, while co-registering biophysical signals unobtrusively using wearables. We then integrate the scores from traditional paper-and-pencil methods with the digital data that we continuously register. We offer a new standardized data type and unifying statistical platform that enables the dynamic tracking of change in the person&#8217;s stochastic signatures under different conditions that probe different functional levels of neuromotor control, ranging from voluntary to autonomic. The protocol and standardized statistical framework offer dynamic digital biomarkers of physical and cognitive function in PD that correspond to validated clinical scales, while significantly improving their precision.&#160;

This protocol offers a digitization of portions of traditional clinical tasks commonly used to measure cognition and motor control in Parkinson&#8217;s disease. Clinical tasks are digitized while biophysical rhythms are co-registered from different functional levels of the nervous systems, ranging from voluntary, spontaneous, automatic to autonomic.

帕金森病中运动与认知功能的动态数字生物标志物

As Parkinson&#8217;s disease (PD) is a heterogeneous disorder, personalized medicine is truly required to optimize care. In their current form, standard ...

Eye-tracking Technology and Data-mining Techniques used for a Behavioral Analysis of Adults engaged in Learning Processes

Behavioral analysis of adults engaged in learning tasks is a major challenge in the field of adult education. Nowadays, in a world of continuous technological changes and scientific advances, there is a need for life-long learning and education within both formal and non-formal educational environments. In response to this challenge, the use of eye-tracking technology and data-mining techniques, respectively, for supervised (mainly prediction) and unsupervised (specifically cluster analysis) learning, provide methods for the detection of forms of learning among users and/or the classification of their learning styles. In this study, a protocol is proposed for the study of learning styles among adults with and without previous knowledge at different ages (18 to 69-year-old) and at different points throughout the learning process (start and end). Statistical analysis-of-variance techniques mean that differences may be detected between the participants by type of learner and previous knowledge of the task. Likewise, the use of&#160;unsupervised learning clustering techniques throws light on similar forms of learning among the participants across different groups. All these data will facilitate personalized proposals from the teacher for the presentation of each task at different points in the chain of information processing. It will likewise be easier for the teacher to adapt teaching materials to the learning needs of each student or group of students with similar characteristics.

We present a protocol for a behavioral analysis of adults (ages 18 to 70-year-old) engaged in learning processes, undertaking tasks designed for Self-Regulated Learning (SRL). The participants, university teachers and students, and adults from the University of Experience, were monitored with eye-tracking devices and the data were analyzed with data-mining techniques.

用于对从事学习过程的成年人进行行为分析的眼跟踪技术和数据挖掘技术

Behavioral analysis of adults engaged in learning tasks is a major challenge in the field of adult education. Nowadays, in a world of continuous ...

提高课堂参与度：基于 AI 的学生注意力水平分析

Artificial Intelligence-Based System for Detecting Attention Levels in Students

The attention level of students in a classroom can be improved through the use of Artificial Intelligence (AI) techniques. By automatically identifying the attention level, teachers can employ strategies to regain students' focus. This can be achieved through various sources of information.
One source is to analyze the emotions reflected on students' faces. AI can detect emotions, such as neutral, disgust, surprise, sadness, fear, happiness, and anger. Additionally, the direction of the students' gaze can also potentially indicate their level of attention. Another source is to observe the students' body posture. By using cameras and deep learning techniques, posture can be analyzed to determine the level of attention. For example, students who are slouching or resting their heads on their desks may have a lower level of attention. Smartwatches distributed to the students can provide biometric and other data, including heart rate and inertial measurements, which can also be used as indicators of attention. By combining these sources of information, an AI system can be trained to identify the level of attention in the classroom. However, integrating the different types of data poses a challenge that requires creating a labeled dataset. Expert input and existing studies are consulted for accurate labeling. In this paper, we propose the integration of such measurements and the creation of a dataset and a potential attention classifier. To provide feedback to the teacher, we explore various methods, such as smartwatches or direct computers. Once the teacher becomes aware of attention issues, they can adjust their teaching approach to re-engage and motivate the students. In summary, AI techniques can automatically identify the students' attention level by analyzing their emotions, gaze direction, body posture, and biometric data. This information can assist teachers in optimizing the teaching-learning process.

作者聚焦：解决测量课堂注意力的技术和主观挑战

This paper proposes an artificial intelligence-based system to automatically detect whether students are paying attention to the class or are distracted. This system is designed to help teachers maintain students' attention, optimize their lessons, and dynamically introduce modifications in order for them to be more engaging.

基于人工智能的系统，用于检测学生的注意力水平

The attention level of students in a classroom can be improved through the use of Artificial Intelligence (AI) techniques. By automatically identifying ...

协议进行数据采集和分析应用到自动人脸表情分析技术和时序分析感官评价

摘要

探索更多视频

此视频中的章节

延髓功能障碍肌萎缩性脊髓侧索硬化症（ALS）的综合评估议定书

使用自动图像处理方法鼠标社交性的实验评估

评估参与的多维度来表征学习:神经生理学角度

探索孤立的表达式和电影剪辑的使用与创伤性脑损伤的人评价情感识别

在阿斯伯格综合症情绪面孔的自觉和不自觉的交涉

面部肌电图对人体有经验和观察到的情感触觉的影响评估

帕金森病中运动与认知功能的动态数字生物标志物

用于对从事学习过程的成年人进行行为分析的眼跟踪技术和数据挖掘技术

基于人工智能的系统，用于检测学生的注意力水平

基于人工智能的系统，用于检测学生的注意力水平