Dissociation of the Confounding Influences of Expectancy and Integrative Difficulty Residing in Anomalous Sentences in Event-related Potential Studies

Summary

We present a protocol to dissociate the intertwining factors of integrative difficulty and unexpectedness in semantically anomalous sentences by applying multiple repetitions to enhance participant's expectancy for anomalous sentences. The dissociation helps to investigate the major contributor of elicited event-related potentials (ERP) effects such as N400 in language studies.

Abstract

The confounding factors of unexpectedness and semantic integration difficulty naturally residing in anomalous sentences in language studies make it difficult to determine the underlying processing mechanism of ERP components. Unlike the traditional static approach of manipulating expectancy through corpus frequency or cloze probability, this protocol proposes a dynamic method to enhance participants' expectancy for rarely-met anomalous sentences by multiple repetitions while maintaining their semantic integration difficulties. To address the time cost increase resulting from multiple repetitions, this protocol proposes to repeat only the strictly simplified core structure extracted from the anomalous sentence before presenting the semantically enriched, much more informative complete anomalous sentence containing the anomalous core structure to reinitiate the semantic integration processing. The complete anomalous sentence elicited a P600 effect. It suggests that the participants did not give up processing the anomalous information after repetitions and the same semantic integration difficulty was successfully reinitiated. Importantly, the representative experimental results reveal that the greatly attenuated N400 effect caused by multiple repetitions was not recovered by the follow-up reinitiated semantic integration difficulty. It suggests that the attenuated N400 effect should be mainly attributed to the enhancement of expectancy for anomalous information by multiple repetitions. The experimental results show that this method can effectively enhance participants' expectancy for anomalous sentences while retaining the semantic integration difficulty.

Introduction

Anomalous sentences are widely used by linguists to study online cognitive processing of normal languages. For example, in event-related potentials (ERP) studies, sentences with semantic anomalies (e.g., "He spread the warm bread with socks.") were reported to elicit an N400 effect¹ (but also see some other studies reporting a semantic P600 effect^2,3), while sentences with syntactic difficulties or anomalies (e.g., "The woman persuaded to answer the door…") were reported to elicit a P600 effect^4,5. These electrophysiological components are widely used as reliable indicators for investigating the normal temporal courses of processing information from different aspects of language, such as syntax and semantics.

Anomalous sentences cause great difficulties for semantic integration processing during comprehension. However, confounding factors such as unexpectedness (i.e., any anomalous expression is naturally an unexpected expression) make it difficult to determine the true cognitive process underlying the observed effect elicited by anomalous sentences. For example, if an N400 effect is elicited by an anomalous sentence, it is unclear whether it is caused by unexpectedness^6,7,8 or integrative difficulty^1,9,10.

To ascertain whether it is expectancy or semantic integration difficulty that contributes to the elicited electrophysiological effect, we need to dissociate these two factors. Traditionally, expectancy is often measured by corpus frequency (in word studies) or cloze probability (in sentence studies). The widely applied method to manipulate expectancy in traditional experiments is by choosing stimuli with high and low scores in expectancy to form expected and unexpected groups. This method is effective in manipulating expectancy and has produced abundant insightful results.

However, as a static approach to manipulating expectancy, it has one limitation: it is hard for the expected and unexpected groups to have the same semantic integration difficulty. With this manipulation, the stimuli selected for the expected and unexpected groups have to be different; thus the semantic integration difficulty is altered when we manipulate expectancy by using different stimuli with different expectancy values or cloze probabilities). While we may find unexpected but reasonable expressions (i.e., unexpected expressions made up of words that can be successfully integrated into a reasonable message), it is possible that the integrative efforts required by these unexpected but reasonable expressions are different from those required by normal expressions. If the differences are not controlled, significant differences in brain responses might ensue, as clear evidence demonstrates that the integrative processing of unexpected but reasonable new metaphorical expressions triggers brain responses quite different from those triggered by conventional metaphors^11,12.

To address this issue, we propose a new method to dynamically enhance participants' expectancy for anomalous sentences while trying to maintain the semantic integration difficulty. Specifically, we quickly familiarize participants with unfamiliar anomalous sentences and thus enhance their expectancy through multiple repetitions. Importantly, multiple repetitions do not change the stimulus itself; therefore, the semantically anomalous information itself remains unchanged (i.e., the critical word still cannot be successfully integrated into the context).

However, the brain might give up integration after learning that the anomalous information cannot be successfully integrated into the previous context at all (i.e., the integration processing might be absent so that there is no integration difficulty). Therefore, this protocol proposes to repeat only the core anomalous information extracted from the anomalous sentence first, and then use the complete anomalous sentence which contains the identical anomalous information as a semantically enriched version of this core anomalous information, to initiate new semantic integration processing in the repetition condition. With the identical anomalous information in the semantically enriched complete anomalous sentence and the initiated new semantic integration processing, we assume that the semantic integration difficulty triggered by the anomalous information in the complete anomalous sentence after multiple repetitions should remain almost the same as that triggered by the anomalous information before repetitions (regardless of the status of semantic integration processing during repetitions). Hence, we assume the factor of semantic integration difficulty in the semantically enriched complete anomalous sentences remains the same after repetitions as in the correspondingly simplified core structures, but the expectancy is greatly enhanced.

Based on these assumptions, we compare the N400 effect elicited by semantically enriched complete anomalous sentences containing the repeated core anomalous information with that elicited by newly-met complete anomalous sentences, to investigate the major contributor of this elicited ERP effect. The working hypotheses are as follows: according to previous studies, the N400 effect would be significantly attenuated by repetitions. On the basis of the attenuated N400, if the newly initiated semantic integration of the same anomalous information causes recovery of the attenuated N400 effect to a level similar to that elicited by the same type of anomalous information contained in newly-met anomalous sentences with no repetitions, then it suggests that the semantic integration difficulty is the dominant contributor of the elicited N400 effect; otherwise it suggests that unexpectedness is the major contributor.

Protocol

The present protocol was approved by the Institutional Review Board of Tsinghua University.

1. Stimuli construction

1. Construct critical anomalous sentences for the repetition and non-repetition groups
   1. Prepare an adequate number of correct sentences for the construction of anomalous sentences (no less than 40). For example, “These two components were separated by a centrifugal device.” Try to use only emotion-free sentences as much as possible.
   2. Construct anomalous sentences based on the correct sentences and then allocate half to the repetition group and half to the non-repetition group.
      1. For example, allocate the sentence “These two components were participled by a centrifugal device” to the repetition group and “These two components were semicoloned by a centrifugal device” to the non-repetition group. Ensure that the two groups of anomalous sentences are highly comparable in elements such as plausibility, violation type and length.
   3. Mix all anomalous sentences with a group of normal sentences (i.e., sentences without integration difficulty) in a random manner to form a list of normal and anomalous sentences.
   4. Invite a group of native speakers (no less than 30 persons) to score the acceptability of the anomalous sentences using a 1–7-point scale, with a high value representing high acceptability.
   5. Select enough pairs of appropriate anomalous sentences (no less than 35) for use as stimuli based on the acceptability scores. Anomalous sentences should be as low as possible in acceptability (e.g., with scores lower than 2). Ensure that the two anomalous sentence groups are not significantly different in the acceptability score (repeated measures analysis of variance is recommended).
2. Prepare corresponding stimuli for these critical anomalous sentences for repetitions
   1. For the repetition group, extract the core anomalous elements from the complete anomalous sentence to form a core structure for repetition (e.g., for studying the N400 effect, extract the core elements causing semantic incongruity).
      1. Ensure that only the most simplified core anomalous elements are kept (i.e., remove any information that is unrelated to elicitation of the effect to be studied). For example, extract only the semantically anomalous core structure “components were participled…” from the complete anomalous sentence “These two components were participled by a centrifugal device.”
   2. For the non-repetition group, change the core structure so that it is different from the one extracted from the complete anomalous sentence, to prevent repetition of the anomalous core structure contained in the complete anomalous sentence (i.e., to prepare a different core structure for the use of repetition).
      1. For example, after extracting the core structure “components were semicoloned…”from the critical anomalous sentence “These two components were semicoloned by a centrifugal device” in the non-repetition group, replace the keyword “semicoloned” in the extracted core structure with another word such as “differenced” (i.e., use “components were differenced…” instead of “components were semicoloned…” for repetition with the non-repetition group).
   3. (Optional minor modifications) Make minor modifications to the extracted core structure (e.g., tense or singular/plural forms), but ensure that the new one can still elicit the same effect as the corresponding complete anomalous sentence.
      1. For example, the past tense and the plural form in the anomalous core structure can be further changed to the present tense and the singular form: “component is participled…” instead of “components were participled.”
         NOTE: The minor modification step is optional. It is highly recommended that the extracted anomalous core structure to be used for repetition have the same form and order as the core structure in the anomalous complete sentence. The purpose of the minor modification is to prevent the participant from memorizing the core structure as an entire unit, so that the follow-up presentation of the anomalous complete sentence can better reinitiate the semantic integration. This protocol chooses to apply minor modification.
   4. Construct a filler short expression for each of the extracted (and modified) core structures in both the repetition and non-repetition groups. For example, coin and introduce a correct but similar filler expression “component is mixed…” for the anomalous core structure “component is participled…” in the repetition group.
   5. Similarly, coin and introduce another correct but similar filler expression “component is discovered…” for the anomalous core structure “component is differenced…” in the non-repetition group.
      NOTE: The filler short expressions are introduced for two purposes: (i) to balance the correctness of the anomalous core structures, and (ii) to serve as the baseline for observing possible elicited effects at different repetition stages (e.g., N400 effect at the first-time, the fifth-time or the tenth-time of repetition), because even the correct stimuli may be influenced by multiple repetitions.

(a) Example of semantically enriched complete anomalous sentences and their preceding core structures
	Repeated Core Structure	Semantically Enriched Complete Sentence
Control		These two components were separated by a centrifugal device.
Repetition Group	* Component is participled…	* These two components were participled by a centrifugal device.
Non-repetition Group	* Component is differenced…	* These two components were semicoloned by a centrifugal device.
(b) Example of the simplified core structures in the repetition part
	Implausible Expressions	Plausible Expressions
For Repetition Group	* Component is participled…	Component is mixed…
For Non-repetition Group	* Component is differenced…	Component is discovered…

Table 1: Examples of stimuli: Complete sentences and simplified core structures. The upper half of the table displays examples of complete sentences (in the control group, the repeated anomalous sentence group,and the unrepeated anomalous sentence group, respectively) in the complete sentence part and their corresponding anomalous core structures to be used in the repetition part; the lower half of the table displays the anomalous core structures and their plausible filler short expressions in the repetition part.

3. Construct filler sentences
   1. Prepare enough filler sentences to enrich the sentence type and to make sure that the normal and anomalous sentences are well-balanced in number.
   2. Select a number of correct filler sentences from the filler sentences (based on the number of all critical anomalous sentences in the repetition group), and create a repetition part for each of the selected filler sentences in the same way as creating the repetition parts for the critical anomalous sentences: extract the core structure from each correct filler sentence and coin a corresponding short filler expression for this core structure.
      NOTE: A repetition part is introduced before each correct filler sentence to avoid exposure of the critical anomalous sentences (which also have a repetition part prior to them). After this step, there are two types of complete sentences: sentences with a preceding repetition part (including all critical sentences and the selected filler sentences preceded by a repetition part) and sentences without a preceding repetition part.

2. Stimuli presentation

1. Presentation of the repetition part
   1. Present the extracted core structure and its correct filler expression (see Table 1b) together for a specific number of times (e.g., seven times) in a random manner (see the repetition part in Figure 1).
2. Presentation of the semantic reinitiating complete sentence
   1. Present the complete anomalous sentence right after its corresponding repetition part (see Figure 1).
      NOTE: Different colors for the repetition part and the complete sentence (e.g., dark blue versus dark green) prove helpful for the participants to distinguish these two parts.
   2. (Optional modifications) If the researcher is concerned only about results of the complete sentence part, then the short expressions can be presented as an entire unit to save time.

Figure 1: Schematic illustration of the stimulus presentation. The upper half of the figure displays the flowchart of presenting the repetition part and its corresponding follow-up complete anomalous sentence, the left side of the lower half displays the detailed manner of presenting each short expression, and the right side of the lower half displays the detailed manner of presenting each complete sentence. Please click here to view a larger version of this figure.

3. Task setting for the repetition part and the complete sentence part
   1. Give a task for each core structure in the repetition part to keep participants focused, for example, a plausibility judging task (or a similar task which requires the participants to respond differently to different core structures, e.g., judging whether an expression has previously been given).
   2. Choose task for the complete sentence part that are widely used in previous related studies (e.g., plausibility judging) so that the results can be compared with results of previous studies.
4. Overall stimuli organization
   1. Present each repetition part (along with its corresponding subsequent complete sentence) only after an anomalous sentence, and present after a normal sentence another complete sentence (whether it is normal or anomalous) (see Figure 2).
      NOTE: This is useful in two ways: (i) providing the participant a reasonable explanation about why some sentences are preceded by a repetition part but others are not (e.g., a researcher may explain that only an incorrect sentence will be followed by a repetition part while a correct sentence will have the power to bypass the repetition part and jump directly to another complete sentence); (ii) making the experiment more interesting by avoiding monotony and thus helping the participant remain focused.

Figure 2: The organization of stimuli in each block. This figure displays the overall flowchart of programming for all anomalous complete sentences (with their corresponding repetition parts) and the correct, complete filler sentences (without any preceding repetition part) in each block. This figure is adopted from Huang et al.¹⁴ with permission. Please click here to view a larger version of this figure.

3. Experiment preparation and electrophysiological recording

1. Recruit native speakers with normal or corrected-to-normal vision and with the same handedness (the Edinburgh handedness test can be used to select the participants¹³). The participants should be free of neurological or psychiatric disorders, and should not be taking any medications affecting the central nervous system.
2. Ensure that participants have no perm or hair dyeing history within two months before the test. Ensure a balanced number of male and female participants in the required age range.
3. Inform participants to have enough sleep and rest before the experiment. Ensure they are in a healthy state (with no cough, fever, headache, etc.) while participating in the experiment.
4. After a participant arrives at the lab, introduce to the participant the equipment and materials that will be used in the experiment, the procedure of the experiment, the tasks and time cost of the experiment, and also the requirements (such as movement and eye blink restriction), in order to help the participant to have basic knowledge about the overall procedure and to eliminate unnecessary concerns.
5. Ask the participant to fill in the Edinburgh Handedness Query Form if the participant has no further questions or worries about the experiment and agrees to participate in the experiment. Confirm that all participants have the same handedness.
6. Give the informed consent form to the participant and ask the participant to read it carefully. Provide necessary explanations if the participant has questions about the content. After the participant understands and agrees to all the contents, ask the participant to sign the form at the designated place, and proceed to the next step.
7. Instruct the participant to clean their head and dry hair properly in the laboratory. While waiting for the participant, keep all the experimental materials ready.
   NOTE: Preparation of experimental materials depends on requirements of the specific recording system.
8. Invite the participant to sit comfortably in the chair of the chamber where the experiment will be conducted.
9. Clean the participant’s skin for corresponding electrode(s) on the forehead (e.g., for the electrodes Fpz, Fp1, Fp2, Af7, Af8), under the left eye (e.g., for the vertical electrooculography [VEOG] electrode), near the outer canthus of the right eye (e.g., for the horizontal electrooculography [HEOG] electrode), and around the left and right mastoid bone (e.g., for the electrodes of Tp9 and Tp10, which will be used as new references offline) with facial scrub and cotton swab.
   1. Scrub the skin gently and do not make the participant feel uncomfortable, but also ensure that the dead skin cells and other substances such as cosmetics are removed.
      NOTE: The distribution of the electrodes might be different depending on the caps used. Please follow the distribution of the electrodes on the cap in use or the guidance of the international 10-20 system of electrode placement to find corresponding locations in the abovementioned areas for skin cleaning.
10. Place the elastic cap with electrodes on the participant's head. Ensure that the electrode Cz on the cap is located at the vertex of the head and the cap has a left-right symmetry, with midline electrodes placed over the midline of the head.
    NOTE: If the electrodes are not fixed on a cap, follow the international 10-20 system of electrode placement on the scalp of the participant
11. Have the electrode(s) recording eye movements placed well and fixed. Fill the two separate HEOG and VEOG electrodes with conductive gel, place them at the outer canthus of the right eye and below the left eye, respectively and fix using adhesive tape.
    NOTE: If the system does not require conductive gel, skip the gel-filling step.
12. Fasten the strap under the chin to prevent the adjacent electrodes from moving during the experiment. The strap should not be too tight or too loose. Connect the cap/electrodes to the recording system.
13. Reduce the impedance of all electrodes to a level below the widely applied impedance threshold for ERP studies (typically below 5 kΩ or 10 kΩ). This can be done as follows.
    1. Switch the recording software to the impedance monitoring interface.
    2. Start with the Ground (GND) and Reference (Ref) electrodes.
    3. Pass the blunt tip needle of the syringe containing conductive gel through the eyelets of the electrodes and hairs until it reaches the skin of the scalp, push the syringe plunger to inject a small dose of the conducting gel onto the skin until the gel connects the metal circle on the electrode and the skin.
       NOTE: Be careful not to inject too much gel in case that superfluous gel connects adjacent electrodes); if the system does not require conductive gel, skip this step, but act according to the requirements of the system.
    4. Check the real-time impedance level displayed in different colors on the monitor until the impedance decreases to below the threshold value.
       NOTE: If the impedance decreases very slowly, the process can be sped up by gently pressing the gel with a cotton swab (or the blunt needle tip) so that the gel can have better contact with the skin or by making circles with the cotton swab (or the syringe) without moving its tip on the scalp.
    5. After the Ground and Reference electrodes are prepared, treat the other electrodes with the same method.
       NOTE: Keep in mind that the impedance reduction process should never make the participant feel uncomfortable. It is helpful to start and maintain a conversation with the participant while preparing the electrodes.
14. Ask the participant to get ready for the experiment. Tell the participant to relax but also try to avoid excessive eye blinks and body movements during the experiment.
15. Inform the participant about the tasks in the repetition part (i.e., to judge the plausibility of the core structure upon seeing the cue “..” following each core structure) and tasks in the main sentence part (i.e., to judge the plausibility of the complete anomalous sentence upon seeing the cue “? ?” following each sentence), and ask the participant to respond accordingly in the experiment.
16. Before the experiment starts, inform the participant explicitly that an incorrect sentence will be followed by a repetition part and a correct sentence by another complete sentence (which may be correct or incorrect).
    NOTE: This helps to direct participants’ attention to the relationship between the repetition part and the sentence before it rather than the subsequent complete sentence.
17. Start the stimuli presentation program and let the participant practice with the practicing section.
    NOTE: Ensure that all types of stimuli included in the experiment appear in this practice session.
18. Start the formal experiment session and record the electrophysiological signals. Divide all the stimuli into different blocks so that the participant can take a break for about 10 min between each block.
19. Monitor the recording system during recording. If any problem arises, pause the experiment and recording until the problem is solved properly.
20. Save the recorded electrophysiological data and help the participant take off the cap. Instruct the participant to wash away the conductive gel on the skin and hair.
21. Provide the participant with the reward and thank the participant for their participation.

Results

The present protocol was used in one of our recent studies to investigate whether the N400 effect reflects semantic integration processing¹⁴. The stimuli used in that study were in Chinese, as shown in Table 2.

(a) Example of the simplified core structures in the repetition part
	Im...

Discussion

Experimental results and significance
In the repetition part, the results demonstrated that the N400 effect became smaller and smaller until almost non-existent. The greatly attenuated N400 effect proved that multiple repetitions did significantly modulate the amplitude of N400. However, the results in this part cannot show whether N400 was actually affected by the change of expectancy or semantic integration. The attenuated N400 effect can still be explained differently. One explanation is that ex...

Materials

Name	Company	Catalog Number	Comments
BrainAmp DC amplifier system (Brain Products GmbH)	Brain Products, Gilching, Germany		BrainAmp S/N AMP13061964DC Input 5.6DC=150mA Operation 7mA Standby
Easycap (Brain Products GmbH)	Brain Products, Gilching, Germany		62 Ag/AgCl electrodes with a configuration of the international 10–20 system of electrode