Experimental Paradigm for Measuring the Effect of Induced Emotion on Grammar Learning

Summary

Here, we present a protocol to measure the effect of positive induced emotion on grammar learning in foreign language learners using a semi-artificial language that integrates the grammatical rules of a foreign language with the lexicon of the learners' native language.

Abstract

Previous studies have found that emotion has significant influence on the learning of foreign language vocabulary and textual comprehension. However, little attention has been given to the effect of induced emotion on grammar learning. This research examined the influence of positive induced emotion on the learning of Japanese grammatical rules among a group of learners with Chinese as their native language, using a semi-artificial language (i.e. Chipanese), which combines the grammatical rules of Japanese and the vocabulary of Chinese. Music was used to invoke positive emotional conditions in participants. Participants were required to learn Chipanese sentences in a training session through practice and then a grammaticality judgment task was administered to measure learning outcomes. We found that participants in positive emotional states performed less accurately and efficiently than those in the control group. The findings suggest that the protocol is effective in identifying the effect of positive induced emotion on grammar learning. The implications of this experimental paradigm for investigating foreign language learning are discussed.

Introduction

Emotions play a crucial role in various cognitive activities such as perception, learning, reasoning, memorization, and problem solving. Because language learning requires attention, reasoning, and memorization, emotions may have significant influence on language learning outcomes¹. Several prior studies have explored the effect of induced emotions on word production or text comprehension^2,3, and have consistently found that emotion had a crucial influence on the two language processes. For example, Egidi and Caramazza found that positive emotion increased the sensitivity to inconsistency in text comprehension in the brain areas specific for inconsistency detection, whereas negative emotion increased the sensitivity to inconsistency in less specific areas². Hinojosa et al. examined the effect of induced mood on word production and discovered that negative mood impaired the retrieval of phonological information during word production³. In spite of the evidence showing that emotions have a marked impact on text comprehension and word production, it is still not clear whether emotions affect grammar learning, one of the essential aspects of language learning. The present study aimed to explore the effect of learners' emotional states on grammar learning.

Language and emotion are two primary components of human experience⁴. Their relationships have mostly been explored by studies in affective neurolinguistics. At the single word level, previous studies have consistently found that emotional features, such as arousal or valence, significantly affect the processing of individual words^5,6,7. Specifically, some studies have identified a significant advantage for positive words⁵, and other studies have found an advantage for both positive and negative words⁷. Although some studies have reported an interaction between valence and arousal, a lack of significant interaction was reported in other research⁴. The picture is more complex at the level of sentence processing. Previous studies have explored issues concerning the interaction between emotional content and syntactic or semantic unification processes during sentence comprehension. Emotional information has been found to exert different influences on the processing of gender or number features⁴. Further, positive and negative emotion was connected with different agreement effects⁴. For instance, positive emotional features facilitated number agreement processing, whereas negative emotional features inhibited these processes⁴. At the semantic level, emotional features influenced semantic unification processes in both sentence and discourse contexts through the activation of the brain regions involved in single word processing and combinatorial semantic processes⁴. A review of the previous literature indicates that most prior research has focused on the effects of emotional information on the comprehension of words, sentences, and texts^8,9, or the neural basis of emotional effects on language production^10,11. However, how individuals' affective states might influence language processing or learning has been largely overlooked.

The most frequently used approach to the studies of emotions in grammar learning is the artificial grammar learning paradigm. Several studies have used artificial grammar tasks to examine the effect of emotion on the learning of a new language¹². First introduced by Reber in 1967¹³, the artificial grammar learning paradigm is characterized by the use of non-meaningful materials, such as number strings or non-word letter strings, which are in fact generated by an underlying grammar. Researchers usually exposed participants in different emotional states (positive, neutral, or negative) to the number strings or letter strings presented either visually or audibly and measured their learning outcomes. Studies with the artificial grammar approach typically consist of a training session and a testing session. In the training session, participants are instructed to observe or memorize a list of symbol sequences that are generated from a finite state grammar. Participants are informed that the sequences follow a particular set of rules, but they are not given any details regarding these rules. In the testing session, participants are presented with new symbol sequences, some of which are grammatical and others are not. They are then required to judge whether the strings are grammatical or not. Artificial grammar tasks allow the instantiation of various theories of learning, such as rules, similarity, and associative learning theories¹⁴. This approach can effectively minimize the influence of lexical factors on the learning of grammatical rules, as artificial languages are made up of numbers, letters, or other meaningless symbols, rather than words in natural languages. However, many researchers have argued that the knowledge acquired in artificial grammar learning may represent statistical properties that are different from the features of natural grammar used by human beings¹⁵. Evidence from neurological studies shows that the grammars in natural languages are processed differently from the finite-state grammars used in artificial grammar learning tasks^16,17. Therefore, artificial grammar learning tasks may not reflect the learning of human languages. Studies of the emotion effect on grammar learning using artificial grammars are more likely to reveal how emotion influences statistical learning, rather than the learning of natural grammars in human languages. It is not entirely clear whether findings from the meaningless stimuli can be generalized to foreign language learning.

The present study intended to adopt a semi-artificial language paradigm to investigate the effect of emotion on grammar learning. Semi-artificial language tasks were first introduced by Williams and Kuribara to examine language learning. A semi-artificial language is generated with the combination of lexicon in the learners' native language and the grammar of a different language. An example of such language can be found in Williams and Kuribara's study¹⁸. Williams and Kuribara designed a novel semi-artificial language, Japlish, which followed the word order and case-marking rules of Japanese but used English vocabulary¹⁸. Sample Japlish sentences in their study are provided in Table 1.

Structure	Examples
SV	Horse-ga when fell?
SOV	Pilot-ga that runway-o saw
SIOV	Student-ga dog-ni what-o offered?
S when what-o V?	Bill-ga when what-o sang?
S who-ni what-o V?	That doctor-ga who-ni what-o showed?
S [SOV]V	John-ga angrily Mary-ga that ring-o lost that said.
OS[SV]V	That disease-o vet-ga cow-ga have that declared.

Table 1: Sample sentences in a semi-artificial language. The sentences were generated with English lexis and Japanese syntax. The sentences in the table are from Williams and Kuribara's study¹⁸.

As shown in the table, although English words are used, they are combined into sentences in accordance with the Japanese word order and case-marking rules. The Japlish sentences are all verb-final and nouns are case-marked for subject (-ga), indirect object (-ni), or object (-o). A detailed description of Japlish can be found in Grey et al.'s study¹⁹. Semi-artificial language tasks involve a training phase and a testing phase. During the training phase, participants are instructed to learn a new language, and in the testing phase, they are required to perform acceptability judgment tasks or sentence-picture matching tasks. The accuracy and reaction times (RTs) of their responses are recorded to assess their learning performance.

Semi-artificial language tasks have mainly three advantages: First, as semi-artificial languages are created using grammatical rules in a new language, the tasks can minimize the influence of prior knowledge of the structures as well as language transfer¹⁹. Second, the tasks enable us to control and manipulate the type and amount of exposure participants receive¹⁹. In this way, they allow for more accurate assessment of the learning effects. Finally, as the grammars used in semi-artificial language tasks are from human languages, the tasks allow us to measure how participants acquire natural grammars, rather than artificial ones. In this aspect, they are more advantageous than artificial grammar tasks in which sequences of numbers or letters are used instead of real words. The use of natural grammar makes us more confident to conclude that the findings obtained are applicable to natural language learning. Given that prior studies have demonstrated learning effects using the semi-artificial language paradigm^20,21,22, it is a useful approach to investigating issues in language learning that are difficult to isolate in the complex context of natural language research. However, semi-artificial language tasks are only applicable to foreign languages that are structurally different from learners' native languages. If the tested language is structurally similar to the learners' native language, it might make the former indistinguishable from the latter.

Compared with the tasks using natural languages, semi-artificial language tasks allow for a more objective assessment of the effects of emotion on grammar learning. This is because words in natural languages are closely associated with specific grammatical functions. For example, inanimate nouns (e.g., desk, nail) are more likely to function as the patients of verbs. Thus, it is difficult to differentiate the performance of vocabulary learning from that of grammar learning because the two are interrelated and inseparable in natural languages. As emotions have been found to have vital influence on word processing^23,24, they may have indirect influence on grammar learning. Therefore, it is not easy to clearly differentiate the effect of emotion on vocabulary learning from that on grammar learning. This problem can be easily solved in semi-artificial language tasks because these tasks allow for the separation of vocabulary from grammar, and thus enable us to identify the effect of emotion on grammar learning, without having to worry about the interference from lexical learning.

Although the semi-artificial language paradigm has been used in some studies to investigate linguistic knowledge in second language acquisition^25,26, this approach has rarely been used to explore learners' individual differences in emotional conditions in foreign language learning. In this study, we intended to explore how positive induced emotion influences grammar learning using a semi-artificial language. Findings from this study have important implications for foreign language teaching and learning.

Protocol

The experiment was approved by the Ethics Committee of Beijing Foreign Studies University and it complied with the guideline for experiments with human subjects. All subjects in this research provided written informed consent.

1. Stimuli construction

1. Design experimental stimuli based on the specific research questions. As this study is intended to examine foreign language learning using a semi-artificial language, create the experimental sentences by rearranging sentences in the participants' native language in accordance with the grammatical rules of the tested foreign language. Sample experimental sentences are provided in Table 2.
   NOTE: As our study was intended to investigate the learning of Japanese grammar by Chinese learners, a semi-artificial language (Chipanese) was generated with the vocabulary of Chinese and the syntax of Japanese. The experimental sentences were adapted from the experimental stimuli used in Liu, Xu, and Wang²⁷.

Table 2: Sample experimental sentences used in this study. Sentence (a) is a Chinese sentence and (b) is its Japanese equivalence. Sentence (c) is the experimental stimuli generated by rearranging sentence (a) in accordance with the syntactic structure of (b). This semi-artificial language was first designed by Liu, Xu, and Wang²⁷.

2. Select representative grammatical structures in the tested language. Make sure to include a variety of grammatical structures to maintain the grammatical complexity and diversity of a natural language.
   NOTE: Four grammatical structures were tested in our study, including two simple grammatical structures (SOV, SIOV) and two complex ones ([SOV]SV, [OSV]SV). The lexical frequency of nouns and verbs was matched between different types of sentences.
3. For the training phase, design at least 20 sentences for each grammatical structure in the tested language for the grammaticality judgment task. The grammatical and ungrammatical sentences should be counterbalanced and controlled for lexical variables such as lexical frequency and number of strokes.
4. For the testing phase, design at least 12 sentences for each grammatical structure in the tested language. Make half of the sentences grammatical and the other half ungrammatical. The grammatical and ungrammatical sentences should be counterbalanced and controlled for lexical variables.
5. Randomize the sentences and design the experiment.
   1. Randomize the sentences before presenting them to the participants in both the training phase and the testing phase. Find the 'Selection' tab on the Property Page in the stimulus presentation software and set the selection method to 'Random'.
   2. Divide the experiment into blocks with each lasting for no more than 10 min.

2. Participant recruitment and preparation for the experiment

1. Recruit participants who have no reading difficulties and no background in the foreign language that will be tested. Make sure participants have normal or corrected-to-normal vision.
2. Randomly assign the participants into two groups (an experimental group and a control group), with each group containing at least 30 members. Ensure the two groups do not differ in years of education or gender ratio.
3. Inform the participants that they should be free from exhaustion, hunger, illness, or other conditions that make them uncomfortable on the day of the experiment.
4. Invite the participants individually or in groups to the laboratory.

3. Procedure

1. Take the participants to the laboratory room and instruct them to sit down at computer workplaces.
2. Ask the participants to read and sign the written informed consent forms.
3. Have the participants complete emotion induction.
   1. Give the participants the pencil-and-paper version of the Self-Assessment Manikin (SAM) pictorial rating scale^28,29. Ask participants to rate their emotion by using pencils to mark the corresponding manikin.
      NOTE: The Self-Assessment Manikin scale is a non-verbal assessment tool to measure the three dimensions of emotion (i.e., valence, arousal, and dominance). The scale we administered was a nine-point scale that ranges from one to nine in each of the three dimensions.
   2. Instruct participants in the experimental group to listen to the positively-valenced music through headphones for 10 min.
      NOTE: In this study, Good Time was used to induce positive emotion. A pilot study was implemented in a group of 20 participants to test the validity of the music in emotional induction, and results indicated that participants rated their emotion as significantly more positive after the induction, which showed that the music was effective in placing participants in the positive affective state. Participants in the control group were not exposed to any emotionally-loaded materials. Rather, they proceeded directly to the learning task.
   3. Give participants the pencil-and-paper version of the Self-Assessment Manikin scale^28,29 and ask them to rate their emotional states.
4. Ask participants to perform the training task.
   1. Present the following written instructions on the computer screens to inform participants that they will learn a language containing the vocabulary in their native language and a new grammar: "Welcome to our experiment! In this experiment, you will be required to learn a new language that contains Chinese words and a new grammar. When you are ready, press any key to proceed to the experiment."
   2. Present the following written instructions on the computer screens to instruct participants to perform the learning task: "Next, you will see some sentences on the screen. Please observe them carefully and decide whether they are grammatical or not. Press '1' if the sentences are grammatical, and '0' if they are ungrammatical. After each response, you will receive a feedback reply ('CORRECT!' or 'INCORRECT!') and the correct sentence will be presented. Press any key to proceed to the learning task."
   3. Have the participants judge the grammatical acceptability of the sentences on the computer screen with a press of the button ('1' for grammatical and '0' for ungrammatical) and present a reply ('CORRECT!' or 'INCORRECT!') on the screen after each response. Present the correct structures after the replies to reinforce the effect of learning.
   4. When all trials are completed, present the following written instructions on the computer screens to inform participants that the training task is over: "The learning task is completed!"
5. Give participants the pencil-and-paper version of the Self-Assessment Manikin scale^28,29 and ask them to rate their emotional conditions.
6. Ask participants to perform the grammaticality judgment task for the testing session.
   1. Present the following written instructions on the computer screens: "Please continue to answer similar questions. Please decide whether the grammatical structures of the sentences are correct. Press '1' if you think they are correct and press '0' if you think they are incorrect. You will be given seven seconds to respond. If you fail to respond within the time limit, the next question starts automatically. Press any key to start the experiment."
   2. Have participants read the sentences shown on the screen and judge the grammaticality of the sentences with a button press ("1" for grammatical and "0" for ungrammatical).
   3. When all trials are completed, present the following written instruction on the computer screens to inform participants that the task is over: "The end. Thank you for your participation!"
7. Ask the participants to fill in a questionnaire about their demographic details. Prepare their payment during the questionnaire.
8. Provide the monetary compensation or reward to participants.

4. Data analysis

1. Collect data from the output files from the stimulus presentation software.
2. Perform an ANOVA analysis with emotion (positive, control) and sentence type (SOV, SIOV, OSVSV, SOVSV) as the independent variables and the average accuracy of learning as the dependent variable to explore the effect of emotion on accuracy.
3. Perform an ANOVA analysis with emotion (positive, control) and sentence type (SOV, SIOV, OSVSV, SOVSV) as the independent variables and the average RTs as the dependent variable to explore the effect of emotion on RTs.

Results

The aim of this study is to explore the effect of positive induced emotion on foreign language grammar learning. For this purpose, two groups of participants were recruited to participate in the experiment, including a positive-emotion group (15 female, M_age = 20.20, age range: 18–27) and a control group (16 female, M_age = 20.33, age range: 18–26). Each group consisted of 30 participants. The two groups did not differ significantly in age, t (58) = -0.215, ...

Discussion

The results indicate that participants rated their emotions to be significantly more positive after being exposed to the positively-valenced music. These subjects were significantly happier than the control group. This suggests that our emotion manipulation was successful. Participants in the positive-emotion group were found to be significantly less accurate and efficient than those in the control group. One possible reason is that participants employed an inductive strategy in grammar learning, resulting in a strong re...

Materials

Name	Company	Catalog Number	Comments
E-prime	PST	2.0.8.22	Stimulus presentation software
Computer	N/A	N/A	Used to present stimuli and record subjects' responses
Self-Assessment Manikin (SAM)	N/A	N/A	Used to assess subjects' affective states. From Lang (1980)29