Interaction between Phonological and Semantic Processes in Visual Word Recognition using Electrophysiology

Summary

We present a protocol to explore the relative activation sequence of phonology and semantics in visual word recognition. The results show that consistent with interactive accounts, semantic and phonological representations may be processed interactively, and higher-level linguistic representations may affect early processing.

Abstract

Controversies have always existed in research related to reading abilities; on whether printed words are perceived in a feedforward manner based on orthographic information after which, other representations, such as phonology and semantics are activated, or whether these are fully interactive and high-level semantic information affects early processing. An interference paradigm was implemented in the presented protocol of phonological and semantic judgment tasks that utilized the same precede-target pairs to explore the relative order of phonological and semantic activation. The high- and low-frequency target words were preceded with three conditions: semantically related, phonological-related (homophones), or unrelated. The results showed that the induced P200 component of low-frequency word pairs was significantly greater than high-frequency words in both the semantic and phonological tasks. In addition, both the homophones in the semantic task and the semantically related pairs in the phonological task caused reduction in N400 when compared to the the control condition, word frequency-independently. It is worth noting that for the low-frequency pairs in the phonological judgment task, the P200 released by the semantically related word pairs was significantly larger than that in the control condition. Overall, semantic processing in phonological tasks and phonological processing in semantic tasks were found in both high- and low-frequency words, suggesting that the interaction between semantics and phonology may operate in a task-independent manner. However, the specific time this interaction occurred may have been affected by the task and frequency.

Introduction

The critical issue in any word recognition model is understanding the role of phonology in the process of semantic access¹. For alphabetic languages, many studies consistently view phonology as playing an important role in semantic access, including English^2,3,4, Hebrew⁵, French⁶, and Spanish⁷. In other words, written word recognition involves not only orthographic but also phonological and semantic processing. This observation in the interactive connectionist model is explained by extensions activated throughout the network, where orthography is associated with phonological and semantic representations through weighted connections⁸. This proliferation of activation provides the core mechanism for the visual word recognition model, which assumes that phonological and semantic representations are automatically activated in response to orthographic input⁹.

However, current empirical evidence supporting the hypothesis of interactive automation remains controversial. Some studies claim that the activation of phonological and semantic representations can be adjusted or prevented by task demands or attention, which implies a certain top-down influence on the high-level processes involved in word perception^10,11. However, the aforementioned description has been questioned by many findings that report phonological and semantic effects in visual word recognition even though these representations are completely irrelevant to the task or cannot be directly accessed¹², thereby supporting the view that semantics and phonology may be accessed automatically and forcibly during the reading process¹³. Therefore, there is uncertainty on whether the phonological and semantic activation in visual word recognition depends on the specific task or whether it occurs forcibly and automatically in a task-independent manner.

The answer to aforementioned question is difficult for Chinese readers. Compared to English, Chinese is a logographic script whose characters represent morphemes instead of phonemes¹⁴. At present, the role of phonology for semantic access to Chinese words remains controversial. Some studies have claimed that phonology plays an important role in semantic access to Chinese words^15,16,17. Others, however, have held the opposite view^18,19. After evaluating the aforementioned research for Chinese phonological processing, we found that the experimental paradigm and specific research methods differ. On the whole, it was mainly divided into two paradigms: word priming^15,18,19 and violation paradigm in the sentence^17,20,21. The target word is usually embedded at the end of the sentence in the violation paradigm²². In terms of language mechanism, a short two-word phrase is a more manageable unit than a complete sentence that is difficult to process²³. In addition, variables that are difficult to control in a sentence, such as syntax, context, or other factors, may lead to different conclusions²⁴. The word priming paradigm is a method commonly used to explore word recognition models, whether in alphabetic languages or Chinese. The task of this paradigm is to judge whether the target word preceded by the primes is a real word or pseudoword; that is, this paradigm usually contains only one lexical task. However, a single lexical decision task may not be the best choice to solve the problem of whether the activation of phonology and semantics depends on the task. Therefore, two different tasks may be more suitable for exploring this question.

Therefore, this research aimed to explore the role of phonology in Chinese word recognition and simultaneously attempt to determine whether the activation of phonology and semantics is in task independent. Our research includes two tasks using the interference paradigm: semantic judgment and phonological judgment. To the best of our knowledge, this is the first event-related potential (ERP) study of Chinese two-character compound recognition using this interference paradigm, and this method rarely appears in studies of alphabetic languages. Specifically, in the semantic judgment task, participants must judge whether the target word and its precedent are semantically related, while in the phonological task, they must judge whether the paired words have the same pronunciation.

The former is a semantic matching task that does not require a priori phonological processing, and the latter is a phonological judgment task that does not require a priori semantic processing. Therefore, we compared homophone pairs and unrelated control groups in the semantic judgment task to reveal whether and how phonology affects semantic processing. Similarly, we compared semantically related word pairs with unrelated control conditions in the phonological judgment task to reveal whether and how semantics affect phonological processing. In addition, the aforementioned problem was verified in high- and low-frequency words. Thus, this complementary semantic and phonological judgment task can not only reveal the importance of phonological processing in Chinese word recognition but also reveal whether and how phonology and semantics interact.

If the phonology and semantic processes are early, automatic, and interactive, the effect of phonological and semantic activation should be observed in the response time of the two tasks. For ERP, phonological and semantic processes trigger two different electrophysiological markers^2,7. In addition, their time courses and their spatial distributions should be different. An early positive component (P200) should reflect phonological processing, and the typical semantic processing marker N400 should also be identified^20,21. We assumed that both the phonologically related pairs in the semantic task and the semantic-related pairs in the phonological task would cause a significant decrease in N400, which would have indicated that phonological processing may lead to some degree of activation at lexical-semantic levels. In addition, we monitored whether the P200, which characterizes phonological processing, appeared in the semantic judgment task or the phonological judgment task. In the phonological judgment task, semantic-related conditions trigger the P200, which can be seen as evidence of the early influence of semantics on phonological processing.

Protocol

The protocol used for this study was approved by the Institutional Review Board of Tsinghua University.

1. Stimuli construction and presentation

1. Stimuli construction
   1. Stimuli preparation: Prepare target words containing approximately 140 Chinese two-character compounds, of which low- and high-frequency words account for half. Precede each target by three analogues: a phonologically identical word (homonymous word), a word with related meaning, and an irrelevant control word.
      1. Ensure that high-frequency targets are always preceded by high-frequency antecedents and low-frequency targets are always preceded by low-frequency precedents, whether in related conditions or unrelated control groups. In addition, ensure the precede-target pairs are similar in the number of strokes and frequency.
         NOTE: All the words for this study were selected from the Modern Chinese Frequency Dictionary (Xiandai Hanyu Pinlu Cidian). The frequency of low-frequency words was less than eight times per million, and the frequency of high-frequency words exceeded 800 times per million.
   2. Stimuli assessment: Recruit a separate group of approximately 30 students to evaluate the degree of semantic relevance between word pairs on a seven-point scale, wherein 1 reflects the lowest correlation and 7 reflects the highest correlation.
   3. Final stimulus determination: Delete inappropriate word pairs, such as word pairs with lower scores in semantically related conditions and word pairs with higher scores in homophone and irrelevant condition scores.
      1. Calculate the respective average scores of high- and low-frequency semantic-related word pairs and ensure no significant difference is present between the two.
      2. In addition, ensure that the scores of homophone pairs and unrelated pairs are not significantly different at both high and low frequency. Finally, determine the final experimental stimulus material (see Table Materials).
         ​NOTE: For the semantic-related values of the semantic-related pairs in this experiment, the final average values were 5.62 and 5.73 for high-frequency and low-frequency pairs, respectively, and there was no significant difference between the two (p > .1). In addition, the semantic relatedness between the homophonic and unrelated pairs was not significantly different (p > .1).
2. Stimuli presentation
   1. Build a program to show the task to the subjects and fill in the aforementioned materials (the program can be written in E-prime or other programming languages).
   2. Ensure that each core structure of the program starts with a screen displaying a " + " sign that lasts for 300 ms, directly after which the preceding word must appear for 140 ms, with no interval between the two.
   3. After that, set a blank screen lasting for 360 ms, and then set the target word, which will appear for 500 ms. Finally, set a question mark (?) that will continue to be displayed until the participant has decided on the word pair just shown and pressed the button as quickly and accurately as possible.
   4. Tell the participants in advance that they need to judge whether the word pair is semantically related in the semantic judgment task and whether the phonology is the same in the phonological judgment task.
   5. Practice session setup: Set up two practice groups to include semantic judgment and phonological judgment tasks, respectively, with no less than 10-word pairs for each task. Inform the participants that they can repeat the exercises to ensure that the accuracy in the practice session is more than 70%.
   6. Formal experiment setup: Divide the whole experiment into 6 blocks, with the semantic judgment task and the phonological judgment task each accounting for half.
      1. Make sure that there are no repeated target words in each block, and that the number of priming types in each block is the same. In addition, set up a few filler trials to reduce the response deviation caused by the unequal number of tests requiring positive or negative reactions.
      2. Randomize the order of items in each block and counterbalance the block order among the subjects.
         ​NOTE: The entire experiment can also be divided into eight or ten or more blocks according to the number of experimental materials to be prepared, which minimizes the repetition of target words in each block.

2. Experiment preparation and electrophysiological recording

1. Recruit right-handed native Chinese speakers with a normal vision that may have been previously corrected.
   1. Exclude participants with any neurological or psychiatric diseases.
   2. Ensure there is a balanced number of female and male participants in the desired age range (18-28 years old).
   3. Ensure that the participants do not have any history of perming or dyeing their hair for the past two months.
   4. Inform the participants that they will need to have sufficient sleep and rest time before the experiment²⁵.
   5. When participating in the experiment, please make sure the participants are in a healthy state at the time of conducting the experiment.
2. When a participant arrives at the lab, introduce the experimental equipment, tasks, and time costs. Explain the requirements (such as not being sleepy, moving, and blinking) to help them understand the entire process and eliminate unnecessary worries.
3. If the participant has no other questions about the experiment, ask them to fill in the Edinburgh Handedness Query Form, which is used to confirm that all participants have the same right-handed habits.
4. Provide the informed consent form to the participants and ask them to read and sign carefully. If participants have questions about the content of the consent form, provide them with the necessary explanations.
5. Instruct the participant to properly clean their scalp and dry their hair in the laboratory. While waiting for participants, please prepare all experimental materials.
   NOTE: The electroencephalogram (EEG) signal is amplified using an amplifier system with a bandpass of 0.01 to 100 Hz and continuously sample at 500 Hz.
6. Invite the participants to sit comfortably on a chair in the chamber where the experiment will be conducted. Instructed them not to move the chair.
7. Use cotton swabs and facial scrubs to clean the skin under the participant's left eye (for the vertical electro-oculographic electrode), near the outer canthus of the right eye (for the horizontal electro-oculographic), and around the right and left mastoid bones (for Tp9 and Tp10, which will be used as new offline references).
   NOTE: The distribution of electrodes may vary depending on the caps used.
8. Place the elastic cap on the participant's head and make sure that the Cz electrode is at the center of the top of the head. Fix the electrode cap strap under the chin with care to ensure that it is not too tight or too loose.
9. Make sure that the cap and amplifier are connected to the recording system. Next, switch the recording software to the impedance monitoring interface.
10. Ensure that the impedance of all electrodes does not exceed 5 kΩ or 10 kΩ, starting with the reference (Ref) and ground (Gnd) electrodes.
11. Pass the syringe filled with conductive gel through the small hole of an electrode to the scalp and then push the plunger to inject a small amount of conductive gel into the scalp while being careful not to cause an overflow. At the same time, monitor the display system that displays the impedance in real-time until the impedance drops to the threshold.
12. After the Ref and Gnd electrodes are prepared, reduce the impedance of the other electrodes in the same way. Treat the impedance reduction of ocular electricity carefully.
    1. Tape the small holes on one side of the two electro-oculographic electrodes to prevent the injected conductive gel from leaking. Fix them to the bottom of the left eye and the outer canthus of the right eye with tape.
13. After all the electrodes are prepared, instruct the participants to be ready for the experiment. Instruct the participants to relax and avoid excessive eye blinking and body movement during the experiment.
14. Present the stimulus via the stimulus demonstration program and let the participants practice in the practice section.
    NOTE: After the practice session, participants can ask questions if they have any doubts or questions about how to proceed.
15. Start the formal experiment and record the EEG information. Monitor the recording system during recording. If an electrode is loose or the resistance exceeds the threshold, refill the electrode when the participant is resting.
    NOTE: Participants can rest for 4-10 min after each block.
16. After the experiment is completed, save the EEG signal and turn off the equipment, such as the recording system and amplifier. Then take off the participant's cap and instruct the participant to wash off the conductive gel from the hair and skin. Finally, reward the participants and thank them for their cooperation.

3. EEG preprocessing

1. Utilize semi-automatic ocular correction with independent component analysis.
2. Compute the ERPs from 100 ms to 600 ms after the onset of the target word (100 ms pre-target baseline).
3. Set the EEG bandpass-filtered offline from 0.05 to 30 Hz (zero phase shift mode, 24 dB/oct).
4. Discard epochs exceeding ±80 µV by artifact rejection and eliminate the trials of erroneous responses.

Results

This protocol was used in a recent study to investigate the role of phonology in Chinese two-character compound recognition and to infer the word recognition model²⁶. All stimuli used in this study were fully disclosed²⁶. Three time windows were selected on the basis of global field power (GFP): at 100-150 ms, 160-280 ms, and 300- 500 ms for N1, P200, and N400 components, respectively²⁶. The average amplitudes of the above two time windows were analy...

Discussion

Experimental results and significance:
The purpose of this protocol was to infer the following: 1) whether the word recognition model is a feedforward model or an interactive model and 2) the interaction between the phonological and semantic patterns in Chinese two-character compound recognition of high and low frequency under different tasks. An interference paradigm of phonological and semantic matching task using the ERP technique was adopted. The ERP responses preceded by homophones and unrelat...

Materials

Name	Company	Catalog Number	Comments
BrainAmp DC amplifier system (Brain Products GmbH)	Brain Products, Gilching, Germany		BrainAmp S/N AMP13061964DC Input 5.6DC=150mA Operation 7mA Standby
Easycap (Brain Products GmbH)	Brain Products, Gilching, Germany		62 Ag/AgCl electrodes with a configuration of the international 10–20 system of electrode