Name: investigating the effect of visual imagery and learning shape-audio regularities on bouba and kiki
Uploaded: 2019-09-13T14:30:00
Duration: 7 min 31 s
Description: Watch this Scientific Journal Video about Investigating the Effect of Visual Imagery and Learning Shape-Audio Regularities on Bouba and Kiki at JoVE.com

Funders: 
This work was funded by the European Union&#8217;s Horizon 2020 research and innovation program under the Marie Sklodowska-Curie Actions (grant agreement number 657440) and the Norwegian Association of the Blind and Partially Sighted.&#160; 
 
Film makeup: 
Makeup artist Rachael Parrey.

<ol>
	<li>Ramachandran, V. S., Hubbard, E. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Synaesthesia+-+A+Window+Into+Perception,+Thought+and+Language.">Synaesthesia - A Window Into Perception, Thought and Language.</a> Journal of Consciousness Studies. 8 (12), 3-34 (2001).</li><li>Bremner, A. J., Caparos, S., Davidoff, J., de Fockert, J., Linnell, K. J., Spence, C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Bouba&#8217;&#8217;and+&#8216;&#8216;Kiki&#8217;&#8217;+in+Namibia?+A+remote+culture+make+similar+sound-shape+matches,+but+different+taste-shape+matches+to+Westerners.">Bouba&#8217;&#8217;and &#8216;&#8216;Kiki&#8217;&#8217; in Namibia? A remote culture make similar sound-shape matches, but different taste-shape matches to Westerners.</a> Cognition. 126 (2), 165-172 (2013).</li><li>Davis, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+fitness+of+names+to+drawings.+A+cross-cultural+study+in+Tanganyika.">The fitness of names to drawings. A cross-cultural study in Tanganyika.</a> British Journal of Psychology. 52 (3), 259-268 (1961).</li><li>Maurer, D., Pathman, T., Mondloch, C. J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+shape+of+boubas:+sound-shape+correspondences+in+toddlers+and+adults.">The shape of boubas: sound-shape correspondences in toddlers and adults.</a> Developmental Science. 9 (3), 316-322 (2006).</li><li>Ozturk, O., Krehm, M., Vouloumanos, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Sound-symbolism+in+infancy:+Evidence+for+sound-Shape+cross-modal+correspondences+in+4-months-olds.">Sound-symbolism in infancy: Evidence for sound-Shape cross-modal correspondences in 4-months-olds.</a> Journal of Experimental Child Psychology. 114 (2), 173-186 (2013).</li><li>Bar, M., Neta, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Humans+Prefer+Curved+Visual+Objects.">Humans Prefer Curved Visual Objects.</a> Psychological Science. 17 (8), 645-648 (2006).</li><li>Bertamini, M., Palumbo, L., Gheorghes, T. N., Galatsidas, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Do+observers+like+curvature+or+do+they+dislike+angularity?.">Do observers like curvature or do they dislike angularity?.</a> British Journal of Psychology. 107 (1), 154-178 (2016).</li><li>Quinn, P. C., Brown, C. R., Streppa, M. L. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Perceptual+Organization+of+Complex+Visual+Configurations+by+Young+Infants.">Perceptual Organization of Complex Visual Configurations by Young Infants.</a> Infant Behavior and Development. 20 (1), 35-46 (1997).</li><li>Silvia, P. J., Barona, C. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Do+people+prefer+curved+objects?+Angularity,+expertise,+and+aesthetic+preference.">Do people prefer curved objects? Angularity, expertise, and aesthetic preference.</a> Empirical Studies of the Arts. 27 (1), 25-42 (2009).</li><li>Nielsen, A., Rendall, D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+Sound+of+Round:+Evaluating+the+Sound-Symbolic+Role+of+Consonants+in+the+Classic+Takete-Maluma+Phenomenon.">The Sound of Round: Evaluating the Sound-Symbolic Role of Consonants in the Classic Takete-Maluma Phenomenon.</a> Canadian Journal of Experimental Psychology. 65 (2), 115-124 (2011).</li><li>D&#8217;Onofrio, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Phonetic+Detail+and+Dimensionality+in+Sound-shape+Correspondences:+Refining+the+Bouba-Kiki+Paradigm.">Phonetic Detail and Dimensionality in Sound-shape Correspondences: Refining the Bouba-Kiki Paradigm.</a> Language and Speech. 57 (3), 367-393 (2014).</li><li>Westbury, C., Hollis, G., Sidhu, D. M., Pexman, P. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Weighing+up+the+evidence+for+sound+symbolism:+Distributional+properties+predict+cue+strength.">Weighing up the evidence for sound symbolism: Distributional properties predict cue strength.</a> Journal of Memory and Language. 99, 122-150 (2018).</li><li>Oberman, L. M., Ramachandran, V. S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Preliminary+evidence+for+deficits+in+multisensory+integration+in+autism+spectrum+disorders:+The+mirror+neuron+hypothesis.">Preliminary evidence for deficits in multisensory integration in autism spectrum disorders: The mirror neuron hypothesis.</a> Social Neuroscience. 3 (3-4), 348-355 (2008).</li><li>K&#246;hler, W. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Gestalt psychology. , (1929).</li><li>Monaghan, P., Mattock, K., Walker, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+Role+of+Sound+Symbolism+in+Language+Learning.">The Role of Sound Symbolism in Language Learning.</a> Journal of Experimental Psychology: Learning, Memory, and Cognition. 38 (5), 1152-1164 (2012).</li><li>Fort, M., Martin, A., Peperkamp, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Consonants+are+More+Important+than+Vowels+in+the+Bouba-kiki+Effect.">Consonants are More Important than Vowels in the Bouba-kiki Effect.</a> Language and Speech. 58 (2), 247-266 (2015).</li><li>Graven, T., Desebrock, C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Bouba+or+kiki+with+and+without+vision:+Shape-audio+regularities+and+mental+images.">Bouba or kiki with and without vision: Shape-audio regularities and mental images.</a> Acta Psychologica. 188, 200-212 (2018).</li><li>Fryer, L., Freeman, J., Pring, L. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Touching+words+is+not+enough:+How+visual+experience+influences+haptic-auditory+associations+in+the+&#8220;Bouba-Kiki&#8221;+effect.">Touching words is not enough: How visual experience influences haptic-auditory associations in the &#8220;Bouba-Kiki&#8221; effect.</a> Cognition. 132 (2), 164-173 (2014).</li><li>Fontana, F. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Association+of+Haptic+Trajectories+to+Takete+and+Maluma.">Association of Haptic Trajectories to Takete and Maluma.</a> Lecture Notes in Computer Science. 7989, 60-68 (2013).</li><li>Graven, T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=How+blind+individuals+discriminate+braille+characters:+An+identification+and+comparison+of+three+discrimination+strategies.">How blind individuals discriminate braille characters: An identification and comparison of three discrimination strategies.</a> British Journal of Visual Impairment. 33 (2), 80-95 (2015).</li><li>Graven, T., Bello, F., Kajimoto, H., Visell, Y. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=How+attention+is+allocated+when+using+haptic+touch:+Shape+feature+distinction+and+discrimination+strategy.">How attention is allocated when using haptic touch: Shape feature distinction and discrimination strategy.</a> EuroHaptics 2016, Part 1. , 380-393 (2016).</li><li>Heller, M. A., Calcaterra, J. A., Burson, L. L., Tyler, L. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Tactual+picture+identification+by+blind+and+sighted+people:+Effects+of+providing+categorical+information.">Tactual picture identification by blind and sighted people: Effects of providing categorical information.</a> Perception &amp; Psychophysics. 58 (2), 310-323 (1996).</li><li>Heller, M. A., Gentaz, E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Psychology of Touch and Blindness. , (2014).</li><li>Pathak, K., Pring, L. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Tactual+picture+recognition+in+congenitally+blind+and+sighted+children.">Tactual picture recognition in congenitally blind and sighted children.</a> Applied Cognitive Psychology. 3 (4), 337-350 (1989).</li></ol>

The authors have nothing to disclose.

The presented protocol is appropriate for investigating the effect of visual imagery and learning shape-audio regularities, when they occur and how robust they are; in specific individuals and groups of individuals.&#160;
This protocol would be appropriate for testing not only the accuracy but also the exploration time: From when the experimenter removes their left hand from the participant&#8217;s fists (in the blind and blindfold-group) / the carton plate or foam board (in the vision-group) to when the participant says &#8216;yes.&#8217; In the statistical analyses, the longest exploration time per trial per experimental group would be divided into four and the number of &#8216;fast,&#8217; &#8216;medium&#8217; and &#8216;slow&#8217; participants in each experimental group counted: &#8216;Fast&#8217; using &#8804;25% of this exploration time, &#8216;medium&#8217; using 26-75%, and &#8216;slow&#8217; using &#8805;75%. Across trials, the participants would be counted as in Experiment 2: That is, as &#8216;fast&#8217; if counted as &#8216;fast&#8217; in &#8805;66.6% of all trials, as &#8216;medium&#8217; if counted as &#8216;medium&#8217; in &#8805;66.6% of all trials, and so on. (For &#8216;inconclusive&#8217;, i.e. the participant cannot be counted as &#8216;fast,&#8217; &#8216;medium&#8217; or &#8216;slow&#8217;, cf. Experiment 3.) It would be possible to test the exploration time for correct and incorrect answers separately, depending on the sample size and/or the number of correct and incorrect answers. Whether testing accuracy, exploration time or both, there are critical protocol steps that must be followed: The three experiments should be conducted in fixed order, and their trials presented in fixed order, for all participants. The participants should not be made aware that the test materials in all three experiments include bouba and kiki. Further, the auditory information from the MP3 player should be played only once and not repeated back orally to the participants (all three experiments). In Experiment 1 and Experiment 2, the tactile/visual bouba and kiki should be presented in the same position and orientation. For the vision-group (in all three experiments), the test materials should be set up behind a carton plate/foam board. Finally, in the blindfold-group, the participants should not remove their blindfold until Experiment 3 has been completed. These steps are to ensure that the amount and type of information about the tactile/visual and the auditory bouba and kiki is kept under control across all experimental groups. In addition, it is crucial in Experiment 2 to present the auditory bouba/kiki as soon as the participant starts exploring the tactile/visual bouba/kiki. This will ensure that multimodal (congruous or incongruous) information is presented. Moreover, this protocol would also be appropriate for testing other groups of individuals. To integrate these, such participant groups could be added as separate experimental groups: For example, a color perception-group and a shape perception-group;22 a sudden vision loss-group and a progressive vision loss-group;20,21 and/or specific groups of individuals with a hearing impairment and/or an Autism Spectrum Disorder13 could be added.&#160;
With this protocol, it would be possible to establish a group norm (for accuracy and/or exploration time) for specific groups on each instant and long term effect, including each repeated within-group measure, and/or task; then to test individuals against their group norm: Whether any particular individual or the entire group is in need of further training in noticing the shape-audio regularities. A group norm for each instant and long term effect, including each repeated within-group measure, and/or task would also make it possible to determine what kind of training is needed to notice the shape-audio regularities: For example, more trials with the tactile/visual-auditory bouba/kiki-effect (Experiment 1) and/or more trials with congruous and incongruous tactile/visual shape and audio (Experiment 2). Indeed, it would be possible to increase both the number of trials and the number of repeated within-group measures in each task as well as altering the time between them, thereby allowing more individualized training and testing.&#160;
However, this protocol would need modifying if testing what type of materials (e.g., bouba or kiki, filled or outlined tactile/visual shape, and/or auditory sound or word) best ensures the learning of shape-audio regularities. A possible way of modifying this protocol could involve presenting the different materials in blocks: For example, filled tactile/visual shapes and auditory sound and/or outlined tactile/visual shapes and auditory word (Experiment 1). These blocks could be presented in random order to all the participants in one experimental group; or alternatively, each one of the four blocks to separate experimental groups (Experiment 1). This protocol would also need modifying if testing even further the robustness of the visual imagery and/or learned shape-audio regularities within and across tasks. The modification could involve adding repeated within-group measures with different tactile/visual shape and/or audio than those included in the training; for example, with the auditory sound for &#8216;maa-boo-maa&#8217; and &#8216;tuh-kee-tee&#8217; after training with tactile/visual bouba and kiki shapes and auditory words (Experiment 1)1,3,4,10,11,12,13,14,15,16. Another option could involve presenting a third tactile/visual shape which is not mainly curved/angular together with the tactile/visual bouba and kiki shapes in Experiment 110,11,12,13,14,15,16. An auditory nonsense word and a sound could be created for this tactile/visual shape to produce a third shape-word pair14,15,16 and a third shape-sound pair17 (Experiment 1). The shape-sound pair could then be included in Experiment 2 as catch-trials; or alternatively as regular congruous trials and with corresponding incongruous trials. Finally, the sound (from Experiment 2 and Experiment 1) and/or the nonsense word (from Experiment 1) could be presented in Experiment 3. A third option could involve not conducting Experiment 2 and Experiment 3 at all, but rather including them as tasks in the repeated within-group measures in Experiment 1, in random order.
The presented protocol is unique in that it keeps under control both the visual imagery and the sensory information during training and testing. It is flexible: Large and small samples and specific individuals can be tested against themselves (cf. the within-group testing) and/or other groups of individuals (cf. the between-group testing). It is possible to test both accuracy and exploration time, change the number of trials and repeated within-group measures in each experiment and the time between them, and change or even collapse the order of trials and/or experiments.

95% of the world&#8217;s population shows an auditory-visual bouba/kiki-effect, associating the visually rounded shape with the spoken word, &#8216;bouba;&#8217; and the visually angular shape with the spoken word &#8216;kiki;&#8217; and they do this even when they have not had any experience with either the shape or the word1. The bouba/kiki-effect precedes language learning; it occurs across languages1,2,3,4,5, with both bouba and kiki (cf. people&#8217;s preference for visual curves over visual angles6,7,8,9), and it depends on the combination of vowels and consonants (e.g., it does not occur with &#8216;bibi&#8217; and &#8216;kuku&#8217;.)1,5,10,11,12. This shape-word association occurs with other shape and word pairs as well, as long as they are mainly curved, as bouba, and mainly angular, as kiki11,13; and/or they have the same combination of vowels and consonants as &#8216;bouba&#8217; and &#8216;kiki,&#8217; for example: &#8216;Baluma&#8217; and &#8216;takete,&#8217; &#8216;maluma&#8217; and &#8216;takete,&#8217; &#8216;uloomo&#8217; and &#8216;takete,&#8217; &#8216;maa-boo-maa&#8217; and &#8216;tuh-kee-tee&#8217;1,3,4,10,11,12,14. In fact, people associate the rounded visual shapes with the spoken words containing continuant consonants (such as /l/ and /m/) and open back vowels (such as /&#593;:/, and /&#596;:/), and the visual angular shapes with the spoken words containing plosive consonants (such as /k/ and /t/) and close front vowels (such as (/&#949;/ and /I/)11,12,15,16. They are influenced more by the consonants than the vowels, especially by the voiceless ones (e.g., /k/ and /t/)11,12,15,16. Indeed, it seems the features of the global shapes &#8211; their curves and angles17 &#8211; and the sound of the spoken words &#8211; their melody17 &#8211; are the most crucial, as opposed to the global shapes and words themselves.
One study has investigated the bouba/kiki-effect with tactile bouba and kiki instead of the visual bouba/kiki shapes and found that participants who were visually impaired showed a significantly less robust bouba/kiki-effect (~64%) than the fully sighted (~90%)18. This study argued that the significantly less robust bouba/kiki-effect amongst the participants who were blind and partially sighted was caused by a lack of visual imagery: The fully sighted participants had noticed regularities in their environment that are not easily accessed when vision is impaired18. It is not clear from this study, however, whether visual imagery is necessary for the bouba/kiki-effect to occur: Only six of the 42 participants with a visual impairment were congenitally totally blind18, thus had no visual imagery at all. Furthermore, the other participants, none of them blindfolded, may have observed the experimenter&#8217;s lip movements when announcing the bouba/kiki word &#8211; rounded lip movements when announcing &#8216;bouba&#8217; and angular lip movements when announcing &#8216;kiki&#8217;1,18 &#8211; showing a tactile-visual-auditory bouba/kiki-effect instead of drawing upon any visual imagery. Additionally, any effect of noticing tactile-auditory regularities was not investigated, for example, by comparing the bouba/kiki-effect on trial one and trial four instead of calculating the effect across all four trials. Moreover, the tactile bouba and kiki were rather different across trials: In trials one and two they were 3D and 2D shapes (curved v angular); in trials three and four, identical in shape (circle) and dissimilar in texture (smooth v rough; smooth v spiky)18. In a related vein, one study has investigated a kinesthetic-auditory bouba/kiki-effect in blindfolded (fully sighted) individuals and found that, after a two-minute training period of holding a robotic stylus programmed to draw trajectories of the bouba and kiki shapes, 82% showed the bouba/kiki-effect19. It is not clear from this study, however, whether the bouba/kiki-effect occurred because of the training period: The study did not include a pre-test, nor a control group.&#160;
To this end, we have investigated the role of visual imagery in the bouba/kiki-effect, whether training in noticing tactile/visual-auditory bouba and kiki regularities affected the bouba/kiki-effect and the recognition of individual tactile/visual bouba and kiki shapes, and finally what mental images these shape-audio regularities produced17. To generate bouba/kiki shape-audio regularities &#8211; not merely, for example, kiki-shape/kiki-word particularities1 &#8211; this study included two types of tactile/visual bouba/kiki shapes (filled; outlined) and two types of auditory bouba/kiki [audio (word; non-word sound)]17. Three groups of individuals participated in three experiments: Blind (N = 12), blindfold (N = 12), and vision (N = 12). The experiments were conducted in fixed order across participants and with no break between them, to keep under control the participants&#8217; amount and type of experience with bouba and kiki. In Experiment 1 (pre-test-post-test design with three repeated within-group measures) the participants were asked to pick out the tactile/visual shape they associated with the auditory bouba/kiki; in Experiment 2 (within-subject design), to name one tactile/visual shape and some audio (sometimes congruous; sometimes incongruous) as &#8216;bouba&#8217; or &#8216;kiki;&#8217; and in Experiment 3, (post-test only design) to draw the tactile/visual shape they associated with the auditory bouba/kiki17. Overall, this study suggested that the effect of visual imagery is robust across tasks, but not long term within each task, whereas the effect of learning shape-audio regularities is robust long term within each task, but not across tasks17. This article presents the protocol from this study17. The presented protocol is appropriate for investigating the effect of visual imagery and learning shape-audio regularities, when they occur and how robust they are; in specific individuals and groups of individuals. This protocol is unique in that it keeps under control both the visual imagery and the sensory information during training and testing.&#160; &#160;

<table>
	<tbody>
		<tr>
			<td>Bouba/kiki filled</td>
			<td></td>
			<td></td>
			<td>30 mm horizontal, 23 mm Vertical and 0.5 mm thickness m vertical, and 0.5 mm line height, and 0.5 mm embossed embossed height</td>
		</tr>
		<tr>
			<td>Bouba/kiki outlined</td>
			<td></td>
			<td></td>
			<td>30 mm horizontal, 23 mm Vertical and 0.5 mm thickness m vertical, and 0.5 mm line height, and 0.5 mm embossed embossed height</td>
		</tr>
		<tr>
			<td>Bouba/kiki sound (sine wave)</td>
			<td></td>
			<td></td>
			<td></td>
		</tr>
		<tr>
			<td>Bouba/kiki word (human voice)</td>
			<td></td>
			<td></td>
			<td></td>
		</tr>
		<tr>
			<td>Carton plate/foam board for covering bouba/kiki (white)</td>
			<td>Ryman</td>
			<td>230461860</td>
			<td>297 mm horizontal x 210 vertical 0 mm vertical</td>
		</tr>
		<tr>
			<td>Copy paper [white (80 g/m2 )]</td>
			<td>Ryman</td>
			<td>250030000</td>
			<td>210 mm horizontal x 148 mm vertical</td>
		</tr>
		<tr>
			<td>Foam board for bouba/kiki picture cards [white (5 mm thick)]</td>
			<td>Ryman</td>
			<td>230461860</td>
			<td>50 mm horizontal x 50 mm vertical</td>
		</tr>
		<tr>
			<td>MP3-player (smartphone with the VLC app for iOS installed)</td>
			<td>Carphone Warehouse</td>
			<td></td>
			<td></td>
		</tr>
		<tr>
			<td>Plastic embossing film</td>
			<td>RNIB Shop</td>
			<td>ZM04</td>
			<td>210 mm horizontal x 148 mm vertical</td>
		</tr>
		<tr>
			<td>Rollerball pen [black (1.0 mm tip)]</td>
			<td>Ryman</td>
			<td>827134001</td>
			<td></td>
		</tr>
		<tr>
			<td>Rubber mat</td>
			<td>RNIB Shop</td>
			<td>LC177</td>
			<td>230 mm horizontal x 150 mm vertical</td>
		</tr>
		<tr>
			<td>Saddleback style</td>
			<td>RNIB Shop</td>
			<td>B511</td>
			<td></td>
		</tr>
		<tr>
			<td>Swell paper</td>
			<td>Zychem Ltd</td>
			<td>Zytex2 paper</td>
			<td>50 mm horizontal x 50 mm vertical</td>
		</tr>
	</tbody>
</table>

investigating the effect of visual imagery and learning shape-audio regularities on bouba and kiki

This article presents a protocol for investigating the role of visual imagery in the bouba/kiki-effect, whether training in noticing the bouba/kiki shape-audio regularities affects the bouba/kiki-effect and the recognition of individual bouba and kiki shapes, and finally what mental images these regularities produce. To generate bouba/kiki shape-audio regularities, there were two types of shapes (filled; outlined) and two types of audio (word; non-word sound). Three groups of individuals participated in three experiments: Blind, blindfold, and vision. The experiments were conducted in fixed order across participants, with no break between them. In Experiment 1 (pre-test-post-test design with three repeated within-group measures) the participants were asked to pick out the shape they associated with the auditory bouba/kiki; in Experiment 2 (within-subject design), to name one shape and some audio (sometimes congruous; sometimes incongruous) as &#8216;bouba&#8217; or &#8216;kiki;&#8217; and in Experiment 3 (post-test only design), to draw the shape they associated with the auditory bouba/kiki. The results suggest that the blindfold-group draw upon visual imagery to solve new problems, but not long term; that training in noticing bouba/kiki shape-audio regularities affects the bouba/kiki-effect and the recognition of individual bouba and kiki shapes, but differently in each experimental group; and that all experimental groups create mental images of the most characteristic shape feature of bouba (curve) and kiki (angle). In fact, the effect of visual imagery is robust across tasks, but not long term; the effect of learning shape-audio regularities is robust long term, but not across tasks. The presented protocol is appropriate for investigating the effect of visual imagery and learning shape-audio regularities, when they occur and how robust they are; in specific individuals and groups of individuals. This protocol is unique in that it keeps under control both the visual imagery and the sensory information during training and testing.

The bouba/kiki-effect
Six of the 12 participants who were congenitally blind (50%), nine of the 12 who were blindfolded (75%), and 10 of the 12 who were fully sighted (~83%) showed an instant tactile/visual-auditory bouba/kiki-effect: That is, both the blindfold and vision-group were significantly above the chance level (of 50%): &#967;2(1. N = 12) = 3.00, p = 0.08 and &#967;2(1. N = 12) = 5.33, p = 0.02 (Experiment 1, Trial 1)17. No significant difference was found by Fisher&#8217;s exact test between the three experimental groups (Experiment 1, pre-test)17. (Cf. Table 1.)
When it comes to the long term tactile/visual-auditory bouba/kiki-effect: On the first repeated within-group measure, nine of the participants who were congenitally blind showed the bouba/kiki-effect along with nine who were fully sighted and seven who were blindfolded: The blind and vision-group significantly above the chance level (of 50%): &#967;2(1. N = 12) = 3.00, p = 0.08 and &#967;2(1. N = 12) = 3.00, p = 0.08 (Experiment 1, Trial 4)17. On the second repeated within-group measure, 11 participants in the blind and vision-group showed the tactile/visual-auditory bouba/kiki-effect: Both experimental groups were again significantly above the chance level (of 50%): &#967;2(1. N = 12) = 8.33, p = 0.00 and &#967;2(1. N = 12) = 8.33, p = 0.00; and seven in the blindfold-group (Experiment 1, Trial 5)17. Finally, on the third repeated within-group measure and post-test, nine of the 12 participants who were congenitally blind (75%), six of the 12 who were blindfolded (50%), and all of the 12 participants who were fully sighted (100%) showed the tactile/visual-auditory bouba/kiki-effect; both the blind and vision-group were again significantly above the chance level (of 50%): &#967;2(1. N = 12) = 3.00, p = 0.08 and &#967;2(1. N = 12) = 12.00, p = 0.00 (Experiment 1, Trial 8)17. Fisher&#8217;s exact test found a significant difference between the blindfold and vision-group [p = 0.01. (Experiment 1, post-test)]17. (Cf. Table 1.)
<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td rowspan="2">Participant group</td>
			<td>Instant bouba/kiki- effect</td>
			<td colspan="3">Long term bouba/kiki-effect</td>
		</tr>
		<tr>
			<td>Pre-test</td>
			<td>Repeated measure 1</td>
			<td>Repeated measure 2</td>
			<td>Repeated measure 3/post-test</td>
		</tr>
		<tr>
			<td>Blind</td>
			<td>50.0%</td>
			<td>75.0%</td>
			<td>91.7%</td>
			<td>75.0%</td>
		</tr>
		<tr>
			<td>Blindfold</td>
			<td>75.0%</td>
			<td>58.3%</td>
			<td>58.3%</td>
			<td>50.0%</td>
		</tr>
		<tr>
			<td>Vision</td>
			<td>83.3%</td>
			<td>75.0%</td>
			<td>91.7%</td>
			<td>100.0%</td>
		</tr>
	</tbody>
</table>
Table 1: The instant and long term bouba/kiki-effect
The recognition of bouba and kiki shapes
Eleven of the 12 participants who were congenitally blind (~92%), nine of the 12 who were blindfolded (75%), and all of the 12 who were fully sighted (100%) instantly recognized the congruous tactile/visual and auditory bouba/kiki; all three experimental groups were significantly above the chance level (of 50%): &#967;2(1. N = 12) = 8.33, p = 0.00, &#967;2(1. N = 12) = 3.00, p = 0.08, and &#967;2(1. N = 12) = 12.00, p = 0.00 (Experiment 2, Trial 1)17. (Cf. Table 2.)
Long term, 11 participants in the blind-group recognized the tactile bouba/kiki shapes together with congruous audio and 10 participants with incongruous audio: Both types of congruousness recognized significantly above the chance level (of 50%): &#967;2(1. N = 12) = 8.33, p = 0.00 and &#967;2(1. N = 12) = 5.33, p = 0.02 (Experiment 2, Trial 1-8)17. Nine participants in the blindfold-group recognized the tactile shapes together with congruous audio and eight participants together with incongruous audio; in other words, the congruous shape and audio were significantly above the chance level (of 50%): &#967;2(1. N = 12) = 3.00, p = 0.08 (Experiment 2, Trial 1-8)17. All 12 participants in the vision-group recognized the visual bouba/kiki shapes together with congruous audio and six participants with incongruous audio: The congruous shape and audio recognized significantly above the chance level (of 50%): &#967;2(1. N = 12) = 12.00, p = 0.00 (Experiment 2, Trial 1-8)17. (Cf. Table 2.)
<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td rowspan="2">Participant group</td>
			<td>Instant recognition of bouba/kiki shape</td>
			<td colspan="2">Long term recognition of bouba and kiki shapes</td>
		</tr>
		<tr>
			<td>Congruous shape and audio</td>
			<td>Congruous shape and audio</td>
			<td>Incongruous shape and audio</td>
		</tr>
		<tr>
			<td>Blind</td>
			<td>91.7%</td>
			<td>91.7%</td>
			<td>83.3%</td>
		</tr>
		<tr>
			<td>Blindfold</td>
			<td>75.0%</td>
			<td>75.0%</td>
			<td>66.7%</td>
		</tr>
		<tr>
			<td>Vision</td>
			<td>100.0%</td>
			<td>100%</td>
			<td>50.0%</td>
		</tr>
	</tbody>
</table>
Table 2: The instant and long term recognition of bouba and kiki shapes
The mental images of bouba and kiki
Eight of the 12 participants who were congenitally blind [~73% (with one &#8216;inconclusive&#8217; participant drawing removed)], eight of the 12 who were blindfolded [~89% (with three &#8216;inconclusive&#8217; participant drawings removed)], and eight of the 12 who were fully sighted [80% (with two &#8216;inconclusive&#8217; participant drawings removed)] instantly drew a mental image: A tactile/visual shape that corresponded to the presented auditory bouba/kiki (Experiment 3, Trial 1)17. Both the blindfold and vision-group were significantly above the chance level (of 50%): &#967;2(1. N = 9) = 5.44, p = 0.02 and &#967;2(1. N = 10) = 3.60, p = 0.06 (Experiment 3, Trial 1)17. (Cf. Table 3.)
Regarding the long-term mental images of bouba and kiki: 11 participants in the blind-group, eight in the blindfold-group, and 12 in the vision-group drew tactile/visual bouba/kiki shapes that corresponded to the presented auditory bouba/kiki (Experiment 3, Trial 1-4)17. Both the blind and vision-group were significantly above the chance level (of 50%): &#967;2(1. N = 12) = 8.33, p = 0.00 and &#967;2(1. N = 12) = 12.00, p = 0.00 (Experiment 3, Trial 1-4)17. Fisher&#8217;s exact test found a significant difference between the blindfold-group and the vision-group [p = 0.09 (Experiment 3, Trial 1-4)]17. (Cf. Table 3.)
<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td>Participant group</td>
			<td>Instant mental image of bouba/kiki</td>
			<td>Long term mental images of bouba and kiki</td>
		</tr>
		<tr>
			<td>Blind</td>
			<td>72.7%</td>
			<td>91.7%</td>
		</tr>
		<tr>
			<td>Blindfold</td>
			<td>88.9%</td>
			<td>66.7%</td>
		</tr>
		<tr>
			<td>Vision</td>
			<td>80.0%</td>
			<td>100.0%</td>
		</tr>
	</tbody>
</table>
Table 3: The instant and long term mental images of bouba and kiki
Furthermore. ~83% of all participant drawings included the most characteristic shape feature of the global bouba and kiki shapes: Curve and angle, respectively (Experiment 3, Trial 1-4)17. The participant drawings differed in the quantity of curves/angles (e.g., one and five angles for the kiki word: cf. Figure 2, Trials 1, 6, and 9), and in the direction of the curves/angles [i.e. horizontal, vertical or diagonal: cf. Figure 2, Trials 2, 5, and 8 (bouba sound)], but typically did not include the global bouba/kiki shape [Experiment 3, Trial 1-4 (cf. Figure 1; Figure 2)]17. Finally, the experimental group was recognized in ~43% of the scores&#8217; answers: Five participants in the blind-group, three in the blindfold-group, and five in the vision-group; no experimental group was significantly above the chance level [of 33.3% (Experiment 3, Trial 1-4)]17.
The effect of visual imagery and learning
The presented protocol succeeded in investigating the role of visual imagery in the bouba/kiki-effect, whether training in noticing tactile/visual-auditory bouba and kiki regularities affected the bouba/kiki-effect and the recognition of individual tactile/visual bouba and kiki shapes, and finally what mental images these shape-audio regularities produced17. By including one experimental group with no visual experience (i.e. the blind-group) and two experimental groups with visual experience (i.e. the blindfold and vision-group), it was possible to test the effect of visual imagery; and by including one experimental group with no visual experience and extensive tactile experience (i.e. the blind-group) and one experimental group with extensive visual experience and no tactile experience (i.e. the blindfold-group), it was possible to test the effect of the training in noticing tactile-auditory regularities. The results clearly suggest that the blindfold-group drew upon visual imagery to solve new tactile-auditory problems, but not long term (Experiment 1&#8211;3); that training in noticing bouba/kiki shape-audio regularities affected the tactile/visual-auditory bouba/kiki-effect (Experiment 1) and the recognition of individual tactile/visual bouba and kiki shapes [i.e. together with congruous audio (Experiment 2)], but differently in each experimental group (Experiment 1-2), and that all experimental groups created mental images of the most characteristic tactile/visual-auditory shape feature of bouba (curve) and kiki [(angle) Experiment 3. Cf. Table 1; Table 2; Table 3]17. Moreover, by including repeated within-group measures of instant and long term effect (Experiment 1-3), it was possible to test when the effect of visual imagery and learning shape-audio regularities actually occurred; and by including three different tasks (i.e. the tactile/visual-auditory bouba/kiki-effect in Experiment 1, recognizing individual tactile/visual bouba/kiki shapes together with congruous and incongruous audio in Experiment 2, and drawing mental images of the bouba/kiki audio in Experiment 3), to test how robust these effects were. The results clearly suggest that the effect of visual imagery was not robust long term within each task, especially when using haptic touch [cf. the blindfold-group (Experiment 1-3)]; whereas the effect of learning shape-audio regularities was, but not across tasks [cf. the blind and vision-group (Experiment 1-3). Cf. Table 1; Table 2; Table 3.]17

Watch this Scientific Journal Video about Investigating the Effect of Visual Imagery and Learning Shape-Audio Regularities on Bouba and Kiki at JoVE.com

Investigating the Effect of Visual Imagery and Learning Shape-Audio Regularities on Bouba and Kiki

The aim of the presented protocol is to investigate the role of visual imagery in the bouba/kiki-effect, whether training in noticing the bouba/kiki shape-audio regularities affects the bouba/kiki-effect and the recognition of individual bouba and kiki shapes, and finally what mental images these regularities produce.

This article presents a protocol for investigating the role of visual imagery in the bouba/kiki-effect, whether training in noticing the bouba/kiki ...

Research

JoVE Journal

Behavior

Methods to Explore the Influence of Top-down Visual Processes on Motor Behavior

Kinesthetic awareness is important to successfully navigate the environment. When we interact with our daily surroundings, some aspects of movement are ...

Investigating the Effects of Antipsychotics and Schizotypy on the N400 Using Event-Related Potentials and Semantic Categorization

Within the field of cognitive neuroscience, functional magnetic resonance imaging (fMRI) is a popular method of visualizing brain function. This is in ...

Quantifying Learning in Young Infants: Tracking Leg Actions During a Discovery-learning Task

Task-specific actions emerge from spontaneous movement during infancy. It has been proposed that task-specific actions emerge through a discovery-learning ...

Place and Response Learning in the Open-field Tower Maze

This protocol describes how the Open-field Tower Maze (OFTM) paradigm is used to study spatial learning in rodents. This maze is especially useful for ...

Investigating Motor Skill Learning Processes with a Robotic Manipulandum

Skilled reaching tasks are commonly used in studies of motor skill learning and motor function under healthy and pathological conditions, but can be ...

Evaluating the Anti-depression Effect of Xiaoyaosan on Chronically-stressed Mice

In addition to the standardized use of antidepressant medications and psychotherapy, the usage of traditional Chinese medicine has lead to an overall ...

Using a Classroom-Based Deese Roediger McDermott Paradigm to Assess the Effects of Imagery on False Memories

Associated word list procedures can elicit false memories in predictable ways by inducing associative processing, thus making it harder to monitor the ...

Using the FishSim Animation Toolchain to Investigate Fish Behavior: A Case Study on Mate-Choice Copying In Sailfin Mollies

Using the FishSim Animation Toolchain to Investigate Fish Behavior: A Case Study on Mate-Choice Copying In Sailfin Mollies

Over the last decade, employing computer animations for animal behavior research has increased due to its ability to non-invasively manipulate the ...

Investigating the Deployment of Visual Attention Before Accurate and Averaging Saccades via Eye Tracking and Assessment of Visual Sensitivity

This experimental protocol was designed to investigate whether visual attention is obligatorily deployed at the endpoint of saccades. To this end, we ...

Exploring Infant Sensitivity to Visual Language using Eye Tracking and the Preferential Looking Paradigm

We discuss the use of the preferential looking paradigm in eye tracking studies in order to study how infants develop, understand, and attend to the world ...