The work is supported by funding from the National Institutes of Health (NIH), grant EY07977. The authors would also like to thank Usman Ayyaz for his assistance in testing the software, and Muhammad Naeem Ayyaz for his comments on the manuscript.

<ol>
	<li>Edelman, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Representation+is+representation+of+similarities.">Representation is representation of similarities.</a> TheBehavioral and Brain Sciences. 21 (4), 449-498 (1998).</li><li>Hahn, U., Chater, N. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Concepts+and+similarity.">Concepts and similarity.</a> Knowledge, Concepts and Categories. , 43-84 (1997).</li><li>Kriegeskorte, N., Kievit, R. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Representational+geometry:+integrating+cognition,+computation,+and+the+brain.">Representational geometry: integrating cognition, computation, and the brain.</a> Trends in Cognitive Sciences. 17 (8), 401-412 (2013).</li><li>Hebart, M. N., Zheng, C. Y., Pereira, F., Baker, C. I. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Revealing+the+multidimensional+mental+representations+of+natural+objects+underlying+human+similarity+judgements.">Revealing the multidimensional mental representations of natural objects underlying human similarity judgements.</a> Nature Human Behaviour. 4 (11), 1173-1185 (2020).</li><li>Deng, W. S., Sloutsky, V. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+development+of+categorization:+Effects+of+classification+and+inference+training+on+category+representation.">The development of categorization: Effects of classification and inference training on category representation.</a> Developmental Psychology. 51 (3), 392-405 (2015).</li><li>Shepard, R. N. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Stimulus+and+response+generalization:+tests+of+a+model+relating+generalization+to+distance+in+psychological+space.">Stimulus and response generalization: tests of a model relating generalization to distance in psychological space.</a> Journal of Experimental Psychology. 55 (6), 509-523 (1958).</li><li>Coombs, C. H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+method+for+the+study+of+interstimulus+similarity.">A method for the study of interstimulus similarity.</a> Psychometrika. 19 (3), 183-194 (1954).</li><li>Ga&#776;rdenfors, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Conceptual Spaces: The Geometry of Thought. , (2000).</li><li>Zaidi, Q., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Perceptual+spaces:+mathematical+structures+to+neural+mechanisms.">Perceptual spaces: mathematical structures to neural mechanisms.</a> The Journal of Neuroscience The Official Journal of the Society for Neuroscience. 33 (45), 17597-17602 (2013).</li><li>Krishnaiah, P. R., Kanal, L. N. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Handbook of Statistics 2. , (1982).</li><li>Tsogo, L., Masson, M. H., Bardot, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Multidimensional+Scaling+Methods+for+Many-Object+Sets:+A+Review.">Multidimensional Scaling Methods for Many-Object Sets: A Review.</a> Multivariate Behavioral Research. 35 (3), 307-319 (2000).</li><li>Kriegeskorte, N., Mur, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Inverse+MDS:+Inferring+dissimilarity+structure+from+multiple+item+arrangements.">Inverse MDS: Inferring dissimilarity structure from multiple item arrangements.</a> Frontiers in Psychology. 3, 245 (2012).</li><li>Rao, V. R., Katz, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Alternative+Multidimensional+Scaling+Methods+for+Large+Stimulus+Sets.">Alternative Multidimensional Scaling Methods for Large Stimulus Sets.</a> Journal of Marketing Research. 8 (4), 488-494 (1971).</li><li>Hoffman, J. I. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Hypergeometric+Distribution.">Hypergeometric Distribution.</a> Biostatistics for Medical and Biomedical Practitioners. , 179-182 (2015).</li><li>Victor, J. D., Rizvi, S. M., Conte, M. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Two+representations+of+a+high-dimensional+perceptual+space.">Two representations of a high-dimensional perceptual space.</a> Vision Research. 137, 1-23 (2017).</li><li>Knoblauch, K., Maloney, L. T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Estimating+classification+images+with+generalized+linear+and+additive+models.">Estimating classification images with generalized linear and additive models.</a> Journal of Vision. 8 (16), 1-19 (2008).</li><li>Maloney, L. T., Yang, J. N. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Maximum+likelihood+difference+scaling.">Maximum likelihood difference scaling.</a> Journal of Vision. 3 (8), 573-585 (2003).</li><li>Logvinenko, A. D., Maloney, L. T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+proximity+structure+of+achromatic+surface+colors+and+the+impossibility+of+asymmetric+lightness+matching.">The proximity structure of achromatic surface colors and the impossibility of asymmetric lightness matching.</a> Perception &amp; Psychophysics. 68 (1), 76-83 (2006).</li><li>Zhou, Y., Smith, B. H., Sharpee, T. O. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Hyperbolic+geometry+of+the+olfactory+space.">Hyperbolic geometry of the olfactory space.</a> Science Advances. 4 (8), (2018).</li><li>Goldstone, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=An+efficient+method+for+obtaining+similarity+data.">An efficient method for obtaining similarity data.</a> Behavior Research Methods, Instruments, &amp; Computers. 26 (4), 381-386 (1994).</li><li>Townsend, J. T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Theoretical+analysis+of+an+alphabetic+confusion+matrix.">Theoretical analysis of an alphabetic confusion matrix.</a> Perception &amp; Psychophysics. 9, 40-50 (1971).</li></ol>

The authors have nothing to disclose.

The protocol outlined here is effective for obtaining and analyzing similarity judgments for stimuli that can be presented visually. The experimental paradigm, the analysis, and possible extensions are discussed first, and later the advantages and disadvantages of the method.
Experimental paradigm: The proposed method is demonstrated using a domain of 37 animal names, and a sample dataset of perceptual judgments is provided so that one can follow the analysis in step 5 and rep...

Humans mentally process and represent incoming sensory information to perform a wide range of tasks, such as object recognition, navigation, making inferences about the environment, and many others. Similarity judgments are commonly used to probe these mental representations1. Understanding the structure of mental representations can provide insight into the organization of conceptual knowledge2. It is also possible to gain insight into neural computations, by relating similarity judgments to brain activation patterns3. Additionally, similarity judgments reveal features that are salient in perception4. Studying how mental representations change during development can shed light on how they are learned5. Thus, similarity judgments provide valuable insight into information processing in the brain.
A common&#160;model of mental representations using similarities is a geometric space model6,7,8. Applied to sensory domains, this kind of model is often referred to as a perceptual space9. Points in the space represent stimuli and distances between points correspond to the perceived dissimilarity between them. From similarity judgments, one can obtain quantitative estimates of dissimilarities. These pairwise dissimilarities (or perceptual distances) can then be used to model the perceptual space via multidimensional scaling10.
There are many methods for collecting similarity judgments, each with its advantages and disadvantages. The most straightforward way of obtaining quantitative measures of dissimilarity is to ask subjects to rate on a scale the degree of dissimilarity between each pair of stimuli. While this is relatively quick, estimates tend to be unstable across long sessions as subjects cannot go back to previous judgments, and context effects, if present, cannot be detected. (Here, a context effect is defined as a change in the judged similarity between two stimuli, based on the presence of other stimuli that are not being compared.) Alternatively, subjects can be asked to compare all pairs of stimuli to all other pairs of stimuli. While this would yield a more reliable rank ordering of dissimilarities, the number of comparisons required scales with the fourth power of the number of stimuli, making it feasible for only small stimulus sets. Quicker alternatives, like sorting into a predefined number of clusters11&#160;or free sorting have their own limitations. Free sorting (into any number of piles) is intuitive, but it forces the subject to categorize the stimuli, even if the stimuli do not easily lend themselves to categorization. The more recent multi-arrangement method, inverse MDS, circumvents many of these limitations and is very efficient12. However, this method requires subjects to project their mental representations onto a 2D Euclidean plane and to consider similarities in a specific geometric manner, making the assumption that similarity structure can be recovered from Euclidean distances on a plane. Thus, there remains a need for an efficient method to collect large amounts of similarity judgments, without making assumptions about the geometry underlying the judgments.
Described here is a method that is both reasonably efficient and also avoids the above potential pitfalls. By asking subjects to rank stimuli in order of similarity to a central reference in each trial13, relative similarity can be probed directly, without assuming anything about the geometric structure of the subjects&#39; responses. The paradigm repeats a subset of comparisons with both identical and different contexts, allowing for direct assessment of context effects as well as the acquisition of graded responses in terms of choice probabilities. The analysis procedure decomposes these rank judgments into multiple pairwise comparisons and uses them to build and search for Euclidean models of perceptual spaces that explain the judgments. The method is suitable for describing in detail the representation of stimulus sets of moderate sizes (e.g., 19 to 49).
To demonstrate the feasibility of the approach, an experiment was conducted, using a set of 37 animals as stimuli. Data was collected over the course of 10 one-hour sessions and then analyzed separately for each subject. Analysis revealed consistency across subjects and negligible context effects. It also assessed consistency of perceived dissimilarities between stimuli with Euclidean models of their perceptual spaces. The paradigm and analysis procedures outlined in this paper are flexible and are expected to be of use to researchers interested in characterizing the geometric properties of a range of perceptual spaces.

<table><tbody><tr><td>Computer Workstation</td><td>N/A</td><td>N/A</td><td>OS: Windows/ MacOS 10 or higher/ Linux; 3.1 GHz Dual-Core Intel Core i5 or similar; 8GB or more memory; User permissions for writing and executing files</td></tr><tr><td>conda</td><td></td><td>Version 4.11</td><td>OS: Windows/ MacOS 10 or higher/ Linux</td></tr><tr><td>Microsoft Excel</td><td>Microsoft</td><td>Any</td><td>To open and shuffle rows and columns in trial conditions files.</td></tr><tr><td>PsychoPy</td><td>N/A</td><td>Version 2021.2</td><td>Framework for running psychophysical studies</td></tr><tr><td>Python 3</td><td>Python Software Foundation</td><td>Python Version 3.8</td><td>Python3 and associated built-in libraries</td></tr><tr><td>Required Python Libraries</td><td>N/A</td><td>numpy version: 1.17.2 or higher; matplotlib version 3.4.3 or higher; scipy version 1.3.1 or higher; pandas version 0.25.3 or higher; seaborn version 0.9.0 or higher; scikit_learn version 0.23.1 or higher; yaml version 6.0 or higher&amp;nbsp;</td><td>numpy, scipy and scikit_learn are computing modules with in-built functions for optimization and vector operations. matplotlib and seaborn are plotting libraries. pandas is used to reading in and edit data from csv files.</td></tr></tbody></table>

a psychophysics paradigm for the collection and analysis of similarity judgments

Similarity judgments are commonly used to study mental representations and their neural correlates. This approach has been used to characterize perceptual spaces in many domains: colors, objects, images, words, and sounds. Ideally, one might want to compare estimates of perceived similarity between all pairs of stimuli, but this is often impractical. For example, if one asks a subject to compare the similarity of two items with the similarity of two other items, the number of comparisons grows with the fourth power of the stimulus set size. An alternative strategy is to ask a subject to rate similarities of isolated pairs, e.g., on a Likert scale. This is much more efficient (the number of ratings grows quadratically with set size rather than quartically), but these ratings tend to be unstable and have limited resolution, and the approach also assumes that there are no context effects.
Here, a novel ranking paradigm for efficient collection of similarity judgments is presented, along with an analysis pipeline (software provided) that tests whether Euclidean distance models account for the data. Typical trials consist of eight stimuli around a central reference stimulus: the subject ranks stimuli in order of their similarity to the reference. By judicious selection of combinations of stimuli used in each trial, the approach has internal controls for consistency and context effects. The approach was validated for stimuli drawn from Euclidean spaces of up to five dimensions. 
The approach is illustrated with an experiment measuring similarities among 37 words. Each trial yields the results of 28 pairwise comparisons of the form, "Was A more similar to the reference than B was to the reference?" While directly comparing all pairs of pairs of stimuli would have required 221445 trials, this design enables reconstruction of the perceptual space from 5994 such comparisons obtained from 222 trials.

Figure 1A shows part of a conditions file generated by the script in step 3.3, for the word experiment. Each row corresponds to a trial. The stimulus in the ref column appears in the center of the display. The column names stim1 to stim8 correspond to eight positions along a circle, running counterclockwise, starting from the position to the right of the central reference. A sample trial from the word experiment is shown in Figure 1B.
<p cla...

Watch this Scientific Journal Video about A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments at JoVE.com

A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments

The protocol presents an experimental psychophysics paradigm to obtain large quantities of similarity judgments, and an accompanying analysis workflow. The paradigm probes context effects and enables modeling of similarity data in terms of Euclidean spaces of at least five dimensions.

Similarity judgments are commonly used to study mental representations and their neural correlates. This approach has been used to characterize perceptual ...

a-psychophysics-paradigm-for-collection-analysis-similarity

Research

JoVE Journal

Neuroscience

2.4K Views.  Weill Cornell Graduate School of Medical Sciences. The protocol presents an experimental psychophysics paradigm to obtain large quantities of similarity judgments, and an accompanying analysis workflow. The paradigm probes context effects and enables modeling of similarity data in terms of Euclidean spaces of at least five dimensions.

Video: A Psychophysics Paradigm for the Collection and Analysis of Similarity Judgments

Perceptual and Category Processing of the Uncanny Valley Hypothesis' Dimension of Human Likeness: Some Methodological Issues

Mori's Uncanny Valley Hypothesis1,2 proposes that the perception of humanlike characters such as robots and, by extension, avatars (computer-generated characters) can evoke negative or positive affect (valence) depending on the object's degree of visual and behavioral realism along a dimension of human likeness (DHL) (Figure 1). But studies of affective valence of subjective responses to variously realistic non-human characters have produced inconsistent findings 3, 4, 5, 6. One of a number of reasons for this is that human likeness is not perceived as the hypothesis assumes. While the DHL can be defined following Mori's description as a smooth linear change in the degree of physical humanlike similarity, subjective perception of objects along the DHL can be understood in terms of the psychological effects of categorical perception (CP) 7. Further behavioral and neuroimaging investigations of category processing and CP along the DHL and of the potential influence of the dimension's underlying category structure on affective experience are needed. This protocol therefore focuses on the DHL and allows examination of CP. Based on the protocol presented in the video as an example, issues surrounding the methodology in the protocol and the use in "uncanny" research of stimuli drawn from morph continua to represent the DHL are discussed in the article that accompanies the video. The use of neuroimaging and morph stimuli to represent the DHL in order to disentangle brain regions neurally responsive to physical human-like similarity from those responsive to category change and category processing is briefly illustrated.

Investigation of the Uncanny Valley Hypothesis and affective experience requires an understanding of the hypothesis' dimension of human likeness (DHL). This protocol allows representation of the DHL and examination of categorical perception. Use of the same stimuli and fMRI to distinguish brain regions responsive to physical and category change is illustrated.

Mori's Uncanny Valley Hypothesis1,2 proposes that the perception of humanlike characters such as robots and, by extension, avatars (computer-generated ...

Behavior

The Crossmodal Congruency Task as a Means to Obtain an Objective Behavioral Measure in the Rubber Hand Illusion Paradigm

The rubber hand illusion (RHI) is a popular experimental paradigm. Participants view touch on an artificial rubber hand while the participants' own hidden hand is touched. If the viewed and felt touches are given at the same time then this is sufficient to induce the compelling experience that the rubber hand is one's own hand. The RHI can be used to investigate exactly how the brain constructs distinct body representations for one's own body. Such representations are crucial for successful interactions with the external world. To obtain a subjective measure of the RHI, researchers typically ask participants to rate statements such as "I felt as if the rubber hand were my hand". Here we demonstrate how the crossmodal congruency task can be used to obtain an objective behavioral measure within this paradigm.
The variant of the crossmodal congruency task we employ involves the presentation of tactile targets and visual distractors. Targets and distractors are spatially congruent (i.e. same finger) on some trials and incongruent (i.e. different finger) on others. The difference in performance between incongruent and congruent trials - the crossmodal congruency effect (CCE) - indexes multisensory interactions. Importantly, the CCE is modulated both by viewing a hand as well as the synchrony of viewed and felt touch which are both crucial factors for the RHI.
The use of the crossmodal congruency task within the RHI paradigm has several advantages. It is a simple behavioral measure which can be repeated many times and which can be obtained during the illusion while participants view the artificial hand. Furthermore, this measure is not susceptible to observer and experimenter biases. The combination of the RHI paradigm with the crossmodal congruency task allows in particular for the investigation of multisensory processes which are critical for modulations of body representations as in the RHI.

We demonstrate how an objective measure can be employed in the widely employed rubber hand illusion paradigm. This measure is obtained by modifying the well-established crossmodal congruency task. This task allows the investigation of multisensory processes which are critical for modulations of body representations as in the rubber hand illusion.

The rubber hand illusion (RHI) is a popular experimental paradigm. Participants view touch on an artificial rubber hand while the participants' own hidden ...

Holistic Facial Composite Creation and Subsequent Video Line-up Eyewitness Identification Paradigm

The paradigm detailed in this manuscript describes an applied experimental method based on real police investigations during which an eyewitness or victim to a crime may create from memory a holistic facial composite of the culprit with the assistance of a police operator. The aim is that the composite is recognized by someone who believes that they know the culprit. For this paradigm, participants view a culprit actor on video and following a delay, participant-witnesses construct a holistic system facial composite. Controls do not construct a composite. From a series of arrays of computer-generated, but realistic faces, the holistic system construction method primarily requires participant-witnesses to select the facial images most closely meeting their memory of the culprit. Variation between faces in successive arrays is reduced until ideally the final image possesses a close likeness to the culprit. Participant-witness directed tools can also alter facial features, configurations between features and holistic properties (e.g., age, distinctiveness, skin tone), all within a whole face context. The procedure is designed to closely match the holistic manner by which humans&#8217; process faces. On completion, based on their memory of the culprit, ratings of composite-culprit similarity are collected from the participant-witnesses. Similar ratings are collected from culprit-acquaintance assessors, as a marker of composite recognition likelihood. Following a further delay, all participants &#8212; including the controls&#160;&#8212; attempt to identify the culprit in either a culprit-present or culprit-absent video line-up, to replicate circumstances in which the police have located the correct culprit, or an innocent suspect. Data of control and participant-witness line-up outcomes are presented, demonstrating the positive influence of holistic composite construction on identification accuracy. Correlational analyses are conducted to measure the relationship between assessor and participant-witness composite-culprit similarity ratings, delay, identification accuracy, and confidence to examine which factors influence video line-up outcomes.

This applied experimental paradigm replicates circumstances by which an eyewitness to a real crime may create a holistic facial composite of the culprit from memory, and then attempt to identify the culprit from a video line-up containing the culprit or one in which he or she is not present.

The paradigm detailed in this manuscript describes an applied experimental method based on real police investigations during which an eyewitness or victim ...

Psychophysically-anchored, Robust Thresholding in Studying Pain-related Lateralization of Oscillatory Prestimulus Activity

In perceptual studies, it is often important to objectively assess the equality of delivered stimulation across participants or to quantify the intra-individual sensation magnitude that is evoked by stimulation over multiple trials. This requires a robust mapping of stimulus magnitude to perceived intensity and is commonly achieved by psychophysical estimation methods such as the staircase procedure. Newer, more efficient procedures like the QUEST algorithm fit a psychophysical function to the data in real time while at the same time maximizing the efficiency of data collection. A robust estimate of the threshold intensity between painful and nonpainful perceptions can then be used to reduce the influence of variations in sensory input in subsequent analyses of oscillatory brain activity. By stimulating at a constant threshold intensity determined by an adaptive estimation procedure, the variance in the ratings can be directly attributed to perceptual processes. Oscillatory activity can then be contrasted between &#34;pain&#34; and &#34;no-pain&#34; trials directly, yielding activity that closely relates to perceptual classification processes in nociception.

Psychophysical methods such as the QUEST estimation procedure can efficiently yield robust estimates of the stimulation intensity at which nonpainful sensations transition into painful sensations. By stimulating repeatedly at the threshold intensity, the variability in rating responses can directly be attributed to perceptual classifications in subsequent analyses.

In perceptual studies, it is often important to objectively assess the equality of delivered stimulation across participants or to quantify the ...

Cross-Modal Multivariate Pattern Analysis

Multivariate pattern analysis (MVPA) is an increasingly popular method of analyzing functional magnetic resonance imaging (fMRI) data1-4. Typically, the method is used to identify a subject's perceptual experience from neural activity in certain regions of the brain. For instance, it has been employed to predict the orientation of visual gratings a subject perceives from activity in early visual cortices5 or, analogously, the content of speech from activity in early auditory cortices6.
Here, we present an extension of the classical MVPA paradigm, according to which perceptual stimuli are not predicted within, but across sensory systems. Specifically, the method we describe addresses the question of whether stimuli that evoke memory associations in modalities other than the one through which they are presented induce content-specific activity patterns in the sensory cortices of those other modalities. For instance, seeing a muted video clip of a glass vase shattering on the ground automatically triggers in most observers an auditory image of the associated sound; is the experience of this image in the "mind's ear" correlated with a specific neural activity pattern in early auditory cortices? Furthermore, is this activity pattern distinct from the pattern that could be observed if the subject were, instead, watching a video clip of a howling dog?
In two previous studies7,8, we were able to predict sound- and touch-implying video clips based on neural activity in early auditory and somatosensory cortices, respectively. Our results are in line with a neuroarchitectural framework proposed by Damasio9,10, according to which the experience of mental images that are based on memories - such as hearing the shattering sound of a vase in the "mind's ear" upon seeing the corresponding video clip - is supported by the re-construction of content-specific neural activity patterns in early sensory cortices.

Classical multivariate pattern analysis predicts sensory stimuli a subject perceives from neural activity in the corresponding cortices (e.g. visual stimuli from activity in visual cortex). Here, we apply pattern analysis cross-modally and show that sound- and touch-implying visual stimuli can be predicted from activity in auditory and somatosensory cortices, respectively.

Multivariate pattern analysis (MVPA) is an increasingly popular method of analyzing functional magnetic resonance imaging (fMRI) data1-4. Typically, the ...

A Two-interval Forced-choice Task for Multisensory Comparisons

We provide a procedure for a psychophysics experiment in humans based on a previously described paradigm aimed to characterize the perceptual duration of intervals within the range of milliseconds of visual, acoustic, and audiovisual aperiodic trains of six pulses. In this task, each of the trials consists of two consecutive intramodal intervals where the participants press the upward arrow key to report that the second stimulus lasted longer than the reference, or the downward arrow key to indicate otherwise. The analysis of the behavior results in psychometric functions of the probability of estimating the comparison stimulus to be longer than the reference, as a function of the comparison intervals. In conclusion, we advance a way of implementing standard programming software to create visual, acoustic, and audiovisual stimuli, and to generate a two-interval forced-choice (2IFC) task by delivering stimuli through noise-blocking headphones and a computer&#39;s monitor.

Psychophysics is essential for studying perception phenomena through sensory information. Here we present a protocol to perform a two-interval forced-choice task as implemented in a previous report on human psychophysics where participants estimated the duration of visual, auditory, or audiovisual intervals of aperiodic trains of pulses.

We provide a procedure for a psychophysics experiment in humans based on a previously described paradigm aimed to characterize the perceptual duration of ...

Interaction between Phonological and Semantic Processes in Visual Word Recognition using Electrophysiology

Controversies have always existed in research related to reading abilities; on whether printed words are perceived in a feedforward manner based on orthographic information after which, other representations, such as phonology and semantics are activated, or whether these are fully interactive and high-level semantic information affects early processing. An interference paradigm was implemented in the presented protocol of phonological and semantic judgment tasks that utilized the same precede-target pairs to explore the relative order of phonological and semantic activation. The high- and low-frequency target words were preceded with three conditions: semantically related, phonological-related (homophones), or unrelated. The results showed that the induced P200 component of low-frequency word pairs was significantly greater than high-frequency words in both the semantic and phonological tasks. In addition, both the homophones in the semantic task and the semantically related pairs in the phonological task caused reduction in N400 when compared to the the control condition, word frequency-independently. It is worth noting that for the low-frequency pairs in the phonological judgment task, the P200 released by the semantically related word pairs was significantly larger than that in the control condition. Overall, semantic processing in phonological tasks and phonological processing in semantic tasks were found in both high- and low-frequency words, suggesting that the interaction between semantics and phonology may operate in a task-independent manner. However, the specific time this interaction occurred may have been affected by the task and frequency.

We present a protocol to explore the relative activation sequence of phonology and semantics in visual word recognition. The results show that consistent with interactive accounts, semantic and phonological representations may be processed interactively, and higher-level linguistic representations may affect early processing.

Controversies have always existed in research related to reading abilities; on whether printed words are perceived in a feedforward manner based on ...

Assessment and Communication for People with Disorders of Consciousness

In this experiment, we demonstrate a suite of hybrid Brain-Computer Interface (BCI)-based paradigms that are designed for two applications: assessing the level of consciousness of people unable to provide motor response and, in a second stage, establishing a communication channel for these people that enables them to answer questions with either &#39;yes&#39; or &#39;no&#39;. The suite of paradigms is designed to test basic responses in the first step and to continue to more comprehensive tasks if the first tests are successful. The latter tasks require more cognitive functions, but they could provide communication, which is not possible with the basic tests. All assessment tests produce accuracy plots that show whether the algorithms were able to detect the patient&#39;s brain&#39;s response to the given tasks. If the accuracy level is beyond the significance level, we assume that the subject understood the task and was able to follow the sequence of commands presented via earphones to the subject. The tasks require users to concentrate on certain stimuli or to imagine moving either the left or right hand. All tasks are designed around the assumption that the user is unable to use the visual modality, and thus, all stimuli presented to the user (including instructions, cues, and feedback) are auditory or tactile.

With this experiment, one might be able to detect consciousness in people with disorders of consciousness. Furthermore, the approach can create a simple communication channel that enables people to give simple YES/NO answers to questions.

In this experiment, we demonstrate a suite of hybrid Brain-Computer Interface (BCI)-based paradigms that are designed for two applications: assessing the ...

The Joint Effect of Social Comparison and Social Distance on Evaluation of Intertemporal Choice Outcomes in Event-related Potential Studies

Intertemporal choice plays a crucial role in our daily lives, influencing decisions related to education, health, consumption, and investment. This research proposes an innovative experimental protocol that examines how social comparison and social distance jointly affect the neural processes involved in outcome assessment for intertemporal choices. The study is based on the theoretical framework of cognitive resource competition. This protocol enables researchers to dynamically establish an indifference point for each participant, effectively eliminating the influence of any biased indifference points on the assessment of intertemporal choices. Consequently, the study solely measures the combined impact of social comparison and social distance on how participants evaluate intertemporal choice outcomes. The findings reveal that individuals are more inclined to opt for immediate outcomes under negative unfair conditions. Moreover, compared to the fair and positive unfair conditions, people tend to undervalue delayed outcomes in the negative unfair condition. The strength of this approach lies in its dynamic indifference point setting, making it an effective method to investigate the influence of various external factors (such as social status and power level) on intertemporal decision-making. While the protocol is designed to measure electrophysiological events like event-related potentials, it can also be tailored for use with fMRI.

This protocol aims to investigate the neural activity related to social comparison and social distance during the processing of intertemporal decision outcomes. Indifference points will be measured using event-related potentials as part of the study.

Intertemporal choice plays a crucial role in our daily lives, influencing decisions related to education, health, consumption, and investment. This ...

Evaluating the Accuracy of Snap Judgments

Source: Diego Reinero &#38; Jay Van Bavel&#8212;New York University
Social psychologists have long been interested in the way people form impressions of others. Much of this work has focused on the errors people make in judging others, such as the exaggerated influence of central traits (such as &#34;warm&#34; and &#34;cold&#34;), the insufficient weight given to the context in which others&#39; behavior takes place, and the tendency for people to make judgments that conform to their initial expectations about another. However, this focus on errors masks the fact that people are quite good at making fairly accurate judgments about other people&#39;s characteristics, an ability that was no doubt important over the course of human evolution.
Indeed, the human ability to make quick sense of social situations and people ranks among our most valuable skills. What is particularly impressive about our ability to make sense of others is not just how little information we need to make inferences, but how well calibrated we can be with so little information. This video shows some experimental techniques used by psychology researchers, including Ambady and Rosenthal in their seminal work,1 and explores the process of making inferences in the context of students&#39; evaluations of their teachers.

Source: Diego Reinero &#38; Jay Van Bavel&#8212;New York University
Social psychologists have long been interested in the way people form impressions of ...