Name: creating objects and object categories for studying perception and perceptual learning
Uploaded: 2012-11-02T15:00:00
Duration: 14 min 38 s
Description: Watch this Scientific Journal Video about Creating Objects and Object Categories for Studying Perception and Perceptual Learning at JoVE.com

This work was supported in part by the U.S. Army Research Laboratory and the U. S. Army Research Office grant W911NF1110105 and NSF grant IOS-1147097to Jay Hegd&eacute;. Support was also provided by a pilot grant to Jay Hegd&eacute; from the Vision Discovery Institute of the Georgia Health Sciences University. Daniel Kersten was supported by grants ONR N00014-05-1-0124 and NIH R01 EY015261 and in part by WCU (World Class University) program funded by the Ministry of Education, Science and Technology through the National Research Foundation of Korea (R31-10008). Karin Hauffen was supported by the Undergraduate Research Apprenticeship Program (URAP) of the U.S. Army.

1. Creating Naturalistic Virtual 3-D Objects using VM
<ol>
 <li>To create digital embryos, use the Digital Embryo Workshop (DEW; see Table 1). Each run generates a single embryo14, the shape of which is unique to a given set of settings (or 'genotype') used for the given run (Figure 1). The 'cells' of the embryo are represented as triangles14.
 <ol>
 <li>Run the program as many times as needed to generate the desired number of embryos.</li>
 <li>If more complex shapes are desired, increase the number of growth cycles, i.e. the number of times the cells of the embryo will divide. Note that this will also slow down the program. If it is necessary to create virtual objects other than digital embryos, use commercially available 3-D modeling tools or obtain virtual objects from commercial vendors (Table 1).</li>
 </ol>
 </li>
 <li>It is generally advisable to save the virtual objects in a commonly used file format, such as OBJ, so that the objects can be readily imported into a commercial 3-D modeling toolkit. To this end, the DEW writes objects in OBJ format by default.</li>
 <li>Visual stimuli can be generated using one or more digital embryos using a 3-D modeling and rendering environment (Table 1). Use standard graphical operations such as varying the orientation, size, lighting, surface texture and background to create the desired stimuli (see Figure 2).</li>
</ol>
2. Creating Naturalistic Object Categories using VP
<ol>
 <li>To generate object categories, generate descendants (or 'children') of the given ancestor (or 'parent') object using a desired combination of the processes in Step 1.1 above (Figure 3)9,10,12,13.
 <ol>
 <li>Some methods described below for creating smooth shape variations, such as morphing or principal components (see Steps 3 and 4 below), work better if all the input objects have the same number of cells and if there is one-to-one correspondence among the vertices of the objects. For creating such objects, use only those VM processes that will not change the number of cells and will preserve the one-to-one correspondence of vertices among the objects (see, e.g., generations G2 to G3 in Figure 3). For instance, cell division and programmed cell death alter the number of cells, and make it much harder (although not impossible17,18) to determine one-to-one correspondence between the vertices of a given pair of objects. 
 Note that the processes that alter the number of cells in a given object also alter its shape complexity. In general, the greater the number of cells, the greater the shape complexity of the object and smoother its surface.</li>
 <li>If necessary, virtual objects other than digital embryos can be used as inputs to VP (Figure 4).</li>
 </ol>
 </li>
 <li>The objects within a given category can be further selected so as to achieve a given distribution of features19. For instance, one can selectively eliminate mid-sized objects from a given category in order to generate a bimodal distribution of object size.</li>
 <li>There is no single method that is universally optimal for measuring the available shape information for all categories, nor is there a single method that is optimal for categorizing all objects20-22. Thus, the experimentor must choose these methods based on the categories and computational goals at hand20-22. Step 4 describes a commonly used method for manipulating various aspects of available shape information.</li>
 <li>The similarity between a given pair of categories can be objectively measured using available phylogenetic methods23,24. For instance, the vertical (or 'evolutionary') distance between a given pair of categories, as measured by hierarchical clustering methods in the R statistical toolkit, is an objective measure of category similarity25,26.</li>
</ol>
3. Additional Methods of Creating Shape Variation: Digital Morphing
<ol>
 <li>Given any pair of objects so that each vertex of one object corresponds to exactly one vertex of the other object (i.e., objects with one-to-one correspondence between vertices), morphing is straightforward17,18,27-29: In this case, smooth variations (or 'morphs') between the two objects are produced by smoothly interpolating between the corresponding vertices and normals (Figure 5). Depending on the pair of objects chosen, morphing will result in new categories or additional children within a category.
 <ol>
 <li>The objects shown in Figure 5 were created using linear morphing27-29. The objects can be morphed (or warped) by a vast array of other available deformation techniques17,18.</li>
 </ol>
 </li>
 <li>To create a desired distribution of morphed shapes, choose the interpolation points accordingly.</li>
</ol>
4. Additional Methods of Creating Shape Variation: Principal Components
<ol>
 <li>In order to use principal components to generate shape variations, one needs to first determine the principal components15. Principal components are specific to the given set of objects used for determining them26. For good results, use at least 30 objects with one-to-one correspondence between vertices26.</li>
 <li>Generate an average object from a desired set of n input objects, by separately averaging the coordinates and the normal of each vertex across all objects. Thus, the x coordinate of a given vertex k of should be the average of the x coordinates of vertex k of all n objects, and so forth.</li>
 <li>Use the Matlab function princomp to determine principal components of the n objects. This will generate n-1 non-zero eigenvectors, along with the corresponding n-1 eigenvalues26.</li>
 <li>To generate a new object Aj from a given principal component Pi , multiply Pi by the corresponding eigenvalue &lambda;i and a desired weight wj and add to the average object: 
 Aj= +wj&lambda;iPi</li>
 <li>Each unique wj will generate a unique object. By smoothly varying w, one can create smooth shape variations along a given principal component.</li>
 <li>To create shape variations along an independent shape dimension, repeat step 4.4 using a different principal component.</li>
 <li>To create a desired distribution of shapes along a given principal component, use the desired distribution of w.</li>
 <li>To create a multi-dimensional grid of shapes, use a set of weights for each of several principal components: 
 <img alt="Equation 2" fo:src="/files/ftp_upload/3358/3358eq2highres.jpg" src="/files/ftp_upload/3358/3358eq2.jpg" /></li>
</ol>
5. Creating Haptic Versions of 3-D Objects
<ol>
 <li>'Print-out' 3-D objects using a 3-D prototyper (or 3-D 'printer'). If necessary, adjust the object's size and smooth the object's surface prior to printing.</li>
</ol>
6. An Exemplar Application: Bayesian Inference of Image Category
<ol>
 <li>An important task in visual processing is inferring the category to which a given observed object belongs. Although the exact mechanism of this inference is unknown, there is both computational and physiological evidence 9,12,13,30-32 that it involves using the information about known features of the object in the given image to infer the category of the object. Here, we will illustrate how this inferential process may work in a Bayesian framework, and how digital embryos may be useful for research in this area.</li>
 <li>For simplicity, we will assume that the categorization task is binary and involves distinguishing category K from category L (Figure 3). Let C be the category variable. We will infer that C = K or C = L according to whether the observed image I belongs to category K or L. A typical approach to categorization involves:
 <ol>
 <li>Computing the probability that the category is K given the information in the image, denoted p(C = K |I);</li>
 <li>Computing the probability that the category is L given the information in the image, denoted p(C = L | I); and</li>
 <li>Picking the category with the higher probability.</li>
 </ol>
 </li>
 <li>Next, we will assume for simplicity that there is exactly one binary feature F. This feature may be either present in the image (denoted F = 1) or absent from the image (denoted F = 0). This example will use the 'informative fragment' feature shown in Figure 8. Informative fragments were first described by Ullman and colleagues33. In the present case, we will use the image template shown in Figure 8 as the feature, and a threshold value of 0.69. To determine whether this feature is present in a given image (say, the rightmost image in row G3 in Figure 3), we will use the following steps:
 <ol>
 <li>Slide this template over all possible locations in the image and compute, at each location, the absolute value of normalized cross-correlation between the template and the underlying sub-image.</li>
 <li>Select the image location with the highest value (0.60 in the present case).</li>
 <li>If this value is above the threshold, conclude that the feature is present; otherwise, conclude that it is absent. In our case, since the highest correlation 0.60 is below the threshold of 0.69, we conclude that the feature is absent in this image.</li>
 <li>The rationale of using such features, and the mechanisms of selecting features and determining their thresholds are beyond the scope of this report, but are described in detail in refs. 33, 30.</li>
 </ol>
 </li>
 <li>Within the framework of feature-based inference, we assume that all the information the observer extracts from the image is contained in the value of this feature, i. e., that p(C | I) = p(C | F). 
 Therefore, the task becomes that of determining the value of F in the given image (present or absent), computing p(C = K | F) and p(C = L | F) for that value F, and selecting the category with the higher probability.</li>
 <li>In the Bayesian framework,</li>
</ol>
<img alt="Equation 3" fo:src="/files/ftp_upload/3358/3358eq3highres.jpg" src="/files/ftp_upload/3358/3358eq3.jpg" /> 
Therefore,
<img alt="equation 4" fo:src="/files/ftp_upload/3358/3358eq4highres.jpg" src="/files/ftp_upload/3358/3358eq4.jpg" />
and
<img alt="Equation 5" fo:src="/files/ftp_upload/3358/3358eq5highres.jpg" src="/files/ftp_upload/3358/3358eq5.jpg" /> 
Note that the denominator in the two equations is the same. Therefore, to compare p(C = K | F) and p(C = L | F), it is not necessary to compute the denominator; rather, it is sufficient to compute the quantities
p(C=K|F) &prop; p(C=K)p(F|C=K)
and
p(C=L|F) &prop; p(C=L)p(F|C=L) 
These are sometimes called 'unnormalized probabilities'. The term p(C) is called 'prior', and the term p(F | C) is called 'likelihood'.
<ol start="6">
 <li>For simplicity, we will assume a 'flat' prior: p(C = K) = p(C = L) = 0.5.</li>
 <li>The task is now to compute p(F|C), the probability of a given feature value in an image of a given category C.
 <ol>
 <li>We will use the six images of category L (Figure 3) as an example to compute p(F = 1|C = L), the probability that the feature is present in an image of category L.</li>
 <li>To compute p(F = 1|C = L), first take all the training images that belong to L. Again, these are shown in Figure 3.</li>
 <li>For each image, determine whether the feature value is 1 (present) or 0 (absent), as described in (6.3.1)-(6.3.3). In our case, for the six images in Figure 3, the values are: [0, 0, 1, 0, 1, 0].</li>
 <li>Compute the fraction of images in which the feature value is 1. In our case, it is 2/6 = 0.33.</li>
 <li>Therefore, p(F = 1|C = L) = 0.33. Note that to obtain accurate estimates, one should use at least 30 images per class.</li>
 <li>In a similar manner, we can compute that p(F = 0|C = L) = 0.67, p(F = 1|C = K) = 0.83, p(F = 0|C = K) = 0.17.</li>
 </ol>
 </li>
 <li>Given these values, inference can be performed. Assume that we are given a new image (Figure 9), and the task is to determine its category label. This is performed as follows:
 <ol>
 <li>We determine whether the feature F is present in the image, as described in (6.3.1)-(6.3.3). In our case, the feature is present, so F = 1.</li>
 <li>Using equations (1) and (2), and the values computed in (6.7.5)-(6.7.6), we can calculate that p(C = K | F = 1) = 0.42 and p(C = L | F = 1) = 0.17.</li>
 <li>Given this information, it is rational to conclude that the image is from category K, with relatively low confidence.</li>
 </ol>
 </li>
 <li>It could be interesting and useful to test such an approach in a biological system (say, psychophysically). In this case, behavioral predictions could be made and compared against the computational results such as those obtained in (6.8.2). To make accurate predictions, good estimates of p(F |C) are necessary. Such estimates, however, may be difficult to obtain for familiar objects. The reason is that a subject's estimate of p(F |C) is not observable directly, and it is extremely difficult to compute it since the subject's prior exposure to natural images is uncontrollable and unknown. In contrast, if digital embryos are used for the experiments, then it is possible to precisely control which embryos and embryo categories the subject is exposed to. This makes it easy to compute the quantities of interest from exactly the same data the subject was exposed to. The fact that embryos' appearance is naturalistic is helpful, since it makes it likely that the subject is using the same categorization strategy as with natural categories.</li>
</ol>

<ol>
	<li>Palmeri, T. J., Gauthier, I. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Visual+object+understanding.">Visual object understanding.</a> Nat. Rev. Neurosci. 5, 291-303 (2004).</li><li>Seger, C. A., Miller, E. K. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Category+learning+in+the+brain.">Category learning in the brain.</a> Annu. Rev. Neurosci. 33, 203-219 (2010).</li><li>Fahle, M., Poggio, T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Perceptual Learning. , (2002).</li><li>Ashby, F. G., Maddox, W. T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Human+category+learning.">Human category learning.</a> Annu. Rev. Psychol. 56, 149-178 (2005).</li><li>Gauthier, I., Tarr, M. J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Becoming+a+"Greeble"+expert:+exploring+mechanisms+for+face+recognition.">Becoming a "Greeble" expert: exploring mechanisms for face recognition.</a> Vision Res. 37, 1673-1682 (1997).</li><li>Op de Beeck, H. P., Baker, C. I., DiCarlo, J. J., Kanwisher, N. G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Discrimination+training+alters+object+representations+in+human+extrastriate+cortex.">Discrimination training alters object representations in human extrastriate cortex.</a> J. Neurosci. 26, 13025-13036 (2006).</li><li>Cui, M. L., Copsey, L., Green, A. A., Bangham, J. A., Coen, E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Quantitative+control+of+organ+shape+by+combinatorial+gene+activity.">Quantitative control of organ shape by combinatorial gene activity.</a> PLoS Biol. 8, e1000538 (2010).</li><li>Green, A. A., Kennaway, J. R., Hanna, A. I., Bangham, J. A., Coen, E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Genetic+control+of+organ+shape+and+tissue+polarity.">Genetic control of organ shape and tissue polarity.</a> PLoS Biol. 8, e1000537 (2010).</li><li>Hegd&#233;, J., Bart, E., Kersten, D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Fragment-based+learning+of+visual+object+categories.">Fragment-based learning of visual object categories.</a> Curr. Biol. 18, 597-601 (2008).</li><li>Bart, E., Hegd&#233;, J., Kersten, D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Fragment-based+learning+of+visual+categories.">Fragment-based learning of visual categories.</a> , 121-12 (2008).</li><li>Ball, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Nature's patterns : a tapestry in three parts. , (2009).</li><li>Vuong, Q. C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Visual+categorization:+when+categories+fall+to+pieces.">Visual categorization: when categories fall to pieces.</a> Curr. Biol. 18, 427-429 (2008).</li><li>Kromrey, S., Maestri, M., Hauffen, K., Bart, E., Hegde, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Fragment-based+learning+of+visual+object+categories+in+non-human+primates.">Fragment-based learning of visual object categories in non-human primates.</a> PLoS One. 5, e15444 (2010).</li><li>Brady, M. J., Kersten, D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Bootstrapped+learning+of+novel+objects.">Bootstrapped learning of novel objects.</a> J. Vis. 3, 413-422 (2003).</li><li>Blanz, V., Vetter, T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+morphable+model+for+the+synthesis+of+3D+faces.">A morphable model for the synthesis of 3D faces.</a> SIGGAPH. 26, 187-194 (1999).</li><li>Freedman, D. J., Riesenhuber, M., Poggio, T., Miller, E. K. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Categorical+representation+of+visual+stimuli+in+the+primate+prefrontal+cortex.">Categorical representation of visual stimuli in the primate prefrontal cortex.</a> Science. 291, 312-316 (2001).</li><li>Lerios, A., Garfinkle, C. D., Levoy, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Feature-based+volume+metamorphosis.">Feature-based volume metamorphosis.</a> SIGGRAPH. , 449-456 (1995).</li><li>Bronstein, A. M., Bronstein, M. M., Kimmel, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Numerical geometry of non-rigid shapes. , (2008).</li><li>Lotka, A. J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Natural+Selection+as+a+Physical+Principle.">Natural Selection as a Physical Principle.</a> Proc. Natl. Acad. Sci. U.S.A. 8, 151-154 (1922).</li><li>Duda, R. O., Hart, P. E., Stork, D. G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Pattern classification. , (2001).</li><li>Bishop, C. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Pattern recognition and machine learning. , (2006).</li><li>Theodoridis, S., Koutroumbas, K. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Pattern recognition. , (2006).</li><li>Sokal, R. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Biometry+:+The+principles+and+practice+of+statistics+in+biological+research.">Biometry : The principles and practice of statistics in biological research.</a> , (2011).</li><li>Tuffery, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Data mining and statistics for decision making. , (2011).</li><li>Crawley, M. J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Statistical Computing: An Introduction to Data Analysis using S-Plus. , (2002).</li><li>Venables, W. N., Ripley, B. D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Modern Applied Statistics with S. , (2003).</li><li>Beier, T., Neely, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Feature-based+image+metamorphosis.">Feature-based image metamorphosis.</a> SIGGRAPH. 26, 35-42 (1992).</li><li>Kent, J. R., Carlson, W. E., Parent, R. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Shape+transformation+for+polyhedral+objects.">Shape transformation for polyhedral objects.</a> SIGGRAPH. 26, 47-54 (1992).</li><li>Gomes, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Warping and morphing of graphical objects. , (1999).</li><li>Ullman, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Object+recognition+and+segmentation+by+a+fragment-based+hierarchy.">Object recognition and segmentation by a fragment-based hierarchy.</a> Trends Cogn. Sci. 11, 58-64 (2007).</li><li>Kobatake, E., Tanaka, K. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Neuronal+selectivities+to+complex+object+features+in+the+ventral+visual+pathway+of+the+macaque+cerebral+cortex.">Neuronal selectivities to complex object features in the ventral visual pathway of the macaque cerebral cortex.</a> J. Neurophysiol. 71, 856-867 (1994).</li><li>Serre, T., Wolf, L., Poggio, T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Proceedings of 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR). , (2005).</li><li>Ullman, S., Vidal-Naquet, M., Sali, E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Visual+features+of+intermediate+complexity+and+their+use+in+classification.">Visual features of intermediate complexity and their use in classification.</a> Nat. Neurosci. 5, 682-687 (2002).</li><li>Davis, M. J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Computer graphics. , (2011).</li><li>Lengyel, E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Mathematics for 3D game programming and computer graphics, third edition. , (2011).</li><li>Gilbert, S. F. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Developmental biology. , (2010).</li><li>Gilbert, S. F., Epel, D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Ecological developmental biology : integrating epigenetics, medicine, and evolution. , (2009).</li><li>Hegd&#233;, J., Thompson, S. K., Brady, M. J., Kersten, D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Object+Recognition+in+Clutter:+Cortical+Responses+Depend+on+the+Type+of+Learning.">Object Recognition in Clutter: Cortical Responses Depend on the Type of Learning.</a> Frontiers in Human Neuroscience. 6, 170 (2012).</li><li>Mervis, C. B., Rosch, E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Categorization+of+natural+objects.">Categorization of natural objects.</a> Annual Review of Psychology. 32, 89-115 (1981).</li><li>Futuyma, D. J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Evolution. , (2009).</li></ol>

No conflicts of interest declared.

Usefulness of VM and VP in Cognitive Science Research
We have previously described the usefulness of VM and VP in detaill9,10,12-14. Briefly, VM, especially the digital embryo methodology, is useful because it provides a principled and flexible method for creating novel, but naturalistic 3-D objects14. Similarly, VP provides a principled method of creating naturalistic categories9,10,12,13. It is worth noting that object categories generated by VP share many features with object categories in nature, including the fact that the categories tend to be hierarchical in nature, and the feature variations within and across categories arise independently of the experimenter and the algorithms for classifying them39.
Current Limitations and Future Directions
Three current limitations of our protocol and the directions for future work they suggest are particularly noteworthy: First, both VM and VP simulate biological processes. While we show that non-biological virtual objects can be used as substrates for these processes, the underlying processes are still biologically motivated. However, natural objects - biological and non-biological alike - undergo shape changes due to non-biological forces. For instance, rocks may change in shape due to geological processes such as erosion or sedimentation. New categories of rock may arise from other such geological processes. It should be relatively straightforward to incorporate these processes into the repertoire of available shape change algorithms.
The second major limitation of our protocol is that its current repertoire of dynamic shape changes is rather limited. It is desirable to incorporate a larger array of shape changes, such as biological motion, or motion due to external forces such as wind, water or gravity. We expect that it will be relatively straightforward to bring to bear the known computer animation algorithms to implement such dynamic shape changes.
The third major limitation of our protocol is that VM currently does not include many other known morphogenetic processes including, most notably, gastrulation36. It also fails to incorporate some known constraints, such as the fact that morphogenesis in plants is mediated entirely by growth, with little or no cell movement possible, because of the cell walls36. Similarly, VP does not include other known phylogenetic processes such as genetic drift40. Addressing these limitations would help greatly facilitate the use of our protocol in developmental, ecological and evolutionary simulations.

<table border="1">
	<tbody>
		<tr>
			<td>Name of toolkit/equipment</td>
			<td>Company / Author</td>
			<td>Catalogue #</td>
			<td>Comments</td>
		</tr>
		<tr>
			<td>Digital Embryo Workshop (DEW)</td>
			<td>Mark Brady and Dan Gu</td>
			<td>&nbsp;</td>
			<td>This user-friendly, menu-driven tool can be downloaded free of charge as Download 1 from <a href="http://www.hegde.us/DigitalEmbryos" target="_blank">http://www.hegde.us/DigitalEmbryos</a>. Currently available only for Windows.</td>
		</tr>
		<tr>
			<td>Digital embryo tools for Cygwin</td>
			<td>Jay Hegd&eacute; and Karin Hauffen</td>
			<td>&nbsp;</td>
			<td>This is a loose collection of not-so-user-friendly programs. They are designed to be run from the command-line interface of the Cygwin Linux emulator for Windows. These programs can be downloaded as Download 2 from <a href="http://www.hegde.us/DigitalEmbryos" target="_blank">http://www.hegde.us/DigitalEmbryos</a>. The Cygwin interface itself can be downloaded free of charge from <a href="http://www.cygwin.com" target="_blank">www.cygwin.com</a>.</td>
		</tr>
		<tr>
			<td>Autodesk 3ds Max, Montreal, Quebec, Canada</td>
			<td>Autodesk Media and Entertainment</td>
			<td>3DS Max</td>
			<td>This is a 3-D modeling, animation and rendering toolkit with a flexible plugin architecture and a built-in scripting language. Available for most of the current operating systems.</td>
		</tr>
		<tr>
			<td>MATLAB</td>
			<td>Mathworks Inc., Natick, MA, USA</td>
			<td>MATLAB</td>
			<td>This is a numerical computing environment and programming language with many useful add-on features. Available for most of the current operating systems.</td>
		</tr>
		<tr>
			<td>R statistical toolkit</td>
			<td>R Project for Statistical Computing</td>
			<td>R</td>
			<td>Can be downloaded free of charge from <a href="http://www.r-project.org/" target="_blank">http://www.r-project.org/</a>. Available for most of the current operating systems.</td>
		</tr>
		<tr>
			<td>OpenGL</td>
			<td>Khronos Group</td>
			<td>OpenGL</td>
			<td>This cross-language, cross-platform graphical toolkit can be downloaded free of charge from <a href="http://www.opengl.org" target="_blank">www.opengl.org</a>.</td>
		</tr>
		<tr>
			<td>V-Flash Personal Printer</td>
			<td>3D Systems Inc., Rock Hill, SC, USA</td>
			<td>V-Flash</td>
			<td>This is a good value for all 3-D printing applications described in this report. The print materials are also vended by 3D Systems, Inc. Less expensive models are available in open source form from RepRap (rapmanusa.com) and MakerGear. More expensive models (&gt; $30 K) are available from Objet Geometries, 3DS Systems, Z-Corp, Dimension Printing etc.</td>
		</tr>
		<tr>
			<td><a href="http://www.TurboSquid.com" target="_blank"> TurboSquid.com</a></td>
			<td>TurboSquid Inc., New York, LA</td>
			<td>(various objects)</td>
			<td>Various virtual 3-D objects can be downloaded from this site free of charge or for a fee.</td>
		</tr>
		<tr>
			<td>&nbsp;</td>
			<td>&nbsp;</td>
			<td>&nbsp;</td>
			<td>Table 1. Table Of Specific Toolkits And Equipment.</td>
		</tr>
	</tbody>
</table>

creating objects and object categories for studying perception and perceptual learning

In order to quantitatively study object perception, be it perception by biological systems or by machines, one needs to create objects and object categories with precisely definable, preferably naturalistic, properties1. Furthermore, for studies on perceptual learning, it is useful to create novel objects and object categories (or object classes) with such properties2.
Many innovative and useful methods currently exist for creating novel objects and object categories3-6 (also see refs. 7,8). However, generally speaking, the existing methods have three broad types of shortcomings.
First, shape variations are generally imposed by the experimenter5,9,10, and may therefore be different from the variability in natural categories, and optimized for a particular recognition algorithm. It would be desirable to have the variations arise independently of the externally imposed constraints.
Second, the existing methods have difficulty capturing the shape complexity of natural objects11-13. If the goal is to study natural object perception, it is desirable for objects and object categories to be naturalistic, so as to avoid possible confounds and special cases.
Third, it is generally hard to quantitatively measure the available information in the stimuli created by conventional methods. It would be desirable to create objects and object categories where the available information can be precisely measured and, where necessary, systematically manipulated (or 'tuned'). This allows one to formulate the underlying object recognition tasks in quantitative terms.
Here we describe a set of algorithms, or methods, that meet all three of the above criteria. Virtual morphogenesis (VM) creates novel, naturalistic virtual 3-D objects called 'digital embryos' by simulating the biological process of embryogenesis14. Virtual phylogenesis (VP) creates novel, naturalistic object categories by simulating the evolutionary process of natural selection9,12,13. Objects and object categories created by these simulations can be further manipulated by various morphing methods to generate systematic variations of shape characteristics15,16. The VP and morphing methods can also be applied, in principle, to novel virtual objects other than digital embryos, or to virtual versions of real-world objects9,13. Virtual objects created in this fashion can be rendered as visual images using a conventional graphical toolkit, with desired manipulations of surface texture, illumination, size, viewpoint and background. The virtual objects can also be 'printed' as haptic objects using a conventional 3-D prototyper.
We also describe some implementations of these computational algorithms to help illustrate the potential utility of the algorithms. It is important to distinguish the algorithms from their implementations. The implementations are demonstrations offered solely as a 'proof of principle' of the underlying algorithms. It is important to note that, in general, an implementation of a computational algorithm often has limitations that the algorithm itself does not have.
Together, these methods represent a set of powerful and flexible tools for studying object recognition and perceptual learning by biological and computational systems alike. With appropriate extensions, these methods may also prove useful in the study of morphogenesis and phylogenesis.

VM can be used to generate a virtually limitless supply of novel 3-D shapes. Some exemplar digital embryos generated using the VM algorithm are shown in the bottom panel of Figure 1. Each of these 16 embryos were generated by using the 'growEmbryos.exe' program in the Digital embryo tools for Cygwin (see Table 1) for 40 growth cycles. All other growth parameters were set internally by the program. Most of these parameters were constant (i.e., identical from one embryo to the next). A few parameters such as the location and strength of morphogen sources, were random parameters set internally by the program independently for each run. The shape variations among these 16 embryos arose solely as a result of the variations in these random parameters.
Some examples of surface texturing34,35 using some arbitrarily chosen textures are shown in Figure 2A. Visual scenes of arbitrary complexity can be created using a commercially available 3-D modeling and rendering environment, as shown in Figure 2B.
A representative 'family tree' generated by VP using digital embryos is shown in Figure 3. Comparable trees can also be constructed using objects other than digital embryos, as shown in Figure 4. Note that in either case, the objects that share a common ancestor straightforwardly constitute a category, although the experimenter may also choose to define a category as any other set of objects. It is worth noting from Figure 4 that our current implementation of the VM and VP algorithms tends to produce relatively smooth, curved surfaces, as opposed to jagged or flat objects. It is also worth noting that this is presumably the limitation of our implementation of these algorithms and not the algorithms themselves, since biolological processes can produce objects with flat surfaces and jagged outlines (e.g., rose leaf).
Figures 5 and 6 illustrate the typical results of two methods that can be used in addition to, or instead of, VP for creating principled variations in object shape and object categories.
The top panel of Figure 7 illustrates visual renderings of two digital embryos, and the bottom panel of Figure 7 illustrates the corresponding printouts generated by a commercially available 3-D prototyper.
Figures 8 and 9 illustrate the procedures described in Section 6 for using image fragments to categorize a given visual object.
<img alt="Figure 1" fo:content-width="4in" fo:src="/files/ftp_upload/3358/3358fig1highres.jpg" src="/files/ftp_upload/3358/3358fig1.jpg" /> 
 Figure 1. Virtual morphogenesis. The bottom panel illustrates a type of novel, naturalistic, virtual 3-D objects called "digital embryos"14. Digital embryos can be generated by simulating one or more of some of the key processes of biological embryogenesis: morphogen-mediated cell division, cell growth, cell movement and programmed cell death7,8,36,37. Each run starts with an icosahedron (shown in the top panel), and generates a unique embryo, depending on the VM parameter settings (or the 'genotype') of that embryo. Thus, the 16 embryos in the bottom panel have different shapes, because they all have different genotypes. Note that simpler or more complex shapes can be generated as needed (e.g., to optimally stimulate neurons at a given level of the visual hierarchy) by manipulating the genotype of the embryo. All of the aforementioned embryogenetic processes except programmed cell death were simulated in generating the embryos shown. Simulated programmed cell death is especially useful for creating targeted indentations (not shown).
<img alt="Figure 2" fo:content-width="3.6in" fo:src="/files/ftp_upload/3358/3358fig2highres.jpg" src="/files/ftp_upload/3358/3358fig2.jpg" /> 
 Figure 2. Creating visual stimuli using digital embryos. Like any virtual 3-D object, digital embryos can be graphically manipulated to create visual scenes of arbitrary complexity using any standard 3-D graphical toolkit. This figure illustrates some common manipulations. (A) The same digital embryo is textured using many different textures, and lit from an invisible light source at top left. (B) A camouflaged scene is created by resizing and re-orienting the digital embryo and digitally placing it against the same background it was textured with. The digital embryo can be found in 'plain view' in the lower right quadrant. For additional examples of visual stimuli created using digital embryos, see refs. 9,10,12-14,38.
<img alt="Figure 3" fo:content-width="5in" fo:src="/files/ftp_upload/3358/3358fig3highres.jpg" src="/files/ftp_upload/3358/3358fig3.jpg" /> 
 Figure 3. Creating digital embryo categories using VP. The VP algorithm emulates biological evolution, in that in both cases, novel objects and object categories emerge as heritable variations accumulate selectively. At each generation Gi , selected embryos procreate, leading to generation Gi+1. The progeny inherit the shape characteristics of their parent, but accrue shape variations of their own (as determined by small variations in their genotype) as they develop. This figure shows a 'family tree' of three generations of descendants starting from a single common ancestor, an icosahedron. Note that, in this case, the shape complexity increases from the icosahedron to generation G1, but not from G1 onward. This is because increase in cell numbers (i.e., cell division) was allowed from the icosahedron to generation G1, but not from G1 onward. In general, cell division tends to increase shape complexity, whereas other morphogenetic processes such as cell movement and cell growth change shape without changing the overall complexity of the shape.
<img alt="Figure 4" fo:content-width="6in" fo:src="/files/ftp_upload/3358/3358fig4highres.jpg" src="/files/ftp_upload/3358/3358fig4.jpg" /> 
 Figure 4. VP using virtual objects other than digital embryos. This figure helps illustrate the general principle that virtual objects other than digital embryos can be used as input to VP. The VP algorithm in its current form can operate on any virtual 3-D object whose surface consists solely of triangles. Generation G1 comprised of (from left to right) a gourd, diamond, face mask, apple, rock, and cactus. Note that the objects in generation G1 in this figure do not have a common ancestor, because VP does not require it. Objects in G2 and G3 represent the descendants of the rock in G1. No cell divisions were allowed in any generation, so that all shape variations arose solely from the movement and/or growth of the individual 'cells' of the given object.
<img alt="Figure 5" fo:content-width="6.5in" fo:src="/files/ftp_upload/3358/3358fig5highres.jpg" src="/files/ftp_upload/3358/3358fig5.jpg" /> 
 Figure 5. Using morphing to create smooth variations in shape. Morphing involves taking two given objects (far left and far right embryo in this figure) and calculating the intermediate objects (intervening embryos) by interpolating between the corresponding vertices of the two designated objects. In the case shown, all vertices were interpolated using the same scalar factor, resulting in a linear morphing. However, it is also possible to morph the objects non-linearly (not shown). Morphing is computationally straightforward when there is an exact one-to-one correspondence between the vertices of two objects, as in the case shown. However it is possible, in principle, to morph between any two given virtual objects regardless of whether their vertices correspond exactly, although there is no unique principled method for doing so17,18.
<img alt="Figure 6" fo:content-width="6in" fo:src="/files/ftp_upload/3358/3358fig6highres.jpg" src="/files/ftp_upload/3358/3358fig6.jpg" /> 
 Figure 6. Using principal components to create smooth variations in shape. (A) Average embryo. This embryo represents the arithmetic average of 400 embryos (200 each from categories K and L in Figure 3). Principal components were calculated as described in Step 4.3. Note that principal components represent mutually independent, abstract shape dimensions of the 400 embryos (not shown)25,26. 400 embryos yield 399 non-zero principal components25,26, which together account for all the variance, or the shape information, available collectively in the embryos. By convention, principal components are arranged in the decreasing order of their eigenvalues, or the proportion of the overall variance they explain25,26. In this case, the first two principal components respectively accounted for 73% and 19% of the shape information available in the 400 embryos. (B) Embryos that represent different weights (or more precisely, weighted eigenvalues) of Principal Component 1. The weights varied from +2 (far left) to -2 (far right) in equal steps of -0.2. (C) Embryos that represent different weights of Principal Component 2. The weights also varied from +2 (far left) to -2 (far right) in equal steps of -0.2. Note that manipulating principal components does not exclusively manipulate any given specific body part of the embryo (e.g., the wings of the embryo in the case shown). However, if necessary, body parts of virtual 3-D objects can be manipulated in any arbitrary user-defined fashion using most of the commercially available 3-D modeling environments (not shown).
<img alt="Figure 7" fo:content-width="4in" fo:src="/files/ftp_upload/3358/3358fig7highres.jpg" src="/files/ftp_upload/3358/3358fig7.jpg" /> 
 Figure 7. Creating haptic objects. Virtual 3-D objects can be 'printed' as haptic objects using a standard, commercially available 3-D 'printer' or prototyper. This figure shows digital embryos rendered as visual objects (top row) or as the corresponding haptic objects (bottom row). The haptic objects shown in this figure were printed to be about 6 cm wide (scale bar = 1 cm), although the objects can be printed at much smaller or larger sizes.
<img alt="Figure 8" src="/files/ftp_upload/3358/3358fig8.jpg" /> 
 Figure 8. A template for an example informative fragment. In this example, the template has a threshold of 0.69 associated with it.
<img alt="Figure 9" fo:content-width="2.5in" fo:src="/files/ftp_upload/3358/3358fig9highres.jpg" src="/files/ftp_upload/3358/3358fig9.jpg" /> 
 Figure 9. A new image for which the object category is not known and needs to be determined.

Watch this Scientific Journal Video about Creating Objects and Object Categories for Studying Perception and Perceptual Learning at JoVE.com

Creating Objects and Object Categories for Studying Perception and Perceptual Learning

We describe a novel methodology for creating naturalistic 3-D objects and object categories with precisely defined feature variations. We use simulations of the biological processes of morphogenesis and phylogenesis to create novel, naturalistic virtual 3-D objects and object categories that can then be rendered as visual images or haptic objects.

In order to quantitatively study object perception, be it perception by biological systems or by machines, one needs to create objects and object ...

creating-objects-object-categories-for-studying-perception-perceptual

Research

JoVE Journal

Neuroscience

C. elegans Positive Butanone Learning, Short-term, and Long-term Associative Memory Assays

The memory of experiences and learned information is critical for organisms to make choices that aid their survival. C. elegans navigates its environment through neuron-specific detection of food and chemical odors1, 2, and can associate nutritive states with chemical odors3, temperature4, and the pathogenicity of a food source5.
Here, we describe assays of C. elegans associative learning and short- and long-term associative memory. We modified an aversive olfactory learning paradigm6 to instead produce a positive response; the assay involves starving ~400 worms, then feeding the worms in the presence of the AWC neuron-sensed volatile chemoattractant butanone at a concentration that elicits a low chemotactic index (similar to Toroyama et al.7). A standard population chemotaxis assay1 tests the worms' attraction to the odorant immediately or minutes to hours after conditioning.
After conditioning, wild-type animals' chemotaxis to butanone increases ~0.6 Chemotaxis Index units, its "Learning Index". Associative learning is dependent on the presence of both food and butanone during training. Pairing food and butanone for a single conditioning period ("massed training") produces short-term associative memory that lasts ~2 hours. Multiple conditioning periods with rest periods between ("spaced training") yields long-term associative memory (&lt;40 hours), and is dependent on the cAMP Response Element Binding protein (CREB),6 a transcription factor required for long-term memory across species.8
Our protocol also includes image analysis methods for quick and accurate determination of chemotaxis indices. High-contrast images of animals on chemotaxis assay plates are captured and analyzed by worm counting software in MatLab. The software corrects for uneven background using a morphological tophat transformation.9 Otsu's method is then used to determine a threshold to separate worms from the background.10 Very small particles are removed automatically and larger non-worm regions (plate edges or agar punches) are removed by manual selection. The software then estimates the size of single worm by ignoring regions that are above a specified maximum size and taking the median size of the remaining regions. The number of worms is then estimated by dividing the total area identified as occupied by worms by the estimated size of a single worm.
We have found that learning and short- and long-term memory can be distinguished, and that these processes share similar key molecules with higher organisms.6,8 Our assays can quickly test novel candidate genes or molecules that affect learning and short- or long-term memory in C. elegans that are relevant across species.

C. elegans Positive Butanone Learning, Short-term, and Long-term Associative Memory Assays

Here we describe methods to test C. elegans associative learning and short- and long-term associative memory. These population assays employ the worms abilities to chemotax toward volatile odorants, and form positive associations upon pairing food with the chemoattractant butanone. Increasing the number of conditioning periods induces long-term memory.

The memory of experiences and learned information is critical for organisms to make choices that aid their survival. C. elegans navigates its environment ...

Using Affordable LED Arrays for Photo-Stimulation of Neurons

Standard slice electrophysiology has allowed researchers to probe individual components of neural circuitry by recording electrical responses of single cells in response to electrical or pharmacological manipulations1,2. With the invention of methods to optically control genetically targeted neurons (optogenetics), researchers now have an unprecedented level of control over specific groups of neurons in the standard slice preparation. In particular, photosensitive channelrhodopsin-2 (ChR2) allows researchers to activate neurons with light3,4. By combining careful calibration of LED-based photostimulation of ChR2 with standard slice electrophysiology, we are able to probe with greater detail the role of adult-born interneurons in the olfactory bulb, the first central relay of the olfactory system. Using viral expression of ChR2-YFP specifically in adult-born neurons, we can selectively control young adult-born neurons in a milieu of older and mature neurons. Our optical control uses a simple and inexpensive LED system, and we show how this system can be calibrated to understand how much light is needed to evoke spiking activity in single neurons. Hence, brief flashes of blue light can remotely control the firing pattern of ChR2-transduced newborn cells.

Adult-born neurons expressing ChR2 can be manipulated in slice electrophysiological preparations in order to examine their contribution towards the function of olfactory neural circuits.

Standard slice electrophysiology has allowed researchers to probe individual components of neural circuitry by recording electrical responses of single ...

Selective Viral Transduction of Adult-born Olfactory Neurons for Chronic in vivo Optogenetic Stimulation

Local interneurons are continuously regenerated in the olfactory bulb of adult rodents1-3. In this process, called adult neurogenesis, neural stem cells in the walls of the lateral ventricle give rise to neuroblasts that migrate for several millimeters along the rostral migratory stream (RMS) to reach and incorporate into the olfactory bulb. To study the different steps and the impact of adult-born neuron integration into preexisting olfactory circuits, it is necessary to selectively label and manipulate the activity of this specific population of neurons. The recent development of optogenetic technologies offers the opportunity to use light to precisely activate this specific cohort of neurons without affecting surrounding neurons4,5. Here, we present a series of procedures to virally express Channelrhodopsin2(ChR2)-YFP in a temporally restricted cohort of neuroblasts in the RMS before they reach the olfactory bulb and become adult-born neurons. In addition, we show how to implant and calibrate a miniature LED for chronic in vivo stimulation of ChR2-expressing neurons.

Selective Viral Transduction of Adult-born Olfactory Neurons for Chronic in vivo Optogenetic Stimulation

Adult-born neurons of the olfactory bulb can be optogenetically controlled using Channelrhodopsin2-expressing lentiviral injection in the rostral migratory stream and chronic photostimulation with an implanted miniature LED.

Local interneurons are continuously regenerated in the olfactory bulb of adult rodents1-3. In this process, called adult neurogenesis, neural stem cells ...

MPI CyberMotion Simulator: Implementation of a Novel Motion Simulator to Investigate Multisensory Path Integration in Three Dimensions

Path integration is a process in which self-motion is integrated over time to obtain an estimate of one's current position relative to a starting point 1. Humans can do path integration based exclusively on visual 2-3, auditory 4, or inertial cues 5. However, with multiple cues present, inertial cues - particularly kinaesthetic - seem to dominate 6-7. In the absence of vision, humans tend to overestimate short distances (&lt;5 m) and turning angles (&lt;30&deg;), but underestimate longer ones 5. Movement through physical space therefore does not seem to be accurately represented by the brain.
Extensive work has been done on evaluating path integration in the horizontal plane, but little is known about vertical movement (see 3 for virtual movement from vision alone). One reason for this is that traditional motion simulators have a small range of motion restricted mainly to the horizontal plane. Here we take advantage of a motion simulator 8-9 with a large range of motion to assess whether path integration is similar between horizontal and vertical planes. The relative contributions of inertial and visual cues for path navigation were also assessed.
16 observers sat upright in a seat mounted to the flange of a modified KUKA anthropomorphic robot arm. Sensory information was manipulated by providing visual (optic flow, limited lifetime star field), vestibular-kinaesthetic (passive self motion with eyes closed), or visual and vestibular-kinaesthetic motion cues. Movement trajectories in the horizontal, sagittal and frontal planes consisted of two segment lengths (1st: 0.4 m, 2nd: 1 m; &plusmn;0.24 m/s2 peak acceleration). The angle of the two segments was either 45&deg; or 90&deg;. Observers pointed back to their origin by moving an arrow that was superimposed on an avatar presented on the screen.
Observers were more likely to underestimate angle size for movement in the horizontal plane compared to the vertical planes. In the frontal plane observers were more likely to overestimate angle size while there was no such bias in the sagittal plane. Finally, observers responded slower when answering based on vestibular-kinaesthetic information alone. Human path integration based on vestibular-kinaesthetic information alone thus takes longer than when visual information is present. That pointing is consistent with underestimating and overestimating the angle one has moved through in the horizontal and vertical planes respectively, suggests that the neural representation of self-motion through space is non-symmetrical which may relate to the fact that humans experience movement mostly within the horizontal plane.

An efficient way to gain insight into how humans navigate themselves in three dimensions is described. The method takes advantage of a motion simulator capable of moving observers in ways unattainable by traditional simulators. Results confirm that movement in the horizontal plane is underestimated, while vertical movement is overestimated.

Path integration is a process in which self-motion is integrated over time to obtain an estimate of one's current position relative to a starting point 1. ...

High-resolution Functional Magnetic Resonance Imaging Methods for Human Midbrain

Functional MRI (fMRI) is a widely used tool for non-invasively measuring correlates of human brain activity. However, its use has mostly been focused upon measuring activity on the surface of cerebral cortex rather than in subcortical regions such as midbrain and brainstem. Subcortical fMRI must overcome two challenges: spatial resolution and physiological noise. Here we describe an optimized set of techniques developed to perform high-resolution fMRI in human SC, a structure on the dorsal surface of the midbrain; the methods can also be used to image other brainstem and subcortical structures.
High-resolution (1.2 mm voxels) fMRI of the SC requires a non-conventional approach. The desired spatial sampling is obtained using a multi-shot (interleaved) spiral acquisition1. Since, T2* of SC tissue is longer than in cortex, a correspondingly longer echo time (TE ~ 40 msec) is used to maximize functional contrast. To cover the full extent of the SC, 8-10 slices are obtained. For each session a structural anatomy with the same slice prescription as the fMRI is also obtained, which is used to align the functional data to a high-resolution reference volume.
In a separate session, for each subject, we create a high-resolution (0.7 mm sampling) reference volume using a T1-weighted sequence that gives good tissue contrast. In the reference volume, the midbrain region is segmented using the ITK-SNAP software application2. This segmentation is used to create a 3D surface representation of the midbrain that is both smooth and accurate3. The surface vertices and normals are used to create a map of depth from the midbrain surface within the tissue4.
Functional data is transformed into the coordinate system of the segmented reference volume. Depth associations of the voxels enable the averaging of fMRI time series data within specified depth ranges to improve signal quality. Data is rendered on the 3D surface for visualization.
In our lab we use this technique for measuring topographic maps of visual stimulation and covert and overt visual attention within the SC1. As an example, we demonstrate the topographic representation of polar angle to visual stimulation in SC.

This article describes techniques to perform high-resolution functional magnetic resonance imaging with 1.2 mm sampling in human midbrain and subcortical structures using a 3T scanner. Use of these techniques to resolve topographic maps of visual stimulation in the human superior colliculus (SC) is given as an example.

Functional MRI (fMRI) is a widely used tool for non-invasively measuring correlates of human brain activity. However, its use has mostly been focused upon ...

Optical Recording of Suprathreshold Neural Activity with Single-cell and Single-spike Resolution

Signaling of information in the vertebrate central nervous system is often carried by populations of neurons rather than individual neurons. Also propagation of suprathreshold spiking activity involves populations of neurons. Empirical studies addressing cortical function directly thus require recordings from populations of neurons with high resolution. Here we describe an optical method and a deconvolution algorithm to record neural activity from up to 100 neurons with single-cell and single-spike resolution. This method relies on detection of the transient increases in intracellular somatic calcium concentration associated with suprathreshold electrical spikes (action potentials) in cortical neurons. High temporal resolution of the optical recordings is achieved by a fast random-access scanning technique using acousto-optical deflectors (AODs)1. Two-photon excitation of the calcium-sensitive dye results in high spatial resolution in opaque brain tissue2. Reconstruction of spikes from the fluorescence calcium recordings is achieved by a maximum-likelihood method. Simultaneous electrophysiological and optical recordings indicate that our method reliably detects spikes (&gt;97% spike detection efficiency), has a low rate of false positive spike detection (&lt; 0.003 spikes/sec), and a high temporal precision (about 3 msec) 3. This optical method of spike detection can be used to record neural activity in vitro and in anesthetized animals in vivo3,4.

Understanding the function of the vertebrate central nervous system requires recordings from many neurons because cortical function arises on the level of populations of neurons. Here we describe an optical method to record suprathreshold neural activity with single-cell and single-spike resolution, dithered random-access scanning. This method records somatic fluorescence calcium signals from up to 100 neurons with high temporal resolution. A maximum-likelihood algorithm deconvolves the underlying suprathreshold neural activity from the somatic fluorescence calcium signals. This method reliably detects spikes with high detection efficiency and a low rate of false positives and can be used to study neural populations in vitro and in vivo.

Signaling of information in the vertebrate central nervous system is often carried by populations of neurons rather than individual neurons. Also ...

Mapping Cortical Dynamics Using Simultaneous MEG/EEG and Anatomically-constrained Minimum-norm Estimates: an Auditory Attention Example

Magneto- and electroencephalography (MEG/EEG) are neuroimaging techniques that provide a high temporal resolution particularly suitable to investigate the cortical networks involved in dynamical perceptual and cognitive tasks, such as attending to different sounds in a cocktail party. Many past studies have employed data recorded at the sensor level only, i.e., the magnetic fields or the electric potentials recorded outside and on the scalp, and have usually focused on activity that is time-locked to the stimulus presentation. This type of event-related field / potential analysis is particularly useful when there are only a small number of distinct dipolar patterns that can be isolated and identified in space and time. Alternatively, by utilizing anatomical information, these distinct field patterns can be localized as current sources on the cortex. However, for a more sustained response that may not be time-locked to a specific stimulus (e.g., in preparation for listening to one of the two simultaneously presented spoken digits based on the cued auditory feature) or may be distributed across multiple spatial locations unknown a priori, the recruitment of a distributed cortical network may not be adequately captured by using a limited number of focal sources.
Here, we describe a procedure that employs individual anatomical MRI data to establish a relationship between the sensor information and the dipole activation on the cortex through the use of minimum-norm estimates (MNE). This inverse imaging approach provides us a tool for distributed source analysis. For illustrative purposes, we will describe all procedures using FreeSurfer and MNE software, both freely available. We will summarize the MRI sequences and analysis steps required to produce a forward model that enables us to relate the expected field pattern caused by the dipoles distributed on the cortex onto the M/EEG sensors. Next, we will step through the necessary processes that facilitate us in denoising the sensor data from environmental and physiological contaminants. We will then outline the procedure for combining and mapping MEG/EEG sensor data onto the cortical space, thereby producing a family of time-series of cortical dipole activation on the brain surface (or "brain movies") related to each experimental condition. Finally, we will highlight a few statistical techniques that enable us to make scientific inference across a subject population (i.e., perform group-level analysis) based on a common cortical coordinate space.

We use magneto- and electroencephalography (MEG/EEG), combined with anatomical information captured by magnetic resonance imaging (MRI), to map the dynamics of the cortical network associated with auditory attention.

Magneto- and electroencephalography (MEG/EEG) are neuroimaging techniques that provide a high temporal resolution particularly suitable to investigate the ...

Drosophila Adult Olfactory Shock Learning

Drosophila have been used in classical conditioning experiments for over 40 years, thus greatly facilitating our understanding of memory, including the elucidation of the molecular mechanisms involved in cognitive diseases1-7. Learning and memory can be assayed in larvae to study the effect of neurodevelopmental genes8-10 and in flies to measure the contribution of adult plasticity genes1-7. Furthermore, the short lifespan of Drosophila facilitates the analysis of genes mediating age-related memory impairment5,11-13. The availability of many inducible promoters that subdivide the Drosophila nervous system makes it possible to determine when and where a gene of interest is required for normal memory as well as relay of different aspects of the reinforcement signal3,4,14,16.
Studying memory in adult Drosophila allows for a detailed analysis of the behavior and circuitry involved and a measurement of long-term memory15-17. The length of the adult stage accommodates longer-term genetic, behavioral, dietary and pharmacological manipulations of memory, in addition to determining the effect of aging and neurodegenerative disease on memory3-6,11-13,15-21.
Classical conditioning is induced by the simultaneous presentation of a neutral odor cue (conditioned stimulus, CS+) and a reinforcement stimulus, e.g., an electric shock or sucrose, (unconditioned stimulus, US), that become associated with one another by the animal1,16. A second conditioned stimulus (CS-) is subsequently presented without the US. During the testing phase, Drosophila are simultaneously presented with CS+ and CS- odors. After the Drosophila are provided time to choose between the odors, the distribution of the animals is recorded. This procedure allows associative aversive or appetitive conditioning to be reliably measured without a bias introduced by the innate preference for either of the conditioned stimuli. Various control experiments are also performed to test whether all genotypes respond normally to odor and reinforcement alone.

Drosophila Adult Olfactory Shock Learning

The method to measure adult Drosophila associative memory is described. The assay is based on the ability of the fly to associate an odor presented with a negative reinforcer (electric shock) and then recall this information at a later time, allowing memory to be measured.

Drosophila have been used in classical conditioning experiments for over 40 years, thus greatly facilitating our understanding of memory, including the ...

Whole Mount Immunolabeling of Olfactory Receptor Neurons in the Drosophila Antenna

Odorant molecules bind to their target receptors in a precise and coordinated manner. Each receptor recognizes a specific signal and relays this information to the brain. As such, determining how olfactory information is transferred to the brain, modifying both perception and behavior, merits investigation. Interestingly, there is emerging evidence that cellular transduction and transcriptional factors are involved in the diversification of olfactory receptor neuron. Here we provide a robust whole mount immunological labeling method to assay in vivo olfactory receptor neuron organization. Using this method, we identified all olfactory receptor neurons with anti-ELAV antibody, a known pan-neural marker and Or49a-mCD8::GFP, an olfactory receptor neuron specifically expressed in Nba neuron using anti-GFP antibody.

Whole Mount Immunolabeling of Olfactory Receptor Neurons in the Drosophila Antenna

Herein we describe the process of whole mount immunostaining of Drosophila antennae, which enables us to better understand the molecular mechanisms involved in the diversification of olfactory receptor neurons (ORN)s.

Odorant molecules bind to their target receptors in a precise and coordinated manner. Each receptor recognizes a specific signal and relays this ...

Electrophysiological Method for Recording Intracellular Voltage Responses of Drosophila Photoreceptors and Interneurons to Light Stimuli In Vivo

Voltage responses of insect photoreceptors and visual interneurons can be accurately recorded with conventional sharp microelectrodes. The method described here enables the investigator to measure long-lasting (from minutes to hours) high-quality intracellular responses from single Drosophila R1-R6 photoreceptors and Large Monopolar Cells (LMCs) to light stimuli. Because the recording system has low noise, it can be used to study variability among individual cells in the fly eye, and how their outputs reflect the physical properties of the visual environment. We outline all key steps in performing this technique. The basic steps in constructing an appropriate electrophysiology set-up for recording, such as design and selection of the experimental equipment are described. We also explain how to prepare for recording by making appropriate (sharp) recording and (blunt) reference electrodes. Details are given on how to fix an intact fly in a bespoke fly-holder, prepare a small window in its eye and insert a recording electrode through this hole with minimal damage. We explain how to localize the center of a cell's receptive field, dark- or light-adapt the studied cell, and to record its voltage responses to dynamic light stimuli. Finally, we describe the criteria for stable normal recordings, show characteristic high-quality voltage responses of individual cells to different light stimuli, and briefly define how to quantify their signaling performance. Many aspects of the method are technically challenging and require practice and patience to master. But once learned and optimized for the investigator's experimental objectives, it grants outstanding in vivo neurophysiological data.

Electrophysiological Method for Recording Intracellular Voltage Responses of Drosophila Photoreceptors and Interneurons to Light Stimuli In Vivo

Sharp microelectrodes enable accurate electrophysiological characterization of photoreceptor and visual interneuron output in living Drosophila. Here we show how to use this method to record high-quality voltage responses of individual cells to controlled light stimulation. This method is ideal for studying neural information processing in insect compound eyes.