Name: design and use of an apparatus for presenting graspable objects in 3d workspace
Uploaded: 2019-08-08T14:45:00.000+00:00
Duration: 9 min 11 s
Description: Watch this Scientific Journal Video about Design and Use of an Apparatus for Presenting Graspable Objects in 3D Workspace at JoVE.com

We thank Mr. Shijiang Shen for his advice on apparatus design and Ms. Guihua Wang for her assistance with animal care and training. This work was supported by National Key Research and Development Program of China (2017YFC1308501), the National Natural Science Foundation of China (31627802), the Public Projects of Zhejiang Province (2016C33059), and the Fundamental Research Funds for the Central Universities.

All behavioral and surgical procedures conformed to the Guide for the Care and Use of Laboratory Animals (China Ministry of Health) and were approved by the Animal Care Committee at Zhejiang University, China.
1.Assembling the 3D translational device
<ol>
	<li>Build a frame of size 920 mm x 690 mm x 530 mm with aluminum construction rails (cross section: 40 mm x 40 mm).</li>
	<li>Secure four pedestals to the two ends of the Y-rails with screws (M4) (Figure 1B).</li>
	<li>Fix two Y-rails onto the top surface of the frame in parallel by securing the four pedestals to the four corners of the top surface with screws (M6) (Figure 1B).</li>
	<li>Connect two Y-rails with a connecting shaft and two diaphragm couplings. Tighten the lock screws of couplings to synchronize the shafts of two rails (Figure 1B).</li>
	<li>Put six nuts (M4) into the back grooves of the Z-rail. Attach one side of the right triangle frame to the back of the Z-rail with screws.</li>
	<li>Pull the triangle frame to the end that is distal to the shaft and tighten the screws. Attach the other right triangle frame to the other Z-rail in the same way (Figure 1C).</li>
	<li>Secure the other right-angled sides of two triangle frames to the sliders of two Y-rails with screws (M6) (Figure 1C).</li>
	<li>Connect two Z-rails with a connecting shaft and diaphragm couplings and tighten the lock screws of coupling (Figure 1C).</li>
	<li>Attach the two T-shaped connecting boards to the back of the X-rail with nuts and screws (M4). Then pull the two T-shaped boards to the two ends of X-rail and tighten the screws (Figure 1D).</li>
	<li>Secure the two T-shaped connecting boards onto the sliders of two Z-rails with screws (M6), respectively (Figure 1D).</li>
	<li>Insert the stepping motor into the shaft hole of the gear reducer and screw their flanges together&#160;(Figure 1E).</li>
	<li>Secure the connecting ring to the shaft end of the active X-rail with screws (M4).</li>
	<li>Insert the shaft of X-rail into the coupling and fix the gear reducer to the connecting ring with screws (M4).&#160; Tighten the lock screws of the coupling (Figure 1E).</li>
	<li>Fix the other two stepping motors and gear reducers to the active Y-rail and Z-rail using the methods described in steps 1.11&#8211;1.12.</li>
	<li>Insert the power and control cables of the three stepping&#160;motors to the power and control ports of their&#160;drivers, respectively and secure the cables with screws on the driver side.</li>
</ol>
2. Assembling the turning table
<ol>
	<li>Download the .DWG design files from the Supplemental Files of this paper. Prepare the objects, mental shaft, locating bar, rotator and case by 3D printing or mechanical processing.</li>
	<li>Put the touch sensors into the groove of the object body and stick them onto the predefined touch areas with double sided tape (Figure 2B). 
	NOTE: Each object consists of four subcomponents: a backboard, object body with groove inside, cover board, and touch sensors.</li>
	<li>Pass the wires through the hole of the object backboard and secure the cover board onto the object body with screws (Figure 2B).</li>
	<li>Pass wires of touch sensors through the holes on the sides of rotator&#160;and fix the objects onto the rotator with screws.&#160;(Figure 2C).</li>
	<li>Solder the wire ends of touch sensors to the rotating wire ends of the electric slip ring and wrap the joints with electrical tape (Figure 2D).</li>
	<li>Secure the case to the slider of the X-rail with screws. Place the bearing in the bottom hole of the box and secure the locating bar to the top surface of case with screws (Figure 2E).</li>
	<li>Place the rotator into the case from side, coinciding the axes of rotator, bearing and box. Pass the wires of the electric slip ring through the top hole of the case (Figure 2F).</li>
	<li>Insert the metal shaft into the bearing from the top hole of case and fit the shaft key to the keyway of the rotator (Figure 2G).</li>
	<li>Set the electric slip ring around the metal shaft. Place the end of locating bar into the notch of electric slip ring to prevent the outer ring from rotating (Figure 2G).</li>
	<li>Insert the shaft of stepping motor into the hole of metal shaft and secure the motor on the top of the box with screws. (Figure 2H).</li>
	<li>Insert the power and control cables of the motor into the power and control ports of its driver and secure them with screws.</li>
	<li>Stick a tricolor LED (RGB) onto the front side of the case with tape and fix the right side board onto the case.</li>
</ol>
3. Setup of the control system
<ol>
	<li>Insert the direction and pulse control wires of the four motor drivers into the digital I/O ports (pins 81, 83, 85, 87) and digital counter ports (pins 89, 91, 93, 95) of the data acquisition (DAQ) board, respectively. Secure the wires with screws.</li>
	<li>Insert the control wires of LED (green color used for the &#8220;go&#8221; cue, blue color used for the &#8220;error&#8221; cue, and red color representing idle) into the digital I/O ports (pin 65 and 66) of the DAQ card and secure them with screws.</li>
	<li>Insert the output wires of touch sensors and switch button into the digital I/O ports (pin 67&#8211;77) of the DAQ board and secure the wires with screws.</li>
	<li>Insert the start-stop and direction control wires of the peristaltic pump into the digital I/O pins 1 and 80, respectively. Insert the flow velocity control wire into the analog I/O port AO2. Secure the wires with screws.</li>
	<li>Setup a motion capture system as described by the manufacturer to record the hand trajectory in 3D space. 
	NOTE: A commercial motion capture system (see Table of Materials) was used, which consists of eight cameras, a power hub, an Ethernet switch and a supporting software (e.g., Cortex). Please refer to the manual to get more details about setup of the system.</li>
	<li>Setup a neural signal acquisition system as described by the manufacturer to record electrophysiology signal from subject. 
	NOTE: A commercial data acquisition system (Table of Materials) was used, which consists of a neural signal processor (NSP), front-end amplifier (FEA), amplifier power supply (ASP), head stages, and its supporting software (e.g., Central). Refer to the manual for more details about the setup of the system.</li>
</ol>
4. Preparation of the experimental session
<ol>
	<li>Initialize the 3D translational device and the turning table. Specifically, pull the sliders of all linear slide rail to the starting point (lower left corner) and turn the first object (i.e., the vertically placed handle) of turning table to face the front side of the turning table.</li>
	<li>Power on the experimental devices, including motion capture system, neural signal acquisition, DAQ board, peristaltic pump, and four motors.</li>
	<li>Setup the paradigm software (Figure 3A).
	<ol>
		<li>Double click Paradigm.exe to open the paradigm software (available on request).</li>
		<li>Define the number of the reaching positions and their 3D coordinates (x, y, and z, in millimeters) relative to initial positions (step 4.2).</li>
		<li>Write coordinates of all positions in the form of matrix in a .txt document. Make sure that each row includes the x-, y-, and z-coordinates of one position separated with a space. Save the txt document.</li>
		<li>Click Open File in the Pool panel of paradigm software and select the .txt document saved before to load the presentation positions into the paradigm software. 
		NOTE: In this study, eight target positions were set according to animal&#8217;s reaching range, which are located at vertices of a cuboid workspace9,10 (90 mm x 60 mm x 90 mm).</li>
		<li>Check the objects to be presented in the experiment in the Object Pool of paradigm software.</li>
		<li>Adjust experimental parameters in the Time Parameters panel of paradigm software. Set Baseline = 400 ms, Motor Run = 2,000 ms, Planning = 1,000 ms, max Reaction Time = 500 ms, max Reach Time = 1,000 ms, min Hold Time = 500 ms, Reward = 60 ms, and Error Cue = 1,000 ms.</li>
	</ol>
	</li>
	<li>Seat the rhesus monkey (with a micro-electrode array implanted in motor cortex) on the monkey chair by inserting its collar into the groove of chair and fixing its head.</li>
	<li>Fix the monkey chair to the aluminum construction frame. Keep the head 250 mm away from the front side of the cuboid and keep the eyes 50 mm above the top side of the cuboid workspace (horizontal visual angle: 20&#176;; vertical visual angle: 18&#176;).</li>
	<li>Construct a tracking template of motion capture system.
	<ol>
		<li>Attach three reflective markers at the end of the arm (close to wrist) with double-sided tape. Make sure that the three markers form a scalene triangle.</li>
		<li>Click the Run button of the paradigm software to start the task.</li>
		<li>Click the Record button on the Motion Capture panel of Cortex software to record trajectories of three markers for 60 s when the monkey is doing the task. Click the Stop button to suspend the experiment.</li>
		<li>Build a tracking template of three markers on the Cortex software using the recorded trajectories and save the template. 
		NOTE: Please refer to the manual of Cortex to get more details about how to build a model.</li>
	</ol>
	</li>
	<li>Connect the GND ports of FEA and micro-electrode array implanted in the monkey&#8217;s motor cortex with a wire and pinch cocks. Then insert the head stages into the connector of the micro-electrode array31.</li>
	<li>Open the Central software of neural signal acquisition system and set recording parameters including storage path, line noise cancellation, spike filter, spike threshold, etc. 
	NOTE: Please refer to the manual of neural signal acquisition system for more details of software setting.</li>
	<li>Open the synchronization software (Figure 3B, available on request). Click the three Connect buttons in the Cerebus, Motion Capture, and Paradigm panels to connect the synchronization software with the neural signal acquisition system, motion capture system and paradigm software, respectively.</li>
	<li>Click the Run button of the paradigm software to continue the experiment.</li>
	<li>Click the Record button on the File Storage panel of Central software to start recording the neural signals.</li>
	<li>Check the saved tracking template and click the Record button on the Motion Capture panel of Cortex software to start recording the trajectory of monkey&#8217;s wrist.</li>
</ol>

<ol>
	<li>Leone, F. T., Monaco, S., Henriques, D. Y., Toni, I., Medendorp, W. P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Flexible+Reference+Frames+for+Grasp+Planning+in+Human+Parietofrontal+Cortex.">Flexible Reference Frames for Grasp Planning in Human Parietofrontal Cortex.</a> eNeuro. 2 (3), (2015).</li><li>Caminiti, R., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Early+coding+of+reaching:+frontal+and+parietal+association+connections+of+parieto-occipital+cortex.">Early coding of reaching: frontal and parietal association connections of parieto-occipital cortex.</a> European Journal of Neuroscience. 11 (9), 3339-3345 (1999).</li><li>Georgopoulos, A. P., Schwartz, A. B., Kettner, R. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Neuronal+population+coding+of+movement+direction.">Neuronal population coding of movement direction.</a> Science. 233 (4771), 1416-1419 (1986).</li><li>Fu, Q. G., Flament, D., Coltz, J. D., Ebner, T. J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Temporal+encoding+of+movement+kinematics+in+the+discharge+of+primate+primary+motor+and+premotor+neurons.">Temporal encoding of movement kinematics in the discharge of primate primary motor and premotor neurons.</a> Journal of Neurophysiology. 73 (2), 836-854 (1995).</li><li>Moran, D. W., Schwartz, A. B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Motor+cortical+representation+of+speed+and+direction+during+reaching.">Motor cortical representation of speed and direction during reaching.</a> Journal of Neurophysiology. 82 (5), 2676-2692 (1999).</li><li>Carmena, J. M., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Learning+to+control+a+brain-machine+interface+for+reaching+and+grasping+by+primates.">Learning to control a brain-machine interface for reaching and grasping by primates.</a> PLoS Biology. 1 (2), E42 (2003).</li><li>Li, H., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Prior+Knowledge+of+Target+Direction+and+Intended+Movement+Selection+Improves+Indirect+Reaching+Movement+Decoding.">Prior Knowledge of Target Direction and Intended Movement Selection Improves Indirect Reaching Movement Decoding.</a> Behavioral Neurology. , 2182843 (2017).</li><li>Reina, G. A., Moran, D. W., Schwartz, A. B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=On+the+relationship+between+joint+angular+velocity+and+motor+cortical+discharge+during+reaching.">On the relationship between joint angular velocity and motor cortical discharge during reaching.</a> Journal of Neurophysiology. 85 (6), 2576-2589 (2001).</li><li>Taylor, D. M., Tillery, S. I., Schwartz, A. B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Direct+cortical+control+of+3D+neuroprosthetic+devices.">Direct cortical control of 3D neuroprosthetic devices.</a> Science. 296 (5574), 1829-1832 (2002).</li><li>Wang, W., Chan, S. S., Heldman, D. A., Moran, D. W. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Motor+cortical+representation+of+hand+translation+and+rotation+during+reaching.">Motor cortical representation of hand translation and rotation during reaching.</a> Journal of Neuroscience. 30 (3), 958-962 (2010).</li><li>Murata, A., Gallese, V., Luppino, G., Kaseda, M., Sakata, H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Selectivity+for+the+shape,+size,+and+orientation+of+objects+for+grasping+in+neurons+of+monkey+parietal+area+AIP.">Selectivity for the shape, size, and orientation of objects for grasping in neurons of monkey parietal area AIP.</a> Journal of Neurophysiology. 83 (5), 2580-2601 (2000).</li><li>Raos, V., Umilt&#225;, M. A., Murata, A., Fogassi, L., Gallese, V. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Functional+Properties+of+Grasping-Related+Neurons+in+the+Ventral+Premotor+Area+F5+of+the+Macaque+Monkey.">Functional Properties of Grasping-Related Neurons in the Ventral Premotor Area F5 of the Macaque Monkey.</a> Journal of Neurophysiology. 95 (2), 709 (2006).</li><li>Schaffelhofer, S., Scherberger, H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Object+vision+to+hand+action+in+macaque+parietal,+premotor,+and+motor+cortices.">Object vision to hand action in macaque parietal, premotor, and motor cortices.</a> eLife. 5, (2016).</li><li>Baumann, M. A., Fluet, M. C., Scherberger, H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Context-specific+grasp+movement+representation+in+the+macaque+anterior+intraparietal+area.">Context-specific grasp movement representation in the macaque anterior intraparietal area.</a> Journal of Neuroscience. 29 (20), 6436-6448 (2009).</li><li>Riehle, A., Wirtssohn, S., Grun, S., Brochier, T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Mapping+the+spatio-temporal+structure+of+motor+cortical+LFP+and+spiking+activities+during+reach-to-grasp+movements.">Mapping the spatio-temporal structure of motor cortical LFP and spiking activities during reach-to-grasp movements.</a> Frontiers in Neural Circuits. 7, 48 (2013).</li><li>Michaels, J. A., Scherberger, H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Population+coding+of+grasp+and+laterality-related+information+in+the+macaque+fronto-parietal+network.">Population coding of grasp and laterality-related information in the macaque fronto-parietal network.</a> Scientific Reports. 8 (1), 1710 (2018).</li><li>Fattori, P., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Hand+orientation+during+reach-to-grasp+movements+modulates+neuronal+activity+in+the+medial+posterior+parietal+area+V6A.">Hand orientation during reach-to-grasp movements modulates neuronal activity in the medial posterior parietal area V6A.</a> Journal of Neuroscience. 29 (6), 1928-1936 (2009).</li><li>Asher, I., Stark, E., Abeles, M., Prut, Y. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Comparison+of+direction+and+object+selectivity+of+local+field+potentials+and+single+units+in+macaque+posterior+parietal+cortex+during+prehension.">Comparison of direction and object selectivity of local field potentials and single units in macaque posterior parietal cortex during prehension.</a> Journal of Neurophysiology. 97 (5), 3684-3695 (2007).</li><li>Stark, E., Asher, I., Abeles, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Encoding+of+reach+and+grasp+by+single+neurons+in+premotor+cortex+is+independent+of+recording+site.">Encoding of reach and grasp by single neurons in premotor cortex is independent of recording site.</a> Journal of Neurophysiology. 97 (5), 3351-3364 (2007).</li><li>Velliste, M., Perel, S., Spalding, M. C., Whitford, A. S., Schwartz, A. B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Cortical+control+of+a+prosthetic+arm+for+self-feeding.">Cortical control of a prosthetic arm for self-feeding.</a> Nature. 453 (7198), 1098-1101 (2008).</li><li>Vargas-Irwin, C. E., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Decoding+complete+reach+and+grasp+actions+from+local+primary+motor+cortex+populations.">Decoding complete reach and grasp actions from local primary motor cortex populations.</a> Journal of Neuroscience. 30 (29), 9659-9669 (2010).</li><li>Mollazadeh, M., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Spatiotemporal+variation+of+multiple+neurophysiological+signals+in+the+primary+motor+cortex+during+dexterous+reach-to-grasp+movements.">Spatiotemporal variation of multiple neurophysiological signals in the primary motor cortex during dexterous reach-to-grasp movements.</a> Journal of Neuroscience. 31 (43), 15531-15543 (2011).</li><li>Saleh, M., Takahashi, K., Hatsopoulos, N. G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Encoding+of+coordinated+reach+and+grasp+trajectories+in+primary+motor+cortex.">Encoding of coordinated reach and grasp trajectories in primary motor cortex.</a> Journal of Neuroscience. 32 (4), 1220-1232 (2012).</li><li>Collinger, J. L., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=High-performance+neuroprosthetic+control+by+an+individual+with+tetraplegia.">High-performance neuroprosthetic control by an individual with tetraplegia.</a> The Lancet. 381 (9866), 557-564 (2013).</li><li>Lehmann, S. J., Scherberger, H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Reach+and+gaze+representations+in+macaque+parietal+and+premotor+grasp+areas.">Reach and gaze representations in macaque parietal and premotor grasp areas.</a> Journal of Neuroscience. 33 (16), 7038-7049 (2013).</li><li>Rouse, A. G., Schieber, M. H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Spatiotemporal+distribution+of+location+and+object+effects+in+reach-to-grasp+kinematics.">Spatiotemporal distribution of location and object effects in reach-to-grasp kinematics.</a> Journal of Neuroscience. 114 (6), 3268-3282 (2015).</li><li>Rouse, A. G., Schieber, M. H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Spatiotemporal+Distribution+of+Location+and+Object+effects+in+Primary+Motor+Cortex+Neurons+during+Reach-to-Grasp.">Spatiotemporal Distribution of Location and Object effects in Primary Motor Cortex Neurons during Reach-to-Grasp.</a> Journal of Neuroscience. 36 (41), 10640-10653 (2016).</li><li>Hao, Y., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Neural+synergies+for+controlling+reach+and+grasp+movement+in+macaques.">Neural synergies for controlling reach and grasp movement in macaques.</a> Neuroscience. 357, 372-383 (2017).</li><li>Takahashi, K., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Encoding+of+Both+Reaching+and+Grasping+Kinematics+in+Dorsal+and+Ventral+Premotor+Cortices.">Encoding of Both Reaching and Grasping Kinematics in Dorsal and Ventral Premotor Cortices.</a> Journal of Neuroscience. 37 (7), 1733-1746 (2017).</li><li>Chen, J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=An+automated+behavioral+apparatus+to+combine+parameterized+reaching+and+grasping+movements+in+3D+space.">An automated behavioral apparatus to combine parameterized reaching and grasping movements in 3D space.</a> Journal of Neuroscience Methods. 312, 139-147 (2019).</li><li>Zhang, Q., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Development+of+an+invasive+brain-machine+interface+with+a+monkey+model.">Development of an invasive brain-machine interface with a monkey model.</a> Chinese Science Bulletin. 57 (16), 2036 (2012).</li><li>Hao, Y., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Distinct+neural+patterns+enable+grasp+types+decoding+in+monkey+dorsal+premotor+cortex.">Distinct neural patterns enable grasp types decoding in monkey dorsal premotor cortex.</a> Journal of Neural Engineering. 11 (6), 066011 (2014).</li></ol>

The authors have nothing to disclose.

The behavioral apparatus is described here enables a trial-wise combination of different reaching and grasping movements (i.e., the monkey can grasp differently shaped objects in any arbitrary 3D locations in each trial). This is accomplished through the combination of a custom turning table that switches different objects and a linear translational device that transports the turning table to multiple positions in 3D space. In addition, the neural signals from the monkey, trajectory of wrist, and hand shapes were able to...

Various apparatuses have been developed to study the neural principles underlying reaching and grasping movement in non-human primate. In reaching tasks, touch screen1,2, screen cursor controlled by a joystick3,4,5,6,7, and virtual reality technology8,9,10 have all been employed to present 2D and 3D targets, respectively. To introduce different grip types, differently shaped objects fixed in one position or rotating around an axis were widely used in the grasping tasks11,12,13. An alternative is to use visual cues to inform subjects to grasp the same object with different grip types14,15,16,17. More recently, reaching and grasping movements have been studied together (i.e., subjects reach multiple positions and grasp with different grip types in an experimental session)18,19,20,21,22,23,24,25,26,27,28,29. Early experiments have presented objects manually, which inevitably lead to low time and spatial precision20,21. To improve experimental precision and save manpower, automatic presentation devices controlled by programs have been widely used. To vary the target position and grip type, experimenters have exposed multiple objects simultaneously, but the relative (or absolute) position of targets and the grip types are bound together, which causes rigid firing patterns through long-term training22,27,28. Objects are usually presented in a 2D plane, which limits the diversity of reaching movement and neural activity19,25,26. Recently, virtual reality24 and robot arm23,29 have been introduced to present objects in 3D space.
Presented here are detailed protocols for building and using an automated apparatus30 that can achieve any combination of multiple target positions and grip types in 3D space. We designed a turning table to switch objects and 3D translational device to transport the turning table in 3D space. Both the turning table and translational device are driven by independent motors. Meanwhile, the 3D trajectory of subject&rsquo;s wrist and neural signals are recorded simultaneously throughout the experiment. The apparatus provides a valuable platform for the study of upper limb function in the rhesus monkey.

<table><tbody><tr><td>Active X-rail</td><td>CCM Automation technology Inc., China</td><td>W50-25</td><td>Effective travel, 600 mm; Load, 25 kg</td></tr><tr><td>Active Y-rail</td><td>CCM Automation technology Inc., China</td><td>W60-35</td><td>Effective travel, 300 mm, Load 35 kg</td></tr><tr><td>Active Z-rail</td><td>CCM Automation technology Inc., China</td><td>W50-25</td><td>Effective travel, 500 mm; Load 25 kg</td></tr><tr><td>Bearing</td><td>Taobao.com</td><td>6004-2RSH</td><td>Acrylic</td></tr><tr><td>Case</td><td>Custom mechanical processing</td><td>TT-C</td><td>Acrylic</td></tr><tr><td>Connecting ring</td><td>CCM Automation technology Inc., China</td><td>57/60-W50</td><td></td></tr><tr><td>Connecting shaft</td><td>CCM Automation technology Inc., China</td><td>D12-700</td><td>Diam., 12 mm;Length, 700 mm</td></tr><tr><td>Diaphragm coupling</td><td>CCM Automation technology Inc., China</td><td>CCM 12-12</td><td>Inner diam., 12-12mm</td></tr><tr><td>Diaphragm coupling</td><td>CCM Automation technology Inc., China</td><td>CCM 12-14</td><td>Inner diam., 14-12mm</td></tr><tr><td>Electric slip ring</td><td>Semring Inc., China</td><td>SNH020a-12</td><td>Acrylic</td></tr><tr><td>Locating bar</td><td>Custom mechanical processing</td><td>TT-L</td><td>Acrylic</td></tr><tr><td>Motion capture system</td><td>Motion Analysis Corp. US</td><td>Eagle-2.36</td><td></td></tr><tr><td>Neural signal acquisition system</td><td>Blackrock Microsystems Corp. US</td><td>Cerebus</td><td></td></tr><tr><td>NI DAQ device</td><td>National Instruments, US</td><td>USB-6341</td><td></td></tr><tr><td>Object</td><td>Custom mechanical processing</td><td>TT-O</td><td>Acrylic</td></tr><tr><td>Passive Y-rail</td><td>CCM Automation technology Inc., China</td><td>W60-35</td><td>Effective travel, 300 mm; Load 35 kg</td></tr><tr><td>Passive Z-rail</td><td>CCM Automation technology Inc., China</td><td>W50-25</td><td>Effective travel, 500 mm; Load 25 kg</td></tr><tr><td>Pedestal</td><td>CCM Automation technology Inc., China</td><td>80-W60</td><td></td></tr><tr><td>Peristaltic pump</td><td>Longer Inc., China</td><td>BT100-1L</td><td></td></tr><tr><td>Planetary gearhead</td><td>CCM Automation technology Inc., China</td><td>PLF60-5</td><td>Flange, 60&amp;times;60 mm; Reduction ratio, 1:5</td></tr><tr><td>Right triangle frame</td><td>CCM Automation technology Inc., China</td><td>290-300</td><td></td></tr><tr><td>Rotator</td><td>Custom mechanical processing</td><td>TT-R</td><td>Acrylic</td></tr><tr><td>Servo motor</td><td>Yifeng Inc., China</td><td>60ST-M01930</td><td>Flange, 60&amp;times;60 mm; Torque, 1.91 N&amp;middot;m; for Y- and Z-rail</td></tr><tr><td>Servo motor</td><td>Yifeng Inc., China</td><td>60ST-M01330</td><td>Flange, 60&amp;times;60 mm; Torque, 1.27 N&amp;middot;m; for X-rail</td></tr><tr><td>Shaft</td><td>Custom mechanical processing</td><td>TT-S</td><td>Acrylic</td></tr><tr><td>Stepping motor</td><td>Taobao.com</td><td>86HBS120</td><td>Flange, 86&amp;times;86 mm; Torque, 1.27 N&amp;middot;m; Driving turning table</td></tr><tr><td>Touch sensor</td><td>Taobao.com</td><td>CM-12X-5V</td><td></td></tr><tr><td>Tricolor LED</td><td>Taobao.com</td><td>CK017, RGB</td><td></td></tr><tr><td>T-shaped connecting board</td><td>CCM Automation technology Inc., China</td><td>110-120</td><td></td></tr></tbody></table>

design and use of an apparatus for presenting graspable objects in 3d workspace

Reaching and grasping are highly-coupled movements, and their underlying neural dynamics have been widely studied in the last decade. To distinguish reaching and grasping encodings, it is essential to present different object identities independent of their positions. Presented here is the design of an automatic apparatus that is assembled with a turning table and three-dimensional (3D) translational device to achieve this goal. The turning table switches different objects corresponding to different grip types while the 3D translational device transports the turning table in 3D space. Both are driven independently by motors so that the target position and object are combined arbitrarily. Meanwhile, wrist trajectory and grip types are recorded via the motion capture system and touch sensors, respectively. Furthermore, representative results that demonstrate successfully trained monkey using this system are described. It is expected that this apparatus will facilitate researchers to study kinematics, neural principles, and brain-machine interfaces related to upper limb function.

The size of complete workspace of the apparatus is 600 mm, 300 mm, and 500&#8201;mm in x-, y-, and z-axes, respectively. The maximum load of the 3D translational device is 25 kg, while the turning table (including the stepping motor) is weighted 15&#8201;kg and can be transported at a speed of up to 500&#8201;mm/s. The kinematic precision of the 3D translational device is less than 0.1&#8201;mm and the noise of the apparatus is less than 60 dB.
To demonstrate the utility of the system, the mon...

Watch this Scientific Journal Video about Design and Use of an Apparatus for Presenting Graspable Objects in 3D Workspace at JoVE.com

Design and Use of an Apparatus for Presenting Graspable Objects in 3D Workspace

Presented here is a protocol to build an automatic apparatus that guides a monkey to perform the flexible reach-to-grasp task. The apparatus combines a 3D translational device and turning table to present multiple objects in an arbitrary position in 3D space.

Reaching and grasping are highly-coupled movements, and their underlying neural dynamics have been widely studied in the last decade. To distinguish ...

design-use-an-apparatus-for-presenting-graspable-objects-3d

Research

JoVE Journal

Behavior

5.7K Views.  Zhejiang University. Presented here is a protocol to build an automatic apparatus that guides a monkey to perform the flexible reach-to-grasp task. The apparatus combines a 3D translational device and turning table to present multiple objects in an arbitrary position in 3D space.

Video: Design and Use of an Apparatus for Presenting Graspable Objects in 3D Workspace

Haptic/Graphic Rehabilitation: Integrating a Robot into a Virtual Environment Library and Applying it to Stroke Therapy

Recent research that tests interactive devices for prolonged therapy practice has revealed new prospects for robotics combined with graphical and other forms of biofeedback. Previous human-robot interactive systems have required different software commands to be implemented for each robot leading to unnecessary developmental overhead time each time a new system becomes available. For example, when a haptic/graphic virtual reality environment has been coded for one specific robot to provide haptic feedback, that specific robot would not be able to be traded for another robot without recoding the program. However, recent efforts in the open source community have proposed a wrapper class approach that can elicit nearly identical responses regardless of the robot used. The result can lead researchers across the globe to perform similar experiments using shared code. Therefore modular "switching out"of one robot for another would not affect development time. In this paper, we outline the successful creation and implementation of a wrapper class for one robot into the open-source H3DAPI, which integrates the software commands most commonly used by all robots.

Recently, a vast amount of prospects have come available for human-robot interactive systems. In this paper we outline the integration of a new robotic device with open source software that can rapidly make possible a library of interactive functionality. We then outline a clinical application for a neurorehabilitation application.

Recent research that tests interactive devices for prolonged therapy practice has revealed new prospects for robotics combined with graphical and other ...

Bioengineering

Novel 3D/VR Interactive Environment for MD Simulations, Visualization and Analysis

The increasing development of computing (hardware and software) in the last decades has impacted scientific research in many fields including materials science, biology, chemistry and physics among many others. A new computational system for the accurate and fast simulation and 3D/VR visualization of nanostructures is presented here, using the open-source molecular dynamics (MD) computer program LAMMPS. This alternative computational method uses modern graphics processors, NVIDIA CUDA technology and specialized scientific codes to overcome processing speed barriers common to traditional computing methods. In conjunction with a virtual reality system used to model materials, this enhancement allows the addition of accelerated MD simulation capability. The motivation is to provide a novel research environment which simultaneously allows visualization, simulation, modeling and analysis. The research goal is to investigate the structure and properties of inorganic nanostructures (e.g., silica glass nanosprings) under different conditions using this innovative computational system. The work presented outlines a description of the 3D/VR Visualization System and basic components, an overview of important considerations such as the physical environment, details on the setup and use of the novel system, a general procedure for the accelerated MD enhancement, technical information, and relevant remarks. The impact of this work is the creation of a unique computational system combining nanoscale materials simulation, visualization and interactivity in a virtual environment, which is both a research and teaching instrument at UC Merced.

A new computational system featuring GPU-accelerated molecular dynamics simulation and 3D/VR visualization, analysis and manipulation of nanostructures has been implemented, representing a novel approach to advance materials research and promote innovative investigation and alternative methods to learn about material structures with dimensions invisible to the human eye.

The increasing development of computing (hardware and software) in the last decades has impacted scientific research in many fields including materials ...

Engineering

Methods for Presenting Real-world Objects Under Controlled Laboratory Conditions

Our knowledge of human object vision is based almost exclusively on studies in which the stimuli are presented in the form of computerized two-dimensional (2-D) images. In everyday life, however, humans interact predominantly with real-world solid objects, not images. Currently, we know very little about whether images of objects trigger similar behavioral or neural processes as do real-world exemplars. Here, we present methods for bringing the real-world into the laboratory. We detail methods for presenting rich, ecologically-valid real-world stimuli under tightly-controlled viewing conditions. We describe how to match closely the visual appearance of real objects and their images, as well as novel apparatus and protocols that can be used to present real objects and computerized images on successively interleaved trials. We use a decision-making paradigm as a case example in which we compare willingness-to-pay (WTP) for real snack foods versus 2-D images of the same items. We show that WTP increases by 6.6% for food items displayed as real objects versus high-resolution 2-D colored&#160;images of the same foods -suggesting that real foods are perceived as being more valuable than their images. Although presenting real object stimuli under controlled conditions presents several practical challenges for the experimenter, this approach will fundamentally expand our understanding of the cognitive and neural processes that underlie naturalistic vision.

We describe methods for presenting real-world objects and matched images of the same objects under tightly-controlled experimental conditions. The methods are described in the context of a decision-making task, but the same real-world approach can be extended to other cognitive domains such as perception, attention, and memory.

Our knowledge of human object vision is based almost exclusively on studies in which the stimuli are presented in the form of computerized two-dimensional ...

Estimation of Contact Regions Between Hands and Objects During Human Multi-Digit Grasping

To grasp an object successfully, we must select appropriate contact regions for our hands on the surface of the object. However, identifying such regions is challenging. This paper describes a workflow to estimate the contact regions from marker-based tracking data. Participants grasp real objects, while we track the 3D position of both the objects and the hand, including the fingers' joints. We first determine the joint Euler angles from a selection of tracked markers positioned on the back of the hand. Then, we use state-of-the-art hand mesh reconstruction algorithms to generate a mesh model of the participant's hand in the current pose and the 3D position. 
Using objects that were either 3D printed or 3D scanned-and are, thus, available as both real objects and mesh data-allows the hand and object meshes to be co-registered. In turn, this allows the estimation of approximate contact regions by calculating the intersections between the hand mesh and the co-registered 3D object mesh. The method may be used to estimate where and how humans grasp objects under a variety of conditions. Therefore, the method could be of interest to researchers studying visual and haptic perception, motor control, human-computer interaction in virtual and augmented reality, and robotics.

When we grasp an object, multiple regions of the fingers and hand typically make contact with the object's surface. Reconstructing such contact regions is challenging. Here, we present a method for approximately estimating the contact regions by combining marker-based motion capture with existing deep learning-based hand mesh reconstruction.

To grasp an object successfully, we must select appropriate contact regions for our hands on the surface of the object. However, identifying such regions ...

An Emerging Target Paradigm to Evoke Fast Visuomotor Responses on Human Upper Limb Muscles

To reach towards a seen object, visual information has to be transformed into motor commands. Visual information such as the object&#8217;s color, shape, and size are processed and integrated within numerous brain areas, then ultimately relayed to the motor periphery. In some instances, a reaction is needed as fast as possible. These fast visuomotor transformations, and their underlying neurological substrates, are poorly understood in humans as they have lacked a reliable biomarker. Stimulus-locked responses (SLRs) are short latency (&#60;100 ms) bursts of electromyographic (EMG) activity representing the first wave of muscle recruitment influenced by visual stimulus presentation. SLRs provide a quantifiable output of rapid visuomotor transformations, but SLRs have not been consistently observed in all subjects in past studies. Here we describe a new, behavioral paradigm featuring the sudden emergence of a moving target below an obstacle that consistently evokes robust SLRs. Human participants generated visually guided reaches toward or away from the emerging target using a robotic manipulandum while surface electrodes recorded EMG activity from the pectoralis major muscle. In comparison to previous studies that investigated SLRs using static stimuli, the SLRs evoked with this emerging target paradigm were larger, evolved earlier, and were present in all participants. Reach reaction times (RTs) were also expedited in the emerging target paradigm. This paradigm affords numerous opportunities for modification that could permit systematic study of the impact of various sensory, cognitive, and motor manipulations on fast visuomotor responses. Overall, our results demonstrate that an emerging target paradigm is capable of consistently and robustly evoking activity within a fast visuomotor system.

Presented here is a behavioral paradigm that elicits robust fast visuomotor responses on human upper limb muscles during visually guided reaches.

To reach towards a seen object, visual information has to be transformed into motor commands. Visual information such as the object&#8217;s color, shape, ...

Neuroscience

Combining Augmented Reality and 3D Printing to Display Patient Models on a Smartphone

Augmented reality (AR) has great potential in education, training, and surgical guidance in the medical field. Its combination with three-dimensional (3D) printing (3DP) opens new possibilities in clinical applications. Although these technologies have grown exponentially in recent years, their adoption by physicians is still limited, since they require extensive knowledge of engineering and software development. Therefore, the purpose of this protocol is to describe a step-by-step methodology enabling inexperienced users to create a smartphone app, which combines AR and 3DP for the visualization of anatomical 3D models of patients with a 3D-printed reference marker. The protocol describes how to create 3D virtual models of a patient&rsquo;s anatomy derived from 3D medical images. It then explains how to perform positioning of the 3D models with respect to marker references. Also provided are instructions for how to 3D print the required tools and models. Finally, steps to deploy the app are provided. The protocol is based on free and multi-platform software and can be applied to any medical imaging modality or patient. An alternative approach is described to provide automatic registration between a 3D-printed model created from a patient&rsquo;s anatomy and the projected holograms. As an example, a clinical case of a patient suffering from distal leg sarcoma is provided to illustrate the methodology. It is expected that this protocol will accelerate the adoption of AR and 3DP technologies by medical professionals.

Presented here is a method to design an augmented reality smartphone application for the visualization of anatomical three-dimensional models of patients using a 3D-printed reference marker.

Augmented reality (AR) has great potential in education, training, and surgical guidance in the medical field. Its combination with three-dimensional (3D) ...

Medicine

Robotized Testing of Camera Positions to Determine Ideal Configuration for Stereo 3D Visualization of Open-Heart Surgery

Stereo 3D video from surgical procedures can be highly valuable for medical education and improve clinical communication. But access to the operating room and the surgical field is restricted. It is a sterile environment, and the physical space is crowded with surgical staff and technical equipment. In this setting, unobscured capture and realistic reproduction of the surgical procedures are difficult. This paper presents a method for rapid and reliable data collection of stereoscopic 3D videos at different camera baseline distances and distances of convergence. To collect test data with minimum interference during surgery, with high precision and repeatability, the cameras were attached to each hand of a dual-arm robot. The robot was ceiling-mounted in the operating room. It was programmed to perform a timed sequence of synchronized camera movements stepping through a range of test positions with baseline distance between 50-240 mm at incremental steps of 10 mm, and at two convergence distances of 1100 mm and 1400 mm. Surgery was paused to allow 40 consecutive 5-s video samples. A total of 10 surgical scenarios were recorded.

The human depth perception of 3D stereo videos depends on the camera separation, point of convergence, distance to, and familiarity of the object. This paper presents a robotized method for rapid and reliable test data collection during live open-heart surgery to determine the ideal camera configuration.

Stereo 3D video from surgical procedures can be highly valuable for medical education and improve clinical communication. But access to the operating room ...

Real-Time Recording and Mapping of Human Interaction with 3D Virtual Objects

Three-Dimensional Mapping of the Rotation of Interactive Virtual Objects with Eye-Tracking Data

We present a method for real-time recording of human interaction with three-dimensional (3D) virtual objects. The approach consists of associating rotation data of the manipulated object with behavioral measures, such as eye tracking, to make better inferences about the underlying cognitive processes.
The task consists of displaying two identical models of the same 3D object (a molecule), presented on a computer screen: a rotating, interactive object (iObj) and a static, target object (tObj). Participants must rotate iObj using the mouse until they consider its orientation to be identical to that of tObj. The computer tracks all interaction data in real time. The participant&#39;s gaze data are also recorded using an eye tracker. The measurement frequency is 10 Hz on the computer and 60 Hz on the eye tracker.
The orientation data of iObj with respect to tObj are recorded in rotation quaternions. The gaze data are synchronized to the orientation of iObj and referenced using this same system. This method enables us to obtain the following visualizations of the human interaction process with iObj and tObj: (1) angular disparity synchronized with other time-dependent data; (2) 3D rotation trajectory inside what we decided to call a &#34;ball of rotations&#34;; (3) 3D fixation heatmap. All steps of the protocol have used free software, such as GNU Octave and Jmol, and all scripts are available as supplementary material.
With this approach, we can conduct detailed quantitative studies of the task-solving process involving mental or physical rotations, rather than only the outcome reached. It is possible to measure precisely how important each part of the 3D models is for the participant in solving tasks, and thus relate the models to relevant variables such as the characteristics of the objects, cognitive abilities of the individuals, and the characteristics of human-machine interface.

Author Spotlight: Insights into the Analysis of Human Interaction with 3D Virtual Objects

We have developed a simple, customizable, and efficient method for recording quantitative processual data from interactive spatial tasks and mapping these rotation data with eye-tracking data.

We present a method for real-time recording of human interaction with three-dimensional (3D) virtual objects. The approach consists of associating ...

Virtual Reality-3D Modeling for Minimally Invasive Neuro-Interventional Planning

Pioneering Patient-Specific Approaches for Precision Surgery Using Imaging and Virtual Reality

Endovascular treatment of complex vascular anomalies shifts the risk of open surgical procedures to the benefit of minimally invasive endovascular procedural solutions. Complex open surgical procedures used to be the only option for the treatment of a myriad of conditions like pulmonary and aortic valve replacement as well as cerebral aneurysm repair. However, due to advancements in catheter-delivered devices and operator expertise, these procedures (along with many others) can now be performed through minimally invasive procedures delivered through a central or peripheral vein or artery. The decision to shift from an open procedure to an endovascular approach is based on multi-modal imaging, often including 3D Digital Imaging and Communications in Medicine (DICOM) imaging datasets. Utilizing these 3D images, our lab generates 3D models of the pathologic anatomy, thereby allowing the pre-procedural analysis necessary to pre-plan critical components of the catheterization lab procedure, namely, C-arm positioning, 3D measurement, and idealized road-map generation. This article describes how to take segmented 3D models of patient-specific pathology and predict generalized C-arm positions, how to measure critical two-dimensional (2D) measurements of 3D structures relevant to the 2D fluoroscopy projections, and how to generate 2D fluoroscopy roadmap analogs that can assist in proper C-arm positioning during catheterization lab procedures.

Author Spotlight: Segmentation and VR for Advanced Neurovascular Interventions

Advancements in endovascular treatment have replaced complex open surgical procedures with minimally invasive options, like valve replacement and aneurysm repair. This paper proposes using three-dimensional (3D) modeling and virtual reality to aid in C-arm positioning, angle measurements, and roadmap generation for neuro-interventional catheterization lab procedural planning, minimizing procedure time.

Endovascular treatment of complex vascular anomalies shifts the risk of open surgical procedures to the benefit of minimally invasive endovascular ...

Pedicle Screw Placement Using an Augmented Reality Head-Mounted Display in a Porcine Model

This protocol helps assess the accuracy and workflow of an augmented reality (AR) hybrid navigation system using the Magic Leap head-mounted display (HMD) for minimally invasive pedicle screw placement. The cadaveric porcine specimens were placed on a surgical table and draped with sterile covers. The levels of interest were identified using fluoroscopy, and a dynamic reference frame was attached to the spinous process of a vertebra in the region of interest. Cone beam computerized tomography (CBCT) was performed, and a 3D rendering was automatically generated, which was used for the subsequent planning of the pedicle screw placements. Each surgeon was fitted with an HMD that was individually eye-calibrated and connected to the spinal navigation system.
Navigated instruments, tracked by the navigation system and displayed in 2D and 3D in the HMD, were used for 33 pedicle cannulations, each with a diameter of 4.5 mm. Postprocedural CBCT scans were assessed by an independent reviewer to measure the technical (deviation from the planned path) and clinical (Gertzbein grade) accuracy of each cannulation. The navigation time for each cannulation was measured. The technical accuracy was 1.0 mm &#177; 0.5 mm at the entry point and 0.8 mm &#177; 0.1 mm at the target. The angular deviation was 1.5&#176; &#177; 0.6&#176;, and the mean insertion time per cannulation was 141 s &#177; 71 s. The clinical accuracy was 100% according to the Gertzbein grading scale (32 grade 0; 1 grade 1). When used for minimally invasive pedicle cannulations in a porcine model, submillimeter technical accuracy and 100% clinical accuracy could be achieved with this protocol.

The augmented reality head-mounted display, Magic Leap, was used in combination with a conventional navigation system to place pedicle screws in a porcine model by adhering to a novel workflow. With a median insertion time of &lt;2.5 min, submillimeter technical accuracy and 100% clinical accuracy were achieved according to Gertzbein.

This protocol helps assess the accuracy and workflow of an augmented reality (AR) hybrid navigation system using the Magic Leap head-mounted display (HMD) ...