Name: combining augmented reality and 3d printing to display patient models on a smartphone
Uploaded: 2020-01-02T13:00:00
Duration: 9 min 26 s
Description: Watch this Scientific Journal Video about Combining Augmented Reality and 3D Printing to Display Patient Models on a Smartphone at JoVE.com

This report was supported by projects PI18/01625 and PI15/02121 (Ministerio de Ciencia, Innovaci&#243;n y Universidades, Instituto de Salud Carlos III and European Regional Development Fund "Una manera de hacer Europa") and IND2018/TIC-9753 (Comunidad de Madrid).

<ol>
	<li>Coles, T. R., John, N. W., Gould, D., Caldwell, D. G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Integrating+Haptics+with+Augmented+Reality+in+a+Femoral+Palpation+and+Needle+Insertion+Training+Simulation.">Integrating Haptics with Augmented Reality in a Femoral Palpation and Needle Insertion Training Simulation.</a> IEEE Transactions on Haptics. 4 (3), 199-209 (2011).</li><li>Pelargos, P. E., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Utilizing+virtual+and+augmented+reality+for+educational+and+clinical+enhancements+in+neurosurgery.">Utilizing virtual and augmented reality for educational and clinical enhancements in neurosurgery.</a> Journal of Clinical Neuroscience. 35, 1-4 (2017).</li><li>Abhari, K., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Training+for+Planning+Tumour+Resection:+Augmented+Reality+and+Human+Factors.">Training for Planning Tumour Resection: Augmented Reality and Human Factors.</a> IEEE Transactions on Biomedical Engineering. 62 (6), 1466-1477 (2015).</li><li>Uppot, R., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Implementing+Virtual+and+Augmented+Reality+Tools+for+Radiology+Education+and+Training,+Communication,+and+Clinical+Care.">Implementing Virtual and Augmented Reality Tools for Radiology Education and Training, Communication, and Clinical Care.</a> Radiology. 291, 182210 (2019).</li><li>Pratt, P., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Through+the+HoloLensTM+looking+glass:+augmented+reality+for+extremity+reconstruction+surgery+using+3D+vascular+models+with+perforating+vessels.">Through the HoloLensTM looking glass: augmented reality for extremity reconstruction surgery using 3D vascular models with perforating vessels.</a> European Radiology Experimental. 2 (1), 2 (2018).</li><li>Rose, A. S., Kim, H., Fuchs, H., Frahm, J. -. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Development+of+augmented-reality+applications+in+otolaryngology-head+and+neck+surgery.">Development of augmented-reality applications in otolaryngology-head and neck surgery.</a> The Laryngoscope. , (2019).</li><li>Zhou, C., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Robot-Assisted+Surgery+for+Mandibular+Angle+Split+Osteotomy+Using+Augmented+Reality:+Preliminary+Results+on+Clinical+Animal+Experiment.">Robot-Assisted Surgery for Mandibular Angle Split Osteotomy Using Augmented Reality: Preliminary Results on Clinical Animal Experiment.</a> Aesthetic Plastic Surgery. 41 (5), 1228-1236 (2017).</li><li>Heinrich, F., Joeres, F., Lawonn, K., Hansen, C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Comparison+of+Projective+Augmented+Reality+Concepts+to+Support+Medical+Needle+Insertion.">Comparison of Projective Augmented Reality Concepts to Support Medical Needle Insertion.</a> IEEE Transactions on Visualization and Computer Graphics. 25 (6), 1 (2019).</li><li>Deng, W., Li, F., Wang, M., Song, Z. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Easy-to-Use+Augmented+Reality+Neuronavigation+Using+a+Wireless+Tablet+PC.">Easy-to-Use Augmented Reality Neuronavigation Using a Wireless Tablet PC.</a> Stereotactic and Functional Neurosurgery. 92 (1), 17-24 (2014).</li><li>Fan, Z., Weng, Y., Chen, G., Liao, H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=3D+interactive+surgical+visualization+system+using+mobile+spatial+information+acquisition+and+autostereoscopic+display.">3D interactive surgical visualization system using mobile spatial information acquisition and autostereoscopic display.</a> Journal of Biomedical Informatics. 71, 154-164 (2017).</li><li>Arnal-Burr&#243;, J., P&#233;rez-Ma&#241;anes, R., Gallo-del-Valle, E., Igualada-Blazquez, C., Cuervas-Mons, M., Vaquero-Mart&#237;n, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Three+dimensional-printed+patient-specific+cutting+guides+for+femoral+varization+osteotomy:+Do+it+yourself.">Three dimensional-printed patient-specific cutting guides for femoral varization osteotomy: Do it yourself.</a> The Knee. 24 (6), 1359-1368 (2017).</li><li>Vaquero, J., Arnal, J., Perez-Ma&#241;anes, R., Calvo-Haro, J., Chana, F. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=3D+patient-specific+surgical+printing+cutting+blocks+guides+and+spacers+for+open-+wedge+high+tibial+osteotomy+(HTO)+-+do+it+yourself.">3D patient-specific surgical printing cutting blocks guides and spacers for open- wedge high tibial osteotomy (HTO) - do it yourself.</a> Revue de Chirurgie Orthop&#233;dique et Traumatologique. 102, 131 (2016).</li><li>De La Pe&#241;a, A., De La Pe&#241;a-Brambila, J., P&#233;rez-De La Torre, J., Ochoa, M., Gallardo, G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Low-cost+customized+cranioplasty+using+a+3D+digital+printing+model:+a+case+report.">Low-cost customized cranioplasty using a 3D digital printing model: a case report.</a> 3D Printing in Medicine. 4 (1), 1-9 (2018).</li><li>Kamio, T., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Utilizing+a+low-cost+desktop+3D+printer+to+develop+a+"one-stop+3D+printing+lab"+for+oral+and+maxillofacial+surgery+and+dentistry+fields.">Utilizing a low-cost desktop 3D printer to develop a "one-stop 3D printing lab" for oral and maxillofacial surgery and dentistry fields.</a> 3D Printing in Medicine. 4 (1), 1-7 (2018).</li><li>Punyaratabandhu, T., Liacouras, P., Pairojboriboon, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Using+3D+models+in+orthopedic+oncology:+presenting+personalized+advantages+in+surgical+planning+and+intraoperative+outcomes.">Using 3D models in orthopedic oncology: presenting personalized advantages in surgical planning and intraoperative outcomes.</a> 3D Printing in Medicine. 4 (1), 1-13 (2018).</li><li>Wake, N., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Patient-specific+3D+printed+and+augmented+reality+kidney+and+prostate+cancer+models:+impact+on+patient+education.">Patient-specific 3D printed and augmented reality kidney and prostate cancer models: impact on patient education.</a> 3D Printing in Medicine. 5 (1), 1-8 (2019).</li><li>Barber, S. R., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Augmented+Reality,+Surgical+Navigation,+and+3D+Printing+for+Transcanal+Endoscopic+Approach+to+the+Petrous+Apex.">Augmented Reality, Surgical Navigation, and 3D Printing for Transcanal Endoscopic Approach to the Petrous Apex.</a> OTO Open: The Official Open Access Journal of the American Academy of Otolaryngology-Head and Neck Surgery Foundation. 2 (4), (2018).</li><li>Witowski, J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Augmented+reality+and+three-dimensional+printing+in+percutaneous+interventions+on+pulmonary+arteries.">Augmented reality and three-dimensional printing in percutaneous interventions on pulmonary arteries.</a> Quantitative Imaging in Medicine and Surgery. 9 (1), (2019).</li><li>Moreta-Mart&#237;nez, R., Garc&#237;a-Mato, D., Garc&#237;a-Sevilla, M., P&#233;rez-Ma&#241;anes, R., Calvo-Haro, J., Pascau, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Augmented+reality+in+computer-assisted+interventions+based+on+patient-specific+3D+printed+reference.">Augmented reality in computer-assisted interventions based on patient-specific 3D printed reference.</a> Healthcare Technology Letters. , (2018).</li></ol>

The authors have nothing to disclose.

AR holds great potential in education, training, and surgical guidance in the medical field. Its combination with 3D printing opens may open new possibilities in clinical applications. This protocol describes a methodology that enables inexperienced users to create a smartphone app combining AR and 3DP for the visualization of anatomical 3D models of patients with 3D-printed reference markers.
In general, one of the most interesting clinical applications of AR and 3DP is to improve patient-to-physician communication by giving the patient a different perspective of the case, improving explanations of specific medical conditions or treatments. Another possible application includes surgical guidance for target localization, in which 3D-printed patient-specific tools (with a reference AR marker attached) can be placed on rigid structures (i.e., bone) and used as a reference for navigation. This application is especially useful for orthopedic and maxillofacial surgical procedures, in which bone tissue surface is easily accessed during surgery.
The protocol starts with section 1, describing the workstation set-up and software tools necessary. Section 2 describes how to use 3D Slicer software to easily segment target anatomies of the patient from any medical imaging modality to obtain 3D models. This step is crucial, as the virtual 3D models created are those displayed in the final AR application.
In section 3, 3D Slicer is used to register the 3D models created in the previous section with an AR marker. During this registration procedure, patient 3D models are efficiently and simply positioned with respect to the AR marker. The position defined in this section will determine the hologram relative position in the final app. It is believed that this solution reduces complexity and multiplies the possible applications. Section 3 describes two different options to define the spatial relationships between the models and AR markers: "visualization" and "registration" mode. The first option, "visualization" mode, allows the 3D models to be positioned anywhere with respect to the marker and displayed as the whole biomodel. This mode provides a realistic, 3D perspective of the patient&#39;s anatomy and allows moving and rotating of the biomodels by moving the tracked AR marker. The second option, "registration" mode, allows attachment and combining of a marker adaptor to any part of the biomodel, offering an automatic registration process. With this option, a small section of the 3D model, including the marker adaptor, can be 3D-printed, and the app can display the rest of the model as a hologram.
Section 4 provides guidelines for the 3D printing process. First, the user can choose between two different markers: the "dual color marker" and "sticker marker". The whole "dual color marker" can be 3D-printed but requires a dual extruder 3D printer. In case this printer is not available, the "sticker marker" is proposed. This is a simpler marker that can be obtained by 3D-printing the cubic structure, then pasting the images of the cube with sticker paper or sticker glue. Furthermore, both markers were designed with extensible sections to perfectly fit in a specific adaptor. Thus, the marker can be reused in several cases.
Section 5 describes the process to create a Unity project for AR using the Vuforia software development kit. This step may be the hardest portion for users with no programming experience, but with these guidelines, it should be easier to obtain the final application that is presented in section 6. The app displays the patient&#39;s virtual models over the smartphone screen when the camera recognizes the 3D-printed marker. In order for the app to detect the 3D marker, a minimum distance of approximately 40 cm or less from the phone to the marker as well as good lighting conditions are required.
The final application of this protocol allows the user to choose the specific biomodels to visualize and in which positions. Addtionally, the app can perform automatic patient-hologram registration using a 3D-printed marker and adaptor attached to the biomodel. This solves the challenge of registering virtual models with the environment in a direct and convenient manner. Moreover, this methodology does not require broad knowledge of medical imaging or software development, does not depend on complex hardware and expensive software, and can be implemented over a short time period. It is expected that this method will help accelerate the adoption of AR and 3DP technologies by medical professionals.

AR and 3DP are technologies that provide increasing numbers of applications in the medical field. In the case of AR, its interaction with virtual 3D models and the real environment benefits physicians in regards to education and training1,2,3, communication and interactions with other physicians4, and guidance during clinical interventions5,6,7,8,9,10. Likewise, 3DP has become a powerful solution for physicians when developing patient-specific customizable tools11,12,13 or creating 3D models of a patient&rsquo;s anatomy, which can help improve preoperative planning and clinical interventions14,15.
Both AR and 3DP technologies help to improve orientation, guidance, and spatial skills in medical procedures; thus, their combination is the next logical step. Previous work has demonstrated that their joint use can increase value in patient education16, facilitating explanations of medical conditions and proposed treatment, optimizing surgical workflow17,18 and improving patient-to-model registration19. Although these technologies have grown exponentially in recent years, their adoption by physicians is still limited, since they require extensive knowledge of engineering and software development. Therefore, the purpose of this work is to describe a step-by-step methodology that enables the use of AR and 3DP by inexperienced users without the need for broad technical knowledge.
This protocol describes how to develop an AR smartphone app that allows the superimposing of any patient-based 3D model onto a real-world environment using a 3D-printed marker tracked by the smartphone camera. In addition, an alternative approach is described to provide automatic registration between a 3D-printed biomodel (i.e., a 3D model created from a patient&rsquo;s anatomy) and the projected holograms. The protocol described is entirely based on free and multi-platform software.
In previous work, AR patient-to-image registration has been calculated manually5 with surface recognition algorithms10 or has been unavailable2. These methods have been considered somewhat limited when an accurate registration is required19. To overcome these limitations, this work provides tools to perform accurate and simple patient-to-image registration in AR procedures by combining AR technology and 3DP.
The protocol is generic and can be applied to any medical imaging modality or patient. As an example, a real clinical case of a patient suffering from distal leg sarcoma is provided to illustrate the methodology. The first step describes how to easily segment the affected anatomy from computed tomography (CT) medical images to generate 3D virtual models. Afterward, positioning of the 3D models is performed, then the required tools and models are 3D-printed. Finally, the desired AR app is deployed. This app allows for the visualization of patient 3D models overlaid on a smartphone camera in real-time.

<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td>3D Printing material: Acrylonitrile Butadiene Styrene (ABS)</td>
			<td></td>
			<td></td>
			<td>Thermoplastic polymer material usually used in domestic 3D printers.</td>
		</tr>
		<tr>
			<td>3D Printing material: Polylactic Acid (PLA)</td>
			<td></td>
			<td></td>
			<td>Bioplastic material usually used in domestic 3D printers.</td>
		</tr>
		<tr>
			<td>3D Slicer</td>
			<td></td>
			<td></td>
			<td>Open-source software platform for medical image informatics, image processing, and three-dimensional visualization</td>
		</tr>
		<tr>
			<td>Android</td>
			<td>Alphabet, Inc.</td>
			<td></td>
			<td>Android is a mobile operating system developed by Google. It is based on a modified version of the Linux kernel and other open source software, and is designed primarily for touchscreen mobile devices such as smartphones and tablets.</td>
		</tr>
		<tr>
			<td>Autodesk Meshmixer</td>
			<td>Autodesk, Inc.</td>
			<td></td>
			<td>Meshmixer is state-of-the-art software for working with triangle meshes. Free software.</td>
		</tr>
		<tr>
			<td>iPhone OS</td>
			<td>Apple, Inc.</td>
			<td></td>
			<td>iPhone OS is a mobile operating system created and developed by Apple Inc. exclusively for its hardware.</td>
		</tr>
		<tr>
			<td>Ultimaker 3 Extended</td>
			<td>Ultimaker BV</td>
			<td></td>
			<td>Fused deposition modeling 3D printer.</td>
		</tr>
		<tr>
			<td>Unity</td>
			<td>Unity Technologies</td>
			<td></td>
			<td>Unity is a real-time development platform to create 3D, 2D VR &#38; AR visualizations for Games, Auto, Transportation, Film, Animation, Architecture, Engineering &#38; more. Free software.</td>
		</tr>
		<tr>
			<td>Xcode</td>
			<td>Apple, Inc.</td>
			<td></td>
			<td>Xcode is a complete developer toolset for creating apps for Mac, iPhone, iPad, Apple Watch, and Apple TV. Free software.</td>
		</tr>
	</tbody>
</table>

combining augmented reality and 3d printing to display patient models on a smartphone

Augmented reality (AR) has great potential in education, training, and surgical guidance in the medical field. Its combination with three-dimensional (3D) printing (3DP) opens new possibilities in clinical applications. Although these technologies have grown exponentially in recent years, their adoption by physicians is still limited, since they require extensive knowledge of engineering and software development. Therefore, the purpose of this protocol is to describe a step-by-step methodology enabling inexperienced users to create a smartphone app, which combines AR and 3DP for the visualization of anatomical 3D models of patients with a 3D-printed reference marker. The protocol describes how to create 3D virtual models of a patient&rsquo;s anatomy derived from 3D medical images. It then explains how to perform positioning of the 3D models with respect to marker references. Also provided are instructions for how to 3D print the required tools and models. Finally, steps to deploy the app are provided. The protocol is based on free and multi-platform software and can be applied to any medical imaging modality or patient. An alternative approach is described to provide automatic registration between a 3D-printed model created from a patient&rsquo;s anatomy and the projected holograms. As an example, a clinical case of a patient suffering from distal leg sarcoma is provided to illustrate the methodology. It is expected that this protocol will accelerate the adoption of AR and 3DP technologies by medical professionals.

The protocol was applied to data from a patient suffering from distal leg sarcoma in order to visualize the affected anatomical region from a 3D perspective. Using the method described in section 2, the portion of the affected bone (here, the tibia and fibula) and tumor were segmented from the patient&#39;s CT scan. Then, using the segmentation tools from 3D Slicer, two biomodels were created: the bone (section of the tibia and fibula) (Figure 1A) and tumor (Figure 1B).
Next, the two 3D models were positioned virtually with respect to the marker for optimal visualization. Both modes described in section 3 were followed for this example. For visualization mode, the models were centered in the upper face of the marker (Figure 2). For registration mode, the marker adaptor was positioned in the bone (specifically, the tibia [Figure 3]). Then, a small section of the tibia was selected to be 3D-printed with a 3D marker adaptor (Figure 4). An Ultimaker 3 extended 3D printer with PLA material was used to create the 3D-printed markers (Figure 5A, B), marker holder base (Figure 5C) for the "visualization" mode, and section of the tibia for "registration" mode (Figure 5D). Figure 5E shows how the marker was attached to the "visualization" mode 3D-printed base. Figure 5F shows the attachment with the "registration" mode 3D-printed biomodel. Finally, Unity was used to create the app and deploy it on the smartphone.
Figure 6 shows how the app worked for "visualization" mode. The hologram was accurately located in the upper part of the cube as previously defined. Figure 7 shows the application for "registration" mode, in which the app positioned the complete bone model on top of the 3D-printed section. The final visualization of the holograms was clear and realistic, maintained the real sizes of the biomodels, and positioned accurately. When using the smartphone application, the AR marker needs to be visible by the camera for the app to correctly display the holograms. In addition, the light conditions in the scene must be of good quality and constant for proper marker detection. Bad light conditions or reflections on the marker surface hinder the tracking of the AR marker and cause malfunctioning of the app.
The time required to create the app depends on several factors. The duration of section 1 is limited by the download speed. Regarding anatomy segmentation (section 2), factors affecting segmentation time include complexity of the region and medical imaging modality (i.e., CT is easily segmented, while MRI is more difficult). For the representative example of the tibia, approximately 10 min was required to generate both 3D models from the CT scan. Biomodel positioning (section 3) is simple and straightforward. Here, it took approximately 5 min to define the biomodel position with respect to the AR marker. For the 3D printing step, the duration is highly dependent on the selected mode. The "dual color marker" was manufactured at high quality in a period of 5 h and 20 min. The "sticker marker" was manufactured in a period of 1 h and 30 min, plus the time required to paste the stickers. The final step for app development can be time-consuming for those with no previous experience in Unity, but it can be easily completed following the protocol steps. Once the AR markers have been 3D-printed, the development of an entirely new AR app can be performed in less than 1 h. This duration can be further reduced with additional experience.
<img alt="Figure 1" class="xfigimg" src="/files/ftp_upload/60618/60618fig01.jpg" /> 
Figure 1: Representation of 3D models created from a CT image of a patient suffering from distal leg sarcoma. (A) Bone tissue represented in white (tibia and fibula). (B) Tumor represented in red. <a href="https://www.jove.com/files/ftp_upload/60618/60618fig01large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Figure 2" class="xfigimg" src="/files/ftp_upload/60618/60618fig02.jpg" /> 
Figure 2: Results showing how "visualization" mode in 3D Slicer positions the virtual 3D models of the bone and tumor with respect to 3D-printed marker reference. The patient 3D models (A) are positioned above the upper face of the marker cube (B). <a href="https://www.jove.com/files/ftp_upload/60618/60618fig02large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Figure 3" class="xfigimg" src="/files/ftp_upload/60618/60618fig03.jpg" /> 
Figure 3: Results showing how "registration" mode in 3D Slicer positions the virtual 3D models of the bone and tumor (A) with respect to 3D-printed marker reference (B). The marker adaptor is attached to the bone tissue model. <a href="https://www.jove.com/files/ftp_upload/60618/60618fig03large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Figure 4" class="xfigimg" src="/files/ftp_upload/60618/60618fig04.jpg" /> 
Figure 4: Small section of the bone tissue and 3D marker adaptor. The two components are combined then 3D-printed. <a href="https://www.jove.com/files/ftp_upload/60618/60618fig04large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Figure 5" class="xfigimg" src="/files/ftp_upload/60618/60618fig05.jpg" /> 
Figure 5: 3D printed tools required for the final application. (A) "Two color cube marker" 3D-printed with two colors of materials. (B) "Sticker cube marker" 3D-printed, with stickers pasted. (C) Marker base cube adaptor. (D) Section of the patient&#39;s bone tissue 3D model and marker cube adaptor. (E) "Sticker cube marker" placed in the marker base cube adaptor. (F) "Two color cube marker" placed in the marker adaptor attached to the patient&#39;s anatomy. <a href="https://www.jove.com/files/ftp_upload/60618/60618fig05large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Figure 6" class="xfigimg" src="/files/ftp_upload/60618/60618fig06.jpg" /> 
Figure 6: App display when using "visualization" mode. The patient&#39;s affected anatomy 3D models are positioned above the upper face of the 3D-printed cube. <a href="https://www.jove.com/files/ftp_upload/60618/60618fig06large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>
<img alt="Figure 7" class="xfigimg" src="/files/ftp_upload/60618/60618fig07.jpg" /> 
Figure 7: AR visualization when using "registration" mode. The 3D-printed marker enables registration of the 3D-printed biomodel with the virtual 3D models. <a href="https://www.jove.com/files/ftp_upload/60618/60618fig07large.jpg" target="_blank">Please click here to view a larger version of this figure.</a>

Watch this Scientific Journal Video about Combining Augmented Reality and 3D Printing to Display Patient Models on a Smartphone at JoVE.com

Combining Augmented Reality and 3D Printing to Display Patient Models on a Smartphone

Presented here is a method to design an augmented reality smartphone application for the visualization of anatomical three-dimensional models of patients using a 3D-printed reference marker.

Augmented reality (AR) has great potential in education, training, and surgical guidance in the medical field. Its combination with three-dimensional (3D) ...

Research

JoVE Journal

Medicine

Step By Step: Microsurgical training method combining two nonliving animal models

The learning of microsurgical techniques and the maintenance of microsurgical skills have been traditionally based on the use of living animals, mainly ...

Smartphone Fundus Photography

Smartphone fundus photography is a simple technique to obtain ocular fundus pictures using a smartphone camera and a conventional handheld indirect ...

Integrating Augmented Reality Tools in Breast Cancer Related Lymphedema Prognostication and Diagnosis

Breast cancer related lymphedema (BCRL) is a detrimental condition characterized by fluid accumulation in the upper limb in breast cancer patients ...

Multicolor 3D Printing of Complex Intracranial Tumors in Neurosurgery

Three-dimensional (3D) printing technologies offer the possibility of visualizing patient-specific pathologies in a physical model of correct dimensions. ...

3D Planning and Printing of Patient Specific Implants for Reconstruction of Bony Defects

We are in the midst of the 3D era in most aspects of life, and especially in medicine. The surgical discipline is one of the major players in the medical ...

Treatment of Facial Deformities using 3D Planning and Printing of Patient-Specific Implants

Technological advancements in surgical planning and patient-specific implants are constantly evolving. One can either adopt the technology to achieve ...

Combining 3D-Printing and Electrospinning to Manufacture Biomimetic Heart Valve Leaflets

Electrospinning has become a widely used technique in cardiovascular tissue engineering as it offers the possibility to create (micro-)fibrous scaffolds ...

Augmented Reality Navigation-Guided Core Decompression for Osteonecrosis of Femoral Head

Osteonecrosis of the femoral head (ONFH) is a common joint disease in young and middle-aged patients, which seriously burdens their lives and work. For ...

3D Printing Model of a Patient's Specific Lumbar Vertebra

Selective dorsal rhizotomy (SDR) is a difficult, risky, and sophisticated operation, in which a laminectomy should not only expose an adequate surgical ...

Plasma Polishing as a New Polishing Option to Reduce the Surface Roughness of Porous Titanium Alloy for 3D Printing

Porous titanium alloy implants with simulated trabecular bone fabricated by 3D printing technology have broad prospects. However, due to the fact that ...