Name: Pleins feux sur l’auteur : Révolutionner la chirurgie à distance grâce à la réalité augmentée et à la robotique pour une précision et une accessibilité accrues
Uploaded: 2024-08-09T14:25:00
Description: Augmented Reality (AR) is in high demand in medical applications. The aim of the paper is to provide automatic surgery using AR for the Transcatheter ...

Les auteurs ne reconnaissent aucun financement pour cette recherche.

1. Environnement chirurgical
<ol><li>Concevez un environnement chirurgical comme illustré à la figure 1. Assurez-vous que l’environnement dispose d’un plateau de transport d’objets, d’un bras robotique et de deux bras suspendus sur le côté, l’un pour tenir un espace réservé à la caméra et l’autre pour avoir un fond blanc cohérent avec le module de pesée pour l’équilibre.</li>
	<li>Développer deux pilotes, l’un pour l’instantané de l’environ...

Reconstruction 3D optimisée et cartographie du mouvement pour la chirurgie assistée par AR à distance

<ol>
	<li>Wang, L. J., Casto, B., Reyes-Molyneux, N., Chance, W. W., Wang, S. J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Smartphone-based+augmented+reality+patient+education+in+radiation+oncology.">Smartphone-based augmented reality patient education in radiation oncology.</a> Tech Innov Patient Supp Radiation Oncol. 29, 100229 (2024).</li><li>Guerroudji, M. A., Amara, K., Lichouri, M., Zenati, N., Masmoudi, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+3D+visualization-based+augmented+reality+application+for+brain+tumor+segmentation.">A 3D visualization-based augmented reality application for brain tumor segmentation.</a> Comput Anim Virtual Worlds. 35 (1), e2223 (2024).</li><li>Xia, L., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Augmented+reality+and+indoor+positioning+based+mobile+production+monitoring+system+to+support+workers+with+human-in-the-loop.">Augmented reality and indoor positioning based mobile production monitoring system to support workers with human-in-the-loop.</a> Robotic Comput Integ Manufac. 86, 102664 (2024).</li><li>Preece, C., Skandalis, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Time+to+imagine+an+escape:+investigating+the+consumer+timework+at+play+in+augmented+reality.">Time to imagine an escape: investigating the consumer timework at play in augmented reality.</a> Eur J Market. 58, 92-118 (2024).</li><li>Suresh, D., Aydin, A., James, S., Ahmed, K., Dasgupta, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+role+of+augmented+reality+in+surgical+training:+a+systematic+review.">The role of augmented reality in surgical training: a systematic review.</a> Surg Innov. 30, 366-382 (2023).</li><li>Moreta-Martinez, R., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Combining+augmented+reality+and+3D+printing+to+display+patient+models+on+a+smartphone.">Combining augmented reality and 3D printing to display patient models on a smartphone.</a> J Vis Exp. (155), 60618 (2020).</li><li>Ma, L., Huang, T., Wang, J., Liao, H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Visualization,+registration+and+tracking+techniques+for+augmented+reality+guided+surgery:+a+review.">Visualization, registration and tracking techniques for augmented reality guided surgery: a review.</a> Phys Med Biol. 68, 04TR02 (2023).</li><li>Hofman, J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=First-in-human+real-time+ai-assisted+instrument+deocclusion+during+augmented+reality+robotic+surgery.">First-in-human real-time ai-assisted instrument deocclusion during augmented reality robotic surgery.</a> Healthc Technol Lett. 11 (2-3), 33-39 (2023).</li><li>Thiene, G., Rizzo, S., Basso, C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Bicuspid+aortic+valve:+The+most+frequent+and+not+so+benign+congenital+heart+disease.">Bicuspid aortic valve: The most frequent and not so benign congenital heart disease.</a> Cardiovasc Pathol. 70, 107604 (2024).</li><li>Mack, M. J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Transcatheter+aortic-valve+replacement+in+low-risk+patients+at+five+years.">Transcatheter aortic-valve replacement in low-risk patients at five years.</a> New Engl J Med. 389, 1949-1960 (2023).</li><li>Vitanova, K., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Aortic+valve+versus+root+surgery+after+failed+transcatheter+aortic+valve+replacement.">Aortic valve versus root surgery after failed transcatheter aortic valve replacement.</a> J Thorac Cardiovas Surg. 166, 1418-1430 (2023).</li><li>Bydlon, T. M., Torjesen, A., Fokkenrood, S., Di Tullio, A., Flexman, M. L. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=3d+visualisation+of+navigation+catheters+for+endovascular+procedures+using+a+3d+hub+and+fiber+optic+realshape+technology:+phantom+study+results.">3d visualisation of navigation catheters for endovascular procedures using a 3d hub and fiber optic realshape technology: phantom study results.</a> EJVES Vascular Forum. 59, 24-30 (2023).</li><li>Faris, H., Harfouche, C., Bandle, J., Wisbach, G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Surgical+tele-mentoring+using+a+robotic+platform:+initial+experience+in+a+military+institution.">Surgical tele-mentoring using a robotic platform: initial experience in a military institution.</a> Surg Endosc. 37, 9159-9166 (2023).</li><li>Fitzgerald, L., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Mentoring+approaches+in+a+safe+surgery+program+in+tanzania:+Lessons+learned+during+covid-19+and+recommendations+for+the+future.">Mentoring approaches in a safe surgery program in tanzania: Lessons learned during covid-19 and recommendations for the future.</a> Surg Open Sci. 14, 109-113 (2023).</li><li>de Turenne, A., Eug&#232;ne, F., Blanc, R., Szewczyk, J., Haigron, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Catheter+navigation+support+for+mechanical+thrombectomy+guidance:+3d/2d+multimodal+catheter-based+registration+with+no+contrast+dye+fluoroscopy.">Catheter navigation support for mechanical thrombectomy guidance: 3d/2d multimodal catheter-based registration with no contrast dye fluoroscopy.</a> Int J Comput Assist Radiol Surg. 19 (3), 459-468 (2023).</li><li>Tang, H., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+multiple+catheter+tips+tracking+method+in+x-ray+fluoroscopy+images+by+a+new+lightweight+segmentation+network+and+bayesian+filtering.">A multiple catheter tips tracking method in x-ray fluoroscopy images by a new lightweight segmentation network and bayesian filtering.</a> Int J Med Robotics Comput Assist Surg. 19, e2569 (2023).</li><li>Annabestani, M., Caprio, A., Wong, S. C., Mosadegh, B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+machine+learning-based+roll+angle+prediction+for+intracardiac+echocardiography+catheter+during+bi-plane+fluoroscopy.">A machine learning-based roll angle prediction for intracardiac echocardiography catheter during bi-plane fluoroscopy.</a> Appl Sci. 13, 3483 (2023).</li><li>Thourani, V. H., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Survival+after+surgical+aortic+valve+replacement+in+low-risk+patients:+a+contemporary+trial+benchmark.">Survival after surgical aortic valve replacement in low-risk patients: a contemporary trial benchmark.</a> Ann Thorac Surg. 117, 106-112 (2024).</li><li>Dom&#237;nguez-Velasco, C. F., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Augmented+reality+simulation+as+training+model+of+ventricular+puncture:+Evidence+in+the+improvement+of+the+quality+of+punctures.">Augmented reality simulation as training model of ventricular puncture: Evidence in the improvement of the quality of punctures.</a> Int J Med Robotics Comput Assist Surg. 19, e2529 (2023).</li><li>Wang, Q., Xie, Z. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Arias:+An+ar-based+interactive+advertising+system.">Arias: An ar-based interactive advertising system.</a> Plos One. 18, e0285838 (2023).</li><li>Ji, F., Chong, F., Wang, F., Bai, D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Augmented+reality+platform+for+the+unmanned+mining+process+in+underground+mines.">Augmented reality platform for the unmanned mining process in underground mines.</a> Mining,Metal Explor. 39 (2), 385-395 (2022).</li><li>Indhumathi, S., Clement, J. C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Directional+intensified+feature+description+using+tertiary+filtering+for+augmented+reality+tracking.">Directional intensified feature description using tertiary filtering for augmented reality tracking.</a> Sci Rep. 13, 20311 (2023).</li><li>Gao, L., Zhao, Y., Han, J., Liu, H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Research+on+multi-view+3D+reconstruction+technology+based+on+SFM.">Research on multi-view 3D reconstruction technology based on SFM.</a> Sensors. 22 (12), 4366 (2022).</li><li>Su&#225;rez, I., Sfeir, G., Buenaposada, J. M., Baumela, L. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=BEBLID:+Boosted+efficient+binary+local+image+descriptor.">BEBLID: Boosted efficient binary local image descriptor.</a> Pattern Recog Lett. 133, 366-372 (2020).</li><li>Borman, R. I., Harjoko, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Improved+ORB+algorithm+through+feature+point+optimization+and+Gaussian+pyramid.">Improved ORB algorithm through feature point optimization and Gaussian pyramid.</a> Int J Adv Comp Sci Appl. 15 (2), 268-275 (2024).</li></ol>

Dans un travail existant15, des radiographies et des tomodensitogrammes sont examinés pour localiser le cathéter dans le cœur. Cependant, le remplacement de l’AR TAVI ouvre de nouvelles possibilités dans la procédure chirurgicale TAVR18 par la mise en œuvre d’un modèle automatisé utilisant la reconstruction 3D. Comme mentionné dans la section protocole, la conception de ce travail comporte cinq étapes. La première étape de DITF22, men...

La RA peut superposer le modèle 3D dans un environnement réel. Le développement technologique vers la RA a opéré un changement de paradigme dans de nombreux domaines, à savoir l’éducation1, la médecine2, la fabrication3 et le divertissement4. La technologie AR, associée à une communication ultra-fiable à faible latence, prouve son rôle incontournable dans le domaine médical. De l’étape d’apprentissage de l’anatomie humaine au guidage chirurgical, les étapes de l’apprentissage peuvent être visualisées à l’aidede logiciels et de matériel alimentés par la RA. La RA fournit une solution cruciale et fiable au médecin dans un environnement chirurgical 7,8.
La sténose de la valve aortique est la maladie des valves cardiaques, qui est la plus fréquente chez l’humanité9. La cause profonde de la maladie est de mauvaises habitudes alimentaires et des routines irrégulières de la vie quotidienne. Le symptôme et le résultat de la maladie est le rétrécissement de la valve cardiaque, suivi d’une réduction du flux sanguin. Ce problème doit être résolu avant que tout dommage ne soit causé au cœur humain. Ainsi, le cœur est surchargé pour traiter le flux sanguin. Ainsi, avant que tout dommage ne se produise, une intervention chirurgicale doit être effectuée, ce qui, en raison des développements technologiques de ces derniers jours, peut également être effectué à l’aide de la procédure TAVI. La procédure peut être adoptée en fonction de l’état du cœur et d’autres parties du corps des patients. Dans ce TAVI10,11, le cathéter est inséré pour remplacer la valve endommagée dans le cœur. Cependant, placer le cathéter en position12 pour remplacer la valve est fastidieux pour le praticien. Cette idée nous a motivés à concevoir un modèle de chirurgie automatisé basé sur l’AR13,14, qui aide le chirurgien à positionner précisément la valve pendant le processus de remplacement. De plus, l’opération peut être réalisée par un algorithme de cartographie du mouvement, qui cartographie le mouvement du chirurgien capturé à distance jusqu’au bras robotique.
Dans le travail existant 15,16,17, la visualisation de la procédure TAVI 18 est surveillée par fluoroscopie. Par conséquent, il est difficile d’analyser la valve cardiaque et fastidieux de trouver l’emplacement de remplacement. Cela crée un obstacle au positionnement du cathéter dans le cœur humain. De plus, le mouvement à distance est mappé au champ opératoire pour automatiser le processus. Cependant, pour combler le manque de recherche, nous proposons une chirurgie robotisée automatisée pour le remplacement des valves à l’aide de la technologie assistée par AR.
Le protocole est un modèle générique qui peut être appliqué à tous les environnements chirurgicaux. Au début du travail, des images 2D sont capturées tout autour de l’environnement chirurgical avec la résolution spatiale la plus complète et le plus grand degré de liberté. Cela signifie qu’un nombre suffisant d’images sont capturées pour la reconstruction 3D19 objectif, suivie d’une cartographie de mouvement par le suivi des gestes de la main20.

<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td>android IDE</td>
			<td>software</td>
			<td>https://developer.android.com/studio</td>
			<td>software can be downloaded from this link</td>
		</tr>
		<tr>
			<td>Arduino Board</td>
			<td>Ardunio Uno</td>
			<td>Ardunio Uno</td>
			<td>Microcontroller for processing</td>
		</tr>
		<tr>
			<td>arduino software</td>
			<td>software</td>
			<td>https://www.arduino.cc/en/software.</td>
			<td>software can be downloaded from this link</td>
		</tr>
		<tr>
			<td>Human Heart phantom model</td>
			<td>Biology Lab Equipment Manufacturer and Exporter</td>
			<td>B071YBLX2V(8B-ZB2Q-H3MS-1)</td>
			<td>light weight model with 3parts to the deep analysis of heart.</td>
		</tr>
		<tr>
			<td>mobile holder</td>
			<td>Humble universal monopoad holder</td>
			<td>B07S9KNGVS</td>
			<td>To carry the mobile in surgical field</td>
		</tr>
		<tr>
			<td>pycharm IDE</td>
			<td>software</td>
			<td>https://www.jetbrains.com/pycharm/</td>
			<td>software can be downloaded from this link</td>
		</tr>
		<tr>
			<td>Robot arm</td>
			<td>Printed-bots</td>
			<td>B08R2JLKYM(P0-E2UT-JSOU)</td>
			<td>arm can be controlled through control signal.it has 5 degree of freedom to access.</td>
		</tr>
		<tr>
			<td>servo motor</td>
			<td>Kollmorgen Co-Engineers Motors</td>
			<td>MG-966R</td>
			<td>high-torque servo motor,servo pulses ranging from 500 to 2500 microseconds (&#181;s), with a frequency of 50Hz to 333Hz.&#160;</td>
		</tr>
		<tr>
			<td>servomotor</td>
			<td>Kollmorgen Co-Engineers&#160;Motors</td>
			<td>SG-90R</td>
			<td>1.8 kg-cm to 2.5 kg-cm load can be applied to SG-90R servo.</td>
		</tr>
		<tr>
			<td>Stepper Motor</td>
			<td>28BYJ-48</td>
			<td>28BYJ-48</td>
			<td>Steper motor, 5V DC, 100 Hz frequency, torque 1200 Gf.cm</td>
		</tr>
		<tr>
			<td>Stepper Motor</td>
			<td>Nema 23</td>
			<td>Nema</td>
			<td>Steper motor, 9V - 42 V DC, 100 Hz frequency</td>
		</tr>
	</tbody>
</table>

automatic surgery in transcatheter aortic valve replacement using augmented reality

La réalité augmentée (RA) est très demandée dans les applications médicales. L’objectif de l’article est de fournir une chirurgie automatique utilisant l’AR pour le remplacement transcathéter de la valve aortique (TAVI). Le TAVI est la procédure médicale alternative pour la chirurgie à cœur ouvert. Le TAVI remplace la valve blessée par la nouvelle à l’aide d’un cathéter. Dans le modèle existant, le guidage à distance est donné, tandis que la chirurgie n’est pas automatisée sur la base de la RA. Dans cet article, nous avons déployé une caméra alignée spatialement qui est reliée à un moteur pour l’automatisation de la capture d’images dans l’environnement chirurgical. La caméra suit l’image 2D haute résolution du cœur du patient ainsi que le banc d’essai du cathéter. Ces images capturées sont téléchargées à l’aide de l’application mobile vers un chirurgien à distance qui est un expert en cardiologie. Cette image est utilisée pour la reconstruction 3D à partir du suivi d’images 2D. Ceci est affiché dans un HoloLens comme un émulateur dans un ordinateur portable. Le chirurgien peut inspecter à distance les images reconstruites en 3D avec des fonctions de transformation supplémentaires telles que la rotation et la mise à l’échelle. Ces fonctionnalités de transformation sont activées par des gestes de la main. Les conseils du chirurgien sont transmis à l’environnement chirurgical pour automatiser le processus en temps réel. Le banc d’essai du cathéter dans le domaine chirurgical est contrôlé par le guidage gestuel de la main du chirurgien à distance. Le modèle prototype développé démontre l’efficacité du guidage chirurgical à distance par AR.

AR can superimpose the 3D model in a real-world environment. The technological development towards AR has made a paradigm shift in many fields, namely education1, medical2, manufacturing3, and entertainment4. AR technology, along with ultra-reliable low-latency communication, proves its inevitable role in the medical field. From the learning stage of human anatomy to surgical guidance, the stages of learning can be visualized with AR-powered software5,6 and hardware. AR provides a crucial and reliable solution to the medical practitioner in a surgical environment7,8.
Aortic valve stenosis is the heart valve disease, which is most common among mankind9. The root cause of the disease is poor food habits and irregular routines of day-to-day life. The symptom and result of the disease is the narrowing of the heart valve, followed by a reduction in the blood flow. This problem needs to be addressed before any damage takes place to the human heart. Thus, the heart is overburdened to process the blood flow. So, before any damage happens, surgery needs to be done, which, owing to technological developments in recent days, can also be done using the TAVR procedure. The procedure can be adopted based on the condition of the heart and other body parts of patients. In this TAVR10,11, the catheter is inserted to replace the damaged valve in the heart. However, placing the catheter position12 to replace the valve is tedious for the practitioner. This idea motivated us to design an automated surgery model based on AR13,14, which helps the surgeon to precisely position the valve during the replacement process. Moreover, the surgery can be performed by a motion mapping algorithm, which maps the surgeon&#39;s movement captured from a remote location to the robotic arm.
In the existing work15,16,17, the visualization of the TAVR18 procedure is monitored through fluoroscopy. Hence, it is difficult to analyze the heart valve and tedious to find the replacement location. This sets up a barrier to positioning the catheter in the human heart. In addition, the remote motion is mapped to the surgical field to make the process automated. However, to overcome the research gap, we propose an automated robotic-based surgery for valve replacement using AR-assisted technology.
The protocol is a generic model that can be applied to all surgical environments. In the beginning stage of the work, 2D images are captured all around the surgical environment with the fullest spatial resolution of the largest degree of freedom. This means that enough images are captured for 3D reconstruction19 purpose, followed by motion mapping through hand gesture tracking20.

Pleins feux sur l’auteur : Révolutionner la chirurgie à distance grâce à la réalité augmentée et à la robotique pour une précision et une accessibilité accrues

Chirurgie automatique dans le remplacement transcathéter de la valve aortique à l’aide de la réalité augmentée

Our research focuses on revolutionizing remote surgery through augmented reality and robotics, aiming to enhance precision and accessibility in medical procedures. We are exploring how real-time 3D reconstruction and feature extraction can provide surgeons with unprecedented control and visualization, even from a distance. We have implemented our own directional intensified feature extraction to ensure precise and accurate detection of surgical features, enhancing the surgeon's control over the robotic arm.By using 3D reconstruction from minimal 2D images, we have significantly reduced the hardware complexity and improved the efficiency of our system. Our system provides the best surveillance of the surgical environment using a single mobile camera synchronized to take snaps every two seconds, ensuring real-time updates for the remote surgeon. We have developed seamless communication protocols with low latency and high reliability using Bluetooth and LAN, ensuring uninterrupted and accurate data transmission.Our approach to achieve precise, real-time synchronization between video feeds and robotic arm responses can serve as a benchmark for future research in remote surgery and telemedicine. It offers a model for integrating real-time data with robotic control systems. Our approach to achieve precise real-time synchronization between video feeds and robotic arm responses can serve as a benchmark for future research in remote surgery and telemedicine.It offers a model for integrating real-time data with robotic control systems.

Le protocole a été testé avec le modèle du fantôme cardiaque. La figure 2 montre la configuration attendue pour la surveillance en direct du champ opératoire à l’aide de caméras distribuées dans l’espace. Les caméras distribuées, comme le montre la figure 2, aident à augmenter la résolution spatiale du champ pour une reconstruction 3D efficace. Cependant, la réalisation de l’emplacement physique de ces caméras dans divers endroits spatiaux i...

Cet article présente la conception et la mise en œuvre d’un module de chirurgie automatique basé sur la reconstruction 3D basée sur la réalité augmentée (RA). Le système permet la chirurgie à distance en permettant aux chirurgiens d’inspecter les caractéristiques reconstruites et de reproduire les mouvements chirurgicaux de la main comme s’ils effectuaient la chirurgie à proximité.

Augmented Reality (AR) is in high demand in medical applications. The aim of the paper is to provide automatic surgery using AR for the Transcatheter ...

automatic-surgery-transcatheter-aortic-valve-replacement-using

Research

JoVE Journal

Medicine

Automatic Surgery in Transcatheter Aortic Valve Replacement Using Augmented Reality

Augmented Reality (AR) is in high demand in medical applications. The aim of the paper is to provide automatic surgery using AR for the Transcatheter Aortic Valve Replacement (TAVR). TAVR is the alternate medical procedure for open-heart surgery. TAVR replaces the injured valve with the new one using a catheter. In the existing model, remote guidance is given, while the surgery is not automated based on AR. In this article, we deployed a spatially aligned camera that is connected to a motor for the automation of image capture in the surgical environment. The camera tracks the 2D high-resolution image of the patient's heart along with the catheter testbed. These captured images are uploaded using the mobile app to a remote surgeon who is a cardiology expert. This image is utilized for the 3D reconstruction from 2D image tracking. This is viewed in a HoloLens like an emulator in a laptop. The surgeon can remotely inspect the 3D reconstructed images with additional transformation features such as rotation and scaling. These transformation features are enabled through hand gestures. The surgeon's guidance is transmitted to the surgical environment to automate the process in real-time scenarios. The catheter testbed in the surgical field is controlled by the hand gesture guidance of the remote surgeon. The developed prototype model demonstrates the effectiveness of remote surgical guidance through AR.

This article presents the design and implementation of an automatic surgery module based on augmented reality (AR)- based 3D reconstruction. The system enables remote surgery by allowing surgeons to inspect reconstructed features and replicate surgical hand movements as if they were performing the surgery in close proximity.

Recherche

Médecine

Creating an Advanced Automated Surgical Environment with Surveillance and Real-Time Image Transmission

To prepare the surgery environment, arrange an object-carrying platter, a robotic arm, and two side-hanging arms on a working platform. After locking the mobile in the mobile holder, enable Bluetooth of the mobile device and the laptop, which serve as the surgeon's HoloLens emulator. Pair the devices for uninterrupted image transmission.Ensure that the hanging arms are controlled by a stepper motor for a flawless revolution of 360 degrees. Connect the motor to the microcontroller board using the TB6600 driver. Click the Download button to download the microcontroller integrated development environment from the browser.Then go to File and open a new sketch to write the code. Connect the microcontroller board to interface with the new sketch through a communication port number for connection port. Check the communication port and verify that it shows the microcontroller board.Check the hardware switch settings of the stepper motor driver TB6600 to ensure a current flow of two amperes. Set SW4 to on and SW5 and SW6 to off. Set SW1 and SW3 to off and SW2 to on to achieve 1/8 micro steps, meeting the required revolution steps.Connect RTC 3231 with the microcontroller to have real global time synchronization. Ensure the revolution step size is 12 degrees and the motor step increments only when the seconds read from the RTC module are odd. Connect the 5-volt pin of the microcontroller board to the RTCVCC and the microcontroller's ground to the RTC's ground.Connect the SCL pin of RTC to the A0 pin and the SDA to the A1 pin of the microcontroller. This module ensures a 12-degree step size, completing 30 steps per revolution with increments occurring every odd second. Run the code available on the GitHub page to verify the setup is working correctly.Download Android Studio to develop the automatic camera application. Connect the mobile phone to the system. In Android Studio, click New, New Project, and choose Empty Views Activity.Click on Next to develop an Android code available at the given site. Confirm that the app captures the images automatically and sends them to a remote device consistently. Transmit snapshots from the mobile app immediately to the remote surgeon system through Bluetooth.

Création d’un environnement chirurgical automatisé avancé avec surveillance et transmission d’images en temps réel

Developing Interactive Surgery Environment with Real-Time Surveillance and Augmented Reality Integration

To begin, open the graphical user interface server module. Enter the VVID port number in the VVID text field. Click on Create Socket to create and bind the socket.Then click on Connect to establish a connection with the mobile app. Now click on Capture to capture and save the scene surveillance images in the local folder. Let the client module include a robotic arm designed to have rotational movement in its base, shoulder, elbow, wrist, and fingers.Ensure that MG996R servos are used to govern the rotational movement at the base, shoulder, and elbow. Use the SG90 servo motor to control the rotational movement at the wrist and fingers. Compile the code given in the microcontroller integrated development environment to drive the robotic arm based on the commands received from the remote surgeon.Read two images at a time in a sequence, one by one, from the local folder to identify the possible overlapping between them. Extract the features using the DITF method. Reconstruct three-dimensional images from the collected features using SFM.Enable the surgeon to inspect the three-dimensional reconstructed image features using hand gesture controls for rotation and zoom in or out, allowing visualization from all perspectives. Normalize and map the distance between the tip of the surgeon's thumb and the index finger of the right hand into a corresponding angle of rotation. Transmit the hand gesture control through Bluetooth to the remote surgery environment.Ensure that the rotation of the object platter at the remote site mirrors the rotation of the three-dimensional reconstructed features at the surgeon's end. The accuracy of motion mapping decreased as the distance between two fingers increased. The DITF algorithm achieved the lowest latency compared to ORB and BEBLID.