Name: a swin transformer-based model for thyroid nodule detection in ultrasound images
Uploaded: 2023-04-21T14:15:00
Description: Watch this Scientific Journal Video about A Swin Transformer-Based Model for Thyroid Nodule Detection in Ultrasound Images at JoVE.com

This study was supported by the National Natural Science Foundation of China (Grant No.32101188) and the General Project of Science and Technology Department of Sichuan Province (Grant No. 2021YFS0102), China.

This retrospective study was approved by the institutional review board of the West China Hospital, Sichuan University, Sichuan, China, and the requirement to obtain informed consent was waived.
1. Environment setup
<ol>
	<li>Graphic processing unit (GPU) software
	<ol>
		<li>To implement deep learning applications, first configure the GPU-related environment. Download and install GPU-appropriate software and drivers from the GPU&#39;s website. 
		&#8203;NOTE: See the <...

<ol>
	<li>Grant, E. G., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Thyroid+ultrasound+reporting+lexicon:+White+paper+of+the+ACR+Thyroid+Imaging,+Reporting+and+Data+System+(TIRADS)+committee.">Thyroid ultrasound reporting lexicon: White paper of the ACR Thyroid Imaging, Reporting and Data System (TIRADS) committee.</a> Journal of the American College of Radiology. 12 (12 Pt A), 1272-1279 (2015).</li><li>Zhao, J., Zheng, W., Zhang, L., Tian, H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Segmentation+of+ultrasound+images+of+thyroid+nodule+for+assisting+fine+needle+aspiration+cytology.">Segmentation of ultrasound images of thyroid nodule for assisting fine needle aspiration cytology.</a> Health Information Science and Systems. 1, 5 (2013).</li><li>Haugen, B. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=American+Thyroid+Association+management+guidelines+for+adult+patients+with+thyroid+nodules+and+differentiated+thyroid+cancer:+What+is+new+and+what+has+changed.">American Thyroid Association management guidelines for adult patients with thyroid nodules and differentiated thyroid cancer: What is new and what has changed.</a> Cancer. 123 (3), 372-381 (2017).</li><li>Shin, J. H., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Ultrasonography+diagnosis+and+imaging-based+management+of+thyroid+nodules:+Revised+Korean+Society+of+Thyroid+Radiology+consensus+statement+and+recommendations.">Ultrasonography diagnosis and imaging-based management of thyroid nodules: Revised Korean Society of Thyroid Radiology consensus statement and recommendations.</a> Korean Journal of Radiology. 17 (3), 370-395 (2016).</li><li>Horvath, E., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=An+ultrasonogram+reporting+system+for+thyroid+nodules+stratifying+cancer+risk+for+clinical+management.">An ultrasonogram reporting system for thyroid nodules stratifying cancer risk for clinical management.</a> The Journal of Clinical Endocrinology &amp; Metabolism. 94 (5), 1748-1751 (2009).</li><li>Park, J. -. Y., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+proposal+for+a+thyroid+imaging+reporting+and+data+system+for+ultrasound+features+of+thyroid+carcinoma.">A proposal for a thyroid imaging reporting and data system for ultrasound features of thyroid carcinoma.</a> Thyroid. 19 (11), 1257-1264 (2009).</li><li>Moon, W. -. J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Benign+and+malignant+thyroid+nodules:+US+differentiation-Multicenter+retrospective+study.">Benign and malignant thyroid nodules: US differentiation-Multicenter retrospective study.</a> Radiology. 247 (3), 762-770 (2008).</li><li>Park, C. S., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Observer+variability+in+the+sonographic+evaluation+of+thyroid+nodules.">Observer variability in the sonographic evaluation of thyroid nodules.</a> Journal of Clinical Ultrasound. 38 (6), 287-293 (2010).</li><li>Kim, S. H., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Observer+variability+and+the+performance+between+faculties+and+residents:+US+criteria+for+benign+and+malignant+thyroid+nodules.">Observer variability and the performance between faculties and residents: US criteria for benign and malignant thyroid nodules.</a> Korean Journal of Radiology. 11 (2), 149-155 (2010).</li><li>Choi, Y. J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+computer-aided+diagnosis+system+using+artificial+intelligence+for+the+diagnosis+and+characterization+of+thyroid+nodules+on+ultrasound:+initial+clinical+assessment.">A computer-aided diagnosis system using artificial intelligence for the diagnosis and characterization of thyroid nodules on ultrasound: initial clinical assessment.</a> Thyroid. 27 (4), 546-552 (2017).</li><li>Chang, T. -. C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+role+of+computer-aided+detection+and+diagnosis+system+in+the+differential+diagnosis+of+thyroid+lesions+in+ultrasonography.">The role of computer-aided detection and diagnosis system in the differential diagnosis of thyroid lesions in ultrasonography.</a> Journal of Medical Ultrasound. 23 (4), 177-184 (2015).</li><li>Li, X. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Fully+convolutional+networks+for+ultrasound+image+segmentation+of+thyroid+nodules.">Fully convolutional networks for ultrasound image segmentation of thyroid nodules.</a> , 886-890 (2018).</li><li>Nguyen, D. T., Choi, J., Park, K. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Thyroid+nodule+segmentation+in+ultrasound+image+based+on+information+fusion+of+suggestion+and+enhancement+networks.">Thyroid nodule segmentation in ultrasound image based on information fusion of suggestion and enhancement networks.</a> Mathematics. 10 (19), 3484 (2022).</li><li>Ma, J., Wu, F., Jiang, T. A., Zhu, J., Kong, D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Cascade+convolutional+neural+networks+for+automatic+detection+of+thyroid+nodules+in+ultrasound+images.">Cascade convolutional neural networks for automatic detection of thyroid nodules in ultrasound images.</a> Medical Physics. 44 (5), 1678-1691 (2017).</li><li>Song, W., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Multitask+cascade+convolution+neural+networks+for+automatic+thyroid+nodule+detection+and+recognition.">Multitask cascade convolution neural networks for automatic thyroid nodule detection and recognition.</a> IEEE Journal of Biomedical and Health Informatics. 23 (3), 1215-1224 (2018).</li><li>Wang, J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Learning+from+weakly-labeled+clinical+data+for+automatic+thyroid+nodule+classification+in+ultrasound+images.">Learning from weakly-labeled clinical data for automatic thyroid nodule classification in ultrasound images.</a> , 3114-3118 (2018).</li><li>Wang, L., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+multi-scale+densely+connected+convolutional+neural+network+for+automated+thyroid+nodule+classification.">A multi-scale densely connected convolutional neural network for automated thyroid nodule classification.</a> Frontiers in Neuroscience. 16, 878718 (2022).</li><li>Krizhevsky, A., Sutskever, I., Hinton, G. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Imagenet+classification+with+deep+convolutional+neural+networks.">Imagenet classification with deep convolutional neural networks.</a> Communications of the ACM. 60 (6), 84-90 (2017).</li><li>He, K., Zhang, X., Ren, S., Sun, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Deep+residual+learning+for+image+recognition.">Deep residual learning for image recognition.</a> Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. , 770-778 (2016).</li><li>Hu, H., Gu, J., Zhang, Z., Dai, J., Wei, Y. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Relation+networks+for+object+detection.">Relation networks for object detection.</a> Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. , 3588-3597 (2018).</li><li>Szegedy, C., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Going+deeper+with+convolutions.">Going deeper with convolutions.</a> Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. , 1-9 (2015).</li><li>Dosovitskiy, A., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=An+image+is+worth+16x16+words:+Transformers+for+image+recognition+at+scale.">An image is worth 16x16 words: Transformers for image recognition at scale.</a> arXiv preprint arXiv:2010.11929. , (2020).</li><li>Touvron, H., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Training+data-efficient+image+transformers+&amp;+distillation+through+attention.">Training data-efficient image transformers &amp; distillation through attention.</a> arXiv:2012.12877. , (2021).</li><li>Liu, Z., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Swin+Transformer:+Hierarchical+vision+transformer+using+shifted+windows.">Swin Transformer: Hierarchical vision transformer using shifted windows.</a> 2021 IEEE/CVF International Conference on Computer Vision (ICCV). , 9992-10002 (2021).</li><li>Vaswani, A., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Attention+is+all+you+need.">Attention is all you need.</a> Advances in Neural Information Processing Systems. 30, (2017).</li><li>Chen, J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=TransUNet:+Transformers+make+strong+encoders+for+medical+image+segmentation.+arXiv.">TransUNet: Transformers make strong encoders for medical image segmentation. arXiv.</a> arXiv:2102.04306. , (2021).</li><li>Ren, S., He, K., Girshick, R., Sun, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Faster+r-cnn:+Towards+real-time+object+detection+with+region+proposal+networks.">Faster r-cnn: Towards real-time object detection with region proposal networks.</a> Advances in Neural Information Processing Systems. 28, 91-99 (2015).</li><li>Li, H., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=An+improved+deep+learning+approach+for+detection+of+thyroid+papillary+cancer+in+ultrasound+images.">An improved deep learning approach for detection of thyroid papillary cancer in ultrasound images.</a> Scientific Reports. 8, 6600 (2018).</li><li>Lin, T. -. Y., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Feature+pyramid+networks+for+object+detection.">Feature pyramid networks for object detection.</a> Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. , 2117-2125 (2017).</li><li>Ouahabi, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+review+of+wavelet+denoising+in+medical+imaging.">A review of wavelet denoising in medical imaging.</a> 2013 8th International Workshop on Systems, Signal Processing and their Applications. , 19-26 (2013).</li><li>Mahdaoui, A. E., Ouahabi, A., Moulay, M. S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Image+denoising+using+a+compressive+sensing+approach+based+on+regularization+constraints.">Image denoising using a compressive sensing approach based on regularization constraints.</a> Sensors. 22 (6), 2199 (2022).</li><li>Castleman, K. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Digital Image Processing. , (1996).</li><li>Liu, W., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Ssd:+Single+shot+multibox+detector.">Ssd: Single shot multibox detector.</a> European Conference on Computer Vision. , 21-37 (2016).</li><li>Redmon, J., Farhadi, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Yolov3:+An+incremental+improvement.">Yolov3: An incremental improvement.</a> arXiv. arXiv:1804.02767. , (2018).</li><li>Lin, T. -. Y., Goyal, P., Girshick, R., He, K., Doll&#225;r, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Focalloss+for+dense+object+detection.">Focalloss for dense object detection.</a> arXiv. arXiv:1708.02002. , (2017).</li><li>Carion, N., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=End-to-end+object+detection+with+transformers.">End-to-end object detection with transformers.</a> Computer Vision-ECCV 2020: 16th European Conference. , 23-28 (2020).</li></ol>

This paper describes in detail how to perform the environment setup, data preparation, model configuration, and network training. In the environment setup phase, one needs to pay attention to ensure that the dependent libraries are compatible and matched. Data processing is a very important step; time and effort must be spent to ensure the accuracy of the annotations. When training the model, a &#34;ModuleNotFoundError&#34; may be encountered. In this case, it is necessary to use the &#34;pip install&#34; command to inst...

The incidence of thyroid cancer has increased rapidly since 1970, especially among middle-aged women1. Thyroid nodules may predict the emergence of thyroid cancer, and most thyroid nodules are asymptomatic2. The early detection of thyroid nodules is very helpful in curing thyroid cancer. Therefore, according to current practice guidelines, all patients with suspected nodular goiter on physical examination or with abnormal imaging findings should undergo further examination3,4.
Thyroid ultrasound (US) is a common method used to detect and characterize thyroid lesions5,6. US is a convenient, inexpensive, and radiation-free technology. However, the application of US is easily affected by the operator7,8. Features such as the shape, size, echogenicity, and texture of thyroid nodules are easily distinguishable on US images. Although certain US features-calcifications, echogenicity, and irregular borders-are often considered criteria for identifying thyroid nodules, the presence of interobserver variability is unavoidable8,9. The diagnosis results of radiologists with different levels of experience are different. Inexperienced radiologists are more likely to misdiagnose than experienced radiologists. Some characteristics of US such as reflections, shadows, and echoes can degrade the image quality. This degradation in image quality caused by the nature of US imaging makes it difficult for even experienced physicians to locate nodules accurately.
Computer-aided diagnosis (CAD) for thyroid nodules has developed rapidly in recent years and can effectively reduce errors caused by different physicians and help radiologists diagnose nodules quickly and accurately10,11. Various CNN-based CAD systems have been proposed for thyroid US nodule analysis, including segmentation12,13, detection14,15, and classification16,17. CNN is a multilayer, supervised learning model18, and the core modules of CNN are the convolution and pooling layers. The convolution layers are used for feature extraction, and the pooling layers are used for downsampling. The shadow convolutional layers can extract primary features such as the texture, edges, and contours, while deep convolutional layers learn high-level semantic features.
CNNs have had great success in computer vision19,20,21. However, CNNs fail to capture long-range contextual dependencies due to the limited valid receptive field of the convolutional layers. In the past, backbone architectures for image classification mostly used CNNs. With the advent of Vision Transformer (ViT)22,23, this trend has changed, and now many state-of-the-art models use transformers as backbones. Based on non-overlapping image patches, ViT uses a standard transformer encoder25 to globally model spatial relationships. The Swin Transformer24 further introduces shift windows to learn features. The shift windows not only bring greater efficiency but also greatly reduce the length of the sequence because self-attention is calculated in the window. At the same time, the interaction between two adjacent windows can be made through the operation of shifting (movement). The successful application of the Swin Transformer in computer vision has led to the investigation of transformer-based architectures for ultrasound image analysis26.
Recently, Li et al. proposed a deep learning approach28 for thyroid papillary cancer detection inspired by Faster R-CNN27. Faster R-CNN is a classic CNN-based object detection architecture. The original Faster R-CNN has four modules-the CNN backbone, the region proposal network (RPN), the ROI pooling layer, and the detection head. The CNN backbone uses a set of basic conv+bn+relu+pooling layers to extract feature maps from the input image. Then, the feature maps are fed into the RPN and the ROI pooling layer. The role of the RPN network is to generate region proposals. This module uses softmax to determine whether anchors are positive and generates accurate anchors by bounding box regression. The ROI pooling layer extracts the proposal feature maps by collecting the input feature maps and proposals and feeds the proposal feature maps into the subsequent detection head. The detection head uses the proposal feature maps to classify objects and obtain accurate positions of the detection boxes by bounding box regression.
This paper presents a new thyroid nodule detection network called Swin Faster R-CNN formed by replacing the CNN backbone in Faster R-CNN with the Swin Transformer, which results in the better extraction of features for nodule detection from ultrasound images. In addition, the feature pyramid network (FPN)29 is used to improve the detection performance of the model for nodules of different sizes by aggregating features of different scales.

<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td>GPU RTX3090</td>
			<td>Nvidia</td>
			<td>1</td>
			<td>24G GPU</td>
		</tr>
		<tr>
			<td>mmdetection2.11.0</td>
			<td>SenseTime</td>
			<td>4</td>
			<td>https://github.com/open-mmlab/mmdetection.git</td>
		</tr>
		<tr>
			<td>python3.8</td>
			<td>&#8212;</td>
			<td>2</td>
			<td>https://www.python.org</td>
		</tr>
		<tr>
			<td>pytorch1.7.1</td>
			<td>Facebook</td>
			<td>3</td>
			<td>https://pytorch.org</td>
		</tr>
	</tbody>
</table>

a swin transformer-based model for thyroid nodule detection in ultrasound images

In recent years, the incidence of thyroid cancer has been increasing. Thyroid nodule detection is critical for both the detection and treatment of thyroid cancer. Convolutional neural networks (CNNs) have achieved good results in thyroid ultrasound image analysis tasks. However, due to the limited valid receptive field of convolutional layers, CNNs fail to capture long-range contextual dependencies, which are important for identifying thyroid nodules in ultrasound images. Transformer networks are effective in capturing long-range contextual information. Inspired by this, we propose a novel thyroid nodule detection method that combines the Swin Transformer backbone and Faster R-CNN. Specifically, an ultrasound image is first projected into a 1D sequence of embeddings, which are then fed into a hierarchical Swin Transformer. 
The Swin Transformer backbone extracts features at five different scales by utilizing shifted windows for the computation of self-attention. Subsequently, a feature pyramid network (FPN) is used to fuse the features from different scales. Finally, a detection head is used to predict bounding boxes and the corresponding confidence scores. Data collected from 2,680 patients were used to conduct the experiments, and the results showed that this method achieved the best mAP score of 44.8%, outperforming CNN-based baselines. In addition, we gained better sensitivity (90.5%) than the competitors. This indicates that context modeling in this model is effective for thyroid nodule detection.

The thyroid US images were collected from two hospitals in China from September 2008 to February 2018. The eligibility criteria for including the US images in this study were conventional US examination before biopsy and surgical treatment, diagnosis with biopsy or postsurgical pathology, and age &#8805; 18 years. The exclusion criteria were images without thyroid tissues.
The 3,000 ultrasound images included 1,384 malignant and 1,616 benign nodules. The majority (90%) of the malignant nodules...

Watch this Scientific Journal Video about A Swin Transformer-Based Model for Thyroid Nodule Detection in Ultrasound Images at JoVE.com

A Swin Transformer-Based Model for Thyroid Nodule Detection in Ultrasound Images

Here, a new model for thyroid nodule detection in ultrasound images is proposed, which uses Swin Transformer as the backbone to perform long-range context modeling. Experiments prove that it performs well in terms of sensitivity and accuracy.

In recent years, the incidence of thyroid cancer has been increasing. Thyroid nodule detection is critical for both the detection and treatment of thyroid ...

This program implements an automatic and efficient measure for detecting thyroid nodules, which may reduce the workload of doctors in the future. This technique applies a Swin Transformer to a nodule detection network. Swin Transformers are effective in capturing long-range content information, which is very helpful in improving the sensitivity of nodule detention.To begin, open a terminal on the machine and install Python 3.8 by typing sudo apt-get install python3.8 in the command line. To install PyTorch 1.7, go to the official website of Miniconda and install it. Then create a Conda environment by typing the command shown on the screen in the command line, and activate it by typing conda activate SwinFasterRCNN.Install PyTorch by writing the shown command in the command line. For the MM detection installation, clone from the official GitHub repository, and then install MM Detection by writing the command pip install v e in the command line. Open the schedule_1x.py file in the Vim text editor and edit the file to set the default training-related parameters, including the optimizer, learning rate, and epic. Begin the training by typing the command shown on the screen. Wait for the network to begin the training for 48 epics, and for the resulting trained weights of the SwinFasterRCNN network to be generated in the output folder.Save the model weights with the highest accuracy on the validation set. After training, select the model with the best performance on the validation set for thyroid nodule detection in the new images. Run the test script to resize the image to 512 pixels by 512 pixels and normalize it.Wait for the script to automatically load the pre-trained model parameters to the SwinFasterRCNN, and feed the pre-processed image into the SwinFasterRCNN for inference. Wait for the SwinFasterRCNN to output the prediction box for each image. Finally, allow the script to automatically perform NMS post-processing on each image to remove the duplicate detection boxes.The detection results for the same image are shown in a given row. The columns are the detection results for SwinFasterRCNN, FasterRCNN, YOLOv3, SSD, RetinaNet, and DETR. The ground truths of the regions are marked with green rectangular boxes.The detection results are framed by the red rectangular boxes. It is very important to select the appropriate hyper parameter to ensure those converges which needs to be determined by many experiments.

a-swin-transformer-based-model-for-thyroid-nodule-detection

Research

JoVE Journal

Medicine

Cerenkov Luminescence Imaging (CLI) for Cancer Therapy Monitoring

In molecular imaging, positron emission tomography (PET) and optical imaging (OI) are two of the most important and thus most widely used modalities1-3. PET is characterized by its excellent sensitivity and quantification ability while OI is notable for non-radiation, relative low cost, short scanning time, high throughput, and wide availability to basic researchers. However, both modalities have their shortcomings as well. PET suffers from poor spatial resolution and high cost, while OI is mostly limited to preclinical applications because of its limited tissue penetration along with prominent scattering optical signals through the thickness of living tissues.
 Recently a bridge between PET and OI has emerged with the discovery of Cerenkov Luminescence Imaging (CLI)4-6. CLI is a new imaging modality that harnesses Cerenkov Radiation (CR) to image radionuclides with OI instruments. Russian Nobel laureate Alekseyevich Cerenkov and his colleagues originally discovered CR in 1934. It is a form of electromagnetic radiation emitted when a charged particle travels at a superluminal speed in a dielectric medium7,8. The charged particle, whether positron or electron, perturbs the electromagnetic field of the medium by displacing the electrons in its atoms. After passing of the disruption photons are emitted as the displaced electrons return to the ground state. For instance, one 18F decay was estimated to produce an average of 3 photons in water5. 
 Since its emergence, CLI has been investigated for its use in a variety of preclinical applications including in vivo tumor imaging, reporter gene imaging, radiotracer development, multimodality imaging, among others4,5,9,10,11. The most important reason why CLI has enjoyed much success so far is that this new technology takes advantage of the low cost and wide availability of OI to image radionuclides, which used to be imaged only by more expensive and less available nuclear imaging modalities such as PET.
 Here, we present the method of using CLI to monitor cancer drug therapy. Our group has recently investigated this new application and validated its feasibility by a proof-of-concept study12. We demonstrated that CLI and PET exhibited excellent correlations across different tumor xenografts and imaging probes. This is consistent with the overarching principle of CR that CLI essentially visualizes the same radionuclides as PET. We selected Bevacizumab (Avastin; Genentech/Roche) as our therapeutic agent because it is a well-known angiogenesis inhibitor13,14. Maturation of this technology in the near future can be envisioned to have a significant impact on preclinical drug development, screening, as well as therapy monitoring of patients receiving treatments.

Cerenkov Luminescence Imaging CLI for Cancer Therapy Monitoring

Use of Cerenkov Luminescence Imaging (CLI) for monitoring preclinical cancer treatment is described here. This method takes advantage of Cerenkov Radiation (CR) and optical imaging (OI) to visualize radiolabeled probes and thus provides an alternative to PET in preclinical therapeutic monitoring and drug screening.

In  molecular imaging, positron emission tomography (PET) and optical imaging (OI)  are two of the most important and thus most widely used modalities1-3. ...

An Orthotopic Mouse Model of Anaplastic Thyroid Carcinoma

Several types of animal models of human thyroid carcinomas have been established, including subcutaneous xenograft and orthotopic implantation of cancer cells into immunodeficient mice. Subcutaneous xenograft models have been valuable for preclinical screening and evaluation of new therapeutic treatments. There are a number of advantages to using a subcutaneous model; 1) rapid, 2) reproducible, and 3) tumor establishment, growth, and response to therapeutic agents may be monitored by visual inspection. However, substantial evidence has shed light on the short-comings of subcutaneous xenograft models1-3. For instance, medicinal treatments demonstrating curative properties in subcutaneous xenograft models often have no notable impact on the human disease. The microenvironment of the site of xenographic transplantation or injection lies at the heart of this dissimilarity.
Orthotopic tumor xenograft models provide a more biologically relevant context in which to study the disease. The advantages of implanting diseased cells or tissue into their anatomical origin equivalent within a host animal includes a suitable site for tumor-host interactions, development of disease-related metastases and the ability to examine site-specific influence on investigational therapeutic remedies. Therefore, orthotopic xenograft models harbor far more clinical value because they closely reproduce human disease. For these reasons, a number of groups have taken advantage of an orthotopic thyroid cancer model as a research tool4-7.
Here, we describe an approach that establishes an orthotopic model for the study of anaplastic thyroid carcinoma (ATC), which is highly invasive, resists treatment, and is virtually fatal in all diagnosed patients. Cultured ATC cells are prepared as a dissociated cellular suspension in a solution containing a basement membrane matrix. A small volume is slowly injected into the right thyroid gland. Overall appearance and health of the mice are monitored to ensure minimal post-operative complications and to gauge pathological penetrance of the cancer. Mice are sacrificed at 4 weeks, and tissue is collected for histological analysis. Animals may be taken at later time-points to examine more advance progression of the disease. Production of this orthotopic mouse model establishes a platform that accomplishes two objectives: 1) further our understanding of ATC pathology, and 2) screen current and future therapeutic agents for efficacy in combating ATC.

Generation of an orthotopic mouse model of anaplastic thyroid carcinoma is described here. This technique employs surgical placement of human anaplastic thyroid cancer cells into the thyroid of immunodeficient mice, thus creating a more clinically relevant setting to study disease progression as well as screen innovative therapeutic interventions.

Several types of animal models of human thyroid carcinomas have been established, including subcutaneous xenograft and orthotopic implantation of cancer ...

Substernal Thyroid Biopsy Using Endobronchial Ultrasound-guided Transbronchial Needle Aspiration

Substernal thyroid goiter (STG) represents about 5.8% of all mediastinal lesions1. There is a wide variation in the published incidence rates due to the lack of a standardized definition for STG. Biopsy is often required to differentiate benign from malignant lesions. Unlike cervical thyroid, the overlying sternum precludes ultrasound-guided percutaneous fine needle aspiration of STG. Consequently, surgical mediastinoscopy is performed in the majority of cases, causing significant procedure related morbidity and cost to healthcare. Endobronchial Ultrasound-guided Transbronchial Needle Aspiration (EBUS-TBNA) is a frequently used procedure for diagnosis and staging of non-small cell lung cancer (NSCLC). Minimally invasive needle biopsy for lesions adjacent to the airways can be performed under real-time ultrasound guidance using EBUS. Its safety and efficacy is well established with over 90% sensitivity and specificity. The ability to perform EBUS as an outpatient procedure with same-day discharges offers distinct morbidity and financial advantages over surgery. As physicians performing EBUS gained procedural expertise, they have attempted to diversify its role in the diagnosis of non-lymph node thoracic pathologies. We propose here a role for EBUS-TBNA in the diagnosis of substernal thyroid lesions, along with a step-by-step protocol for the procedure.

Substernal thyroid lesions are common, and need to be differentiated from malignancy. Obtaining percutaneous fine needle biopsy is not possible due to its retrosternal location. This article proposes a protocol for biopsy of substernal thyroid lesions using Endobronchial Ultrasound-guided Transbronchial Needle Aspiration (EBUS-TBNA).

Substernal thyroid goiter (STG) represents about 5.8% of all mediastinal lesions1. There is a wide variation in the published incidence rates due to the ...

Intra-Operative Neural Monitoring of Thyroid Surgery in a Porcine Model

Intraoperative injury to the recurrent laryngeal nerve (RLN) can cause vocal cord paralysis, which interferes with speech and can potentially interfere with breathing. In recent years, intraoperative neural monitoring (IONM) has been widely adapted as an adjunct technique to localize the RLN, detect RLN injury, and predict vocal cord function during the operations. Many studies have also used animal models to investigate new applications of IONM technology and to develop reliable strategies for preventing intraoperative RLN injury. The aim of this article is to introduce a standard protocol for using a porcine model in IONM research. The article demonstrates the procedures for inducing general anesthesia, performing tracheal intubation, and experimental design to investigate the electrophysiological characteristics of RLN injuries. Applications of this protocol can improve overall efficacy in implementing the 3R principle (replacement, reduction and refinement) in porcine IONM studies.

This study aims to develop a standard protocol of intra-operative neural monitoring of thyroid surgery in a porcine model. Here, we present a protocol to demonstrate general anesthesia, to compare different types of electrodes, and to investigate the electrophysiological characteristics of the normal and injured recurrent laryngeal nerves.

Intraoperative injury to the recurrent laryngeal nerve (RLN) can cause vocal cord paralysis, which interferes with speech and can potentially interfere ...

Electromagnetic Navigation Transthoracic Nodule Localization for Minimally Invasive Thoracic Surgery

The increased use of chest computed tomography (CT) has led to an increased detection of pulmonary nodules requiring diagnostic evaluation and/or excision. Many of these nodules are identified and excised via minimally invasive thoracic surgery; however, subcentimeter and subsolid nodules are frequently difficult to identify intra-operatively. This can be mitigated by the use of electromagnetic transthoracic needle localization. This protocol delineates the step-by-step process of electromagnetic localization from the pre-operative period to the postoperative period and is an adaptation of the electromagnetically guided percutaneous biopsy previously described by Arias et al. Pre-operative steps include obtaining a same day CT followed by the generation of a three-dimensional virtual map of the lung. From this map, the target lesion(s) and an entry site are chosen. In the operating room, the virtual reconstruction of the lung is then calibrated with the patient and the electromagnetic navigation platform. The patient is then sedated, intubated, and placed in the lateral decubitus position. Using a sterile technique and visualization from multiple views, the needle is inserted into the chest wall at the prechosen skin entry site and driven down to the target lesion. Dye is then injected into the lesion and, then, continuously during needle withdrawal, creating a tract for visualization intra-operatively. This method has many potential benefits when compared to the CT-guided localization, including a decreased radiation exposure and decreased time between the dye injection and the surgery. Dye diffusion from the pathway occurs over time, thereby limiting intra-operative nodule identification. By decreasing the time to surgery, there is a decrease in wait time for the patient, and less time for dye diffusion to occur, resulting in an improvement in nodule localization. When compared to electromagnetic bronchoscopy, airway architecture is no longer a limitation as the target nodule is accessed via a transparenchymal approach. Details of this procedure are described in a step-by-step fashion.

Presented here is a protocol for lung nodule localization using dye marking via electromagnetically navigated transthoracic needle access. The technique described here can be accomplished in the peri-operative period to optimize nodule localization and to successful resection when performing minimally invasive thoracic surgery.

The increased use of chest computed tomography (CT) has led to an increased detection of pulmonary nodules requiring diagnostic evaluation and/or ...

A Pleural Effusion Model in Rats by Intratracheal Instillation of Polyacrylate/Nanosilica

Pleural effusion is a prevalent clinical finding of many pulmonary diseases. Having a useful animal pleural effusion model is very important to study these pulmonary diseases. Previous pleural effusion models paid more attention to biological factors rather than nanoparticles in the environment. Here, we introduce a model to make pleural effusion in rats by intratracheal instillation of polyacrylate/nanosilica, and a method of nanoparticle isolation in the pleural effusion. By intratracheal instillation of polyacrylate/nanosilica with concentrations of 3.125, 6.25 and 12.5 mg/kg&#8729;mL, the pleural effusion in rats presented on day 3, peaked at days 7-10 in 6.25 and 12.5 mg/kg&#8729;mL groups, then slowly decreased and disappeared on day 14. When the concentration of polyacrylate/nanosilica increased, the pleural effusion is produced more and faster. This pleural fluid was detected by ultrasound examination or CT chest scanning and confirmed by dissection of rats. Silica nanoparticles were observed in the rats' pleural effusion by transmission electron microscope. These results showed that the exposure to polyacrylate/nanosilica leads to the induction of pleural effusion, which was consistent with our previous report in humans. Additionally, this model is beneficial for the further study of nanotoxicology and the pleural effusion diseases.

Here, we present a protocol to construct a pleural effusion model in rats by intratracheal instillation of polyacrylate/nanosilica.

Pleural effusion is a prevalent clinical finding of many pulmonary diseases. Having a useful animal pleural effusion model is very important to study ...

Investigation of the Electrophysiological and Thermographic Safety Parameters of Surgical Energy Devices During Thyroid and Parathyroid Surgery in a Porcine Model

In thyroid and parathyroid surgery, surgical energy devices (SEDs) provide more efficient hemostasis than conventional clamp-and-tie hemostasis in areas with rich blood supply. However, when a SED is activated near the recurrent laryngeal nerve (RLN), the heat generated by the SED may injure the nerve irreversibly. To safely apply SEDs in thyroid/parathyroid surgery, this article introduces experimental porcine model studies to investigate the activation and cooling safety parameters of SEDs in standardized electrophysiological (EP) and thermographic (TG) procedures, respectively. In the EP safety parameter experiments, continuous intraoperative neuromonitoring (C-IONM) is applied to demonstrate the RLN function in real-time. The EP activation study evaluates the safe activation distance of SEDs; the EP cooling study evaluates the safe cooling time of SEDs. In the TG safety parameter experiment, a thermal imaging camera is used to record the temperature change after activating the SED. The TG activation study evaluates the lateral thermal spread distance after SED activation in a dry or humid environment and whether smoke and splashing are generated; the TG cooling study evaluates the cooling time. This will help establish the safety parameters of newly developed SEDs used in thyroid/parathyroid surgery and provide safety guidelines to avoid RLN injury and related complications.

The safe application of newly developed surgical energy devices in thyroid/parathyroid surgery attracts the attention of surgeons. Animal experimental models can avoid unnecessary trials and errors in human surgery. This report aims to demonstrate electrophysiological and thermographic methods to evaluate the safety parameters of SEDs in thyroid/parathyroid surgery.

In thyroid and parathyroid surgery, surgical energy devices (SEDs) provide more efficient hemostasis than conventional clamp-and-tie hemostasis in areas ...

A Personalized 3D-Printed Model for Preoperative Evaluation in Thyroid Surgery

The anatomic structure of the surgical area of thyroid cancer is complex. It is very important to comprehensively and carefully evaluate the tumor location and its relation with the capsule, trachea, esophagus, nerves, and blood vessels before operation. This paper introduces an innovative 3D-printed model establishment method based on computerized tomography (CT) DICOM images. We established a personalized 3D-printed model of the cervical thyroid surgery field for each patient who needed thyroid surgery to help clinicians evaluate the key points and difficulties of the surgery and select the operation methods of key parts as a basis. The results showed that this model is conducive to preoperative discussion and the formulation of operation strategies. In particular, as a result of the clear display of the recurrent laryngeal nerve and parathyroid gland locations in the thyroid operation field, injury to them can be avoided during surgery, the difficulty of thyroid surgery reduced, and the incidence of postoperative hypoparathyroidism and complications related to recurrent laryngeal nerve injury reduced too. Moreover, this 3D-printed model is intuitive and aids communication for the signing of informed consent by patients before surgery.

Here, a new method of establishing a personalized 3D-printed model for preoperative evaluation of thyroid surgery is proposed. It is conducive to preoperative discussion, reducing the difficulty of thyroid surgery.

The anatomic structure of the surgical area of thyroid cancer is complex. It is very important to comprehensively and carefully evaluate the tumor ...

Constructing a 3D Digital Model for any Specific Nodule

Use the three dimensional nodules function with two parameters, including the slice number and the thoracic volume to reconstruct a three dimensional digital model for specific nodules. If the three dimensional nodules function correctly executes, press and hold the left mouse button, and drag it in any direction to alter the perspective of the pulmonary nodules. It is important to note that the observation angle should account for anatomical considerations and attempt to display both the medical characteristics of the pulmonary nodules and the relationship between the nodules and the surrounding tissues.Use the zoom and move icons in the upper right corner. Alternatively, use the middle mouse button to zoom in on the model or out of the view. The GUI displays the coordinate indication in the lower left corner with the positive direction on the Z axis being the scanning direction in the horizontal position.Utilize the screenshot tool for the operating system to save the necessary three dimensional projection of the nodules. With this three dimensional construction, the GUI allows the user to observe the pulmonary nodules of interest from any perspective.

Technique for Salivary Gland Ultrasound Examination of Sjogren&#039;s Syndrome

Ultrasonographic Evaluation of Salivary Glands for Sjogren's Syndrome: Diagnostic and Monitoring Insights

Sjogren's syndrome (SS) is a chronic autoimmune condition commonly affecting the exocrine glands, causing oral or ocular dryness and extraglandular manifestations including arthralgia, cytopenia, and lymphoma. The presence of autoantibodies against SSA/Ro, labial salivary gland biopsy, ocular staining, Schirmer's test, or salivary flow assessment are included in the current classification criteria of SS. However, the availability and invasiveness of these procedures limit their widespread use in clinical settings. Salivary gland ultrasonography (SGUS) is a non-invasive imaging modality for the evaluation of the salivary gland parenchyma and is increasingly utilized to aid diagnosis and monitoring in SS. 
This article presents the protocol of SGUS for image acquisition at the parotid and submandibular glands. The objective is to present a standardized, reproducible, and practical approach to diagnostic SGUS for SS in daily clinical settings. Major salivary glands are scanned in a stepwise approach, beginning at the angle of the mandible for the superficial lobe of the parotid gland, followed by the deep lobe below the ramus of the mandible. Submandibular areas are then scanned for the submandibular glands. The steps in obtaining salivary gland images at each anatomical site are explained in the accompanying video. The echogenicity and echotexture at the thyroid gland are taken as a reference. The homogeneity, the presence and distribution of hypoechoic areas within the glands, and the border of the salivary glands are examined. The sizes and features of intra-/peri-glandular lymph nodes are recorded. The most distinctive sonographic feature in SS is glandular heterogeneity with the presence of hypoechoic/hyperechoic areas within the glands. 
In summary, while SGUS cannot diagnose SS on its own, it can supplement the current classification criteria of SS and guide the clinical decision for salivary gland biopsy to support the diagnosis of SS in patients with sicca syndrome or suspicious systemic features, combined with autoantibody testing.

Author Spotlight: A Focus on Standardized Salivary Gland Ultrasound Protocol in Connective Tissue Disease Research

Salivary gland ultrasonography is a promising tool for the diagnosis and evaluation of disease activity and complications in Sjogren's syndrome (SS). The scanning technique for salivary gland ultrasound examination relevant to SS is described in the article.

Sjogren's syndrome (SS) is a chronic autoimmune condition commonly affecting the exocrine glands, causing oral or ocular dryness and extraglandular ...