This study was supported by the National Natural Science Foundation of China (Grant No.32101188) and the General Project of Science and Technology Department of Sichuan Province (Grant No. 2021YFS0102), China.

This retrospective study was approved by the institutional review board of the West China Hospital, Sichuan University, Sichuan, China, and the requirement to obtain informed consent was waived.
1. Environment setup
<ol>
	<li>Graphic processing unit (GPU) software
	<ol>
		<li>To implement deep learning applications, first configure the GPU-related environment. Download and install GPU-appropriate software and drivers from the GPU&#39;s website. 
		&#8203;NOTE: See the Table of Materials for those used in this study.</li>
	</ol>
	</li>
	<li>Python3.8 installation
	<ol>
		<li>Open a terminal on the machine. Type the following: 
		Command line: sudo apt-get install python3.8 python-dev python-virtualenv</li>
	</ol>
	</li>
	<li>Pytorch1.7 installation
	<ol>
		<li>Follow the steps on the official website to download and install Miniconda.</li>
		<li>Create a conda environment and activate it. 
		Command line: conda create --name SwinFasterRCNN python=3.8 -y 
		Command line: conda activate SwinFasterRCNN</li>
		<li>Install Pytorch. 
		Command line: conda install pytorch==1.7.1 torchvision==0.8.2 torchaudio==0.7.2 </li>
	</ol>
	</li>
	<li>MMDetection installation
	<ol>
		<li>Clone from the official Github repository. 
		Command line: git clone https://github.com/open-mmlab/mmdetection.git</li>
		<li>Install MMDetection. 
		Command line: cd mmdetection 
		Command line: pip install -v -e .</li>
	</ol>
	</li>
</ol>
2. Data preparation
<ol>
	<li>Data collection
	<ol>
		<li>Collected the ultrasound images (here, 3,000 cases from a Grade-A tertiary hospital). Ensure that each case has diagnostic records, treatment plans, US reports, and the corresponding US images.</li>
		<li>Place all the US images in a folder named &#34;images.&#34; 
		NOTE: The data used in this study included 3,853 US images from 3,000 cases.</li>
	</ol>
	</li>
	<li>Data cleaning
	<ol>
		<li>Manually check the dataset for images of non-thyroid areas, such as lymph images.</li>
		<li>Manually check the dataset for images containing color Doppler flow.</li>
		<li>Delete the images selected in the previous two steps. 
		NOTE: After data cleaning, 3,000 images were left from 2,680 cases.</li>
	</ol>
	</li>
	<li>Data annotation
	<ol>
		<li>Have a senior doctor locate the nodule area in the US image and outline the nodule boundary. 
		NOTE: The annotation software and process can be found in Supplemental File 1.</li>
		<li>Have another senior doctor review and revise the annotation results.</li>
		<li>Place the annotated data in a separate folder called &#34;Annotations.&#34;</li>
	</ol>
	</li>
	<li>Data split
	<ol>
		<li>Run the python script, and set the path of the image in step 2.1.2 and the paths of the annotations in step 2.3.3. Randomly divide all the images and the corresponding labeled files into training and validation sets at a ratio of 8:2. Save the training set data in the &#34;Train&#34; folder and the validation set data in the &#34;Val&#34; folder. 
		NOTE: Python scripts are provided in Supplemental File 2.</li>
	</ol>
	</li>
	<li>Converting to the CoCo dataset format 
	NOTE: To use MMDetection, process the data into a CoCo dataset format, which includes a json file that holds the annotation information and an image folder containing the US images.
	<ol>
		<li>Run the python script, and input the annotations folder paths (step 2.3.3) to extract the nodule areas outlined by the doctor and convert them into masks. Save all the masks in the &#34;Masks&#34; folder. 
		NOTE: The Python scripts are provided in Supplemental File 3.</li>
		<li>Run the python script, and set the path of the masks folder in step 2.5.1 to make the data into a dataset in CoCo format and generate a json file with the US images. 
		NOTE: Python scripts are provided in Supplemental File 4.</li>
	</ol>
	</li>
</ol>
3. Swin Faster RCNN configuration
<ol>
	<li>Download the Swin Transformer model file (https://github.com/microsoft/Swin-Transformer/blob/main/models/swin_transformer.py), modify it, and place it in the &#8220;mmdetection/mmdet/models/backbones/&#8221; folder. Open the &#8220;swin_transformer.py&#8221; file in a vim text editor, and modify it as the Swin Transformer model file provided in&#160;Supplemental File 5. 
	Command line: vim swin_transformer.py</li>
	<li>Make a copy of the Faster R-CNN configuration file, change the backbone to Swin Transformer, and set up the FPN parameters. 
	Command line: cd mmdetection/configs/faster_rcnn 
	Command line: cp faster_rcnn_r50_fpn_1x_coco.py swin_faster_rcnn_swin.py 
	NOTE: The Swin Faster R-CNN configuration file (swin_faster_rcnn_swin.py) is provided in Supplemental File 6. The Swin Faster R-CNN network structure is shown in Figure 1.</li>
	<li>Set the dataset path to the CoCo format dataset path (step 2.5.2) in the configuration file. Open the &#34;coco_detection.py&#34; file in the vim text editor, and modify the following line: 
	data_root = &#34;dataset path(step 2.5.2)&#34; 
	Command line:vim mmdetection/configs/_base_/datasets/coco_detection.py</li>
</ol>
4. Training the Swin Faster R-CNN
<ol>
	<li>Edit mmdetection/configs/_base_/schedules/schedule_1x.py, and set the default training-related parameters, including the learning rate, optimizer, and epoch. Open the &#34;schedule_1x.py&#34; file in the vim text editor, and modify the following lines: 
	optimizer = dict(type=&#34;AdamW&#34;, lr=0.001, momentum=0.9, weight_decay=0.0001) 
	runner = dict(type=&#39;EpochBasedRunner&#39;, max_epochs=48) 
	Command line:vim mmdetection/configs/_base_/schedules/schedule_1x.py 
	NOTE: In this protocol for this paper, the learning rate was set to 0.001, AdamW optimizer was used, the maximum training epoch was set to 48, and the batch size was set to 16.</li>
	<li>Begin training by typing the following commands. Wait for the network to begin training for 48 epochs and for the resulting trained weights of the Swin Faster R-CNN network to be generated in the output folder. Save the model weights with the highest accuracy on the validation set. 
	Command line: cd mmdetection 
	Command line: python tools/train.py congfigs/faster_rcnn/swin_faster_rcnn_swin.py --work-dir ./work_dirs 
	NOTE: The model was trained on an &#34;NVIDIA GeForce RTX3090 24G&#34; GPU. The central processing unit used was the &#34;AMD Epyc 7742 64-core processor &#215; 128&#34;, and the operating system was Ubuntu 18.06. The overall training time was ~2 h.</li>
</ol>
5. Performing thyroid nodule detection on new images
<ol>
	<li>After training, select the model with the best performance on the validation set for thyroid nodule detection in the new images.
	<ol>
		<li>First, resize the image to 512 pixels x 512 pixels, and normalize it. These operations are performed automatically when the test script is run. 
		Command line: python tools/test.py congfigs/faster_rcnn/swin_faster_rcnn_swin.py --out ./output</li>
		<li>Wait for the script to automatically load the pretrained model parameters to the Swin Faster R-CNN, and feed the preprocessed image into the Swin Faster R-CNN for inference. Wait for the Swin Faster R-CNN to output the prediction box for each image.</li>
		<li>Finally, allow the script to automatically perform NMS postprocessing on each image to remove duplicate detection boxes. 
		NOTE: The detection results are output to the specified folder, which contains the images with the detection boxes and the bounding box coordinates in a packed file.</li>
	</ol>
	</li>
</ol>

<ol>
	<li>Grant, E. G., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Thyroid+ultrasound+reporting+lexicon:+White+paper+of+the+ACR+Thyroid+Imaging,+Reporting+and+Data+System+(TIRADS)+committee.">Thyroid ultrasound reporting lexicon: White paper of the ACR Thyroid Imaging, Reporting and Data System (TIRADS) committee.</a> Journal of the American College of Radiology. 12 (12 Pt A), 1272-1279 (2015).</li><li>Zhao, J., Zheng, W., Zhang, L., Tian, H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Segmentation+of+ultrasound+images+of+thyroid+nodule+for+assisting+fine+needle+aspiration+cytology.">Segmentation of ultrasound images of thyroid nodule for assisting fine needle aspiration cytology.</a> Health Information Science and Systems. 1, 5 (2013).</li><li>Haugen, B. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=American+Thyroid+Association+management+guidelines+for+adult+patients+with+thyroid+nodules+and+differentiated+thyroid+cancer:+What+is+new+and+what+has+changed.">American Thyroid Association management guidelines for adult patients with thyroid nodules and differentiated thyroid cancer: What is new and what has changed.</a> Cancer. 123 (3), 372-381 (2017).</li><li>Shin, J. H., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Ultrasonography+diagnosis+and+imaging-based+management+of+thyroid+nodules:+Revised+Korean+Society+of+Thyroid+Radiology+consensus+statement+and+recommendations.">Ultrasonography diagnosis and imaging-based management of thyroid nodules: Revised Korean Society of Thyroid Radiology consensus statement and recommendations.</a> Korean Journal of Radiology. 17 (3), 370-395 (2016).</li><li>Horvath, E., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=An+ultrasonogram+reporting+system+for+thyroid+nodules+stratifying+cancer+risk+for+clinical+management.">An ultrasonogram reporting system for thyroid nodules stratifying cancer risk for clinical management.</a> The Journal of Clinical Endocrinology &amp; Metabolism. 94 (5), 1748-1751 (2009).</li><li>Park, J. -. Y., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+proposal+for+a+thyroid+imaging+reporting+and+data+system+for+ultrasound+features+of+thyroid+carcinoma.">A proposal for a thyroid imaging reporting and data system for ultrasound features of thyroid carcinoma.</a> Thyroid. 19 (11), 1257-1264 (2009).</li><li>Moon, W. -. J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Benign+and+malignant+thyroid+nodules:+US+differentiation-Multicenter+retrospective+study.">Benign and malignant thyroid nodules: US differentiation-Multicenter retrospective study.</a> Radiology. 247 (3), 762-770 (2008).</li><li>Park, C. S., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Observer+variability+in+the+sonographic+evaluation+of+thyroid+nodules.">Observer variability in the sonographic evaluation of thyroid nodules.</a> Journal of Clinical Ultrasound. 38 (6), 287-293 (2010).</li><li>Kim, S. H., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Observer+variability+and+the+performance+between+faculties+and+residents:+US+criteria+for+benign+and+malignant+thyroid+nodules.">Observer variability and the performance between faculties and residents: US criteria for benign and malignant thyroid nodules.</a> Korean Journal of Radiology. 11 (2), 149-155 (2010).</li><li>Choi, Y. J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+computer-aided+diagnosis+system+using+artificial+intelligence+for+the+diagnosis+and+characterization+of+thyroid+nodules+on+ultrasound:+initial+clinical+assessment.">A computer-aided diagnosis system using artificial intelligence for the diagnosis and characterization of thyroid nodules on ultrasound: initial clinical assessment.</a> Thyroid. 27 (4), 546-552 (2017).</li><li>Chang, T. -. C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+role+of+computer-aided+detection+and+diagnosis+system+in+the+differential+diagnosis+of+thyroid+lesions+in+ultrasonography.">The role of computer-aided detection and diagnosis system in the differential diagnosis of thyroid lesions in ultrasonography.</a> Journal of Medical Ultrasound. 23 (4), 177-184 (2015).</li><li>Li, X. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Fully+convolutional+networks+for+ultrasound+image+segmentation+of+thyroid+nodules.">Fully convolutional networks for ultrasound image segmentation of thyroid nodules.</a> , 886-890 (2018).</li><li>Nguyen, D. T., Choi, J., Park, K. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Thyroid+nodule+segmentation+in+ultrasound+image+based+on+information+fusion+of+suggestion+and+enhancement+networks.">Thyroid nodule segmentation in ultrasound image based on information fusion of suggestion and enhancement networks.</a> Mathematics. 10 (19), 3484 (2022).</li><li>Ma, J., Wu, F., Jiang, T. A., Zhu, J., Kong, D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Cascade+convolutional+neural+networks+for+automatic+detection+of+thyroid+nodules+in+ultrasound+images.">Cascade convolutional neural networks for automatic detection of thyroid nodules in ultrasound images.</a> Medical Physics. 44 (5), 1678-1691 (2017).</li><li>Song, W., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Multitask+cascade+convolution+neural+networks+for+automatic+thyroid+nodule+detection+and+recognition.">Multitask cascade convolution neural networks for automatic thyroid nodule detection and recognition.</a> IEEE Journal of Biomedical and Health Informatics. 23 (3), 1215-1224 (2018).</li><li>Wang, J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Learning+from+weakly-labeled+clinical+data+for+automatic+thyroid+nodule+classification+in+ultrasound+images.">Learning from weakly-labeled clinical data for automatic thyroid nodule classification in ultrasound images.</a> , 3114-3118 (2018).</li><li>Wang, L., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+multi-scale+densely+connected+convolutional+neural+network+for+automated+thyroid+nodule+classification.">A multi-scale densely connected convolutional neural network for automated thyroid nodule classification.</a> Frontiers in Neuroscience. 16, 878718 (2022).</li><li>Krizhevsky, A., Sutskever, I., Hinton, G. E. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Imagenet+classification+with+deep+convolutional+neural+networks.">Imagenet classification with deep convolutional neural networks.</a> Communications of the ACM. 60 (6), 84-90 (2017).</li><li>He, K., Zhang, X., Ren, S., Sun, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Deep+residual+learning+for+image+recognition.">Deep residual learning for image recognition.</a> Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. , 770-778 (2016).</li><li>Hu, H., Gu, J., Zhang, Z., Dai, J., Wei, Y. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Relation+networks+for+object+detection.">Relation networks for object detection.</a> Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. , 3588-3597 (2018).</li><li>Szegedy, C., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Going+deeper+with+convolutions.">Going deeper with convolutions.</a> Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. , 1-9 (2015).</li><li>Dosovitskiy, A., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=An+image+is+worth+16x16+words:+Transformers+for+image+recognition+at+scale.">An image is worth 16x16 words: Transformers for image recognition at scale.</a> arXiv preprint arXiv:2010.11929. , (2020).</li><li>Touvron, H., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Training+data-efficient+image+transformers+&amp;+distillation+through+attention.">Training data-efficient image transformers &amp; distillation through attention.</a> arXiv:2012.12877. , (2021).</li><li>Liu, Z., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Swin+Transformer:+Hierarchical+vision+transformer+using+shifted+windows.">Swin Transformer: Hierarchical vision transformer using shifted windows.</a> 2021 IEEE/CVF International Conference on Computer Vision (ICCV). , 9992-10002 (2021).</li><li>Vaswani, A., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Attention+is+all+you+need.">Attention is all you need.</a> Advances in Neural Information Processing Systems. 30, (2017).</li><li>Chen, J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=TransUNet:+Transformers+make+strong+encoders+for+medical+image+segmentation.+arXiv.">TransUNet: Transformers make strong encoders for medical image segmentation. arXiv.</a> arXiv:2102.04306. , (2021).</li><li>Ren, S., He, K., Girshick, R., Sun, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Faster+r-cnn:+Towards+real-time+object+detection+with+region+proposal+networks.">Faster r-cnn: Towards real-time object detection with region proposal networks.</a> Advances in Neural Information Processing Systems. 28, 91-99 (2015).</li><li>Li, H., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=An+improved+deep+learning+approach+for+detection+of+thyroid+papillary+cancer+in+ultrasound+images.">An improved deep learning approach for detection of thyroid papillary cancer in ultrasound images.</a> Scientific Reports. 8, 6600 (2018).</li><li>Lin, T. -. Y., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Feature+pyramid+networks+for+object+detection.">Feature pyramid networks for object detection.</a> Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. , 2117-2125 (2017).</li><li>Ouahabi, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+review+of+wavelet+denoising+in+medical+imaging.">A review of wavelet denoising in medical imaging.</a> 2013 8th International Workshop on Systems, Signal Processing and their Applications. , 19-26 (2013).</li><li>Mahdaoui, A. E., Ouahabi, A., Moulay, M. S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Image+denoising+using+a+compressive+sensing+approach+based+on+regularization+constraints.">Image denoising using a compressive sensing approach based on regularization constraints.</a> Sensors. 22 (6), 2199 (2022).</li><li>Castleman, K. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=.">.</a> Digital Image Processing. , (1996).</li><li>Liu, W., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Ssd:+Single+shot+multibox+detector.">Ssd: Single shot multibox detector.</a> European Conference on Computer Vision. , 21-37 (2016).</li><li>Redmon, J., Farhadi, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Yolov3:+An+incremental+improvement.">Yolov3: An incremental improvement.</a> arXiv. arXiv:1804.02767. , (2018).</li><li>Lin, T. -. Y., Goyal, P., Girshick, R., He, K., Doll&#225;r, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Focalloss+for+dense+object+detection.">Focalloss for dense object detection.</a> arXiv. arXiv:1708.02002. , (2017).</li><li>Carion, N., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=End-to-end+object+detection+with+transformers.">End-to-end object detection with transformers.</a> Computer Vision-ECCV 2020: 16th European Conference. , 23-28 (2020).</li></ol>

The authors declare no conflicts of interest.

This paper describes in detail how to perform the environment setup, data preparation, model configuration, and network training. In the environment setup phase, one needs to pay attention to ensure that the dependent libraries are compatible and matched. Data processing is a very important step; time and effort must be spent to ensure the accuracy of the annotations. When training the model, a &#34;ModuleNotFoundError&#34; may be encountered. In this case, it is necessary to use the &#34;pip install&#34; command to inst...

The incidence of thyroid cancer has increased rapidly since 1970, especially among middle-aged women1. Thyroid nodules may predict the emergence of thyroid cancer, and most thyroid nodules are asymptomatic2. The early detection of thyroid nodules is very helpful in curing thyroid cancer. Therefore, according to current practice guidelines, all patients with suspected nodular goiter on physical examination or with abnormal imaging findings should undergo further examination3,4.
Thyroid ultrasound (US) is a common method used to detect and characterize thyroid lesions5,6. US is a convenient, inexpensive, and radiation-free technology. However, the application of US is easily affected by the operator7,8. Features such as the shape, size, echogenicity, and texture of thyroid nodules are easily distinguishable on US images. Although certain US features-calcifications, echogenicity, and irregular borders-are often considered criteria for identifying thyroid nodules, the presence of interobserver variability is unavoidable8,9. The diagnosis results of radiologists with different levels of experience are different. Inexperienced radiologists are more likely to misdiagnose than experienced radiologists. Some characteristics of US such as reflections, shadows, and echoes can degrade the image quality. This degradation in image quality caused by the nature of US imaging makes it difficult for even experienced physicians to locate nodules accurately.
Computer-aided diagnosis (CAD) for thyroid nodules has developed rapidly in recent years and can effectively reduce errors caused by different physicians and help radiologists diagnose nodules quickly and accurately10,11. Various CNN-based CAD systems have been proposed for thyroid US nodule analysis, including segmentation12,13, detection14,15, and classification16,17. CNN is a multilayer, supervised learning model18, and the core modules of CNN are the convolution and pooling layers. The convolution layers are used for feature extraction, and the pooling layers are used for downsampling. The shadow convolutional layers can extract primary features such as the texture, edges, and contours, while deep convolutional layers learn high-level semantic features.
CNNs have had great success in computer vision19,20,21. However, CNNs fail to capture long-range contextual dependencies due to the limited valid receptive field of the convolutional layers. In the past, backbone architectures for image classification mostly used CNNs. With the advent of Vision Transformer (ViT)22,23, this trend has changed, and now many state-of-the-art models use transformers as backbones. Based on non-overlapping image patches, ViT uses a standard transformer encoder25 to globally model spatial relationships. The Swin Transformer24 further introduces shift windows to learn features. The shift windows not only bring greater efficiency but also greatly reduce the length of the sequence because self-attention is calculated in the window. At the same time, the interaction between two adjacent windows can be made through the operation of shifting (movement). The successful application of the Swin Transformer in computer vision has led to the investigation of transformer-based architectures for ultrasound image analysis26.
Recently, Li et al. proposed a deep learning approach28 for thyroid papillary cancer detection inspired by Faster R-CNN27. Faster R-CNN is a classic CNN-based object detection architecture. The original Faster R-CNN has four modules-the CNN backbone, the region proposal network (RPN), the ROI pooling layer, and the detection head. The CNN backbone uses a set of basic conv+bn+relu+pooling layers to extract feature maps from the input image. Then, the feature maps are fed into the RPN and the ROI pooling layer. The role of the RPN network is to generate region proposals. This module uses softmax to determine whether anchors are positive and generates accurate anchors by bounding box regression. The ROI pooling layer extracts the proposal feature maps by collecting the input feature maps and proposals and feeds the proposal feature maps into the subsequent detection head. The detection head uses the proposal feature maps to classify objects and obtain accurate positions of the detection boxes by bounding box regression.
This paper presents a new thyroid nodule detection network called Swin Faster R-CNN formed by replacing the CNN backbone in Faster R-CNN with the Swin Transformer, which results in the better extraction of features for nodule detection from ultrasound images. In addition, the feature pyramid network (FPN)29 is used to improve the detection performance of the model for nodules of different sizes by aggregating features of different scales.

<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td>GPU RTX3090</td>
			<td>Nvidia</td>
			<td>1</td>
			<td>24G GPU</td>
		</tr>
		<tr>
			<td>mmdetection2.11.0</td>
			<td>SenseTime</td>
			<td>4</td>
			<td>https://github.com/open-mmlab/mmdetection.git</td>
		</tr>
		<tr>
			<td>python3.8</td>
			<td>&#8212;</td>
			<td>2</td>
			<td>https://www.python.org</td>
		</tr>
		<tr>
			<td>pytorch1.7.1</td>
			<td>Facebook</td>
			<td>3</td>
			<td>https://pytorch.org</td>
		</tr>
	</tbody>
</table>

a swin transformer-based model for thyroid nodule detection in ultrasound images

In recent years, the incidence of thyroid cancer has been increasing. Thyroid nodule detection is critical for both the detection and treatment of thyroid cancer. Convolutional neural networks (CNNs) have achieved good results in thyroid ultrasound image analysis tasks. However, due to the limited valid receptive field of convolutional layers, CNNs fail to capture long-range contextual dependencies, which are important for identifying thyroid nodules in ultrasound images. Transformer networks are effective in capturing long-range contextual information. Inspired by this, we propose a novel thyroid nodule detection method that combines the Swin Transformer backbone and Faster R-CNN. Specifically, an ultrasound image is first projected into a 1D sequence of embeddings, which are then fed into a hierarchical Swin Transformer. 
The Swin Transformer backbone extracts features at five different scales by utilizing shifted windows for the computation of self-attention. Subsequently, a feature pyramid network (FPN) is used to fuse the features from different scales. Finally, a detection head is used to predict bounding boxes and the corresponding confidence scores. Data collected from 2,680 patients were used to conduct the experiments, and the results showed that this method achieved the best mAP score of 44.8%, outperforming CNN-based baselines. In addition, we gained better sensitivity (90.5%) than the competitors. This indicates that context modeling in this model is effective for thyroid nodule detection.

The thyroid US images were collected from two hospitals in China from September 2008 to February 2018. The eligibility criteria for including the US images in this study were conventional US examination before biopsy and surgical treatment, diagnosis with biopsy or postsurgical pathology, and age &#8805; 18 years. The exclusion criteria were images without thyroid tissues.
The 3,000 ultrasound images included 1,384 malignant and 1,616 benign nodules. The majority (90%) of the malignant nodules...

Watch this Scientific Journal Video about A Swin Transformer-Based Model for Thyroid Nodule Detection in Ultrasound Images at JoVE.com

A Swin Transformer-Based Model for Thyroid Nodule Detection in Ultrasound Images

Here, a new model for thyroid nodule detection in ultrasound images is proposed, which uses Swin Transformer as the backbone to perform long-range context modeling. Experiments prove that it performs well in terms of sensitivity and accuracy.

In recent years, the incidence of thyroid cancer has been increasing. Thyroid nodule detection is critical for both the detection and treatment of thyroid ...

a-swin-transformer-based-model-for-thyroid-nodule-detection

Research

JoVE Journal

Medicine

1.7K Views.  West China Hospital of Sichuan University. Here, a new model for thyroid nodule detection in ultrasound images is proposed, which uses Swin Transformer as the backbone to perform long-range context modeling. Experiments prove that it performs well in terms of sensitivity and accuracy.

Video: A Swin Transformer-Based Model for Thyroid Nodule Detection in Ultrasound Images

Cerenkov Luminescence Imaging (CLI) for Cancer Therapy Monitoring

In molecular imaging, positron emission tomography (PET) and optical imaging (OI) are two of the most important and thus most widely used modalities1-3. PET is characterized by its excellent sensitivity and quantification ability while OI is notable for non-radiation, relative low cost, short scanning time, high throughput, and wide availability to basic researchers. However, both modalities have their shortcomings as well. PET suffers from poor spatial resolution and high cost, while OI is mostly limited to preclinical applications because of its limited tissue penetration along with prominent scattering optical signals through the thickness of living tissues.
 Recently a bridge between PET and OI has emerged with the discovery of Cerenkov Luminescence Imaging (CLI)4-6. CLI is a new imaging modality that harnesses Cerenkov Radiation (CR) to image radionuclides with OI instruments. Russian Nobel laureate Alekseyevich Cerenkov and his colleagues originally discovered CR in 1934. It is a form of electromagnetic radiation emitted when a charged particle travels at a superluminal speed in a dielectric medium7,8. The charged particle, whether positron or electron, perturbs the electromagnetic field of the medium by displacing the electrons in its atoms. After passing of the disruption photons are emitted as the displaced electrons return to the ground state. For instance, one 18F decay was estimated to produce an average of 3 photons in water5. 
 Since its emergence, CLI has been investigated for its use in a variety of preclinical applications including in vivo tumor imaging, reporter gene imaging, radiotracer development, multimodality imaging, among others4,5,9,10,11. The most important reason why CLI has enjoyed much success so far is that this new technology takes advantage of the low cost and wide availability of OI to image radionuclides, which used to be imaged only by more expensive and less available nuclear imaging modalities such as PET.
 Here, we present the method of using CLI to monitor cancer drug therapy. Our group has recently investigated this new application and validated its feasibility by a proof-of-concept study12. We demonstrated that CLI and PET exhibited excellent correlations across different tumor xenografts and imaging probes. This is consistent with the overarching principle of CR that CLI essentially visualizes the same radionuclides as PET. We selected Bevacizumab (Avastin; Genentech/Roche) as our therapeutic agent because it is a well-known angiogenesis inhibitor13,14. Maturation of this technology in the near future can be envisioned to have a significant impact on preclinical drug development, screening, as well as therapy monitoring of patients receiving treatments.

Cerenkov Luminescence Imaging CLI for Cancer Therapy Monitoring

Use of Cerenkov Luminescence Imaging (CLI) for monitoring preclinical cancer treatment is described here. This method takes advantage of Cerenkov Radiation (CR) and optical imaging (OI) to visualize radiolabeled probes and thus provides an alternative to PET in preclinical therapeutic monitoring and drug screening.

In  molecular imaging, positron emission tomography (PET) and optical imaging (OI)  are two of the most important and thus most widely used modalities1-3. ...

An Orthotopic Mouse Model of Anaplastic Thyroid Carcinoma

Several types of animal models of human thyroid carcinomas have been established, including subcutaneous xenograft and orthotopic implantation of cancer cells into immunodeficient mice. Subcutaneous xenograft models have been valuable for preclinical screening and evaluation of new therapeutic treatments. There are a number of advantages to using a subcutaneous model; 1) rapid, 2) reproducible, and 3) tumor establishment, growth, and response to therapeutic agents may be monitored by visual inspection. However, substantial evidence has shed light on the short-comings of subcutaneous xenograft models1-3. For instance, medicinal treatments demonstrating curative properties in subcutaneous xenograft models often have no notable impact on the human disease. The microenvironment of the site of xenographic transplantation or injection lies at the heart of this dissimilarity.
Orthotopic tumor xenograft models provide a more biologically relevant context in which to study the disease. The advantages of implanting diseased cells or tissue into their anatomical origin equivalent within a host animal includes a suitable site for tumor-host interactions, development of disease-related metastases and the ability to examine site-specific influence on investigational therapeutic remedies. Therefore, orthotopic xenograft models harbor far more clinical value because they closely reproduce human disease. For these reasons, a number of groups have taken advantage of an orthotopic thyroid cancer model as a research tool4-7.
Here, we describe an approach that establishes an orthotopic model for the study of anaplastic thyroid carcinoma (ATC), which is highly invasive, resists treatment, and is virtually fatal in all diagnosed patients. Cultured ATC cells are prepared as a dissociated cellular suspension in a solution containing a basement membrane matrix. A small volume is slowly injected into the right thyroid gland. Overall appearance and health of the mice are monitored to ensure minimal post-operative complications and to gauge pathological penetrance of the cancer. Mice are sacrificed at 4 weeks, and tissue is collected for histological analysis. Animals may be taken at later time-points to examine more advance progression of the disease. Production of this orthotopic mouse model establishes a platform that accomplishes two objectives: 1) further our understanding of ATC pathology, and 2) screen current and future therapeutic agents for efficacy in combating ATC.

Generation of an orthotopic mouse model of anaplastic thyroid carcinoma is described here. This technique employs surgical placement of human anaplastic thyroid cancer cells into the thyroid of immunodeficient mice, thus creating a more clinically relevant setting to study disease progression as well as screen innovative therapeutic interventions.

Several types of animal models of human thyroid carcinomas have been established, including subcutaneous xenograft and orthotopic implantation of cancer ...

Substernal Thyroid Biopsy Using Endobronchial Ultrasound-guided Transbronchial Needle Aspiration

Substernal thyroid goiter (STG) represents about 5.8% of all mediastinal lesions1. There is a wide variation in the published incidence rates due to the lack of a standardized definition for STG. Biopsy is often required to differentiate benign from malignant lesions. Unlike cervical thyroid, the overlying sternum precludes ultrasound-guided percutaneous fine needle aspiration of STG. Consequently, surgical mediastinoscopy is performed in the majority of cases, causing significant procedure related morbidity and cost to healthcare. Endobronchial Ultrasound-guided Transbronchial Needle Aspiration (EBUS-TBNA) is a frequently used procedure for diagnosis and staging of non-small cell lung cancer (NSCLC). Minimally invasive needle biopsy for lesions adjacent to the airways can be performed under real-time ultrasound guidance using EBUS. Its safety and efficacy is well established with over 90% sensitivity and specificity. The ability to perform EBUS as an outpatient procedure with same-day discharges offers distinct morbidity and financial advantages over surgery. As physicians performing EBUS gained procedural expertise, they have attempted to diversify its role in the diagnosis of non-lymph node thoracic pathologies. We propose here a role for EBUS-TBNA in the diagnosis of substernal thyroid lesions, along with a step-by-step protocol for the procedure.

Substernal thyroid lesions are common, and need to be differentiated from malignancy. Obtaining percutaneous fine needle biopsy is not possible due to its retrosternal location. This article proposes a protocol for biopsy of substernal thyroid lesions using Endobronchial Ultrasound-guided Transbronchial Needle Aspiration (EBUS-TBNA).

Substernal thyroid goiter (STG) represents about 5.8% of all mediastinal lesions1. There is a wide variation in the published incidence rates due to the ...

Intra-Operative Neural Monitoring of Thyroid Surgery in a Porcine Model

Intraoperative injury to the recurrent laryngeal nerve (RLN) can cause vocal cord paralysis, which interferes with speech and can potentially interfere with breathing. In recent years, intraoperative neural monitoring (IONM) has been widely adapted as an adjunct technique to localize the RLN, detect RLN injury, and predict vocal cord function during the operations. Many studies have also used animal models to investigate new applications of IONM technology and to develop reliable strategies for preventing intraoperative RLN injury. The aim of this article is to introduce a standard protocol for using a porcine model in IONM research. The article demonstrates the procedures for inducing general anesthesia, performing tracheal intubation, and experimental design to investigate the electrophysiological characteristics of RLN injuries. Applications of this protocol can improve overall efficacy in implementing the 3R principle (replacement, reduction and refinement) in porcine IONM studies.

This study aims to develop a standard protocol of intra-operative neural monitoring of thyroid surgery in a porcine model. Here, we present a protocol to demonstrate general anesthesia, to compare different types of electrodes, and to investigate the electrophysiological characteristics of the normal and injured recurrent laryngeal nerves.

Intraoperative injury to the recurrent laryngeal nerve (RLN) can cause vocal cord paralysis, which interferes with speech and can potentially interfere ...

Electromagnetic Navigation Transthoracic Nodule Localization for Minimally Invasive Thoracic Surgery

The increased use of chest computed tomography (CT) has led to an increased detection of pulmonary nodules requiring diagnostic evaluation and/or excision. Many of these nodules are identified and excised via minimally invasive thoracic surgery; however, subcentimeter and subsolid nodules are frequently difficult to identify intra-operatively. This can be mitigated by the use of electromagnetic transthoracic needle localization. This protocol delineates the step-by-step process of electromagnetic localization from the pre-operative period to the postoperative period and is an adaptation of the electromagnetically guided percutaneous biopsy previously described by Arias et al. Pre-operative steps include obtaining a same day CT followed by the generation of a three-dimensional virtual map of the lung. From this map, the target lesion(s) and an entry site are chosen. In the operating room, the virtual reconstruction of the lung is then calibrated with the patient and the electromagnetic navigation platform. The patient is then sedated, intubated, and placed in the lateral decubitus position. Using a sterile technique and visualization from multiple views, the needle is inserted into the chest wall at the prechosen skin entry site and driven down to the target lesion. Dye is then injected into the lesion and, then, continuously during needle withdrawal, creating a tract for visualization intra-operatively. This method has many potential benefits when compared to the CT-guided localization, including a decreased radiation exposure and decreased time between the dye injection and the surgery. Dye diffusion from the pathway occurs over time, thereby limiting intra-operative nodule identification. By decreasing the time to surgery, there is a decrease in wait time for the patient, and less time for dye diffusion to occur, resulting in an improvement in nodule localization. When compared to electromagnetic bronchoscopy, airway architecture is no longer a limitation as the target nodule is accessed via a transparenchymal approach. Details of this procedure are described in a step-by-step fashion.

Presented here is a protocol for lung nodule localization using dye marking via electromagnetically navigated transthoracic needle access. The technique described here can be accomplished in the peri-operative period to optimize nodule localization and to successful resection when performing minimally invasive thoracic surgery.

The increased use of chest computed tomography (CT) has led to an increased detection of pulmonary nodules requiring diagnostic evaluation and/or ...

A Pleural Effusion Model in Rats by Intratracheal Instillation of Polyacrylate/Nanosilica

Pleural effusion is a prevalent clinical finding of many pulmonary diseases. Having a useful animal pleural effusion model is very important to study these pulmonary diseases. Previous pleural effusion models paid more attention to biological factors rather than nanoparticles in the environment. Here, we introduce a model to make pleural effusion in rats by intratracheal instillation of polyacrylate/nanosilica, and a method of nanoparticle isolation in the pleural effusion. By intratracheal instillation of polyacrylate/nanosilica with concentrations of 3.125, 6.25 and 12.5 mg/kg&#8729;mL, the pleural effusion in rats presented on day 3, peaked at days 7-10 in 6.25 and 12.5 mg/kg&#8729;mL groups, then slowly decreased and disappeared on day 14. When the concentration of polyacrylate/nanosilica increased, the pleural effusion is produced more and faster. This pleural fluid was detected by ultrasound examination or CT chest scanning and confirmed by dissection of rats. Silica nanoparticles were observed in the rats' pleural effusion by transmission electron microscope. These results showed that the exposure to polyacrylate/nanosilica leads to the induction of pleural effusion, which was consistent with our previous report in humans. Additionally, this model is beneficial for the further study of nanotoxicology and the pleural effusion diseases.

Here, we present a protocol to construct a pleural effusion model in rats by intratracheal instillation of polyacrylate/nanosilica.

Pleural effusion is a prevalent clinical finding of many pulmonary diseases. Having a useful animal pleural effusion model is very important to study ...

A Personalized 3D-Printed Model for Preoperative Evaluation in Thyroid Surgery

The anatomic structure of the surgical area of thyroid cancer is complex. It is very important to comprehensively and carefully evaluate the tumor location and its relation with the capsule, trachea, esophagus, nerves, and blood vessels before operation. This paper introduces an innovative 3D-printed model establishment method based on computerized tomography (CT) DICOM images. We established a personalized 3D-printed model of the cervical thyroid surgery field for each patient who needed thyroid surgery to help clinicians evaluate the key points and difficulties of the surgery and select the operation methods of key parts as a basis. The results showed that this model is conducive to preoperative discussion and the formulation of operation strategies. In particular, as a result of the clear display of the recurrent laryngeal nerve and parathyroid gland locations in the thyroid operation field, injury to them can be avoided during surgery, the difficulty of thyroid surgery reduced, and the incidence of postoperative hypoparathyroidism and complications related to recurrent laryngeal nerve injury reduced too. Moreover, this 3D-printed model is intuitive and aids communication for the signing of informed consent by patients before surgery.

Here, a new method of establishing a personalized 3D-printed model for preoperative evaluation of thyroid surgery is proposed. It is conducive to preoperative discussion, reducing the difficulty of thyroid surgery.

The anatomic structure of the surgical area of thyroid cancer is complex. It is very important to comprehensively and carefully evaluate the tumor ...

Constructing a 3D Digital Model for any Specific Nodule

Technique for Salivary Gland Ultrasound Examination of Sjogren's Syndrome

Ultrasonographic Evaluation of Salivary Glands for Sjogren's Syndrome: Diagnostic and Monitoring Insights

Sjogren's syndrome (SS) is a chronic autoimmune condition commonly affecting the exocrine glands, causing oral or ocular dryness and extraglandular manifestations including arthralgia, cytopenia, and lymphoma. The presence of autoantibodies against SSA/Ro, labial salivary gland biopsy, ocular staining, Schirmer's test, or salivary flow assessment are included in the current classification criteria of SS. However, the availability and invasiveness of these procedures limit their widespread use in clinical settings. Salivary gland ultrasonography (SGUS) is a non-invasive imaging modality for the evaluation of the salivary gland parenchyma and is increasingly utilized to aid diagnosis and monitoring in SS. 
This article presents the protocol of SGUS for image acquisition at the parotid and submandibular glands. The objective is to present a standardized, reproducible, and practical approach to diagnostic SGUS for SS in daily clinical settings. Major salivary glands are scanned in a stepwise approach, beginning at the angle of the mandible for the superficial lobe of the parotid gland, followed by the deep lobe below the ramus of the mandible. Submandibular areas are then scanned for the submandibular glands. The steps in obtaining salivary gland images at each anatomical site are explained in the accompanying video. The echogenicity and echotexture at the thyroid gland are taken as a reference. The homogeneity, the presence and distribution of hypoechoic areas within the glands, and the border of the salivary glands are examined. The sizes and features of intra-/peri-glandular lymph nodes are recorded. The most distinctive sonographic feature in SS is glandular heterogeneity with the presence of hypoechoic/hyperechoic areas within the glands. 
In summary, while SGUS cannot diagnose SS on its own, it can supplement the current classification criteria of SS and guide the clinical decision for salivary gland biopsy to support the diagnosis of SS in patients with sicca syndrome or suspicious systemic features, combined with autoantibody testing.

Author Spotlight: A Focus on Standardized Salivary Gland Ultrasound Protocol in Connective Tissue Disease Research

Salivary gland ultrasonography is a promising tool for the diagnosis and evaluation of disease activity and complications in Sjogren's syndrome (SS). The scanning technique for salivary gland ultrasound examination relevant to SS is described in the article.

Sjogren's syndrome (SS) is a chronic autoimmune condition commonly affecting the exocrine glands, causing oral or ocular dryness and extraglandular ...

Novel PLL-AF Suture Technique for Annulus Fibrosus Repair in Lumbar Endoscopy

A Suture Technique for Ruptured Annulus Fibrosus Following Decompression Under Percutaneous Transforaminal Endoscopic Discectomy

The suture technique for a ruptured annulus fibrosus (AF) under full-endoscopy remains challenging. Direct suturing of a ruptured annular tear after full decompression has been shown to decrease the recurrence rate of lumbar disc herniation during endoscopic surgery. Traditional suture operations under endoscopy involve only simple suturing of the ruptured AF. Due to the weak and poor quality of the AF tissue around the tear portal, using this area as needle insertion points during suturing may lead to insufficient tension and a low success rate of AF closure. Currently, there is no detailed technical illustration based on video for AF tear suturing under lumbar full-endoscopy. We innovatively propose a method of covering and suturing the AF tear by pulling up the posterior longitudinal ligament (PLL) under lumbar endoscopy and using three stitches (PLL-AF suture technique). The patient who received the novel suture technique achieved satisfactory results. Six months after the operation, lumbar MRI showed no evidence of recurrence in the outpatient clinic.

Author Spotlight: Development and Application of a Novel Suture Technique for Annular Fibrosus Repair in Percutaneous Transforaminal Endoscopic Discectomy

This protocol describes an innovative suture technique for ruptured annular fibrosus during percutaneous transforaminal endoscopic discectomy.

The suture technique for a ruptured annulus fibrosus (AF) under full-endoscopy remains challenging. Direct suturing of a ruptured annular tear after full ...