This work was supported by the National Nature Foundation of China (grant no.82004504), the National Key Research and Development Program of the Ministry of Science and Technology of China (grant no.2018YFC1707606), Chinese Medicine Administration of Sichuan Province (grant no.2021MS199) and National Nature Foundation of China (grant no.82174236).

This study has been approved by the National Natural Science Foundation of China project, Constructing Dynamic Change rules of TCM Facial image Based on Association Analysis. The ethics approval number is 2021KL-027, and the ethics committee has approved the clinical study to be carried out in accordance with the approved documents which include clinical research protocol (2021.04.12, V2.0), informed consent (2021.04.12, V2.0), subject recruitment materials (2021.04.12, V2.0), study cases and/or case reports, subject dia...

<ol>
	<li>Jiu, G., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Effects+of+herbal+therapy+on+intestinal+microbiota+and+serum+metabolomics+in+different+rat+models+of+Mongolian+medicine.">Effects of herbal therapy on intestinal microbiota and serum metabolomics in different rat models of Mongolian medicine.</a> Evidence-Based Complementary and Alternative. 2022, 7255780 (2022).</li><li>Xi, J., Xin, Y., Teregle, <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Study+on+the+correlation+between+the+animal+model+of+Mongolian+medicine+34;Xieri+disease&amp;#34;+and+serum+ALT+and+AST.">Study on the correlation between the animal model of Mongolian medicine 34;Xieri disease&amp;#34; and serum ALT and AST.</a> Electronic Journal of Cardiovascular Diseases in Combination of Traditional Chinese. 4 (33), 134-135 (2016).</li><li>Yin, L., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Study+on+the+mechanism+of+serum+differential+protein+changes+in+bronchial+asthma+based+on+proteomics.">Study on the mechanism of serum differential protein changes in bronchial asthma based on proteomics.</a> Chinese Journal of Traditional Chinese Medicine. 47 (22), 6227-6234 (2022).</li><li>Wang, X. H., Bao, L. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Band+Seed.+The+origin+and+development+of+tongue+diagnosis+in+Mongolian+medicine.">Band Seed. The origin and development of tongue diagnosis in Mongolian medicine.</a> Chinese Ethnic Folk Medicine. (1), 64-65 (2008).</li><li>Yang, S., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+comparative+study+on+the+feature+parameters+of+tongue+diagnosis+images+in+five+nationalities.">A comparative study on the feature parameters of tongue diagnosis images in five nationalities.</a> Chinese Journal of Traditional Chinese Medicine. 36 (11), 6428-6430 (2021).</li><li>He, J. Y., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Efficacy+and+safety+of+Chou-Ling-Dan+granules+in+the+treatment+of+seasonal+influenza+via+combining+Western+and+traditional+Chinese+medicine,+protocol+for+a+multicentre,+randomised+controlled+clinical+trial.">Efficacy and safety of Chou-Ling-Dan granules in the treatment of seasonal influenza via combining Western and traditional Chinese medicine, protocol for a multicentre, randomised controlled clinical trial.</a> BMJ Open. 9 (4), e024800 (2019).</li><li>Wang, D. J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Scientific+knowledge+mapping+and+visualization+analysis+in+the+field+of+Chinese+medicine+tongue+feature+objectification+research.">Scientific knowledge mapping and visualization analysis in the field of Chinese medicine tongue feature objectification research.</a> World Science and Technology - Modernization of Chinese Medicine. 23 (9), 3032-3040 (2021).</li><li>Yuan, S. M., Qian, P., Li, F. F. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Research+progress+of+color+correction+methods+for+tongue+and+face+diagnosis+in+traditional+Chinese+Medicine.">Research progress of color correction methods for tongue and face diagnosis in traditional Chinese Medicine.</a> Chinese Journal of Traditional Chinese Medicine. 34 (9), 4183-4185 (2019).</li><li>Kanawong, R., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Tongue+image+analysis+and+its+mobile+app+development+for+health+diagnosis.">Tongue image analysis and its mobile app development for health diagnosis.</a> Advances in Experimental Medicine and Biology. 1005, 99-121 (2017).</li><li>Yu, Y., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Semantic+segmentation+evaluation+index+and+evaluation+method.">Semantic segmentation evaluation index and evaluation method.</a> Computer Engineering and Application. , (2023).</li><li>Sehyung, L., Negishi, M., Urakubo, H., Kasai, H., Ishii, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Mu-net:+Multi-scale+U-net+for+two-photon+microscopy+image+denoising+and+restoration.">Mu-net: Multi-scale U-net for two-photon microscopy image denoising and restoration.</a> Neural Networks. 125, 92-103 (2020).</li><li>Huang, X. M., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+review+on+the+application+of+U-Net+and+its+variants+in+medical+image+segmentation.">A review on the application of U-Net and its variants in medical image segmentation.</a> Chinese Journal of Biomedical Engineering. 41 (5), 567-576 (2022).</li><li>Lu, J. H., Xu, Y. F., Wang, Y. Q., Hao, Y. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Research+overview+of+tongue+objectification+in+traditional+Chinese+medicine+based+on+computer+image+technology.">Research overview of tongue objectification in traditional Chinese medicine based on computer image technology.</a> World Science and Technology - Modernization of Traditional Chinese Medicine. 24 (11), 4568-4573 (2022).</li><li>Mansour, R. F., Althobaiti, M. M., Ashour, A. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Internet+of+things+and+synergic+deep+learning+based+biomedical+tongue+color+image+analysis+for+disease+diagnosis+and+classification.">Internet of things and synergic deep learning based biomedical tongue color image analysis for disease diagnosis and classification.</a> IEEE Access. 9, 94769-94779 (2021).</li><li>Cai, Y. H., Hu, S. B., Guan, J., Zhang, X. F. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Analysis+of+the+development+and+application+of+tongue+diagnosis+objectification+techniques+in+Chinese+medicine.">Analysis of the development and application of tongue diagnosis objectification techniques in Chinese medicine.</a> World Science and Technology - Modernization of Chinese Medicine. 23 (7), 2447-2453 (2021).</li><li>Ghosh, S., Das, N., Nasipuri, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Reshaping+inputs+for+convolutional+neural+network:+some+common+and+uncommon+methods.">Reshaping inputs for convolutional neural network: some common and uncommon methods.</a> Pattern Recognition. 93, 79-94 (2019).</li><li>Shang, Z. M., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Research+progress+of+digital+acquisition+and+characterization+of+tongue+diagnosis+information.">Research progress of digital acquisition and characterization of tongue diagnosis information.</a> Chinese Journal of Traditional Chinese Medicine. 36 (10), 6010-6013 (2021).</li><li>Ning, J., Zhang, D., Wu, C., Yue, F. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Automatic+tongue+image+segmentation+based+on+gradient+vector+flow+and+region+merging.">Automatic tongue image segmentation based on gradient vector flow and region merging.</a> Neural Computing and Applications. 21, 1819-1826 (2012).</li><li>Zheng, F., Huang, X. Y., Wang, B. L., Wang, Y. H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+method+for+tongue+detection+based+on+image+segmentation.">A method for tongue detection based on image segmentation.</a> Journal of Xiamen University. 55 (6), 895-900 (2016).</li><li>Li, Y. T., Luo, Y. S., Zhu, Z. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Deep+learning-based+tongue+feature+analysis.">Deep learning-based tongue feature analysis.</a> Computer Science. 47 (11), 148-158 (2020).</li><li>Wang, Y. D., Sun, C. H., Cui, J. L., Wu, X. R., Qin, Y. X. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Research+on+deep+learning-based+tongue+fissure+segmentation+algorithm.">Research on deep learning-based tongue fissure segmentation algorithm.</a> World Science and Technology - Modernization of Chinese Medicine. 23 (9), 3065-3073 (2021).</li><li>Wang, X. M., Wang, R. Y., Guo, D., Lu, S. Z., Zhou, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Research+on+the+identification+method+of+tongue+punctures+based+on+auxiliary+light+source.">Research on the identification method of tongue punctures based on auxiliary light source.</a> Journal of Sensing Technology. 29 (10), 1553-1559 (2016).</li><li>Yang, J. D., Zhang, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+fully+connected+neural+network+based+on+migration+learning+for+tongue+image+classification.">A fully connected neural network based on migration learning for tongue image classification.</a> Journal of the Second Military Medical University. 39 (8), 897-902 (2018).</li><li>Song, C., Wang, B., Xu, J. T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Research+on+tongue+feature+classification+method+based+on+deep+migration+learning.">Research on tongue feature classification method based on deep migration learning.</a> Computer Engineering and Science. 43 (8), 1488-1496 (2021).</li><li>Ding, H. J., He, J. C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Study+on+modern+techniques+and+methods+of+tongue+diagnosis.">Study on modern techniques and methods of tongue diagnosis.</a> Shi Zhen Chinese Medicine. 21 (5), 1230-1232 (2010).</li><li>Zhai, P. B., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+multi-stage+tongue+image+classification+algorithm+incorporating+attention+mechanism.">A multi-stage tongue image classification algorithm incorporating attention mechanism.</a> Computer Engineering and Design. 42 (6), 1606-1613 (2021).</li><li>Hou, Y. S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+new+clustering+analysis+algorithm+based+on+deep+learning.">A new clustering analysis algorithm based on deep learning.</a> Journal of Xinxiang University. 35 (12), 4 (2018).</li><li>Shi, Y. L., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+decision+tree+algorithm+for+classification+of+non-small+cell+lung+cancer+evidence+based+on+tongue+and+pulse+data.">A decision tree algorithm for classification of non-small cell lung cancer evidence based on tongue and pulse data.</a> World Science and Technology - Modernization of Chinese Medicine. 24 (7), 2766-2775 (2022).</li><li>Li, J., Hu, X. J., Zhou, C. L., Xu, J. T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Study+on+the+feature+analysis+and+diagnosis+model+of+diabetic+tongue+based+on+random+forest+algorithm.">Study on the feature analysis and diagnosis model of diabetic tongue based on random forest algorithm.</a> Chinese Journal of Traditional Chinese Medicine. 37 (3), 1639-1643 (2022).</li><li>Chen, C. H., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+characteristics+of+the+combination+of+the+four+diagnostic+methods+of+traditional+Chinese+medicine+from+the+perspective+of+the+differences+between+Chinese+and+Western+medical+diagnosis+methods.">The characteristics of the combination of the four diagnostic methods of traditional Chinese medicine from the perspective of the differences between Chinese and Western medical diagnosis methods.</a> Journal of Guangzhou University of Traditional Chinese Medicine. 28 (3), 332-334 (2011).</li><li>Chen, R., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Correlation+analysis+of+tongue+image+and+western+medicine+typing+in+382+patients+with+novel+coronavirus+pneumonia+based+on+Lab+colour+model+and+imaging+histology.">Correlation analysis of tongue image and western medicine typing in 382 patients with novel coronavirus pneumonia based on Lab colour model and imaging histology.</a> Chinese Journal of Traditional Chinese Medicine. 36 (12), 7010-7014 (2021).</li><li>Ju, J. W., Jung, H., Lee, Y. J., Mun, S. W., Lee, J. H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Semantic+segmentation+dataset+for+AI-based+quantification+of+clean+mucosa+in+capsule+endoscopy.">Semantic segmentation dataset for AI-based quantification of clean mucosa in capsule endoscopy.</a> Medicina. 58 (3), 397 (2022).</li><li>Wu, X., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+review+of+research+on+deep+learning+in+tongue+image+classification.">A review of research on deep learning in tongue image classification.</a> Computer Science and Exploration. , 1-23 (2022).</li></ol>

The authors have no conflict of interest to declare.

Based on the comparison results presented above, it is evident that the characteristics of the four algorithms under consideration are varied, and their distinct advantages and disadvantages are described below. The U-Net structure, based on the modification and expansion of a full convolution network, can obtain contextual information and precise positioning through a contracting path and a symmetrical expanding path. By classifying each pixel point, this algorithm achieves a higher segmentation accuracy and segments th...

Tongue observation is a widely utilized technique in traditional Chinese ethnic medicine (TCM). The color and shape of the tongue can reflect the physical condition and various disease properties, severities, and prognoses. For instance, in traditional Hmong medicine, the tongue&#39;s color is used to identify body temperature e.g., a red or purple tongue indicates pathological factors related to heat. In Tibetan medicine, a condition is judged by observing the tongue of a patient, paying attention to the color, shape, and moisture of the mucus. For instance, the tongues of patients with Heyi disease become red and rough or black and dry1; patients with Xieri disease2 have yellow and dry tongues; meanwhile, patients with Badakan disease3 have a white, humid, and soft tongue4. These observations reveal the close relationship between tongue features and physiology and pathology. Overall, the state of the tongue plays a vital role in diagnosis, disease identification, and evaluation of the treatment effect.
Simultaneously, owing to diverse living conditions and dietary practices among different ethnic groups, variations in tongue images are evident. The Lab model, established on the basis of an international standard for the determination of color, was formulated by the Commission International Eclairage (CIE) in 1931. In 1976, a color pattern was modified and named. The Lab color model is composed of three elements: L corresponds to brightness, while a and b are two color channels. a includes colors from dark green (low brightness value) to gray (medium brightness value) to bright pink (high brightness value); b goes from bright blue (low brightness value) to gray (medium brightness value) to yellow (high brightness value). By comparing the L x a x b values of the tongue color of five ethnic groups, Yang et al.5 found that the characteristics of tongue images of the Hmong, Hui, Zhuang, Han, and Mongolian groups were significantly distinct from each other. For example, the Mongolians have dark tongues with a yellow tongue coating, while the Hmong have light tongues with a white tongue coating, suggesting that tongue features can be used as a diagnostic indicator for assessing the health status of a population. Moreover, tongue images can function as an evaluation index for evidence-based medicine in clinical research of ethnic medicine. He et al.6 employed tongue images as a foundation for TCM diagnosis and systematically evaluated the safety and efficacy of Chou-Ling-Dan pellets (CLD granules-used to treat inflammatory and febrile diseases, including seasonal influenza in TCM) combined with Chinese and Western medicine. The results established the scientific validity of tongue images as an evaluation index for clinical studies. Nevertheless, traditional medical practitioners generally rely on subjectivity to observe tongue characteristics and assess patients&#39; physiological and pathological conditions, requiring more precise indicators.
The emergence of the internet and artificial intelligence technology has paved the way for digitizing and objectifying tongue diagnosis. This process involves using mathematical models to provide a qualitative and objective description of tongue images7, reflecting the content of the tongue image. The process includes several steps: image acquisition, optical compensation, color correction, and geometric transformation. The pre-processed images are then fed into an algorithmic model for image positioning and segmentation, feature extraction, pattern recognition, etc. The output of this process is a highly efficient and precise diagnosis of tongue image data, thereby achieving the goal of objectification, quantification, and informatization of tongue diagnosis8. Thus, the purpose of high efficiency and high precision processing of tongue diagnosis data is achieved. Based on tongue diagnosis knowledge and deep learning technology, this study automatically separated the tongue body and tongue coating from tongue images using a computer algorithm, in order to extract the quantitative features of tongues for doctors, improve the reliability and consistency of diagnosis, and provide methods for subsequent tongue diagnosis objectification research9.

<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td>CPU</td>
			<td>Intel(R) Core(TM) i7-9700K</td>
			<td></td>
			<td></td>
		</tr>
		<tr>
			<td>GPU</td>
			<td>&#160;NVIDIA GeForce RTX 3070 Ti (8192MB)</td>
			<td></td>
			<td></td>
		</tr>
		<tr>
			<td>Operating systems</td>
			<td>Microsoft Windows 10 Professional Edition (64-bit)</td>
			<td></td>
			<td></td>
		</tr>
		<tr>
			<td>Programming language</td>
			<td>Python</td>
			<td></td>
			<td></td>
		</tr>
		<tr>
			<td>RAM</td>
			<td>16G</td>
			<td></td>
			<td></td>
		</tr>
	</tbody>
</table>

objectification of tongue diagnosis in traditional medicine, data analysis, and study application

Tongue diagnosis is an essential technique of traditional Chinese medicine (TCM) diagnosis, and the need for objectifying tongue images through image processing technology is growing. The present study provides an overview of the progress made in tongue objectification over the past decade and compares segmentation models. Various deep learning models are constructed to verify and compare algorithms using real tongue image sets. The strengths and weaknesses of each model are analyzed. The findings indicate that the U-Net algorithm outperforms other models regarding precision accuracy (PA), recall, and mean intersection over union (MIoU) metrics. However, despite the significant progress in tongue image acquisition and processing, a uniform standard for objectifying tongue diagnosis has yet to be established. To facilitate the widespread application of tongue images captured using mobile devices in tongue diagnosis objectification, further research could address the challenges posed by tongue images captured in complex environments.

For the comparison results, see Figure 12, Figure 13, and Table 1, where the environment constructed by this study uses the same samples to train and test the algorithm model. MIoU indicator: U-Net &#62; Seg-Net &#62; PSPNet &#62; DeeplabV3; MPA indicator: U-Net &#62; Seg-Net &#62; PSPNet &#62; DeeplabV3; precision indicator: U-Net &#62; Seg-Net &#62; DeeplabV3 &#62; PSPNet; recall: U-Net &#62; Seg-Net &#62; PSPNet &#62; DeeplabV3. The larger th...

Watch this Scientific Journal Video about Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application at JoVE.com

Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application

The present study employed U-Net and other deep learning algorithms to segment a tongue image and compared the segmentation results to investigate the objectification of tongue diagnosis.

Tongue diagnosis is an essential technique of traditional Chinese medicine (TCM) diagnosis, and the need for objectifying tongue images through image ...

This protocol uses deep learning algorithm to segment tongue images and automatically separates the tongue from the whole image, which can be used for further research. This protocol compares the performance of four-tongue image segmentation algorithm to select the base algorithm that can segment a tongue with a clear counter. Patented instruments and algorithm are used in this study.It is recommended to use professional instruments and master computer technology. To begin, prepare the self-developed handheld lingual face diagnostic instrument to collect lingual face images of patients. Fill in the patient's name, gender, age, and disease on the computer page.Next, ask the patient to sit upright and position the image acquisition device connected to the computer. Then, position the face of the patient in the image acquisition instrument and ask them to extend their tongue out of their mouth to the maximum extent. Verify through the images on the computer screen that the patient is in the correct position and that the tongue and face are fully exposed.Press the shoot button on the computer screen three times to capture three images. Then, manually select the collected images of the face and the tongue. Filter them according to the Image Acquisition page of the software.Collect three images from each patient at a time and export the images to a folder for manual screening. Next, select a standard, fully exposed, well-illuminated, and clear images, the sample for algorithm training and testing. Delete the images that do not meet the criteria of a standard image.The criteria for exclusion are incomplete tongue and face exposure and images that are too dark. To perform tongue image segmentation, open LabelMe and click on the Open button in the upper left corner of the interface. Select the folder containing the image to be segmented and open the photos.Click the Create Polygon button to track the points. Track the tongue and lingual shapes, name them according to the selected areas, and save. On finishing all tracking, click Save to save the image to the data folder.Set the device to capture images with a size of 1080x1920 pixels and ensure the size of the filled image is 1920x1920 pixels. Manually screen the the images of each patient to select and retain one from the three images that were taken for uncontrollable factors, such as subject blinking and lens blocking. For training the model, collect data from 200 individuals or 600 images and retain about 200 images that are usable after the screening.Randomly divide all the tongue images according to the image number, placing 70%of them into the Training Set and 30%of the images into the Test Set folder. Download and install Anaconda Python and LabelMe from their official websites. Activate the environment and complete the installation and adjustment of the overall environment.Choose the appropriate model for the research purpose. In this study, UNet, SegNet, DeepLab Version 3, and PSPNet were selected for validation. Next, build the deep learning algorithm model in the installed environment.Tune the parameters by adjusting the values and complete the model training using the Training Set. Using LabelMe annotation and uniform image size methods, construct the required data set in conjunction with the research content. Examine the results of the segmentation and evaluate the model performance based on four metrics, precision, recall, mean pixel accuracy or MPA, and MIOU, to get a more comprehensive evaluation.Compare the generated values of the four models. The higher the value, the higher the segmentation accuracy and the better the performance of the model. According to the index results, the UNet algorithm is superior to the other algorithms in MIOU, MPA, precision, and recall, and its segmentation accuracy is also higher.PSPNet is better than DeepLab Version 3 in MIOU, MPA, and recall, while the DeepLab Version 3 model is lower than the segment model in all indexes. In tongue segmentation, the outer red area in the image results from tongue segmentation and the inner green area results from tongue coating segmentation. In this study, the unit algorithm performed well in tongue segmentation, and further research can be conducted based on the unit algorithms.

objectification-tongue-diagnosis-traditional-medicine-data-analysis

Research

JoVE Journal

Medicine

2.1K Views.  Chengdu University of Traditional Chinese Medicine. This protocol uses deep learning algorithm to segment tongue images and automatically separates the tongue from the whole image, which can be used for further research. This protocol compares the performance of four-tongue image segmentation algorithm to select the base algorithm that can segment a tongue with a clear counter. Patented instruments and algorithm are used in this study.It is recommended to use professional instruments and master computer technology. To begin, prepare the...