Name: objectification of tongue diagnosis in traditional medicine, data analysis, and study application
Uploaded: 2023-04-14T14:15:00.000+00:00
Duration: 5 min 56 s
Description: Watch this Scientific Journal Video about Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application at JoVE.com

This work was supported by the National Nature Foundation of China (grant no.82004504), the National Key Research and Development Program of the Ministry of Science and Technology of China (grant no.2018YFC1707606), Chinese Medicine Administration of Sichuan Province (grant no.2021MS199) and National Nature Foundation of China (grant no.82174236).

This study has been approved by the National Natural Science Foundation of China project, Constructing Dynamic Change rules of TCM Facial image Based on Association Analysis. The ethics approval number is 2021KL-027, and the ethics committee has approved the clinical study to be carried out in accordance with the approved documents which include clinical research protocol (2021.04.12, V2.0), informed consent (2021.04.12, V2.0), subject recruitment materials (2021.04.12, V2.0), study cases and/or case reports, subject diary cards and other questionnaires (2021.04.12, V2.0), a list of participants in the clinical trial, research project approval, etc. Informed consent from the patients participating in the study was obtained. The main experimental approach of this study is to use real tongue images to validate and compare the model segmentation effects. Figure 1 presents the components of tongue diagnosis objectification.
1. Image acquisition
<ol>
	<li>Use the self-developed hand-held lingual face diagnostic instrument to collect lingual face images of patients.</li>
	<li>Fill in the patient&#39;s name, gender, age, and disease on the computer page. Images included here are from patients who came to the clinic and agreed to be photographed after being informed of the purpose and content of the study. Confirm that the patient is sitting upright, place the whole face in the image acquisition instrument, and instruct the patient to extend their tongue out of their mouth to the maximum extent.</li>
	<li>Hold the image acquisition device connected to a computer and verify through the images on the computer screen that the patient is in the correct position and that the tongue and face are fully exposed.</li>
	<li>Press the Shoot button on the computer screen three times to take three pictures. 
	NOTE: The image acquisition instrument is currently only at the patent application stage and is not for commercial use, so it is not for sale.</li>
	<li>Manually select and filter the collected tongue and face images. Filter and exclude images that have incomplete tongue and face exposure, as well as images that are too dark due to insufficient light. Figure 2 shows the image acquisition page of the software.</li>
	<li>In the experimental design, collect three images from each patient at a time as alternatives and select a relatively standard, fully exposed, well-illuminated, and clear image as the sample for subsequent algorithm training and testing.</li>
	<li>Collect data after the shooting, export the data for manual screening, and delete the non-standard images visible to the naked eye. Use the following filtering and exclusion criteria: incomplete tongue and face exposure, and images that are too dark as a result of insufficient light. An example of an under-lit, an incomplete, and a standard image is shown in Figure 3. 
	&#8203;NOTE: Insufficient light is generally caused by failure of the patient to place the face entirely into the instrument. Complete exposure is usually only obtained by correctly photographing the patient.</li>
</ol>
2. Tongue segmentation
<ol>
	<li>Perform tongue image segmentation using an online annotation tool, as described below.
	<ol>
		<li>Install Labelme, click on the Open button in the upper left corner of the label interface, select the folder where the image is located, and open the photos.</li>
		<li>Click on create polygon to start tracking points, track the tongue and lingual shapes, name them according to the selected areas (e.g., tongue and lingual surface), and save them.</li>
		<li>When all the marks are complete, click Save to save the image to the data folder. See Figure 4 for a detailed flow chart. 
		NOTE: As the images may have pixel differences, the images cannot be directly used for algorithm training and testing.</li>
	</ol>
	</li>
	<li>Unify the images to the same size by edge-filling the images, with the long side of the image as the target fill length and performing white edge-filling to fill the images to a square, with the long side of the image as the edge length. The image size captured by the device is 1080 x 1920 pixels, and the size of the filled image is 1920 x 1920 pixels. See Figure 5.</li>
	<li>Apply image enhancement if needed. No enhancement was applied in this study, as the images used were taken in a fixed scene and were less affected by the environment, lighting, and other factors.</li>
	<li>Because three images were collected for each patient during the shooting process to account for uncontrollable factors, such as subject blinking and lens blocking, manually screen the images from each patient to retain one image per patient.</li>
	<li>For the purpose of training the model, collect data from 200 people, or 600 images. After the screening, retain about 200 usable images.</li>
	<li>According to the image number, randomly divide all the tongue images, placing 70% of them into the training set and 30% into the test set in a spreadsheet.</li>
</ol>
3. Tongue classification
<ol>
	<li>Go to the official websites and download and install Anaconda, Python, and Labelme. Activate the environment and complete the installation and adjustment of the overall environment. See Figure 6 for a flow chart describing the installing and setting up of the software.</li>
	<li>Build the deep learning algorithm model in the installed environment, tune the parameters, and complete the model training using the training set. Perform model selection and tuning as described in the following steps.
	<ol>
		<li>Model selection: Choose the appropriate model based on the purpose of the research. After reviewing research on tongue image processing in the last 5 years, four algorithms, U-Net, Seg-Net, DeeplabV3, and PSPNet, were selected for validation in this study (see Supplementary Coding File 1, Supplementary Coding File 2, Supplementary Coding File 3, and Supplementary Coding File 4 for model codes).</li>
		<li>Data set construction: After completing the model selection, construct the required data set in conjunction with the research content, mainly using Labelme annotation and the uniform image size methods, as described above.</li>
	</ol>
	</li>
	<li>Perform model training as described below. Figure 7 shows details of the algorithm training operation.
	<ol>
		<li>Input the data into the neural network for forward propagation, with each neuron first inputting a weighted accumulation of values and then inputting an activation function as the output value of that neuron to obtain the result.</li>
		<li>Input the result into the error function and compare it with the expected value to get the error and judge the degree of recognition by mistake. The smaller the loss function is, the better the model will be.</li>
		<li>Reduce the error by back propagation and determine the gradient vector. Adjust the weights by the gradient vector to the trend toward results so that the error tends to zero or shrinks.</li>
		<li>Repeat this training process until the set is completed or the error value no longer declines, at which point the model training is complete. See Figure 8 for a flow chart of the algorithm model in training and testing.</li>
	</ol>
	</li>
	<li>Test the four models using the same test data for segmentation and judge the model performance according to the segmentation effect. The four metrics of precision, recall, mean pixel accuracy (MPA), and MIoU provide a more comprehensive model performance evaluation.</li>
	<li>After the results of the four models are generated, compare their values horizontally; the higher the value is, the higher the segmentation accuracy and the better the model&#39;s performance. See Figure 9, Figure 10, and Figure 11.</li>
</ol>

<ol>
	<li>Jiu, G., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Effects+of+herbal+therapy+on+intestinal+microbiota+and+serum+metabolomics+in+different+rat+models+of+Mongolian+medicine.">Effects of herbal therapy on intestinal microbiota and serum metabolomics in different rat models of Mongolian medicine.</a> Evidence-Based Complementary and Alternative. 2022, 7255780 (2022).</li><li>Xi, J., Xin, Y., Teregle, <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Study+on+the+correlation+between+the+animal+model+of+Mongolian+medicine+34;Xieri+disease&amp;#34;+and+serum+ALT+and+AST.">Study on the correlation between the animal model of Mongolian medicine 34;Xieri disease&amp;#34; and serum ALT and AST.</a> Electronic Journal of Cardiovascular Diseases in Combination of Traditional Chinese. 4 (33), 134-135 (2016).</li><li>Yin, L., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Study+on+the+mechanism+of+serum+differential+protein+changes+in+bronchial+asthma+based+on+proteomics.">Study on the mechanism of serum differential protein changes in bronchial asthma based on proteomics.</a> Chinese Journal of Traditional Chinese Medicine. 47 (22), 6227-6234 (2022).</li><li>Wang, X. H., Bao, L. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Band+Seed.+The+origin+and+development+of+tongue+diagnosis+in+Mongolian+medicine.">Band Seed. The origin and development of tongue diagnosis in Mongolian medicine.</a> Chinese Ethnic Folk Medicine. (1), 64-65 (2008).</li><li>Yang, S., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+comparative+study+on+the+feature+parameters+of+tongue+diagnosis+images+in+five+nationalities.">A comparative study on the feature parameters of tongue diagnosis images in five nationalities.</a> Chinese Journal of Traditional Chinese Medicine. 36 (11), 6428-6430 (2021).</li><li>He, J. Y., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Efficacy+and+safety+of+Chou-Ling-Dan+granules+in+the+treatment+of+seasonal+influenza+via+combining+Western+and+traditional+Chinese+medicine,+protocol+for+a+multicentre,+randomised+controlled+clinical+trial.">Efficacy and safety of Chou-Ling-Dan granules in the treatment of seasonal influenza via combining Western and traditional Chinese medicine, protocol for a multicentre, randomised controlled clinical trial.</a> BMJ Open. 9 (4), e024800 (2019).</li><li>Wang, D. J., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Scientific+knowledge+mapping+and+visualization+analysis+in+the+field+of+Chinese+medicine+tongue+feature+objectification+research.">Scientific knowledge mapping and visualization analysis in the field of Chinese medicine tongue feature objectification research.</a> World Science and Technology - Modernization of Chinese Medicine. 23 (9), 3032-3040 (2021).</li><li>Yuan, S. M., Qian, P., Li, F. F. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Research+progress+of+color+correction+methods+for+tongue+and+face+diagnosis+in+traditional+Chinese+Medicine.">Research progress of color correction methods for tongue and face diagnosis in traditional Chinese Medicine.</a> Chinese Journal of Traditional Chinese Medicine. 34 (9), 4183-4185 (2019).</li><li>Kanawong, R., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Tongue+image+analysis+and+its+mobile+app+development+for+health+diagnosis.">Tongue image analysis and its mobile app development for health diagnosis.</a> Advances in Experimental Medicine and Biology. 1005, 99-121 (2017).</li><li>Yu, Y., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Semantic+segmentation+evaluation+index+and+evaluation+method.">Semantic segmentation evaluation index and evaluation method.</a> Computer Engineering and Application. , (2023).</li><li>Sehyung, L., Negishi, M., Urakubo, H., Kasai, H., Ishii, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Mu-net:+Multi-scale+U-net+for+two-photon+microscopy+image+denoising+and+restoration.">Mu-net: Multi-scale U-net for two-photon microscopy image denoising and restoration.</a> Neural Networks. 125, 92-103 (2020).</li><li>Huang, X. M., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+review+on+the+application+of+U-Net+and+its+variants+in+medical+image+segmentation.">A review on the application of U-Net and its variants in medical image segmentation.</a> Chinese Journal of Biomedical Engineering. 41 (5), 567-576 (2022).</li><li>Lu, J. H., Xu, Y. F., Wang, Y. Q., Hao, Y. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Research+overview+of+tongue+objectification+in+traditional+Chinese+medicine+based+on+computer+image+technology.">Research overview of tongue objectification in traditional Chinese medicine based on computer image technology.</a> World Science and Technology - Modernization of Traditional Chinese Medicine. 24 (11), 4568-4573 (2022).</li><li>Mansour, R. F., Althobaiti, M. M., Ashour, A. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Internet+of+things+and+synergic+deep+learning+based+biomedical+tongue+color+image+analysis+for+disease+diagnosis+and+classification.">Internet of things and synergic deep learning based biomedical tongue color image analysis for disease diagnosis and classification.</a> IEEE Access. 9, 94769-94779 (2021).</li><li>Cai, Y. H., Hu, S. B., Guan, J., Zhang, X. F. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Analysis+of+the+development+and+application+of+tongue+diagnosis+objectification+techniques+in+Chinese+medicine.">Analysis of the development and application of tongue diagnosis objectification techniques in Chinese medicine.</a> World Science and Technology - Modernization of Chinese Medicine. 23 (7), 2447-2453 (2021).</li><li>Ghosh, S., Das, N., Nasipuri, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Reshaping+inputs+for+convolutional+neural+network:+some+common+and+uncommon+methods.">Reshaping inputs for convolutional neural network: some common and uncommon methods.</a> Pattern Recognition. 93, 79-94 (2019).</li><li>Shang, Z. M., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Research+progress+of+digital+acquisition+and+characterization+of+tongue+diagnosis+information.">Research progress of digital acquisition and characterization of tongue diagnosis information.</a> Chinese Journal of Traditional Chinese Medicine. 36 (10), 6010-6013 (2021).</li><li>Ning, J., Zhang, D., Wu, C., Yue, F. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Automatic+tongue+image+segmentation+based+on+gradient+vector+flow+and+region+merging.">Automatic tongue image segmentation based on gradient vector flow and region merging.</a> Neural Computing and Applications. 21, 1819-1826 (2012).</li><li>Zheng, F., Huang, X. Y., Wang, B. L., Wang, Y. H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+method+for+tongue+detection+based+on+image+segmentation.">A method for tongue detection based on image segmentation.</a> Journal of Xiamen University. 55 (6), 895-900 (2016).</li><li>Li, Y. T., Luo, Y. S., Zhu, Z. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Deep+learning-based+tongue+feature+analysis.">Deep learning-based tongue feature analysis.</a> Computer Science. 47 (11), 148-158 (2020).</li><li>Wang, Y. D., Sun, C. H., Cui, J. L., Wu, X. R., Qin, Y. X. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Research+on+deep+learning-based+tongue+fissure+segmentation+algorithm.">Research on deep learning-based tongue fissure segmentation algorithm.</a> World Science and Technology - Modernization of Chinese Medicine. 23 (9), 3065-3073 (2021).</li><li>Wang, X. M., Wang, R. Y., Guo, D., Lu, S. Z., Zhou, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Research+on+the+identification+method+of+tongue+punctures+based+on+auxiliary+light+source.">Research on the identification method of tongue punctures based on auxiliary light source.</a> Journal of Sensing Technology. 29 (10), 1553-1559 (2016).</li><li>Yang, J. D., Zhang, P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+fully+connected+neural+network+based+on+migration+learning+for+tongue+image+classification.">A fully connected neural network based on migration learning for tongue image classification.</a> Journal of the Second Military Medical University. 39 (8), 897-902 (2018).</li><li>Song, C., Wang, B., Xu, J. T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Research+on+tongue+feature+classification+method+based+on+deep+migration+learning.">Research on tongue feature classification method based on deep migration learning.</a> Computer Engineering and Science. 43 (8), 1488-1496 (2021).</li><li>Ding, H. J., He, J. C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Study+on+modern+techniques+and+methods+of+tongue+diagnosis.">Study on modern techniques and methods of tongue diagnosis.</a> Shi Zhen Chinese Medicine. 21 (5), 1230-1232 (2010).</li><li>Zhai, P. B., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+multi-stage+tongue+image+classification+algorithm+incorporating+attention+mechanism.">A multi-stage tongue image classification algorithm incorporating attention mechanism.</a> Computer Engineering and Design. 42 (6), 1606-1613 (2021).</li><li>Hou, Y. S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+new+clustering+analysis+algorithm+based+on+deep+learning.">A new clustering analysis algorithm based on deep learning.</a> Journal of Xinxiang University. 35 (12), 4 (2018).</li><li>Shi, Y. L., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+decision+tree+algorithm+for+classification+of+non-small+cell+lung+cancer+evidence+based+on+tongue+and+pulse+data.">A decision tree algorithm for classification of non-small cell lung cancer evidence based on tongue and pulse data.</a> World Science and Technology - Modernization of Chinese Medicine. 24 (7), 2766-2775 (2022).</li><li>Li, J., Hu, X. J., Zhou, C. L., Xu, J. T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Study+on+the+feature+analysis+and+diagnosis+model+of+diabetic+tongue+based+on+random+forest+algorithm.">Study on the feature analysis and diagnosis model of diabetic tongue based on random forest algorithm.</a> Chinese Journal of Traditional Chinese Medicine. 37 (3), 1639-1643 (2022).</li><li>Chen, C. H., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=The+characteristics+of+the+combination+of+the+four+diagnostic+methods+of+traditional+Chinese+medicine+from+the+perspective+of+the+differences+between+Chinese+and+Western+medical+diagnosis+methods.">The characteristics of the combination of the four diagnostic methods of traditional Chinese medicine from the perspective of the differences between Chinese and Western medical diagnosis methods.</a> Journal of Guangzhou University of Traditional Chinese Medicine. 28 (3), 332-334 (2011).</li><li>Chen, R., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Correlation+analysis+of+tongue+image+and+western+medicine+typing+in+382+patients+with+novel+coronavirus+pneumonia+based+on+Lab+colour+model+and+imaging+histology.">Correlation analysis of tongue image and western medicine typing in 382 patients with novel coronavirus pneumonia based on Lab colour model and imaging histology.</a> Chinese Journal of Traditional Chinese Medicine. 36 (12), 7010-7014 (2021).</li><li>Ju, J. W., Jung, H., Lee, Y. J., Mun, S. W., Lee, J. H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Semantic+segmentation+dataset+for+AI-based+quantification+of+clean+mucosa+in+capsule+endoscopy.">Semantic segmentation dataset for AI-based quantification of clean mucosa in capsule endoscopy.</a> Medicina. 58 (3), 397 (2022).</li><li>Wu, X., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+review+of+research+on+deep+learning+in+tongue+image+classification.">A review of research on deep learning in tongue image classification.</a> Computer Science and Exploration. , 1-23 (2022).</li></ol>

The authors have no conflict of interest to declare.

Based on the comparison results presented above, it is evident that the characteristics of the four algorithms under consideration are varied, and their distinct advantages and disadvantages are described below. The U-Net structure, based on the modification and expansion of a full convolution network, can obtain contextual information and precise positioning through a contracting path and a symmetrical expanding path. By classifying each pixel point, this algorithm achieves a higher segmentation accuracy and segments th...

Tongue observation is a widely utilized technique in traditional Chinese ethnic medicine (TCM). The color and shape of the tongue can reflect the physical condition and various disease properties, severities, and prognoses. For instance, in traditional Hmong medicine, the tongue&#39;s color is used to identify body temperature e.g., a red or purple tongue indicates pathological factors related to heat. In Tibetan medicine, a condition is judged by observing the tongue of a patient, paying attention to the color, shape, and moisture of the mucus. For instance, the tongues of patients with Heyi disease become red and rough or black and dry1; patients with Xieri disease2 have yellow and dry tongues; meanwhile, patients with Badakan disease3 have a white, humid, and soft tongue4. These observations reveal the close relationship between tongue features and physiology and pathology. Overall, the state of the tongue plays a vital role in diagnosis, disease identification, and evaluation of the treatment effect.
Simultaneously, owing to diverse living conditions and dietary practices among different ethnic groups, variations in tongue images are evident. The Lab model, established on the basis of an international standard for the determination of color, was formulated by the Commission International Eclairage (CIE) in 1931. In 1976, a color pattern was modified and named. The Lab color model is composed of three elements: L corresponds to brightness, while a and b are two color channels. a includes colors from dark green (low brightness value) to gray (medium brightness value) to bright pink (high brightness value); b goes from bright blue (low brightness value) to gray (medium brightness value) to yellow (high brightness value). By comparing the L x a x b values of the tongue color of five ethnic groups, Yang et al.5 found that the characteristics of tongue images of the Hmong, Hui, Zhuang, Han, and Mongolian groups were significantly distinct from each other. For example, the Mongolians have dark tongues with a yellow tongue coating, while the Hmong have light tongues with a white tongue coating, suggesting that tongue features can be used as a diagnostic indicator for assessing the health status of a population. Moreover, tongue images can function as an evaluation index for evidence-based medicine in clinical research of ethnic medicine. He et al.6 employed tongue images as a foundation for TCM diagnosis and systematically evaluated the safety and efficacy of Chou-Ling-Dan pellets (CLD granules-used to treat inflammatory and febrile diseases, including seasonal influenza in TCM) combined with Chinese and Western medicine. The results established the scientific validity of tongue images as an evaluation index for clinical studies. Nevertheless, traditional medical practitioners generally rely on subjectivity to observe tongue characteristics and assess patients&#39; physiological and pathological conditions, requiring more precise indicators.
The emergence of the internet and artificial intelligence technology has paved the way for digitizing and objectifying tongue diagnosis. This process involves using mathematical models to provide a qualitative and objective description of tongue images7, reflecting the content of the tongue image. The process includes several steps: image acquisition, optical compensation, color correction, and geometric transformation. The pre-processed images are then fed into an algorithmic model for image positioning and segmentation, feature extraction, pattern recognition, etc. The output of this process is a highly efficient and precise diagnosis of tongue image data, thereby achieving the goal of objectification, quantification, and informatization of tongue diagnosis8. Thus, the purpose of high efficiency and high precision processing of tongue diagnosis data is achieved. Based on tongue diagnosis knowledge and deep learning technology, this study automatically separated the tongue body and tongue coating from tongue images using a computer algorithm, in order to extract the quantitative features of tongues for doctors, improve the reliability and consistency of diagnosis, and provide methods for subsequent tongue diagnosis objectification research9.

<table><tbody><tr><td>CPU</td><td>Intel(R) Core(TM) i7-9700K</td><td></td><td></td></tr><tr><td>GPU</td><td>&amp;nbsp;NVIDIA GeForce RTX 3070 Ti (8192MB)</td><td></td><td></td></tr><tr><td>Operating systems</td><td>Microsoft Windows 10 Professional Edition (64-bit)</td><td></td><td></td></tr><tr><td>Programming language</td><td>Python</td><td></td><td></td></tr><tr><td>RAM</td><td>16G</td><td></td><td></td></tr></tbody></table>

objectification of tongue diagnosis in traditional medicine, data analysis, and study application

Tongue diagnosis is an essential technique of traditional Chinese medicine (TCM) diagnosis, and the need for objectifying tongue images through image processing technology is growing. The present study provides an overview of the progress made in tongue objectification over the past decade and compares segmentation models. Various deep learning models are constructed to verify and compare algorithms using real tongue image sets. The strengths and weaknesses of each model are analyzed. The findings indicate that the U-Net algorithm outperforms other models regarding precision accuracy (PA), recall, and mean intersection over union (MIoU) metrics. However, despite the significant progress in tongue image acquisition and processing, a uniform standard for objectifying tongue diagnosis has yet to be established. To facilitate the widespread application of tongue images captured using mobile devices in tongue diagnosis objectification, further research could address the challenges posed by tongue images captured in complex environments.

For the comparison results, see Figure 12, Figure 13, and Table 1, where the environment constructed by this study uses the same samples to train and test the algorithm model. MIoU indicator: U-Net &#62; Seg-Net &#62; PSPNet &#62; DeeplabV3; MPA indicator: U-Net &#62; Seg-Net &#62; PSPNet &#62; DeeplabV3; precision indicator: U-Net &#62; Seg-Net &#62; DeeplabV3 &#62; PSPNet; recall: U-Net &#62; Seg-Net &#62; PSPNet &#62; DeeplabV3. The larger th...

Watch this Scientific Journal Video about Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application at JoVE.com

Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application

The present study employed U-Net and other deep learning algorithms to segment a tongue image and compared the segmentation results to investigate the objectification of tongue diagnosis.

Tongue diagnosis is an essential technique of traditional Chinese medicine (TCM) diagnosis, and the need for objectifying tongue images through image ...

objectification-tongue-diagnosis-traditional-medicine-data-analysis

Research

JoVE Journal

Medicine

2.3K Views.  Chengdu University of Traditional Chinese Medicine. Dit protocol maakt gebruik van deep learning-algoritme om tongbeelden te segmenteren en scheidt automatisch de tong van het hele beeld, dat kan worden gebruikt voor verder onderzoek. Dit protocol vergelijkt de prestaties van het viertongs beeldsegmentatiealgoritme om het basisalgoritme te selecteren dat een tong kan segmenteren met een duidelijke teller. In dit onderzoek worden gepatenteerde instrumenten en algoritmes gebruikt.Het wordt aanbevolen om professionele instrumenten te gebru...

Video: Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application

Technique to Collect Fungiform (Taste) Papillae from Human Tongue

The sense of taste is critical for human life. It informs the body about the quality of food that will be potentially ingested and stimulates metabolic processes that prepare the alimentary canal for digestion. Steady progress is being made towards understanding the early biochemical and molecular events underlying taste transduction (for a review, Breslin and Spector, 20081). However, progress to date has largely resulted from animal models. Yet, since marked differences in receptor specificity and receptor density vary among species, human taste transduction will only be understood by using human taste tissue. Here we describe a biopsy technique to collect human fungiform papillae, visible as rounded pink anterior structures, about 0.5 mm in diameter that contain taste buds. These biopsied papillae are used for several purposes including the isolation of viable taste bud cells, in situ hybridization, immunohistochemistry and, through techniques of molecular biology, the identification of taste-specific novel proteins.

Technique to Collect Fungiform Taste Papillae from Human Tongue

Knowledge of molecular mechanisms underlying gustatory transduction has recently enjoyed significant advances, largely due to using animal models. However, the wide diversity in taste sensitivity and specificity among mammals warrants studies in human tissue. We describe a biopsy technique to collect living taste cells from the papillae on human tongue.

The sense of taste is critical for human life. It informs the body about the quality of food that will be potentially ingested and stimulates metabolic ...

Fabrication of Tongue Extracellular Matrix and Reconstitution of Tongue Squamous Cell Carcinoma In Vitro

In order to construct an effective and realistic model for tongue squamous cell carcinoma (TSCC) in vitro, the methods were created to produce decellularized tongue extracellular matrix (TEM) which provides functional scaffolds for TSCC construction. TEM provides an in vitro niche for cell growth, differentiation, and cell migration. The microstructures of native extracellular matrix (ECM) and biochemical compositions retained in the decellularized matrix provide tissue-specific niches for anchoring cells. The fabrication of TEM can be realized by deoxyribonuclease (DNase) digestion accompanied with a serious of organic or inorganic pretreatment. This protocol is easy to operate and ensures high efficiency for the decellularization. The TEM showed favorable cytocompatibility for TSCC cells under static or stirred culture conditions, which enables the construction of the TSCC model. A self-made bioreactor was also used for the persistent stirred condition for cell culture. Reconstructed TSCC using TEM showed the characteristics and properties resembling clinical TSCC histopathology, suggesting the potential in TSCC research.

Fabrication of Tongue Extracellular Matrix and Reconstitution of Tongue Squamous Cell Carcinoma In Vitro

A method is shown here for the preparation of the tongue extracellular matrix (TEM) with efficient decellularization. The TEM can be used as functional scaffolds for the reconstruction of a tongue squamous cell carcinoma (TSCC) model under static or stirred culture conditions.

In order to construct an effective and realistic model for tongue squamous cell carcinoma (TSCC) in vitro, the methods were created to produce ...

Cancer Research

Ultrasound Images of the Tongue: A Tutorial for Assessment and Remediation of Speech Sound Errors

Diagnostic ultrasound imaging has been a common tool in medical practice for several decades. It provides a safe and effective method for imaging structures internal to the body. There has been a recent increase in the use of ultrasound technology to visualize the shape and movements of the tongue during speech, both in typical speakers and in clinical populations. Ultrasound imaging of speech has greatly expanded our understanding of how sounds articulated with the tongue (lingual sounds) are produced. Such information can be particularly valuable for speech-language pathologists. Among other advantages, ultrasound images can be used during speech therapy to provide (1) illustrative models of typical (i.e.&#160;&#34;correct&#34;) tongue configurations for speech sounds, and (2) a source of insight into the articulatory nature of deviant productions. The images can also be used as an additional source of feedback for clinical populations learning to distinguish their better productions from their incorrect productions, en route to establishing more effective articulatory habits.
Ultrasound feedback is increasingly used by scientists and clinicians as both the expertise of the users increases and as the expense of the equipment declines. In this tutorial, procedures are presented for collecting ultrasound images of the tongue in a clinical context. We illustrate these procedures in an extended example featuring one common error sound, American English /r/. Images of correct and distorted /r/ are used to demonstrate (1) how to interpret ultrasound images, (2) how to assess tongue shape during production of speech sounds, (3), how to categorize tongue shape errors, and (4), how to provide visual feedback to elicit a more appropriate and functional tongue shape. We present a sample protocol for using real-time ultrasound images of the tongue for visual feedback to remediate speech sound errors. Additionally, example data are shown to illustrate outcomes with the procedure.

Ultrasound imaging can be used to display the shape and movements of the tongue in real time during speech. The images can be used to determine the nature of speech sound errors. Visual feedback of the tongue can be used to facilitate improvements in speech sound production in clinical populations.

Diagnostic ultrasound imaging has been a common tool in medical practice for several decades. It provides a safe and effective method for imaging ...

Behavior

Assessment of Midline Lingual Point-Pressure Somatosensation Using Von Frey Hair Monofilaments

Detection and discrimination threshold estimates for oral point pressure are assessed using Von Frey Hair monofilaments. Consistent with previously published protocols, threshold estimates are determined using a two-interval forced choice (2-IFC) paradigm with a three down/one up approach. Detection threshold estimates determine the mean force in which a participant can identify the presence of pressure. During the detection threshold procedure, the participant is instructed to choose which of two sequentially presented observation intervals contained the tactile test stimulus. If the participant performs three correct detections in a row (i.e., 3 &#39;hits&#39;), the researcher decreases the stimulus to the next lower target force level. With one incorrect detection (a &#39;miss&#39;), the researcher increases the force delivered to the next higher level. This threshold estimation approach is known as a 3-down/1-up adaptive staircase. Reponses are recorded on a paper ballot, and a participant&#39;s estimated threshold is defined as the geometric mean of five reversals. During the discrimination threshold procedure, the participant is asked to make a choice between two serially presented stimuli as to which is the &#34;harder&#34; or &#34;stronger&#34; pressure. The same scoring of &#39;hits&#39;, &#39;misses&#39;, and stopping points are used. Detection and discrimination testing for oral point pressure at tongue midline takes approximately 20 min to complete. Using these commercially-available clinical tools, individual touch sensation profiles for the midline tongue can be achieved in a relatively time and cost effective means.

This work describes a standard method to assess tactile sensation at the midline of the tongue tip. Using Von Frey Hair (VFH) monofilaments, this protocol provides estimates of detection and discrimination threshold estimates for oral point pressure (OPP).

Detection and discrimination threshold estimates for oral point pressure are assessed using Von Frey Hair monofilaments. Consistent with previously ...

Measuring Oral Fatty Acid Thresholds, Fat Perception, Fatty Food Liking, and Papillae Density in Humans

Emerging evidence from a number of laboratories indicates that humans have the ability to identify fatty acids in the oral cavity, presumably via fatty acid receptors housed on taste cells. Previous research has shown that an individual&#39;s oral sensitivity to fatty acid, specifically oleic acid (C18:1) is associated with body mass index (BMI), dietary fat consumption, and the ability to identify fat in foods. We have developed a reliable and reproducible method to assess oral chemoreception of fatty acids, using a milk and C18:1 emulsion, together with an ascending forced choice triangle procedure. In parallel, a food matrix has been developed to assess an individual&#39;s ability to perceive fat, in addition to a simple method to assess fatty food liking. As an added measure tongue photography is used to assess papillae density, with higher density often being associated with increased taste sensitivity.

Oral chemoreception of fatty acids and the association with diet and fatty food preferences may enable the identification of mechanisms involved with the development of obesity and why dietary changes may be difficult for many individuals.

Emerging evidence from a number of laboratories indicates that humans have the ability to identify fatty acids in the oral cavity, presumably via fatty ...

Neuroscience

Denver Papillae Protocol for Objective Analysis of Fungiform Papillae

The goal of the Denver Papillae Protocol is to use a dichotomous key to define and prioritize the characteristics of fungiform papillae (FP) to ensure consistent scoring between scorers. This protocol builds off of a need that has arisen from the last two decades of taste research using FP as a proxy for taste pore density. FP density has historically been analyzed using Miller &#38; Reedy&#8217;s 1990 characterizations of their morphology: round, stained lighter, large, and elevated. In this work, the authors forewarned that stricter definitions of FP morphology needed to be outlined. Despite this call to action, follow up literature has been scarce, with most studies continuing to cite Miller &#38; Reedy&#8217;s original work. Consequently, FP density reports have been highly variable and, combined with small sample sizes, may contribute to the discrepant conclusions on the role of FP in taste sensitivity. The Genetics of Taste Lab explored this apparent inconsistency in counting and found that scorers were individually prioritizing the importance of these characteristics differently and had no guidance for when a papilla had some, but not all, of the reported qualities of FP. The result of this subjectivity is highly variable FP counts of the same tongue image. The Denver Papillae Protocol has been developed to remedy this consequence through use of a dichotomous key that further defines and prioritizes the importance of the characteristics put forth by Miller &#38; Reedy. The proposed method could help create a standard way to quantify FP for researchers in the field of taste and nutritional studies.

Here we present a protocol to measure fungiform papilla density from digital photographs. This method builds prioritization and objective characteristic metrics into the original descriptive work on fungiform papillae by Miller &#38; Reedy (1990).

The goal of the Denver Papillae Protocol is to use a dichotomous key to define and prioritize the characteristics of fungiform papillae (FP) to ensure ...

In Vivo Evaluation of the Mechanical and Viscoelastic Properties of the Rat Tongue

The tongue is a highly innervated and vascularized muscle hydrostat on the floor of the mouth of most vertebrates. Its primary functions include supporting mastication and deglutition, as well as taste-sensing and phonetics. Accordingly, the strength and volume of the tongue can impact the ability of vertebrates to accomplish basic activities such as feeding, communicating, and breathing. Human patients with sleep apnea have enlarged tongues, characterized by reduced muscle tone and increased intramuscular fat that can be visualized and quantified by magnetic resonance imaging (MRI). The abilities to measure force generation and viscoelastic properties of the tongue constitute important tools for obtaining functional information to correlate with imaging data. Here, we present techniques for measuring tongue force production in anesthetized Zucker rats via electrical stimulation of the hypoglossal nerves and for determining the viscoelastic properties of the tongue by applying passive Lissajous force/deformation curves.

In Vivo Evaluation of the Mechanical and Viscoelastic Properties of the Rat Tongue

We describe a surgical procedure in an anesthetized rat model for determining the muscle tone and viscoelastic properties of the tongue. The procedure involves specific stimulation of the hypoglossal nerves and application of passive Lissajous force/deformation curves to the muscle.

The tongue is a highly innervated and vascularized muscle hydrostat on the floor of the mouth of most vertebrates. Its primary functions include ...

Taste Exam: A Brief and Validated Test

The emerging importance of taste in medicine and biomedical research, and new knowledge about its genetic underpinnings, has motivated us to supplement classic taste-testing methods in two ways. First, we explain how to do a brief assessment of the mouth, including the tongue, to ensure that taste papillae are present and to note evidence of relevant disease. Second, we draw on genetics to validate taste test data by comparing reports of perceived bitterness intensity and inborn receptor genotypes. Discordance between objective measures of genotype and subjective reports of taste experience can identify data collection errors, distracted subjects or those who have not understood or followed instructions. Our expectation is that fast and valid taste tests may persuade researchers and clinicians to assess taste regularly, making taste testing as common as testing for hearing and vision. Finally, because many tissues of the body express taste receptors, taste responses may provide a proxy for tissue sensitivity elsewhere in the body and, thereby, serve as a rapid, point-of-care test to guide diagnosis and a research tool to evaluate taste receptor protein function.

This protocol measures human taste responses and includes a brief anatomical assessment, a short taste test, and a validation method using the subject&#39;s reported sensation and taste receptor genotype.

The emerging importance of taste in medicine and biomedical research, and new knowledge about its genetic underpinnings, has motivated us to supplement ...

Effective aPDT Using Curcuminoids for Oral Candidiasis in Mice

Curcuminoid-Mediated Antimicrobial Photodynamic Therapy on a Murine Model of Oral Candidiasis

Antimicrobial Photodynamic Therapy (aPDT) has been extensively investigated in vitro, and preclinical animal models of infections are suitable for evaluating alternative treatments prior to clinical trials. This study describes the efficacy of aPDT in a murine model of oral candidiasis. Forty mice were immunosuppressed with subcutaneous injections of prednisolone, and their tongues were inoculated using an oral swab previously soaked in a C. albicans cell suspension. Tetracycline was administered via drinking water during the course of the experiment. Five days after fungal inoculation, mice were randomly distributed into eight groups; a ninth group of untreated uninfected mice was included as a negative control (n = 5). Three concentrations (20 &#181;M, 40 &#181;M, and 80 &#181;M) of a mixture of curcuminoids were tested with a blue LED light (89.2 mW/cm2; ~455 nm) and without light (C+L+ and C+L- groups, respectively). Light alone (C-L+), no treatment (C-L-), and animals without infection were evaluated as controls. Data were analyzed using Welch&#39;s ANOVA and Games-Howell tests (&#945; = 0.05). Oral candidiasis was established in all infected animals and visualized macroscopically through the presence of characteristic white patches or pseudomembranes on the dorsum of the tongues. Histopathological sections confirmed a large presence of yeast and filaments limited to the keratinized layer of the epithelium in the C-L- group, and the presence of fungal cells was visually decreased in the images obtained from mice subjected to aPDT with either 40 &#181;M or 80 &#181;M curcuminoids. aPDT mediated by 80 &#181;M curcuminoids promoted a 2.47 log10 reduction in colony counts in comparison to those in the C-L- group (p = 0.008). All other groups showed no statistically significant reduction in the number of colonies, including photosensitizer (C+L-) or light alone (C-L+) groups. Curcuminoid-mediated aPDT reduced the fungal load from the tongues of mice.

Author Spotlight: Exploring Photodynamic Therapy with Curcumin in a Murine Model for Oral Candidiasis

This protocol describes the application of antimicrobial Photodynamic Therapy (aPDT) in a murine model of oral candidiasis. aPDT was performed using a water-soluble mixture of curcuminoids and blue LED light.

Antimicrobial Photodynamic Therapy (aPDT) has been extensively investigated in vitro, and preclinical animal models of infections are suitable for ...

Immunology and Infection

&#181;Tongue: A Microfluidics-Based Functional Imaging Platform for the Tongue In Vivo

Intravital fluorescence microscopy is a tool used widely to study multicellular dynamics in a live animal. However, it has not been successfully used in the taste sensory organ. By integrating microfluidics into the intravital tongue imaging window, the &#181;Tongue provides reliable functional images of taste cells in vivo under controlled exposure to multiple tastants. In this paper, a detailed step-by-step procedure to utilize the &#181;Tongue system is presented. There are five subsections: preparing of tastant solutions, setting up of a microfluidic module, sample mounting, acquiring functional image data, and data analysis. Some tips and techniques to solve the practical issues that may arise when using the &#181;Tongue are also presented.

 &#181;Tongue: A Microfluidics-Based Functional Imaging Platform for the Tongue In Vivo

The article introduces the &#181;Tongue (microfluidics-on-a-tongue) device for functional taste cell imaging in vivo by integrating microfluidics into an intravital imaging window on the tongue.

Intravital fluorescence microscopy is a tool used widely to study multicellular dynamics in a live animal. However, it has not been successfully used in ...

Objectification of Tongue Diagnosis in Traditional Medicine, Data Analysis, and Study Application

Samenvatting

Meer video's verkennen

Hoofdstukken in deze video