We thank Qiushi Sun, Yao Wang, Yu Xia, Jinyu Yang, Chang Lin, Zimu Cen, Dongdong Liang, Rong Wei, Ze Xu, Guangyin Xi, Gang Xue, Can Ye, Li-Peng Wang, Peng Zou, Shi-Qiang Wang, Pablo Rivera-Fuentes, Salome P&#252;ntener, Zhixing Chen, Yi Liu, and Jue Zhang, for laying the groundwork of this strategy. This work was supported by the National Key R&#38;D Program of China (2018YFA0800504, 2019YFA0110000) and the Space Medical Experiment Project of China Manned Space Program (HYZHXM01020) to Yang Zhao. Figure 1 was created with BioRender.com.

Monitoring Pluripotent Stem Cell Differentiation by Image-Based ML Strategy

Yang Zhao, Jue Zhang, Xiaochun Yang, Yao Wang, and Daichao Chen are filing a patent for the PSC differentiation strategy reported in this paper (202210525166.X).

Here, we described a detailed protocol to overcome one of the major problems in current PSC application and translation&#8212;the variability in cell differentiation. By harnessing live-cell brightfield imaging and ML, we iteratively optimized PSC differentiation to achieve consistently high efficiency across cell lines and batches. However, in the above differentiation process, several critical steps in the protocol have a decisive influence on whether the differentiation would succeed or not. Since the cell state in th...

Pluripotent stem cells (PSCs) possess the remarkable ability to differentiate into many types of cells in vitro. These differentiated functional cells could be used for cell therapy, disease modeling, and drug development, all valuable for research or clinical applications1,2,3. For example, a variety of methods have been developed to differentiate PSCs into cardiomyocytes (CMs)4,5,6,7. These CMs can be applied for cardiotoxicity testing of drugs, modeling of heart disease, and cell transplantation8,9,10,11.
However, the conversion from PSC to the terminal differentiated cells is a stepwise process, and multiple perturbations during the differentiation process may lead cells to divergent cell fates. Different genetic backgrounds and epigenetic marks of PSC lines influence the potential for differentiation to a specific lineage12,13,14,15; the number of PSC passages and accumulated gene mutations are also sources of PSC heterogeneity; differences in the experimental operations employed by different experimenters can also lead to completely different differentiation results16,17,18,19,20. Therefore, currently one of the main problems in PSC-derived cell production is the instability among cell lines and batches21,22,23,24,25. Instability in PSC differentiation often leads to multiple repeated experiments, consuming significant time and labor resources. To address this issue, it is crucial to develop a strategy that minimizes the variability among cell lines and batches, thus enhancing the stability and robustness of the differentiation.
Recently, advances in high-resolution microscopy and machine learning (ML) have facilitated the application of ML-based quantitative image analysis in cell biology, making it possible to utilize valuable information in cell imaging features26,27,28,29,30,31,32,33,34. In our previous work, we proposed a live-cell image-based ML strategy to monitor and intervene in the PSC differentiation status in real time to improve the stability and efficiency of the PSC differentiation (Figure 1)35. Taking PSC-to-cardiomyocyte differentiation as an example, we evaluated the initial PSC state using random forest models, predicted the optimal differentiation condition using logistic regression models, and recognized successfully differentiated cells using deep learning-based Grad-CAM36 and pix2pix37. ML models learned to identify cell lineages from a range of bright-field morphological features, including features about area, circumference, convexity, solidity, brightness, moving velocity, and other implicit features extracted by deep convolutional neural networks. Based on inference from these established ML models, we realized control of the initial PSC state, early assessment and intervention in differentiation conditions, and elimination of the misdifferentiated cell contamination, together providing a comprehensive and accurate modulation of the cardiac differentiation process. Here we provide a step-by-step protocol for developing the strategy.

<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td>0.25% Trypsin-EDTA</td>
			<td>Gibco</td>
			<td>25200056</td>
			<td>Diluted digests were used for CPC and CM digestion</td>
		</tr>
		<tr>
			<td>4% Paraformaldehyde in PBS</td>
			<td>KeyGEN BioTECH</td>
			<td>KGIHC016</td>
			<td></td>
		</tr>
		<tr>
			<td>6-well Cell Culture Plate</td>
			<td>NEST</td>
			<td>703001</td>
			<td></td>
		</tr>
		<tr>
			<td>96-well Cell Culture Plate</td>
			<td>NEST</td>
			<td>701001</td>
			<td></td>
		</tr>
		<tr>
			<td>B27 Supplement</td>
			<td>Gibco</td>
			<td>17504044</td>
			<td></td>
		</tr>
		<tr>
			<td>B27 Supplement Minus Insulin</td>
			<td>Gibco</td>
			<td>A1895601</td>
			<td></td>
		</tr>
		<tr>
			<td>Bovine serum albumin (BSA)</td>
			<td>GPC BIOTECH</td>
			<td>AA904-100G</td>
			<td></td>
		</tr>
		<tr>
			<td>Celldiscoverer 7</td>
			<td>Zeiss</td>
			<td></td>
			<td>Instruments used to take bright-field images throughout differentiation and final cTnT images</td>
		</tr>
		<tr>
			<td>CHIR99021</td>
			<td>Selleck</td>
			<td>S1263</td>
			<td></td>
		</tr>
		<tr>
			<td>DMEM/F12</td>
			<td>Gibco</td>
			<td>12634010</td>
			<td></td>
		</tr>
		<tr>
			<td>Donkey anti-Mouse IgG (H+L) Highly Cross-Adsorbed Secondary Antibody, Alexa Fluor 488</td>
			<td>Thermo</td>
			<td>A-21202</td>
			<td>Secondary Antibody</td>
		</tr>
		<tr>
			<td>FACSAria III</td>
			<td>BD Biosciences</td>
			<td></td>
			<td>Flow cytometry sorter</td>
		</tr>
		<tr>
			<td>Fetal Bovine Serum (FBS)</td>
			<td>VISTECH</td>
			<td>SE100-B</td>
			<td></td>
		</tr>
		<tr>
			<td>Hoechst 33342</td>
			<td>YEASEN</td>
			<td>40732ES03</td>
			<td></td>
		</tr>
		<tr>
			<td>Human Pluripotent Stem Cell Chemical-defined Medium</td>
			<td>Cauliscell Inc</td>
			<td>400105</td>
			<td>Basal medium of PSC preparation medium</td>
		</tr>
		<tr>
			<td>iPS-18</td>
			<td>TaKaRa</td>
			<td>Y00300</td>
			<td></td>
		</tr>
		<tr>
			<td>iPS-B1</td>
			<td>Cellapy</td>
			<td>CA4025106</td>
			<td></td>
		</tr>
		<tr>
			<td>iPS-F</td>
			<td>Nuwacell</td>
			<td>RC01001-B</td>
			<td></td>
		</tr>
		<tr>
			<td>iPS-M</td>
			<td>Nuwacell</td>
			<td>RC01001-A</td>
			<td></td>
		</tr>
		<tr>
			<td>IWR1-1-endo</td>
			<td>Selleck</td>
			<td>S7086</td>
			<td>IWR1</td>
		</tr>
		<tr>
			<td>Jupyter Notebook</td>
			<td>N/A</td>
			<td>Version 6.4.0</td>
			<td><a href="https://jupyter.org/">https://jupyter.org/</a></td>
		</tr>
		<tr>
			<td>MATLAB</td>
			<td>MathWorks</td>
			<td>Version R2020a</td>
			<td>Software for scientific computation and image annotation</td>
		</tr>
		<tr>
			<td>Matrigel Matrix</td>
			<td>Corning</td>
			<td>354230</td>
			<td>Matrigel</td>
		</tr>
		<tr>
			<td>Mouse monoclonal IgG1 anti-cTnT</td>
			<td>Thermo</td>
			<td>MA5-12960</td>
			<td>cTnT primary antibody</td>
		</tr>
		<tr>
			<td>Normal Donkey Serum</td>
			<td>Jackson</td>
			<td>017-000-121</td>
			<td></td>
		</tr>
		<tr>
			<td>ORCA-Flash 4.0 V3 digital CMOS camera</td>
			<td>Hamamatsu</td>
			<td>C13440-20CU</td>
			<td>The digital camera assembled on Celldiscoverer7</td>
		</tr>
		<tr>
			<td>PBS</td>
			<td>NEB</td>
			<td>21-040-CVR</td>
			<td></td>
		</tr>
		<tr>
			<td>Penicillin-Streptomycin</td>
			<td>Gibco</td>
			<td>15140-122</td>
			<td></td>
		</tr>
		<tr>
			<td>Pluripotency Growth Mater 1 basal medium</td>
			<td>Cellapy</td>
			<td>CA1007500-1</td>
			<td>Basal medium of PSC culture medium</td>
		</tr>
		<tr>
			<td>Pluripotency Growth Mater 1 supplement</td>
			<td>Cellapy</td>
			<td>CA1007500-2</td>
			<td>Supplement of PSC culture medium</td>
		</tr>
		<tr>
			<td>Prism</td>
			<td>Graphpad</td>
			<td>Version 8/9</td>
			<td>Statistical software for statistical analysis and plotting</td>
		</tr>
		<tr>
			<td>Python</td>
			<td>N/A</td>
			<td>version 3.6</td>
			<td>Python 3 environment for scientific computation, with packages pytorch (1.9.0), numpy, scipy, pandas, visdom, scikit-learn, scikit-image, opencv-python, and matplotlib software for scientific computation and image annotation.</td>
		</tr>
		<tr>
			<td>RPMI 1640</td>
			<td>Gibco</td>
			<td>11875176</td>
			<td></td>
		</tr>
		<tr>
			<td>Supplement hPSC-CDM (500x)</td>
			<td>Cauliscell Inc</td>
			<td>00015</td>
			<td>Supplement of PSC preparation medium</td>
		</tr>
		<tr>
			<td>TiE</td>
			<td>Nikon</td>
			<td></td>
			<td>An inverted fluorescence microscope (with modification) for region-selevtive purification</td>
		</tr>
		<tr>
			<td>Triton&#160;X-100</td>
			<td>Amresco</td>
			<td>9002-93-1</td>
			<td></td>
		</tr>
		<tr>
			<td>Versene Solution</td>
			<td>Thermo</td>
			<td>15040066</td>
			<td>EDTA solution for PSC digestion</td>
		</tr>
		<tr>
			<td>Y27632</td>
			<td>Selleck</td>
			<td>S6390</td>
			<td></td>
		</tr>
		<tr>
			<td>Zen</td>
			<td>Zeiss</td>
			<td>Version 3.1</td>
			<td>A supporting software of Celldiscoverer7 for&#160; image acquisition, processing and analysis</td>
		</tr>
	</tbody>
</table>

a live-cell image-based machine learning strategy to monitor pluripotent stem cell differentiation

Pluripotent stem cell (PSC) technologies have been widely used in drug discovery, disease modeling, and regenerative medicine.&#160;However, available PSC-to-functional cell differentiation systems are impeded by problems of severe line-to-line and batch-to-batch variability. Precise control of cell differentiation in real time is therefore important. In this protocol, we describe a non-invasive and intelligent strategy that overcomes the variability in cell differentiation by using bright-field image-based machine learning. Taking PSC-to-cardiomyocyte differentiation as an example, this methodology provides detailed information for control of the initial PSC state, early assessment and intervention in differentiation conditions, and elimination of the misdifferentiated cell contamination, together realizing consistently high-quality differentiation from PSCs to functional cells. In principle, this strategy can be extended to other cell differentiation or reprogramming systems with multiple steps to support cell manufacturing, as well as to further our understanding of the mechanisms during cell fate conversion.

Author Spotlight: Enhancing PSC-to-Functional Cell Differentiation Using ML Models Based on Live-Cell Bright-Field Imaging

A Live-cell Image-Based Machine Learning Strategy to Monitor Pluripotent Stem Cell Differentiation

In this study, based on live-cell bright-field images, we developed a strategy, harnessing different machine learning models. This strategy can identify cell lineage non-invasively, modulate the differentiation process in real-time, and optimize the differentiation protocol, improving the invulnerability in PSC-to-functional cell differentiation. Pluripotent stem cells present the ability to differentiate into many types of cells in vitro, which could be used for cell therapy, disease modeling, and drug development.One of the main problems in PSC-derived cell production, is the instability among cell lines and batches. It often leads to multiple repeated experiments, consuming significant time and labor. Currently, state-of-the-art microscopic technologies could support long-term time-lapse, high-throughput image acquisition on live cells.Meanwhile, the fast evolving machine learning method is being increasingly applied in cellular image analysis, which is opening possibilities to recognize specific cellular constituents or cell images during differentiation in cell culture. This method could hopefully be applied to standardized other cell fate induction systems like organoid differentiation, direct transdifferentiation, or cellular programming, especially for unstable systems requiring multiple steps and complex inducers. Also, our strategy is compatible with other technologies, which promisingly can be integrated into a self-adaptive and closed-loop system for full-automatic processing of PSC differentiation in vitro.

&#160;Based on brightfield imaging and ML, the overall differentiation process can be intelligently monitored and optimized. At the PSC stage, we developed an ML model that could predict the final differentiation efficiency according to the morphological features of initial PSC colonies, to determine the most suitable or appropriate time point to initiate differentiation (Figure 4A,B). The differentiation efficiency predicted by the random forest model is highly correlated w...

Available pluripotent stem cell (PSC)-to-functional cell differentiation systems are currently impeded by problems of severe line-to-line and batch-to-batch variability. Here, using cardiac differentiation as the main example, we present a protocol to intelligently monitor and modulate the process of PSC differentiation based on image-based machine learning.

Pluripotent stem cell (PSC) technologies have been widely used in drug discovery, disease modeling, and regenerative medicine.&#160;However, available ...

a-live-cell-image-based-machine-learning-strategy-to-monitor

Research

JoVE Journal

Biology

Cardiac Differentiation and Immunofluorescence Staining of Pluripotent Stem Cells (PSCs)

Maintain pluripotent stem cells or PSCs in six-well culture plates and confirm the cell density under a microscope. For cardiomyocyte or CM differentiation, seed PSCs into a 96-well culture plate in a PSC preparation medium. At stage one of cardiac differentiation, when PSCs reach 80 to 90%confluency, switch the medium to CM differentiation medium with two to 20 micromolar CHIR.After 24 to 48 hours of CHIR treatment, change the medium to a fresh CM differentiation medium. At stage two or day three, replace the medium with the CM differentiation medium supplemented with five micromolar IWR1 in culture until day five. Then change the medium to the CM differentiation medium and keep it until days six to seven when the PSCs differentiate into cardiac progenitor cells, or CPC.At day 10 or day 12, fix the cells with 4%paraformaldehyde in PBS for 15 minutes at room temperature. At the time of staining, treat the cells with a permeabilizing solution for 30 minutes at room temperature. Incubate the sample with CTNT primary antibody overnight at four degrees Celsius.Then treat the sample with secondary antibodies in PBS containing 1%bovine serum albumin for one hour at 37 degrees Celsius. After removing the secondary antibody from the cells, stain the nuclei by Hoechst for five minutes at room temperature. Then rinse the cells three times using PBS before adding 100 microliters of PBS per well to avoid drying.To collect brightfield and CTNT immunofluorescent images of different stages of differentiation using an automated microscope supporting live cell culture, open the software and create a new experimental design. Choose a 5X objective and a 2X2 blends for imaging. In the channels menu, add TL brightfield channel for brightfield and AF 488 and H 33 42 channels for immunofluorescence imaging.Modify the light path in the imaging setup. In the Z stack menu, set the number of slices and intervals when scanning. In the navigation and tiles window, set up the tile regions by carrier and set 25 tiles five columns into five rows for one well.For acquiring images, first put the cell culture plate into the sample tray and load the tray inside the microscope. If the sample consists of live cells, open the heating system and carbon dioxide pump to keep the appropriate conditions for culture at 37 degrees Celsius and 5%carbon dioxide. Open the preset experimental project.Open the tiles menu and calibrate the position of the plate manually. In the navigation and tiles window, select the wells needed and click create to construct tile regions for these wells. Click the verify tile regions in the tiles menu and run auto focus to verify all of the wells.Then manually correct the focus of each well. Click the start experiment button and wait for automatic imaging. Typically, it takes about 1.2 hours to scan a whole 96-well culture plate.In the processing framework, choose image export, select the file type of the uncompressed TIFF format, or PNG format and apply.

Establishment of the Image-Based ML Strategy to Monitor and Modulate Pluripotent Stem Cell (PSC) Differentiation 

To establish the machine learning or ML strategy for the initial PSC colony states, prepare a data set of bright-field images at zero hours or before CHIR treatment and the final cTNT fluorescence images. Quantify the differentiation efficiency of each well by the differentiation Efficiency index computed from its cTNT fluorescence images. Quantify the morphological profiles of zero hour bright-field images by high dimensional features that reveal the properties of the colony's shape.Randomly divide the dataset into a training set and a test set. Train a random forest regression model on the training set to predict differentiation efficiency from zero hour bright-field image features. Evaluate the trained random forest model on the test set.Prepare a dataset consisting of whole well bright-field image streams at zero to 12 hours and the corresponding CHIR doses, which are the combinations of CHIR concentrations and duration. According to the displayed criteria, determine the low, optimal, and high CHIR concentration range under each CHIR duration in each batch. Under each duration, compute delta CHIR concentration for each concentration to quantify its deviation from the optimum.Extract features of the image streams in the dataset. For each CHIR duration, train a logistic regression model to predict the CHIR concentration label from the image stream features on the training set. Evaluate the classification performance of the trained logistic regression model on the test set using accuracy, precision, recall, F1 score, and area under the curve.To evaluate the model performance in CHIR dose assessment, in the test set, merge predicted labels of parallel wells with the same CHIR concentration using deviation scores ranging from minus one to one. Using the Pearson correlation coefficient, confirm that the predicted deviation scores highly correlate with the true delta CHIR concentration for each dose. Next, apply the trained logistic regression models to assess CHIR doses in new batches.With the model-based CHIR dose assessment, rescue wells under each suboptimal CHIR concentration accordingly by adjusting their CHIR duration or concentration towards optimum before 48 hours. Prepare a dataset consisting of bright-field images on day six, and manually annotate CM-committed CPCs by tracking the cTNT-positive cells in the image streams from day 12 back to day six. Crop the bright-field images and the corresponding manual annotation of CM-committed CPCs into patches.Label patches greater than or equal to 30%of CM-committed CPCs as positive, and patches without CM-committed CPCs as negative. Train a deep convolutional neural network, ResNeSt, to learn to classify these patches. Use Grad-CAM to highlight the regions that contribute most to the inference of ResNeSt represented by heat maps.Merge the patch level-binarized heat maps to get the predicted CM-committed CPC regions, which are named imaged-recognized CPC, or IRCPC regions. Compare the IRCPC regions with manually annotated masks on the test set using accuracy, F1 score, precision, recall, specificity, and intersection over union. Prepare a data set consisting of bright-field images of CMs and the corresponding cTNT fluorescence images.Train the pix2pix model on the training set before applying the trained model on the test set. To purify the CM-committed CPCs, use a non-cytotoxic photoactivitable probe, dual-activatable cell tracker 1, or DACT-1, to region selectively label non-CPC. After irradiating the non-CPC's region with a 405 nanometer laser line, detect the DACT-1-labeled cells using a 561 nanometer laser line.After sorting the dissociated cells, seed the sorted cells in a matrigel-coated 96 well plate, and culture DACT-1-labeled non-CPCs, unlabeled IRCPCs, and the control group cells for six days in the medium.

244 Views.  Peking University.  Available pluripotent stem cell (PSC)-to-functional cell differentiation systems are currently impeded by problems of severe line-to-line and batch-to-batch variability. Here, using cardiac differentiation as the main example, we present a protocol to intelligently monitor and modulate the process of PSC differentiation based on image-based machine learning. 

Monitoring Pluripotent Stem Cell Differentiation by Image-Based ML Strategy

Summary