Name: volume segmentation and analysis of biological materials using survos (super-region volume segmentation) workbench
Uploaded: 2017-08-23T15:00:00
Duration: 11 min 38 s
Description: Watch this Scientific Journal Video about Volume Segmentation and Analysis of Biological Materials Using SuRVoS Super-region Volume Segmentation Workbench at JoVE.com

We would like to acknowledge and thank Rui Wang and Wah Chiu from Baylor College of Medicine for providing the cryoET dataset and Andrew Bodey from Diamond Light Source for assisting with the I13 beamtime. Parts of this research were supported by the National Institutes of Health (NIH) grant No. (P41GM103832) We acknowledge Diamond Light Source for jointly funding Imanol Luengo under PhD STU0079.

NOTE: Generally useful ranges of parameters for each processing step and the specific parameters for each data type shown here are provided in Table 1.
1. Preparing a Workspace and Loading Data
<ol>
	<li>Launch SuRVoS Workbench, click the open dataset button, and in the resulting popup, select the data file to be segmented. Choose an appropriate orientation of the dataset. Next, choose or create a folder where the workspace and associated files will be stored. It is recommended that this folder is empty when starting a new segmentation. After the data has been loaded, the workbench will open with the Plugins pane on the left, the Visualization pane on the right and a set of Tool shortcuts between the two panes (Figure 1).</li>
</ol>
2. Preprocessing and Data Representation
<ol>
	<li>In the Select ROI tab, input the z, y, and x start and end coordinates for the region of interest and click add. To choose appropriate y, and x coordinates hover the mouse over a point on the image. Choose z coordinates using the slider at the top of the Visualization pane. Once a region has been added, make sure it is selected by checking the box to the right. All downstream calculations will be performed on the selected region. Generally, starting with a small, representative region of interest (ROI), optimizing parameters and then re-applying these parameters to the whole area to be segmented is advised.</li>
	<li>In the Feature Channels tab, use the drop-down menu at the top to choose a feature/filter and add it to the queue (see Discussion for more info about feature channels). Once a feature/filter has been added and selected by clicking on its name, modify options specific to the feature/filter and choose the input dataset on which to run the feature/filter. Once all options have been chosen, click the checkbox to the right of the feature/filter name to compute.
	<ol>
		<li>In order to optimize parameters for a new dataset, add multiple filters/features and choose the parameters for them, before being computed in order, one after the other. To do this, add each new filter/feature and select appropriate parameters, check the box to the left of each filter/feature to be run, and click the compute features box at the top of the pane. See Discussion for additional information.</li>
	</ol>
	</li>
</ol>
3. Generating Appropriate Super-Regions
<ol>
	<li>In the Super Regions tab, in the Supervoxels section, use the source drop-down menu to select the filtered dataset from which supervoxels will be created. Then specify the shape, spacing and compactness of the supervoxels (see Discussion and Figure 2 for further details). Finally, click on the apply button to generate the supervoxels. Once the supervoxels have been created they can be viewed in the Visualization pane, turned on or off and their transparency controlled in the Visualization tab and Viewer Window shortcut.</li>
	<li>In the Super Regions tab, in the Megavoxels section, use the source drop-down menu to select the filtered dataset from which megavoxels will be created. Next, specify the lambda, numBins and Gamma parameters of the megavoxels (see Discussion for further details). Once the megavoxels have been created they can be viewed in the Visualization pane, turned on or off and their transparency controlled in the Visualization tab and Viewer Window shortcut.</li>
</ol>
4. Introduction to Annotation
<ol>
	<li>In the Annotations tab use the Add Level button to add an annotation level. After a Level has been added, use the Add Label button in that level to add a label for the annotation. Once added, the name and color of the label can be modified for ease of annotation.</li>
	<li>Next, in order to begin annotating, select the pen icon from the tool shortcut section. When this is selected a set of options appear at the top of the Visualization pane. These options control the pen width and whether voxels, supervoxels or megavoxels will be used to annotate.
	<ol>
		<li>For the purpose of model training, generally, select supervoxels in the Annotation Level drop-down box and a middling to large pen width should be used. In the annotations tab, select the label to be annotated by checking the box to the far right of the label information. Next, click in the Visualization pane to annotate a single supervoxel, or click and drag to annotate many. 
		NOTE: Voxels and megavoxels can be selected in the Annotation Level drop-down box and used to annotate in the same way, which, in the case of megavoxels, can enable many thousands of similar voxels to be segmented with a single click of the mouse.</li>
	</ol>
	</li>
</ol>
5. Segmentation Using Model Training demonstrated with a microCT dataset.
NOTE: The first segmentation for many datasets is to differentiate multiple large regions from each other. For example, separating the nucleus from the cytoplasm, or the cell from the external ice and support structure. For this type of segmentation, with clear delineated boundaries and large regions, model training is useful. To demonstrate this, X-ray phase contrast tomographic data of Goosegrass will be used.
<ol>
	<li>Load the data, preprocess using the filter and feature suite, and determine appropriate supervoxels and/or megavoxels as described in the above sections using the parameters in Table 1 as a guide. Continuing to use the parameters from Table 1 and the instructions in Section 4, roughly annotate some large areas of the dataset, as shown in Figure 3. 
	NOTE: The dataset does not need to be completely segmented at this point.</li>
	<li>In the Model Training tab, set the predict level to the level that contains the manual training annotations, and in the descriptor section set the region to Supervoxels. Next, select the descriptors that are to be used to differentiate regions of the data by clicking on the Select Sources drop-down and checking the boxes of the features and filters of choice (see Table 1 and discussion).</li>
	<li>Next, click on the predict button. After computation is complete, the Visualization pane will be updated with the predictions for all of the non-labeled voxels showing which of the annotation classes they are predicted to belong to. Generally, the default parameters for each classifier methodology provide a good starting point and the user should only need to switch between classifiers to find a good fit. However, for expert or experienced users the options for each classifier are available and can be modified.</li>
	<li>After assessing the effect of the training methodologies and choosing one, apply additional refinement by clicking on the Refine drop-down in the Refinement section. At the bottom of the Model Training tab, in the &#34;Update Annotations&#34; section, make sure that the visualization drop-down menu is set to Predictions. Use the confidence slider to assign more or less of the unannotated supervoxels to the selected annotation labels.</li>
	<li>After an appropriate level of confidence has been selected based on visual inspection, use the Save buttons next to the labels at the bottom of the Confidence tool to save the predictions into specific labels. The Visualization pane will update to show the changes. Each label can be saved separately, and indeed, labels can be saved from smaller sub-regions by inputting values in the From and To z, y, and x boxes and clicking on the Save button for each label.</li>
	<li>Address minor mislabeling by providing further training data as described in Section 4. After appropriate predictions are added to labels, repeat the process of model training with refinement and adding high confidence predictions until there are no more unlabeled supervoxels. This is effective because each time the model training process is run there are more assigned supervoxels to train with, and hence the process becomes more robust as the iterations increase.</li>
</ol>
6. Segmentation using Super-Regions, Demonstrated with a CryoET Dataset.
NOTE: Since Super-Region segmentation is useful for smaller, discretely bound areas, the focus here will be on segmentation of the organelles and microtubules within this dataset. Model training was used to quickly segment the platelet from the background ice and carbon; these parameters are not discussed further, but are presented in Table 1.
<ol>
	<li>Load the data, preprocess using the filter and feature suite, and determine appropriate supervoxels and/or megavoxels as described in the above sections using the parameters in Table 1 as a guide.</li>
	<li>Add appropriate levels and labels to the annotation tab, select a label and begin annotating using a middling pen width with supervoxels selected. Be mindful of the need to choose different labels for objects in close proximity to each other in order to avoid labeling them as a single object.</li>
	<li>In order to clean up the annotations further, use morphological refinement methods (dilation, erosion, opening, closing and fill holes). These options can be found at the bottom of the Annotations tab. To use them, select the segmentation label and refinement method. Enter a radius value and choose how to apply the refinement. Then click refine.</li>
</ol>
7. Classification and Analysis of Data Objects Based on Inherent Characteristics, demonstrated with a CryoSXT Dataset
NOTE: Generally, the next step after segmentation is analysis of the data. The label splitter tool in SuRVoS allows for the classification of segmented objects using rules based on intrinsic characteristics of the objects such as average object intensity, variance, volume, or position. The label statistics tool allows for the visualization of relationships between these measures for each new object class. These are powerful new tools for the analysis of complex 3D datasets after segmentation.
<ol>
	<li>Load the data, preprocess using the filter and feature suite, determine appropriate supervoxels and/or megavoxels and segment as described in the above sections using the parameters in Table 1 as a guide.</li>
	<li>After segmentation, click on the second tab of the Visualization pane, called Label Splitter. This will add a new area to the right-hand side of the window - the Rule Creation pane.</li>
	<li>At the top of this area, select an appropriate level and labels for label splitting. Then select the dataset to query and click Label. All of the objects in the selected labels will now be outlined in blue as separate objects in the Visualization pane and a plot showing the average intensity of the objects will be displayed in the Rule Creation pane. To change the measure being shown in the plot, click on the drop-down box at the top of the right-hand side.</li>
	<li>To begin to split the objects into relevant classes, click add new label at the bottom of the Rule Creation pane. The name and color associated with this new label can be changed as described previously.
	<ol>
		<li>Click Add new rule and using the drop-down and freeform entry boxes define the rule to be applied. Click Apply to see the effects of the new rule in the Visualization pane and the plot in the Rule Creation pane. Multiple rules can be applied to a single label and multiple labels can be created within the same dataset. 
		NOTE: In order to gather any unlabeled objects, create a new label and instead of adding a rule to it, click Select Others.</li>
	</ol>
	</li>
	<li>When the objects of interest have been split into new labels, create a new, empty level in the Annotations tab. Then choose this level in the Rule Creation Tab and click Save labels. This will save the new labels to this empty level.</li>
	<li>On the edge of the Visualization pane, click on the Label Statistics tab. This will open a new Visualization pane that can be used to begin to understand relationships between object classes in the data. At the top, select an appropriate level and labels, and the dataset to query.
	<ol>
		<li>Select a few of the measures of interest by checking the boxes next to them. Then click Label. This will produce pairwise comparison plots for each of the selected measures. To update the plots, select or deselect appropriate measures then click Update plot.</li>
	</ol>
	</li>
</ol>
8. Exporting Data and Segmentations
<ol>
	<li>Export the plots and the raw measurement data by clicking on Export plot and Export data, respectively, in the Label Statistics tab of the Visualization pane.</li>
	<li>Export the image data and segmentations by clicking on the Export tab in the Plugins pane. First, click to select a folder where the data will be saved. Next, select the output (Raw Data, Raw Annotations, Segmentation Masks, or Masked Data) and the format (HDF5, MRC, or TIFF). Lastly, select the Annotation level(s) to be exported using the check boxes and click Export. The data can be scaled and inverted prior to exporting as necessary. When exporting the masked data, the dataset that the mask(s) will be applied to can be selected using a drop-down menu.</li>
</ol>

<ol>
	<li>. DiamondLightSource/SuRVoS: Version 1.0. Zenodo Available from: <a target="_blank" href="https://zenodo.org/record/247547">https://zenodo.org/record/247547</a> (2017)</li><li>. SuRVoS Workbench - Super Region Volume Segmentation Available from: <a target="_blank" href="https://diamondlightsource.github.io/SuRVoS/">https://diamondlightsource.github.io/SuRVoS/</a> (2017)</li><li>Frangakis, A. S., F&#246;rster, F. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Computational+exploration+of+structural+information+from+cryo-electron+tomograms.">Computational exploration of structural information from cryo-electron tomograms.</a> Curr Opin Struct Biol. 14 (3), 325-331 (2004).</li><li>Luci&#263;, V., F&#246;rster, F., Baumeister, W. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Structural+studies+by+electron+tomography:+from+cells+to+molecules.">Structural studies by electron tomography: from cells to molecules.</a> Annu Rev Biochem. 74, 833-865 (2005).</li><li>Lu&#269;i&#269;, V., Rigort, A., Baumeister, W. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Cryo-electron+tomography:+the+challenge+of+doing+structural+biology+in+situ.">Cryo-electron tomography: the challenge of doing structural biology in situ.</a> J Cell Biol. 202 (3), 407-419 (2013).</li><li>Milani, M., Drobne, D. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Focused+ion+beam+manipulation+and+ultramicroscopy+of+unprepared+cells.">Focused ion beam manipulation and ultramicroscopy of unprepared cells.</a> Scanning. 28 (3), 148-154 (2006).</li><li>Denk, W., Horstmann, H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Serial+Block-Face+Scanning+Electron+Microscopy+to+Reconstruct+Three-Dimensional+Tissue+Nanostructure.">Serial Block-Face Scanning Electron Microscopy to Reconstruct Three-Dimensional Tissue Nanostructure.</a> PLoS Biol. 2 (11), (2004).</li><li>Rigort, A., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Automated+segmentation+of+electron+tomograms+for+a+quantitative+description+of+actin+filament+networks.">Automated segmentation of electron tomograms for a quantitative description of actin filament networks.</a> J Struct Biol. 177 (1), 135-144 (2012).</li><li>F&#246;rster, F., Han, B. -. G., Beck, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Visual+Proteomics.">Visual Proteomics.</a> Meth Enzymolb. 483, 215-243 (2010).</li><li>Liu, Y., Sigworth, F. J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Automatic+cryo-EM+particle+selection+for+membrane+proteins+in+spherical+liposomes.">Automatic cryo-EM particle selection for membrane proteins in spherical liposomes.</a> J Struct Biol. 185 (3), 295-302 (2014).</li><li>Asano, S., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+molecular+census+of+26S+proteasomes+in+intact+neurons.">A molecular census of 26S proteasomes in intact neurons.</a> Science. 347 (6220), 439-442 (2015).</li><li>Gardu&#241;o, E., Wong-Barnum, M., Volkmann, N., Ellisman, M. H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Segmentation+of+electron+tomographic+data+sets+using+fuzzy+set+theory+principles.">Segmentation of electron tomographic data sets using fuzzy set theory principles.</a> J Struct Biol. 162 (3), 368-379 (2008).</li><li>Sommer, C., Straehle, C., K&#246;the, U., Hamprecht, F. A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Ilastik:+Interactive+learning+and+segmentation+toolkit.">Ilastik: Interactive learning and segmentation toolkit.</a> 2011 IEEE Int Symp Biomed Imaging: From Nano to Macro. , 230-233 (2011).</li><li>Darrow, M. C., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Structural+Mechanisms+of+Mutant+Huntingtin+Aggregation+Suppression+by+the+Synthetic+Chaperonin-like+CCT5+Complex+Explained+by+Cryoelectron+Tomography.">Structural Mechanisms of Mutant Huntingtin Aggregation Suppression by the Synthetic Chaperonin-like CCT5 Complex Explained by Cryoelectron Tomography.</a> J Biol Chem. 290 (28), 17451-17461 (2015).</li><li>Wang, R., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Electron+cryotomography+reveals+ultrastructure+alterations+in+platelets+from+patients+with+ovarian+cancer.">Electron cryotomography reveals ultrastructure alterations in platelets from patients with ovarian cancer.</a> Proc Natl Acad Sci USA. 112 (46), 14266-14271 (2015).</li><li>Shahmoradian, S. H., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=TRiC's+tricks+inhibit+huntingtin+aggregation.">TRiC's tricks inhibit huntingtin aggregation.</a> eLife. 2, (2013).</li><li>Dai, W., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Visualizing+virus+assembly+intermediates+inside+marine+cyanobacteria.">Visualizing virus assembly intermediates inside marine cyanobacteria.</a> Nature. 502 (7473), 707-710 (2013).</li><li>Sandberg, K. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Methods+for+Image+Segmentation+in+Cellular+Tomography.">Methods for Image Segmentation in Cellular Tomography.</a> Methods Cell Biol. 79, 769-798 (2007).</li><li>Volkmann, N. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Methods+for+Segmentation+and+Interpretation+of+Electron+Tomographic+Reconstructions.">Methods for Segmentation and Interpretation of Electron Tomographic Reconstructions.</a> Methods Enzymol. 483, 31-46 (2010).</li><li>Tsai, W. -. T., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=From+Voxels+to+Knowledge:+A+Practical+Guide+to+the+Segmentation+of+Complex+Electron+Microscopy+3D-Data.">From Voxels to Knowledge: A Practical Guide to the Segmentation of Complex Electron Microscopy 3D-Data.</a> J Vis Exp. (90), (2014).</li><li>. Amira &amp; Avizo 3D: Software for Scientific and Industrial Data Available from: <a target="_blank" href="https://www.fei.com/software/amira-avizo/">https://www.fei.com/software/amira-avizo/</a> (2016)</li><li>Kremer, J. R., Mastronarde, D. N., McIntosh, J. R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Computer+Visualization+of+Three-Dimensional+Image+Data+Using+IMOD.">Computer Visualization of Three-Dimensional Image Data Using IMOD.</a> J Struct Biol. 116 (1), 71-76 (1996).</li><li>Nguyen, H., Ji, Q. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Shape-driven+three-dimensional+watersnake+segmentation+of+biological+membranes+in+electron+tomography.">Shape-driven three-dimensional watersnake segmentation of biological membranes in electron tomography.</a> IEEE trans med imaging. 27 (5), 616-628 (2008).</li><li>Rusu, M., Starosolski, Z., Wahle, M., Rigort, A., Wriggers, W. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Automated+tracing+of+filaments+in+3D+electron+tomography+reconstructions+using+Sculptor+and+Situs.">Automated tracing of filaments in 3D electron tomography reconstructions using Sculptor and Situs.</a> J Struct Biol. 178 (2), 121-128 (2012).</li><li>Moussavi, F., Heitz, G., Amat, F., Comolli, L. R., Koller, D., Horowitz, M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=3D+segmentation+of+cell+boundaries+from+whole+cell+cryogenic+electron+tomography+volumes.">3D segmentation of cell boundaries from whole cell cryogenic electron tomography volumes.</a> J Struct Biol. 170 (1), 134-145 (2010).</li><li>Hecksel, C. W., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Quantifying+Variability+of+Manual+Annotation+in+Cryo-Electron+Tomograms.">Quantifying Variability of Manual Annotation in Cryo-Electron Tomograms.</a> Microsc Mpicroanal. 22 (3), 487-496 (2016).</li><li>Achanta, R., Shaji, A., Smith, K., Lucchi, A., Fua, P., S&#252;sstrunk, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=SLIC+superpixels+compared+to+state-of-the-art+superpixel+methods.">SLIC superpixels compared to state-of-the-art superpixel methods.</a> IEEE trans pattern anal mach intell. 34 (11), 2274-2282 (2012).</li><li>Luengo, I., Basham, M., French, A. P. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Fast+global+interactive+volume+segmentation+with+regional+supervoxel+descriptors.">Fast global interactive volume segmentation with regional supervoxel descriptors.</a> Proc. SPIE. 9784, (2016).</li><li>Luengo, I., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=SuRVoS:+Super-Region+Volume+Segmentation+Workbench.">SuRVoS: Super-Region Volume Segmentation Workbench.</a> J Struct Biol. 198 (1), 43-53 (2017).</li><li>. DiamondLightSource/Savu: Version 1.2. Zenodo Available from: <a target="_blank" href="https://zenodo.org/record/198726">https://zenodo.org/record/198726</a> (2014)</li><li>Paganin, D., Mayo, S. C., Gureyev, T. E., Miller, P. R., Wilkins, S. W. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Simultaneous+phase+and+amplitude+extraction+from+a+single+defocused+image+of+a+homogeneous+object.">Simultaneous phase and amplitude extraction from a single defocused image of a homogeneous object.</a> J Microsc. 206 (Pt 1), 33-40 (2002).</li><li>van Aarle, W., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Fast+and+flexible+X-ray+tomography+using+the+ASTRA+toolbox.">Fast and flexible X-ray tomography using the ASTRA toolbox.</a> Opt Express. 24 (22), 25129-25147 (2016).</li><li>Palenstijn, W. J., Batenburg, K. J., Sijbers, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Performance+improvements+for+iterative+electron+tomography+reconstruction+using+graphics+processing+units+(GPUs).">Performance improvements for iterative electron tomography reconstruction using graphics processing units (GPUs).</a> J Struct Biol. 176 (2), 250-253 (2011).</li><li>Mastronarde, D. N. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Dual-Axis+Tomography:+An+Approach+with+Alignment+Methods+That+Preserve+Resolution.">Dual-Axis Tomography: An Approach with Alignment Methods That Preserve Resolution.</a> J Struct Biol. 120 (3), 343-352 (1997).</li><li>Duke, E., Dent, K., Razi, M., Collinson, L. M. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Biological+applications+of+cryo-soft+X-ray+tomography.">Biological applications of cryo-soft X-ray tomography.</a> J Microsc. 255 (2), 65-70 (2014).</li><li>Arbel&#225;ez, P., Maire, M., Fowlkes, C., Malik, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Contour+detection+and+hierarchical+image+segmentation.">Contour detection and hierarchical image segmentation.</a> IEEE trans pattern anal mach intell. 33 (5), 898-916 (2011).</li></ol>

The authors declare they have no competing financial interests.

SuRVoS Workbench differs from other segmentation programs in that optimization of parameters is a necessary and important step prior to beginning the actual segmentation. In some manual or semi-manual segmentation programs, the user begins segmenting within moments of opening a new project. With SuRVoS, because large amounts of the volume will be segmented with very little user input and boundaries are delineated by the program, optimizing the parameters is critical to a successful segmentation. Specifically, feature channels and super-region building are two areas where attention should be paid.
Feature channels and model training
In addition to the raw data, SuRVoS allows the user to create additional datasets or channels derived from an existing dataset. These channels can be created using a selection of computational methodologies or feature extractors. Each of the data representations are available in parallel, and can be individually displayed to assess outcomes of feature or filter application. Because of these characteristics, they are referred to as feature channels in SuRVoS. There are many feature channel options provided within SuRVoS. For information on the options and parameters used here, see Table 1, for a full list and description of available feature channels visit https://diamondlightsource.github.io/SuRVoS/ 2. First, noisy datasets will benefit from denoising with either the Gaussian or total variation filter. It is recommended that further feature channel and supervoxel/megavoxel computations be performed using one of these denoised datasets as the data source. Generally, the total variation denoised dataset is used as the source data for feature channel and supervoxel/megavoxel computations. It is suggested to run with the default values first, assess the result in 3D and finally, iteratively optimize the parameters for the dataset. Additionally, feature channels can be built up into &#34;filter sets&#34; to specifically isolate aspects of the dataset, and these can then be used as data sources to create supervoxels and megavoxels. While this strategy is highly dataset dependent, it can be beneficial.
Feature channels are also used as sources to train the classifier in model training. When deciding on which feature channels to use, it is recommended that a few robust feature channels (e.g., from blob detection, texture and structure, or robust features categories) are used when working with a small amount of annotations to train the classifier. When working with a large amount of training data, it is recommended to use more feature channels overall, from any of the categories as long as they provide varied information to the classifier (e.g., add to the above list feature channels from local features and Gaussian features categories).
There are three main parts to model training: providing input data sources that describe the data, using these inputs to train a classifier, and finally refining the output predictions. Generally, smaller regions of the data will require more user annotations to accurately train the classifier, while larger regions of the data will require fewer user annotations. Model training first without selecting a refinement can be used to find the best predictions. Then include the refinement and optimize the lambda parameter as needed to fix issues with the predictions such as holes or jagged edges.
Supervoxels and megavoxels
Supervoxels are clusters of multiple nearby, similar voxels38,39. Supervoxels begin as a standard 3D grid overlaid on the data that is then iteratively deformed in order to adhere to the underlying boundaries, and thus better represent the data. Supervoxel creation and deformation is controlled by four user inputs: data source, superpixel shape, spacing, and compactness. The data source provides the data inputs that are queried during supervoxel creation. Any source can be used including filtered data sources. The superpixel shape parameters determine the starting 3D grid and the approximate desired shape of the resulting supervoxels. Changing these parameters can increase or decrease the size of the supervoxels before deformation. The spacing parameters define the importance of boundaries in each direction. Changing these parameters can emphasize boundaries in one or two directions at the expense of the other(s), meaning the resulting supervoxels will deform to better follow data boundaries in the given direction(s). The last parameter, compactness, controls how much the supervoxels can deform. A low compactness number allows the supervoxels to deform more. These parameters should be optimized to provide supervoxels that represent the boundaries of the data of interest. Note: Currently, supervoxel shape parameters must equal 1024 or less when multiplied together.
In some ways, supervoxel parameters can compensate for each other, meaning there is no one &#34;right answer&#34; when deciding on parameters. For example, a large starting grid (e.g. superpixel shape: 10 x 10 x 10) and a low compactness number (ex. 20) may give supervoxels with similar boundary adherence as compared to a small starting grid (e.g. superpixel shape 5 x 5 x 5) and a higher compactness number (e.g. 50). Because there are more, smaller supervoxels in the second scenario, they don&#39;t have to deform as much to represent boundaries. Both sets of parameters could be appropriate for segmentation of the dataset.
The biggest consideration when choosing supervoxel parameters is how well the supervoxels represent the data. Displaying the supervoxels alone, without data underneath them, as in Figure 2D, is a good way to assess supervoxel parameters. When displayed this way, the edges and outlines of shapes found in the data should still be visible in the supervoxels.
Megavoxels are conglomerates of multiple neighboring, similar supervoxels38,39. They are again controlled by four user inputs: data source, lambda, numbins, and gamma. As with supervoxels, the data source provides the data inputs that are queried during megavoxel creation. Both lambda and numbins impact the size and boundary adherence of the megavoxels. As the megavoxels grow larger (high lambda, low numbins), their boundary adherence decreases. The converse is also true, boundary adherence will increase with smaller megavoxels (low lambda, high numbins), however as the megavoxel size decreases, so does their usefulness in segmenting large amounts of voxels quickly. The optional gamma parameter controls the smoothness factor versus the cost of merging two supervoxels together. Small values of gamma can enhance the similarity between two supervoxels, at the cost of having fewer megavoxels overall.
As with supervoxels, the biggest consideration when choosing and optimizing megavoxel parameters is how well the megavoxels represent the data. Displaying the megavoxels alone as described for supervoxels can again be used to assess parameters. However, because megavoxels will generally be much larger and are three-dimensional, using the annotation tools to choose single megavoxels to ensure the boundary between regions of interest is tight is also recommended.
Annotation strategy
Two general annotation strategies have been described: a model training approach is useful for separating large regions of a dataset, while a super-region segmentation approach is useful for smaller, more diverse features such as individual organelles. Annotations can be organized in a hierarchical fashion so that it is possible to annotate large regions first, then subdivide them into more specific regions using a parent-child relationship. The parent label for a label can be assigned by clicking on the area to the right of the label color selection and choosing an appropriate parent label from a previous level. In practice, most datasets use both the model training and super-region segmentation strategies to segment specific regions/features of interest.
In the model training example here, a few training inputs (in the form of manual user supervoxel-based annotations) were used on three equally spaced slices of the data. In this way, the model training aspect of SuRVoS vastly increases the speed with which segmentation is possible especially when working with large, differentiated regions such as the divide between the regions in the goosegrass fruiting body as highlighted in Figure 3.
When model training, if the predictions cannot be seen, it may be necessary to go to the Visualization tab and make sure that the Predictions layer is turned on and set to an appropriate amount of transparency. Also, a confidence of 0 will assign every unlabeled supervoxel to a label, based on whatever the closest match is. Confidence of 100 will only assign a label if only one category of label has any proportional match. Everything in between is a tradeoff of these two extremes. When selecting a confidence level it is suggested to check a few slices to visually inspect that there are no incorrectly predicted voxels prior to saving the prediction to a label.
A good strategy for annotating using super-regions is to use the magnification tool to zoom in on the data, annotate a few organelles at once on one slice, using a &#34;quick, messy&#34; approach first (Figure 4C). Next, move up or down a few slices in Z and repeat this process. Because supervoxels are three-dimensional, many of the faults of the &#34;messy&#34; approach are fixed by annotations made in the above or below slices. In this way, segmentation is sped up and the boundaries are provided by the supervoxels rather than manually.
To clean up a label, standard segmentation refinement options have been provided. Dilation causes the selected segmentation label to grow by the given radius, erosion causes it to shrink. Opening and closing are the application of first erosion and then dilation, or vice-versa, respectively. And fill holes does exactly that. The order of these operations does matter. Generally, performing fill holes, then opening, then dilation works well. Each refinement method can be applied on a single slice (&#34;this slice&#34;), on all slices in 2D (&#34;All slices (2D)&#34;) or in 3D (&#34;Whole volume (3D)&#34;). All slices (2D) is recommended.
Significance and future directions
Efficient and accurate segmentation is the next bottleneck in processing of 3D datasets, especially with the routine automated collection of terabytes of image data during long-run sessions. SuRVoS Workbench can speed the segmentation process by a factor of 5 as compared to manual segmentation. Also, because the boundaries are delineated by supervoxels, the variability of the resultant segmentations should improve. In future, we hope to explore ways of using the segmentation of a representative 3D region of interest as training data to apply to the rest of the volume, or even a separate volume, with high confidence. This advance would further decrease the amount of user time and input necessary to segment even complex biological volumes, helping to alleviate the image processing and segmentation bottleneck. This, in turn, will allow the quantitative comparison of biological data in various states (e.g. non-disease, disease, treated) with robust experimental numbers.

The SuRVoS Workbench is a piece of software designed to allow researchers to extract scientifically relevant information from volumetric data from various samples, regardless of the structure of interest, resolution or the imaging modality1,2. Volumetric data such as these are often collected using X-ray or electron tomography systems, routinely based at large laboratories or centralized facilities due to their complexity. Both of these methods, and other techniques, produce large, information rich datasets which prove challenging to segment with either semi-automatic methods or manually. Particularly, near-native state cryo-immobilized datasets require low-dose imaging conditions, resulting in a low signal-to-noise ratio and poor contrast, especially in cryo electron tomography (cryoET)3,4,5. An additional factor in some 3D datasets is the presence of artifacts introduced by the challenging experimental conditions involved, for example missing wedge artifacts due to data collection over a limited tilt range, resulting in missing information and elongation in the direction of the beam3,4,5. Even when low signal-to-noise or missing wedge artifacts are not problematic (e.g. focused ion beam SEM6 or serial block face SEM7), the complexity and three-dimensional nature of the sample, and the large amount of data mean analysis would still benefit from an automated process for data segmentation.
Currently, when considering biological volumes of cells, there are many options for automatically or semi-automatically identifying very specific cellular features, such as actin, microtubules, or specific protein complexes, using a template-based search, or identifying features in specific types of datasets (e.g. high contrast, stained, resin-embedded samples)8,9,10,11,12. However, in these cases a priori information or specific sample preparation protocols are necessary, limiting the broad applicability of these segmentation strategies. There are also tools available that perform model training at the voxel level to learn the appearance of various structures of interest when given user input13. However, at this level the complexity of training and testing the models can be error-prone and computationally expensive. Given the challenging image conditions, and the lack of broadly applicable, semi-automatic segmentation strategies, manual segmentation is common, even when working with complex biological materials14,15,16,17. However, it is generally accepted that the process of manual segmentation is not only time-consuming, but also error-prone, subjective and variable4,5,18,19,20. Some segmentation programs offer tools to ease the manual segmentation process (i.e. interpolation, lasso, or blow tools)21,22, however, in cases of noisy datasets, they are difficult to apply successfully, and even when they are used successfully, the process is still subjective and variable.
Traditionally, segmentations have been used in two distinct ways: qualitatively or quantitatively. As imaging technologies and segmentation strategies improve, it has become more common to use segmentation as a quantitative tool to answer biological questions and as a &#34;ground truth&#34; for algorithm development8,12,15,23,24,25. In order to do this, detailed checks and balances are required to decrease variability and subjectivity throughout the process26. However, these precautions further increase the time-consuming nature of segmentation. Because of this, it is critical to provide a faster and less variable segmentation strategy.
The SuRVoS Workbench begins to address these issues by providing the user with a selection of machine learning and image processing tools that assist the user in the segmentation process, while also guiding the user through the required steps. To achieve this, two key innovations are implemented together in SuRVoS. First, it uses a super-region hierarchy to group similar, nearby regions of the data based on their inherent properties. Each of the regions in the hierarchy represents the same volume using fewer elements, while still providing strong boundary adherence. Thus, super-regions reduce the complexity of segmenting a volume by several orders of magnitude yet still represent the data without significant loss of information27. Second, SuRVoS provides a semi-automated segmentation strategy that uses minimal manual segmentation inputs to train classifiers, which are then used to segment the remaining volume28,29. This strategy reduces manual segmentation, greatly decreasing the amount of user time spent on segmenting and, when using super-regions, removes manual delineation of boundaries, potentially reducing variability and subjectivity.
A further key feature of SuRVoS is the Label Splitter Tool, whereby a user can classify a series of already segmented objects based on their inherent properties. After segmentation of various objects of interest, this tool can be used to divide the set into subclasses based on measures such as average object intensity, variance, size, location, etc. This is useful when classifying large groups of objects with high complexity. For example, a group of cellular organelles can be split into mitochondria, empty vesicles, lipid droplets, etc.; or a set of material inclusions can be separated based on size or shape. Once segmented the individual labels can be split into groups using any number of classifiers, reducing identification bias.
The SuRVoS Workbench has been successfully used to segment data from several imaging techniques. Here, synchrotron X-ray phase contrast tomography (microCT) of the fruiting body of a plant is used to demonstrate segmentation using model training, cryo electron tomography (cryoET) of human platelets is used to demonstrate segmentation using super- and megavoxels, and cryo soft X-ray tomography (cryoSXT) of a mammalian cell line is used to demonstrate the label splitting tools

<table>
	<tbody>
		<tr>
			<td>computer</td>
			<td>n/a</td>
			<td>n/a</td>
			<td>Must be running Linux operating system and have an NVidia GPU with at least 4 GB of memory</td>
		</tr>
	</tbody>
</table>

volume segmentation and analysis of biological materials using survos (super-region volume segmentation) workbench

Segmentation is the process of isolating specific regions or objects within an imaged volume, so that further study can be undertaken on these areas of interest. When considering the analysis of complex biological systems, the segmentation of three-dimensional image data is a time consuming and labor intensive step. With the increased availability of many imaging modalities and with automated data collection schemes, this poses an increased challenge for the modern experimental biologist to move from data to knowledge. This publication describes the use of SuRVoS Workbench, a program designed to address these issues by providing methods to semi-automatically segment complex biological volumetric data. Three datasets of differing magnification and imaging modalities are presented here, each highlighting different strategies of segmenting with SuRVoS. Phase contrast X-ray tomography (microCT) of the fruiting body of a plant is used to demonstrate segmentation using model training, cryo electron tomography (cryoET) of human platelets is used to demonstrate segmentation using super- and megavoxels, and cryo soft X-ray tomography (cryoSXT) of a mammalian cell line is used to demonstrate the label splitting tools. Strategies and parameters for each datatype are also presented. By blending a selection of semi-automatic processes into a single interactive tool, SuRVoS provides several benefits. Overall time to segment volumetric data is reduced by a factor of five when compared to manual segmentation, a mainstay in many image processing fields. This is a significant savings when full manual segmentation can take weeks of effort. Additionally, subjectivity is addressed through the use of computationally identified boundaries, and splitting complex collections of objects by their calculated properties rather than on a case-by-case basis.

Watch this Scientific Journal Video about Volume Segmentation and Analysis of Biological Materials Using SuRVoS Super-region Volume Segmentation Workbench at JoVE.com

Volume Segmentation and Analysis of Biological Materials Using SuRVoS Super-region Volume Segmentation Workbench

Segmentation of three-dimensional data from many imaging techniques is a major bottleneck in analysis of complex biological systems. Here, we describe the use of SuRVoS Workbench to semi-automatically segment volumetric data at various length-scales using example datasets from cryo-electron tomography, cryo soft X-ray tomography, and phase contrast X-ray tomography techniques.

Volume Segmentation and Analysis of Biological Materials Using SuRVoS (Super-region Volume Segmentation) Workbench

Segmentation is the process of isolating specific regions or objects within an imaged volume, so that further study can be undertaken on these areas of ...

Research

JoVE Journal

Biology

Large Volume (20L+) Filtration of Coastal Seawater Samples

Large Volume 20L+ Filtration of Coastal Seawater Samples

The workflow begins with the collection of coastal marine waters for downstream microbial community, nutrient and trace gas analyses. For this method, ...

Small Volume (1-3L) Filtration of Coastal Seawater Samples

Small Volume 1-3L Filtration of Coastal Seawater Samples

The workflow begins with the collection of coastal marine waters for downstream microbial community, nutrient and trace gas analyses. For today s ...

Concentration Determination of Nucleic Acids and Proteins Using the Micro-volume Bio-spec Nano Spectrophotometer

Nucleic Acid quantitation procedures have advanced significantly in the last three decades.  More and more, molecular biologists require consistent ...

Aseptic Laboratory Techniques: Volume Transfers with Serological Pipettes and Micropipettors

Microorganisms are everywhere - in the air, soil, and human body as well as on inanimate surfaces like laboratory benches and computer keyboards. The ...

Reconstruction of 3-Dimensional Histology Volume and its Application to Study Mouse Mammary Glands

Histology volume reconstruction facilitates the study of 3D shape and volume change of an organ at the level of macrostructures made up of cells. It can ...

Cardiac Pressure-Volume Loop Analysis Using Conductance Catheters in Mice

Cardiac pressure-volume loop analysis is the &#8220;gold-standard&#8221; in the assessment of load-dependent and load-independent measures of ventricular ...

Measuring Pressure Volume Loops in the Mouse

Understanding the causes and progression of heart disease presents a significant challenge to the biomedical community. The genetic flexibility of the ...

ACT-PRESTO: Biological Tissue Clearing and Immunolabeling Methods for Volume Imaging

The identification and exploration of the detailed organization of organs or of the whole body at the cellular level are fundamental challenges in ...

Biological Sample Preparation by High-pressure Freezing, Microwave-assisted Contrast Enhancement, and Minimal Resin Embedding for Volume Imaging

The described sample preparation technique is designed to combine the best quality of ultrastructural preservation with the most suitable contrast for the ...

Array Tomography Workflow for the Targeted Acquisition of Volume Information using Scanning Electron Microscopy

Electron microscopy is applied in biology and medicine for imaging of cellular and structural details at nanometer resolution. Historically, Transmission ...