Name: end-to-end deep neural network for salient object detection in complex environments
Uploaded: 2023-12-15T13:10:00.000+00:00
Duration: 3 min 31 s
Description: Watch this Scientific Journal Video about End-To-End Deep Neural Network for Salient Object Detection in Complex Environments at JoVE.com

This work is supported by 2024 Henan Provincial Higher Education Institutions Key Scientific Research Project Funding Program Establishment (Project Number:24A520053). This study is also supported by Specialized Creation and Integration Characteristic Demonstration Course Construction in Henan Province.

1. Experimental setup and procedure
<ol>
	<li>Load the pre-trained VGG16 model. 
	NOTE: The first step is to load the pre-trained VGG16 model from the Keras library6.
	<ol>
		<li>To load a pre-trained VGG16 model in Python using popular deep learning libraries like PyTorch (see Table of Materials), follow these general steps:
		<ol>
			<li>Import torch. Import torchvision.models as models.</li>
			<li>Load the pre-trained VGG16 model. vgg16_model = models.vgg16(pretrained=True).</li>
			<li>Ensure that the summary of the VGG16 model is &#34;print(vgg16_model)&#34;.</li>
		</ol>
		</li>
	</ol>
	</li>
	<li>Define the DCL and DEDN models.
	<ol>
		<li>For the pseudo-code of the DCL algorithm, provide Input: Image dataset SOD and Output: Trained DCL model.
		<ol>
			<li>Initialize the DCL model with the VGG16 backbone network.</li>
			<li>Preprocess the image dataset D (e.g., resize, normalization).</li>
			<li>Split the dataset into training and validation sets.</li>
			<li>Define the loss function for training the DCL model (e.g., binary cross-entropy).</li>
			<li>Set the hyperparameters for training: Learning rate (0.0001), Number of training epochs set (50), Batch size is (8), Optimizer (Adam).</li>
			<li>Train the DCL model: for each epoch in the defined number of epochs, do for each batch in the training set. Input the following:
			<ol>
				<li>Forward pass: Feed batch images to the DCL model. Compute the loss using the predicted saliency maps and ground truth maps.</li>
				<li>Backward pass: Update the model parameters using gradient descent end. Compute the validation loss and other evaluation metrics on the validation set end.</li>
			</ol>
			</li>
			<li>Save the trained DCL model.</li>
			<li>Return the trained DCL model.</li>
		</ol>
		</li>
		<li>For pseudo-code for the DEDN algorithm, input: Image dataset (X), Ground truth saliency maps (Y), Number of training iterations (N).
		<ol>
			<li>For the Encoder Network, ensure that the encoder is based on the VGG16 skeleton with modifications (as mentioned below). 
			NOTE: encoder_input = Input(shape=input_shape) 
			encoder_conv1 = Conv2D(64, (3, 3), activation=&#39;relu&#39;, padding=&#39;same&#39;)(encoder_input) 
			encoder_pool1 = MaxPooling2D((2, 2))(encoder_conv1) 
			encoder_conv2 = Conv2D(128, (3, 3), activation=&#39;relu&#39;, padding=&#39;same&#39;)(encoder_pool1) 
			encoder_pool2 = MaxPooling2D((2, 2))(encoder_conv2) 
			encoder_conv3 = Conv2D(256, (3, 3), activation=&#39;relu&#39;, padding=&#39;same&#39;)(encoder_pool2) 
			encoder_pool3 = MaxPooling2D((2, 2))(encoder_conv3)</li>
			<li>For the Decoder Network, ensure that the decoder is based on the VGG16 skeleton with modifications (as mentioned below). 
			NOTE: decoder_conv1 = Conv2D(256, (3, 3), activation=&#39;relu&#39;, padding=&#39;same&#39;)(encoder_pool3) 
			decoder_upsample1 = UpSampling2D((2, 2))(decoder_conv1) 
			decoder_conv2 = Conv2D(128, (3, 3), activation=&#39;relu&#39;, padding=&#39;same&#39;)(decoder_upsample1) 
			decoder_upsample2 = UpSampling2D((2, 2))(decoder_conv2) 
			decoder_conv3 = Conv2D(64, (3, 3), activation=&#39;relu&#39;, padding=&#39;same&#39;)(decoder_upsample2) 
			decoder_upsample3 = UpSampling2D((2, 2))(decoder_conv3) 
			decoder_output = Conv2D(1, (1, 1), activation=&#39;sigmoid&#39;, padding=&#39;same&#39;)(decoder_upsample3)</li>
		</ol>
		</li>
		<li>Define the DEDN model. model = Model (inputs = encoder_input, outputs = decoder_output).</li>
		<li>Compile the model. model.compile (optimizer = adam, loss = binary_crossentropy).</li>
		<li>Select the Training loop. 
		&#8203;NOTE: For iteration in range(N): # Randomly select a batch of images and ground truth maps; batch_X, batch_Y = randomly_select_batch(X, Y, batch_size).
		<ol>
			<li>Train the model on the batch. loss = model.train_on_batch(batch_X, batch_Y). Print the loss for monitoring.</li>
		</ol>
		</li>
		<li>Save the trained model. model.save (&#39;dedn_model.h5&#39;).</li>
	</ol>
	</li>
	<li>Combine.
	<ol>
		<li>Combine the outputs of the DCL and DEDN networks and refine the saliency map using a fully connected conditional random field (CRF) model.</li>
	</ol>
	</li>
</ol>
2. Image processing
<ol>
	<li>Click on run code to bring up the GUI interface (Figure 4).</li>
	<li>Click on open image to select the path and thus the image to be detected.</li>
	<li>Click on the display image to display the image that has been selected for detection.</li>
	<li>Click on start detection to detect the selected image. 
	NOTE: Detection result will appear with the detected image, i.e., the result of the salient object (Figure 5).</li>
	<li>Click on select the save path to save the image results of the salient object detection.</li>
</ol>

Enhanced Precision in Salient Object Detection Using Deep Neural Networks

<ol>
	<li>Wang, W. G., Shen, J. B., Ling, H. B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+deep+network+solution+for+attention+and+aesthetics+aware+photo+cropping.">A deep network solution for attention and aesthetics aware photo cropping.</a> IEEE Transactions on Pattern Analysis and Machine Intelligence. 41 (7), 1531-1544 (2018).</li><li>Wang, W. G., Sun, G. L., Gool, L. V. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Looking+beyond+single+images+for+weakly+supervised+semantic+segmentation+learning.">Looking beyond single images for weakly supervised semantic segmentation learning.</a> IEEE Transactions on Pattern Analysis and Machine. , (2022).</li><li>Mei, H. L., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Exploring+dense+context+for+salient+object+detection.">Exploring dense context for salient object detection.</a> IEEE Transactions on Circuits and Systems for Video Technology. 32 (3), 1378-1389 (2021).</li><li>Borji, A., Itti, L. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=State-of-the-art+in+visual+attention+modeling.">State-of-the-art in visual attention modeling.</a> IEEE Transactions on Pattern Analysis and Machine Intelligence. 35 (1), 185-207 (2012).</li><li>Long, J., Shelhamer, E., Darrell, T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Fully+convolutional+networks+for+semantic+segmentation.">Fully convolutional networks for semantic segmentation.</a> , 3431-3440 (2015).</li><li>Simonyan, K., Zisserman, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Very+deep+convolutional+networks+for+large-scale+image+recognition.">Very deep convolutional networks for large-scale image recognition.</a> arXiv preprint. , 1409-1556 (2014).</li><li>He, K., Zhang, X., Ren, S., Sun, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Deep+residual+learning+for+image+recognition.">Deep residual learning for image recognition.</a> , 770-778 (2016).</li><li>Liu, N., Han, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Dhsnet:+Deep+hierarchical+saliency+network+for+salient+object+detection.">Dhsnet: Deep hierarchical saliency network for salient object detection.</a> , 678-686 (2016).</li><li>Zhang, L., Dai, J., Lu, H., He, Y., Wang, G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+bi-directional+message+passing+model+for+salient+object+detection.">A bi-directional message passing model for salient object detection.</a> , 1741-1750 (2018).</li><li>Wu, R., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+mutual+learning+method+for+salient+object+detection+with+intertwined+multi-supervision.">A mutual learning method for salient object detection with intertwined multi-supervision.</a> , 8150-8159 (2019).</li><li>Li, G., Yu, Y. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Deep+contrast+learning+for+salient+object+detection.">Deep contrast learning for salient object detection.</a> , 478-487 (2019).</li><li>Ren, Q., Hu, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Multi-scale+deep+encoder-decoder+network+for+salient+object+detection.">Multi-scale deep encoder-decoder network for salient object detection.</a> Neurocomputing. 316, 95-104 (2018).</li><li>Wang, W. G., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Salient+object+detection+in+the+deep+learning+era:+An+in-depth+survey.">Salient object detection in the deep learning era: An in-depth survey.</a> IEEE Transactions on Pattern Analysis and Machine Intelligence. 44 (6), 3239-3259 (2021).</li><li>Zhou, T., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=RGB-D+salient+object+detection:+A+survey.">RGB-D salient object detection: A survey.</a> Computational Visual Media. 7, 37-69 (2021).</li><li>Fu, K., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Siamese+network+for+RGB-D+salient+object+detection+and+beyond.">Siamese network for RGB-D salient object detection and beyond.</a> IEEE Transactions on Pattern Analysis and Machine Intelligence. 44 (9), 5541-5559 (2021).</li><li>Lai, Q., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Weakly+supervised+visual+saliency+prediction.">Weakly supervised visual saliency prediction.</a> IEEE Transactions on Image Processing. 31, 3111-3124 (2022).</li><li>Zhang, L., Dai, J., Lu, H., He, Y., Wang, G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+bi-directional+message+passing+model+for+salient+object+detection.">A bi-directional message passing model for salient object detection.</a> , 1741-1750 (2018).</li><li>Wu, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+mutual+learning+method+for+salient+object+detection+with+intertwined+multi-supervision.">A mutual learning method for salient object detection with intertwined multi-supervision.</a> , 8150-8159 (2019).</li><li>Wang, W., Shen, J., Dong, X., Borji, A., Yang, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Inferring+salient+objects+from+human+fixations.">Inferring salient objects from human fixations.</a> IEEE Transactions on Pattern Analysis and Machine Intelligence. 42 (8), 1913-1927 (2019).</li><li>Movahedi, V., Elder, J. H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Design+and+perceptual+validation+of+performance+measures+for+salient+object+segmentation.">Design and perceptual validation of performance measures for salient object segmentation.</a> , 49-56 (2010).</li><li>Shi, J., Yan, Q., Xu, L., Jia, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Hierarchical+image+saliency+detection+on+extended+CSSD.">Hierarchical image saliency detection on extended CSSD.</a> IEEE Transactions on Pattern Analysis and Machine Intelligence. 38 (4), 717-729 (2015).</li><li>Achanta, R., Hemami, S., Estrada, F., Susstrunk, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Frequency-tuned+salient+region+detection.">Frequency-tuned salient region detection.</a> , 1597-1604 (2009).</li><li>Yang, C., Zhang, L., Lu, H., Ruan, X., Yang, M. H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Saliency+detection+via+graph-based+manifold+ranking.">Saliency detection via graph-based manifold ranking.</a> , 3166-3173 (2013).</li><li>Wei, Y., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Geodesic+saliency+using+background+priors.">Geodesic saliency using background priors.</a> Computer Vision-ECCV 2012. , 29-42 (2012).</li><li>Margolin, R., Tal, A., Zelnik-Manor, L. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=What+makes+a+patch+distinct.">What makes a patch distinct.</a> , 1139-1146 (2013).</li><li>Perazzi, F., Kr&#228;henb&#252;hl, P., Pritch, Y., Hornung, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Saliency+filters:+Contrast+based+filtering+for+salient+region+detection.">Saliency filters: Contrast based filtering for salient region detection.</a> , 733-740 (2012).</li><li>Hou, X., Harel, J., Koch, C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Image+signature:+Highlighting+sparse+salient+regions.">Image signature: Highlighting sparse salient regions.</a> IEEE Transactions on Pattern Analysis and Machine Intelligence. 34 (1), 194-201 (2011).</li><li>Jiang, H., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Salient+object+detection:+A+discriminative+regional+feature+integration+approach.">Salient object detection: A discriminative regional feature integration approach.</a> , 2083-2090 (2013).</li><li>Li, G., Yu, Y. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Visual+saliency+based+on+multiscale+deep+features.">Visual saliency based on multiscale deep features.</a> , 5455-5463 (2015).</li><li>Lee, G., Tai, Y. W., Kim, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Deep+saliency+with+encoded+low+level+distance+map+and+high-level+features.">Deep saliency with encoded low level distance map and high-level features.</a> , 660-668 (2016).</li><li>Liu, N., Han, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Dhsnet:+Deep+hierarchical+saliency+network+for+salient+object+detection.">Dhsnet: Deep hierarchical saliency network for salient object detection.</a> , 678-686 (2016).</li></ol>

The authors have nothing to disclose.

The article introduces an end-to-end deep neural network specifically designed for the detection of salient objects in complex environments. The network is composed of two interconnected components: a pixel-level multiscale fully convolutional network (DCL) and a deep encoder-decoder network (DEDN). These components work synergistically, incorporating contextual semantics to generate visual contrasts within multiscale feature maps. Additionally, they leverage both deep and shallow image features to improve the precision ...

<table><tbody><tr><td>Matlab</td><td>MathWorks</td><td>Matlab R2016a</td><td>MATLAB&#039;s programming interface provides development tools for improving code quality maintainability and maximizing performance.&lt;br/&gt; It provides tools for building applications using custom graphical interfaces.&lt;br/&gt; It provides tools for combining MATLAB-based algorithms with external applications and languages</td></tr><tr><td>Processor&amp;nbsp;</td><td>Intel</td><td>11th Gen Intel(R) Core (TM) i5-1135G7 @ 2.40GHz</td><td>64-bit Win11 processor&amp;nbsp;</td></tr><tr><td>Pycharm</td><td>JetBrains</td><td>PyCharm 3.0</td><td>PyCharm is a Python IDE (Integrated Development Environment)&lt;br/&gt; a list of required python:&lt;br/&gt; modulesmatplotlib&lt;br/&gt; skimage&lt;br/&gt; torch&lt;br/&gt; os&lt;br/&gt; time&lt;br/&gt; pydensecrf&lt;br/&gt; opencv&lt;br/&gt; glob&lt;br/&gt; PIL&lt;br/&gt; torchvision&lt;br/&gt; numpy&lt;br/&gt; tkinter</td></tr><tr><td>PyTorch&amp;nbsp;</td><td>Facebook</td><td>PyTorch 1.4&amp;nbsp;</td><td>PyTorch is an open source Python machine learning library , based on Torch , used for natural language processing and other applications.PyTorch can be viewed both as the addition of GPU support numpy , but also can be viewed as a powerful deep neural network with automatic derivatives .</td></tr></tbody></table>

end-to-end deep neural network for salient object detection in complex environments

Salient object detection has emerged as a burgeoning area of interest within the realm of computer vision. However, prevailing algorithms exhibit diminished precision when tasked with detecting salient objects within intricate and multifaceted environments. In light of this pressing concern, this article presents an end-to-end deep neural network that aims to detect salient objects within complex environments. The study introduces an end-to-end deep neural network that aims to detect salient objects within complex environments. Comprising two interrelated components, namely a pixel-level multiscale full convolutional network and a deep encoder-decoder network, the proposed network integrates contextual semantics to produce visual contrast across multiscale feature maps while employing deep and shallow image features to improve the accuracy of object boundary identification. The integration of a fully connected conditional random field (CRF) model further enhances the spatial coherence and contour delineation of salient maps. The proposed algorithm is extensively evaluated against 10 contemporary algorithms on the SOD and ECSSD databases. The evaluation results demonstrate that the proposed algorithm outperforms other approaches in terms of precision and accuracy, thereby establishing its efficacy in salient object detection within complex environments.

This study introduces an end-to-end deep neural network comprising two complementary networks: a pixel-level multi-scale fully convolutional network and a deep encoder-decoder network. The first network integrates contextual semantics to derive visual contrasts from multi-scale feature maps, addressing the challenge of fixed receptive fields in deep neural networks across different layers. The second network utilizes both deep and shallow image features to mitigate the issue of blurred boundaries in target objects. Final...

Watch this Scientific Journal Video about End-To-End Deep Neural Network for Salient Object Detection in Complex Environments at JoVE.com

Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications

The present protocol describes a novel end-to-end salient object detection algorithm. It leverages deep neural networks to enhance the precision of salient object detection within intricate environmental contexts.

End-To-End Deep Neural Network for Salient Object Detection in Complex Environments

Salient object detection has emerged as a burgeoning area of interest within the realm of computer vision. However, prevailing algorithms exhibit ...

end-to-end-deep-neural-network-for-salient-object-detection-complex

Research

JoVE Journal

Engineering

439 Views.  Zhengzhou University of Economics and Business. The present protocol describes a novel end-to-end salient object detection algorithm. It leverages deep neural networks to enhance the precision of salient object detection within intricate environmental contexts.

Video: Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications

Controlling the Size, Shape and Stability of Supramolecular Polymers in Water

For aqueous based supramolecular polymers, the simultaneous control over shape, size and stability is very difficult1. At the same time, the ability to do so is highly important in view of a number of applications in functional soft matter including electronics, biomedical engineering, and sensors. In the past, successful strategies to control the size and shape of supramolecular polymers typically focused on the use of templates2,3, end cappers4 or selective solvent techniques5. 
Here we disclose a strategy based on self-assembling discotic amphiphiles that leads to the control over stack length and shape of ordered, chiral columnar aggregates. By balancing electrostatic repulsive interactions on the hydrophilic rim and attractive non-covalent forces within the hydrophobic core of the polymerizing building block, we manage to create small and discrete spherical objects6,7. Increasing the salt concentration to screen the charges induces a sphere-to-rod transition. Intriguingly, this transition is expressed in an increase of cooperativity in the temperature-dependent self-assembly mechanism, and more stable aggregates are obtained. 
For our study we select a benzene-1,3,5-tricarboxamide (BTA) core connected to a hydrophilic metal chelate via a hydrophobic, fluorinated L-phenylalanine based spacer (Scheme 1). The metal chelate selected is a Gd(III)-DTPA complex that contains two overall remaining charges per complex and necessarily two counter ions. The one-dimensional growth of the aggregate is directed by &pi;-&pi; stacking and intermolecular hydrogen bonding. However, the electrostatic, repulsive forces that arise from the charges on the Gd(III)-DTPA complex start limiting the one-dimensional growth of the BTA-based discotic once a certain size is reached. At millimolar concentrations the formed aggregate has a spherical shape and a diameter of around 5 nm as inferred from 1H-NMR spectroscopy, small angle X-ray scattering, and cryogenic transmission electron microscopy (cryo-TEM). The strength of the electrostatic repulsive interactions between molecules can be reduced by increasing the salt concentration of the buffered solutions. This screening of the charges induces a transition from spherical aggregates into elongated rods with a length &gt; 25 nm. Cryo-TEM allows to visualise the changes in shape and size. In addition, CD spectroscopy permits to derive the mechanistic details of the self-assembly processes before and after the addition of salt. Importantly, the cooperativity -a key feature that dictates the physical properties of the produced supramolecular polymers- increases dramatically upon screening the electrostatic interactions. This increase in cooperativity results in a significant increase in the molecular weight of the formed supramolecular polymers in water.

The goal of this experiment is to determine and control the size, shape and stability of self-assembled discotic amphiphiles in water. For aqueous based supramolecular polymers such level of control is very difficult. We apply a strategy using both repulsive and attractive interactions. The experimental techniques applied to characterize this system are broadly applicable.

For aqueous based supramolecular polymers, the simultaneous control over shape, size and stability is very difficult1. At the same time, the ability to do ...

Fabrication of Spatially Confined Complex Oxides

Complex materials such as high Tc superconductors, multiferroics, and colossal magnetoresistors have electronic and magnetic properties that arise from the inherent strong electron correlations that reside within them. These materials can also possess electronic phase separation in which regions of vastly different resistive and magnetic behavior can coexist within a single crystal alloy material. By reducing the scale of these materials to length scales at and below the inherent size of the electronic domains, novel behaviors can be exposed. Because of this and the fact that spin-charge-lattice-orbital order parameters each involve correlation lengths, spatially reducing these materials for transport measurements is a critical step in understanding the fundamental physics that drives complex behaviors. These materials also offer great potential to become the next generation of electronic devices 1-3. Thus, the fabrication of low dimensional nano- or micro-structures is extremely important to achieve new functionality. This involves multiple controllable processes from high quality thin film growth to accurate electronic property characterization. Here, we present fabrication protocols of high quality microstructures for complex oxide manganite devices. Detailed descriptions and required equipment of thin film growth, photo-lithography, and wire-bonding are presented.

We describe the use of pulsed laser deposition (PLD), photolithography and wire-bonding techniques to create micrometer scale complex oxides devices. The PLD is utilized to grow epitaxial thin films. Photolithography and wire-bonding techniques are introduced to create practical devices for measurement purposes.

Complex materials such as high Tc superconductors, multiferroics, and colossal magnetoresistors have electronic and magnetic properties that arise from ...

Detection and Recovery of Palladium, Gold and Cobalt Metals from the Urban Mine Using Novel Sensors/Adsorbents Designated with Nanoscale Wagon-wheel-shaped Pores

Developing low-cost, efficient processes for recovering and recycling palladium, gold and cobalt metals from urban mine remains a significant challenge in industrialized countries. Here, the development of optical mesosensors/adsorbents (MSAs) for efficient recognition and selective recovery of Pd(II), Au(III), and Co(II) from urban mine was achieved. A simple, general method for preparing MSAs based on using high-order mesoporous monolithic scaffolds was described. Hierarchical cubic Ia3d wagon-wheel-shaped MSAs were fabricated by anchoring chelating agents (colorants) into three-dimensional pores and micrometric particle surfaces of the mesoporous monolithic scaffolds. Findings show, for the first time, evidence of controlled optical recognition of Pd(II), Au(III), and Co(II) ions and a highly selective system for recovery of Pd(II) ions (up to ~95%) in ores and industrial wastes. Furthermore, the controlled assessment processes described herein involve evaluation of intrinsic properties (e.g., visual signal change, long-term stability, adsorption efficiency, extraordinary sensitivity, selectivity, and reusability); thus, expensive, sophisticated instruments are not required. Results show evidence that MSAs will attract worldwide attention as a promising technological means of recovering and recycling palladium, gold and cobalt metals.

Because of the importance and extensive use of palladium, gold and cobalt metals in high-technology equipment, their recovery and recycling constitute an important industrial challenge. The metal recovery system described herein is a simple, low-cost means for the effective detection, removal, and recovery of these metals from the urban mine.

Developing low-cost, efficient processes for recovering and recycling palladium, gold and cobalt metals from urban mine remains a significant challenge in ...

Combining Eye-tracking Data with an Analysis of Video Content from Free-viewing a Video of a Walk in an Urban Park Environment

As individuals increasingly live in cities, methods to study their everyday movements and the data that can be collected becomes important and valuable. Eye-tracking informatics are known to connect to a range of feelings, health conditions, mental states and actions. But because vision is the result of constant eye-movements, teasing out what is important from what is noise is complex and data intensive. Furthermore, a significant challenge is controlling for what people look at compared to what is presented to them.
The following presents a methodology for combining and analyzing eye-tracking on a video of a natural and complex scene with a machine learning technique for analyzing the content of the video. In the protocol we focus on analyzing data from filmed videos, how a video can be best used to record participants&#39; eye-tracking data, and importantly how the content of the video can be analyzed and combined with the eye-tracking data. We present a brief summary of the results and a discussion of the potential of the method for further studies in complex environments.

The objective of the protocol is to detail how to collect video data for use in the laboratory; how to record eye-tracking data of participants looking at the data and how to efficiently analyze the content of the videos that they were looking at using a machine learning technique.

As individuals increasingly live in cities, methods to study their everyday movements and the data that can be collected becomes important and valuable. ...

Environment

A Step-by-Step Implementation of DeepBehavior, Deep Learning Toolbox for Automated Behavior Analysis

Understanding behavior is the first step to truly understanding neural mechanisms in the brain that drive it. Traditional behavioral analysis methods often do not capture the richness inherent to the natural behavior. Here, we provide detailed step-by-step instructions with visualizations of our recent methodology, DeepBehavior. The DeepBehavior toolbox uses deep learning frameworks built with convolutional neural networks to rapidly process and analyze behavioral videos. This protocol demonstrates three different frameworks for single object detection, multiple object detection, and three-dimensional (3D) human joint pose tracking. These frameworks return cartesian coordinates of the object of interest for each frame of the behavior video. Data collected from the DeepBehavior toolbox contain much more detail than traditional behavior analysis methods and provides detailed insights to the behavior dynamics. DeepBehavior quantifies behavior tasks in a robust, automated, and precise way. Following the identification of behavior, post-processing code is provided to extract information and visualizations from the behavioral videos.

The purpose of this protocol is to utilize pre-built convolutional neural nets to automate behavior tracking and perform detailed behavior analysis. Behavior tracking can be applied to any video data or sequences of images and is generalizable to track any user-defined object.

Understanding behavior is the first step to truly understanding neural mechanisms in the brain that drive it. Traditional behavioral analysis methods ...

Behavior

Deep Neural Networks for Image-Based Dietary Assessment

Due to the issues and costs associated with manual dietary assessment approaches, automated solutions are required to ease and speed up the work and increase its quality. Today, automated solutions are able to record a person's dietary intake in a much simpler way, such as by taking an image with a smartphone camera. In this article, we will focus on such image-based approaches to dietary assessment. For the food image recognition problem, deep neural networks have achieved the state of the art in recent years, and we present our work in this field. In particular, we first describe the method for food and beverage image recognition using a deep neural network architecture, called NutriNet. This method, like most research done in the early days of deep learning-based food image recognition, is limited to one output per image, and therefore unsuitable for images with multiple food or beverage items. That is why approaches that perform food image segmentation are considerably more robust, as they are able to identify any number of food or beverage items in the image. We therefore also present two methods for food image segmentation - one is based on fully convolutional networks (FCNs), and the other on deep residual networks (ResNet).

The goal of the work presented in this article is to develop technology for automated recognition of food and beverage items from images taken by mobile devices. The technology comprises of two different approaches - the first one performs food image recognition while the second one performs food image segmentation.

Due to the issues and costs associated with manual dietary assessment approaches, automated solutions are required to ease and speed up the work and ...

Application of Deep Learning-Based Medical Image Segmentation via Orbital Computed Tomography

Recently, deep learning-based segmentation models have been widely applied in the ophthalmic field. This study presents the complete process of constructing an orbital computed tomography (CT) segmentation model based on U-Net. For supervised learning, a labor-intensive and time-consuming process is required. The method of labeling with super-resolution to efficiently mask the ground truth on orbital CT images is introduced. Also, the volume of interest is cropped as part of the pre-processing of the dataset. Then, after extracting the volumes of interest of the orbital structures, the model for segmenting the key structures of the orbital CT is constructed using U-Net, with sequential 2D slices that are used as inputs and two bi-directional convolutional long-term short memories for conserving the inter-slice correlations. This study primarily focuses on the segmentation of the eyeball, optic nerve, and extraocular muscles. The evaluation of the segmentation reveals the potential application of segmentation to orbital CT images using deep learning methods.

Application of Deep Learning-Based Medical Image Segmentation via Orbital Computed Tomography

An object segmentation protocol for orbital computed tomography (CT) images is introduced. The methods of labeling the ground truth of orbital structures by using super-resolution, extracting the volume of interest from CT images, and modeling multi-label segmentation using 2D sequential U-Net for orbital CT images are explained for supervised learning.

Recently, deep learning-based segmentation models have been widely applied in the ophthalmic field. This study presents the complete process of ...

Swin-PSAxialNet: A Streamlined Approach for Segmenting Multiple Organs

Swin-PSAxialNet: An Efficient Multi-Organ Segmentation Technique

Abdominal multi-organ segmentation is one of the most important topics in the field of medical image analysis, and it plays an important role in supporting clinical workflows such as disease diagnosis and treatment planning. In this study, an efficient multi-organ segmentation method called Swin-PSAxialNet based on the nnU-Net architecture is proposed. It was designed specifically for the precise segmentation of 11 abdominal organs in CT images. &#65279;The proposed network has made the following improvements compared to nnU-Net. Firstly, Space-to-depth (SPD) modules and parameter-shared axial attention (PSAA) feature extraction blocks were introduced, enhancing the capability of 3D image feature extraction. Secondly, a multi-scale image fusion approach was employed to capture detailed information and spatial features, improving the capability of extracting subtle features and edge features. Lastly, a parameter-sharing method was introduced to reduce the model's computational cost and training speed. &#65279;The proposed network achieves an average Dice coefficient of 0.93342 for the segmentation task involving 11 organs. Experimental results indicate the notable superiority of Swin-PSAxialNet over previous mainstream segmentation methods. The method shows excellent accuracy and low computational costs in segmenting major abdominal organs.

The present protocol describes an efficient multi-organ segmentation method called Swin-PSAxialNet, which has achieved excellent accuracy compared to previous segmentation methods. The key steps of this procedure include dataset collection, environment configuration, data preprocessing, model training and comparison, and ablation experiments.

Abdominal multi-organ segmentation is one of the most important topics in the field of medical image analysis, and it plays an important role in ...

AI Program for Trypanosomiasis Detection and Classification

Superior Auto-Identification of Trypanosome Parasites by Using a Hybrid Deep-Learning Model

Trypanosomiasis is a significant public health problem in several regions across the world, including South Asia and Southeast Asia. The identification of hotspot areas under active surveillance is a fundamental procedure for controlling disease transmission. Microscopic examination is a commonly used diagnostic method. It is, nevertheless, primarily reliant on skilled and experienced personnel. To address this issue, an artificial intelligence (AI) program was introduced that makes use of a hybrid deep learning technique of object identification and object classification neural network backbones on the in-house low-code AI platform (CiRA CORE). The program can identify and classify the protozoan trypanosome species, namely Trypanosoma cruzi, T. brucei, and T. evansi, from oil-immersion microscopic images. The AI program utilizes pattern recognition to observe and analyze multiple protozoa within a single blood sample and highlights the nucleus and kinetoplast of each parasite as specific characteristic features using an attention map.
To assess the AI program&#39;s performance, two unique modules are created that provide a variety of statistical measures such as accuracy, recall, specificity, precision, F1 score, misclassification rate, receiver operating characteristics (ROC) curves, and precision versus recall (PR) curves. The assessment findings show that the AI algorithm is effective at identifying and categorizing parasites. By delivering a speedy, automated, and accurate screening tool, this technology has the potential to transform disease surveillance and control. It could also assist local officials in making more informed decisions on disease transmission-blocking strategies.

Author Spotlight: AI-Driven Trypanosome Species Detection from Microscopic Images 

Worldwide medical blood parasites were automatically screened using simple steps on a low-code AI platform. The prospective diagnosis of blood films was improved by using an object detection and classification method in a hybrid deep learning model. The collaboration of active monitoring and well-trained models helps to identify hotspots of trypanosome transmission.

Trypanosomiasis is a significant public health problem in several regions across the world, including South Asia and Southeast Asia. The identification of ...

Bioengineering

A Swin Transformer-Based Model for Thyroid Nodule Detection in Ultrasound Images

In recent years, the incidence of thyroid cancer has been increasing. Thyroid nodule detection is critical for both the detection and treatment of thyroid cancer. Convolutional neural networks (CNNs) have achieved good results in thyroid ultrasound image analysis tasks. However, due to the limited valid receptive field of convolutional layers, CNNs fail to capture long-range contextual dependencies, which are important for identifying thyroid nodules in ultrasound images. Transformer networks are effective in capturing long-range contextual information. Inspired by this, we propose a novel thyroid nodule detection method that combines the Swin Transformer backbone and Faster R-CNN. Specifically, an ultrasound image is first projected into a 1D sequence of embeddings, which are then fed into a hierarchical Swin Transformer. 
The Swin Transformer backbone extracts features at five different scales by utilizing shifted windows for the computation of self-attention. Subsequently, a feature pyramid network (FPN) is used to fuse the features from different scales. Finally, a detection head is used to predict bounding boxes and the corresponding confidence scores. Data collected from 2,680 patients were used to conduct the experiments, and the results showed that this method achieved the best mAP score of 44.8%, outperforming CNN-based baselines. In addition, we gained better sensitivity (90.5%) than the competitors. This indicates that context modeling in this model is effective for thyroid nodule detection.

Here, a new model for thyroid nodule detection in ultrasound images is proposed, which uses Swin Transformer as the backbone to perform long-range context modeling. Experiments prove that it performs well in terms of sensitivity and accuracy.

In recent years, the incidence of thyroid cancer has been increasing. Thyroid nodule detection is critical for both the detection and treatment of thyroid ...

End-To-End Deep Neural Network for Salient Object Detection in Complex Environments

Podsumowanie

Przeglądaj więcej filmów

Rozdziały w tym wideo

Controlling the Size, Shape and Stability of Supramolecular Polymers in Water

Fabrication of Spatially Confined Complex Oxides

Detection and Recovery of Palladium, Gold and Cobalt Metals from the Urban Mine Using Novel Sensors/Adsorbents Designated with Nanoscale Wagon-wheel-shaped Pores

Combining Eye-tracking Data with an Analysis of Video Content from Free-viewing a Video of a Walk in an Urban Park Environment

A Step-by-Step Implementation of DeepBehavior, Deep Learning Toolbox for Automated Behavior Analysis

Deep Neural Networks for Image-Based Dietary Assessment

Application of Deep Learning-Based Medical Image Segmentation via Orbital Computed Tomography

Swin-PSAxialNet: An Efficient Multi-Organ Segmentation Technique

Superior Auto-Identification of Trypanosome Parasites by Using a Hybrid Deep-Learning Model

A Swin Transformer-Based Model for Thyroid Nodule Detection in Ultrasound Images