This work is supported by 2024 Henan Provincial Higher Education Institutions Key Scientific Research Project Funding Program Establishment (Project Number:24A520053). This study is also supported by Specialized Creation and Integration Characteristic Demonstration Course Construction in Henan Province.

1. Experimental setup and procedure
<ol>
	<li>Load the pre-trained VGG16 model. 
	NOTE: The first step is to load the pre-trained VGG16 model from the Keras library6.
	<ol>
		<li>To load a pre-trained VGG16 model in Python using popular deep learning libraries like PyTorch (see Table of Materials), follow these general steps:
		<ol>
			<li>Import torch. Import torchvision.models as models.</li>
			<li>Load the pre-trained VGG16 model. vgg16_model = models.vgg16(pretrained=True).</li>
			<li>Ensure that the summary of the VGG16 model is &#34;print(vgg16_model)&#34;.</li>
		</ol>
		</li>
	</ol>
	</li>
	<li>Define the DCL and DEDN models.
	<ol>
		<li>For the pseudo-code of the DCL algorithm, provide Input: Image dataset SOD and Output: Trained DCL model.
		<ol>
			<li>Initialize the DCL model with the VGG16 backbone network.</li>
			<li>Preprocess the image dataset D (e.g., resize, normalization).</li>
			<li>Split the dataset into training and validation sets.</li>
			<li>Define the loss function for training the DCL model (e.g., binary cross-entropy).</li>
			<li>Set the hyperparameters for training: Learning rate (0.0001), Number of training epochs set (50), Batch size is (8), Optimizer (Adam).</li>
			<li>Train the DCL model: for each epoch in the defined number of epochs, do for each batch in the training set. Input the following:
			<ol>
				<li>Forward pass: Feed batch images to the DCL model. Compute the loss using the predicted saliency maps and ground truth maps.</li>
				<li>Backward pass: Update the model parameters using gradient descent end. Compute the validation loss and other evaluation metrics on the validation set end.</li>
			</ol>
			</li>
			<li>Save the trained DCL model.</li>
			<li>Return the trained DCL model.</li>
		</ol>
		</li>
		<li>For pseudo-code for the DEDN algorithm, input: Image dataset (X), Ground truth saliency maps (Y), Number of training iterations (N).
		<ol>
			<li>For the Encoder Network, ensure that the encoder is based on the VGG16 skeleton with modifications (as mentioned below). 
			NOTE: encoder_input = Input(shape=input_shape) 
			encoder_conv1 = Conv2D(64, (3, 3), activation=&#39;relu&#39;, padding=&#39;same&#39;)(encoder_input) 
			encoder_pool1 = MaxPooling2D((2, 2))(encoder_conv1) 
			encoder_conv2 = Conv2D(128, (3, 3), activation=&#39;relu&#39;, padding=&#39;same&#39;)(encoder_pool1) 
			encoder_pool2 = MaxPooling2D((2, 2))(encoder_conv2) 
			encoder_conv3 = Conv2D(256, (3, 3), activation=&#39;relu&#39;, padding=&#39;same&#39;)(encoder_pool2) 
			encoder_pool3 = MaxPooling2D((2, 2))(encoder_conv3)</li>
			<li>For the Decoder Network, ensure that the decoder is based on the VGG16 skeleton with modifications (as mentioned below). 
			NOTE: decoder_conv1 = Conv2D(256, (3, 3), activation=&#39;relu&#39;, padding=&#39;same&#39;)(encoder_pool3) 
			decoder_upsample1 = UpSampling2D((2, 2))(decoder_conv1) 
			decoder_conv2 = Conv2D(128, (3, 3), activation=&#39;relu&#39;, padding=&#39;same&#39;)(decoder_upsample1) 
			decoder_upsample2 = UpSampling2D((2, 2))(decoder_conv2) 
			decoder_conv3 = Conv2D(64, (3, 3), activation=&#39;relu&#39;, padding=&#39;same&#39;)(decoder_upsample2) 
			decoder_upsample3 = UpSampling2D((2, 2))(decoder_conv3) 
			decoder_output = Conv2D(1, (1, 1), activation=&#39;sigmoid&#39;, padding=&#39;same&#39;)(decoder_upsample3)</li>
		</ol>
		</li>
		<li>Define the DEDN model. model = Model (inputs = encoder_input, outputs = decoder_output).</li>
		<li>Compile the model. model.compile (optimizer = adam, loss = binary_crossentropy).</li>
		<li>Select the Training loop. 
		&#8203;NOTE: For iteration in range(N): # Randomly select a batch of images and ground truth maps; batch_X, batch_Y = randomly_select_batch(X, Y, batch_size).
		<ol>
			<li>Train the model on the batch. loss = model.train_on_batch(batch_X, batch_Y). Print the loss for monitoring.</li>
		</ol>
		</li>
		<li>Save the trained model. model.save (&#39;dedn_model.h5&#39;).</li>
	</ol>
	</li>
	<li>Combine.
	<ol>
		<li>Combine the outputs of the DCL and DEDN networks and refine the saliency map using a fully connected conditional random field (CRF) model.</li>
	</ol>
	</li>
</ol>
2. Image processing
<ol>
	<li>Click on run code to bring up the GUI interface (Figure 4).</li>
	<li>Click on open image to select the path and thus the image to be detected.</li>
	<li>Click on the display image to display the image that has been selected for detection.</li>
	<li>Click on start detection to detect the selected image. 
	NOTE: Detection result will appear with the detected image, i.e., the result of the salient object (Figure 5).</li>
	<li>Click on select the save path to save the image results of the salient object detection.</li>
</ol>

Enhanced Precision in Salient Object Detection Using Deep Neural Networks

<ol>
	<li>Wang, W. G., Shen, J. B., Ling, H. B. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+deep+network+solution+for+attention+and+aesthetics+aware+photo+cropping.">A deep network solution for attention and aesthetics aware photo cropping.</a> IEEE Transactions on Pattern Analysis and Machine Intelligence. 41 (7), 1531-1544 (2018).</li><li>Wang, W. G., Sun, G. L., Gool, L. V. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Looking+beyond+single+images+for+weakly+supervised+semantic+segmentation+learning.">Looking beyond single images for weakly supervised semantic segmentation learning.</a> IEEE Transactions on Pattern Analysis and Machine. , (2022).</li><li>Mei, H. L., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Exploring+dense+context+for+salient+object+detection.">Exploring dense context for salient object detection.</a> IEEE Transactions on Circuits and Systems for Video Technology. 32 (3), 1378-1389 (2021).</li><li>Borji, A., Itti, L. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=State-of-the-art+in+visual+attention+modeling.">State-of-the-art in visual attention modeling.</a> IEEE Transactions on Pattern Analysis and Machine Intelligence. 35 (1), 185-207 (2012).</li><li>Long, J., Shelhamer, E., Darrell, T. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Fully+convolutional+networks+for+semantic+segmentation.">Fully convolutional networks for semantic segmentation.</a> , 3431-3440 (2015).</li><li>Simonyan, K., Zisserman, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Very+deep+convolutional+networks+for+large-scale+image+recognition.">Very deep convolutional networks for large-scale image recognition.</a> arXiv preprint. , 1409-1556 (2014).</li><li>He, K., Zhang, X., Ren, S., Sun, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Deep+residual+learning+for+image+recognition.">Deep residual learning for image recognition.</a> , 770-778 (2016).</li><li>Liu, N., Han, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Dhsnet:+Deep+hierarchical+saliency+network+for+salient+object+detection.">Dhsnet: Deep hierarchical saliency network for salient object detection.</a> , 678-686 (2016).</li><li>Zhang, L., Dai, J., Lu, H., He, Y., Wang, G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+bi-directional+message+passing+model+for+salient+object+detection.">A bi-directional message passing model for salient object detection.</a> , 1741-1750 (2018).</li><li>Wu, R., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+mutual+learning+method+for+salient+object+detection+with+intertwined+multi-supervision.">A mutual learning method for salient object detection with intertwined multi-supervision.</a> , 8150-8159 (2019).</li><li>Li, G., Yu, Y. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Deep+contrast+learning+for+salient+object+detection.">Deep contrast learning for salient object detection.</a> , 478-487 (2019).</li><li>Ren, Q., Hu, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Multi-scale+deep+encoder-decoder+network+for+salient+object+detection.">Multi-scale deep encoder-decoder network for salient object detection.</a> Neurocomputing. 316, 95-104 (2018).</li><li>Wang, W. G., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Salient+object+detection+in+the+deep+learning+era:+An+in-depth+survey.">Salient object detection in the deep learning era: An in-depth survey.</a> IEEE Transactions on Pattern Analysis and Machine Intelligence. 44 (6), 3239-3259 (2021).</li><li>Zhou, T., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=RGB-D+salient+object+detection:+A+survey.">RGB-D salient object detection: A survey.</a> Computational Visual Media. 7, 37-69 (2021).</li><li>Fu, K., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Siamese+network+for+RGB-D+salient+object+detection+and+beyond.">Siamese network for RGB-D salient object detection and beyond.</a> IEEE Transactions on Pattern Analysis and Machine Intelligence. 44 (9), 5541-5559 (2021).</li><li>Lai, Q., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Weakly+supervised+visual+saliency+prediction.">Weakly supervised visual saliency prediction.</a> IEEE Transactions on Image Processing. 31, 3111-3124 (2022).</li><li>Zhang, L., Dai, J., Lu, H., He, Y., Wang, G. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+bi-directional+message+passing+model+for+salient+object+detection.">A bi-directional message passing model for salient object detection.</a> , 1741-1750 (2018).</li><li>Wu, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=A+mutual+learning+method+for+salient+object+detection+with+intertwined+multi-supervision.">A mutual learning method for salient object detection with intertwined multi-supervision.</a> , 8150-8159 (2019).</li><li>Wang, W., Shen, J., Dong, X., Borji, A., Yang, R. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Inferring+salient+objects+from+human+fixations.">Inferring salient objects from human fixations.</a> IEEE Transactions on Pattern Analysis and Machine Intelligence. 42 (8), 1913-1927 (2019).</li><li>Movahedi, V., Elder, J. H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Design+and+perceptual+validation+of+performance+measures+for+salient+object+segmentation.">Design and perceptual validation of performance measures for salient object segmentation.</a> , 49-56 (2010).</li><li>Shi, J., Yan, Q., Xu, L., Jia, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Hierarchical+image+saliency+detection+on+extended+CSSD.">Hierarchical image saliency detection on extended CSSD.</a> IEEE Transactions on Pattern Analysis and Machine Intelligence. 38 (4), 717-729 (2015).</li><li>Achanta, R., Hemami, S., Estrada, F., Susstrunk, S. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Frequency-tuned+salient+region+detection.">Frequency-tuned salient region detection.</a> , 1597-1604 (2009).</li><li>Yang, C., Zhang, L., Lu, H., Ruan, X., Yang, M. H. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Saliency+detection+via+graph-based+manifold+ranking.">Saliency detection via graph-based manifold ranking.</a> , 3166-3173 (2013).</li><li>Wei, Y., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Geodesic+saliency+using+background+priors.">Geodesic saliency using background priors.</a> Computer Vision-ECCV 2012. , 29-42 (2012).</li><li>Margolin, R., Tal, A., Zelnik-Manor, L. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=What+makes+a+patch+distinct.">What makes a patch distinct.</a> , 1139-1146 (2013).</li><li>Perazzi, F., Kr&#228;henb&#252;hl, P., Pritch, Y., Hornung, A. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Saliency+filters:+Contrast+based+filtering+for+salient+region+detection.">Saliency filters: Contrast based filtering for salient region detection.</a> , 733-740 (2012).</li><li>Hou, X., Harel, J., Koch, C. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Image+signature:+Highlighting+sparse+salient+regions.">Image signature: Highlighting sparse salient regions.</a> IEEE Transactions on Pattern Analysis and Machine Intelligence. 34 (1), 194-201 (2011).</li><li>Jiang, H., et al. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Salient+object+detection:+A+discriminative+regional+feature+integration+approach.">Salient object detection: A discriminative regional feature integration approach.</a> , 2083-2090 (2013).</li><li>Li, G., Yu, Y. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Visual+saliency+based+on+multiscale+deep+features.">Visual saliency based on multiscale deep features.</a> , 5455-5463 (2015).</li><li>Lee, G., Tai, Y. W., Kim, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Deep+saliency+with+encoded+low+level+distance+map+and+high-level+features.">Deep saliency with encoded low level distance map and high-level features.</a> , 660-668 (2016).</li><li>Liu, N., Han, J. <a target="_blank" href="http://www.ncbi.nlm.nih.gov/entrez/query.fcgi?db=PubMed&cmd=Search&doptcmdl=Citation&defaultField=Title+Word&term=Dhsnet:+Deep+hierarchical+saliency+network+for+salient+object+detection.">Dhsnet: Deep hierarchical saliency network for salient object detection.</a> , 678-686 (2016).</li></ol>

The authors have nothing to disclose.

The article introduces an end-to-end deep neural network specifically designed for the detection of salient objects in complex environments. The network is composed of two interconnected components: a pixel-level multiscale fully convolutional network (DCL) and a deep encoder-decoder network (DEDN). These components work synergistically, incorporating contextual semantics to generate visual contrasts within multiscale feature maps. Additionally, they leverage both deep and shallow image features to improve the precision ...

<table border="1" fo:keep-together.within-page="1" fo:keep-with-next.within-page="always">
	<tbody>
		<tr>
			<td>Matlab</td>
			<td>MathWorks</td>
			<td>Matlab R2016a</td>
			<td>MATLAB&#39;s programming interface provides development tools for improving code quality maintainability and maximizing performance. 
			It provides tools for building applications using custom graphical interfaces. 
			It provides tools for combining MATLAB-based algorithms with external applications and languages</td>
		</tr>
		<tr>
			<td>Processor&#160;</td>
			<td>Intel</td>
			<td>11th Gen Intel(R) Core (TM) i5-1135G7 @ 2.40GHz</td>
			<td>64-bit Win11 processor&#160;</td>
		</tr>
		<tr>
			<td>Pycharm</td>
			<td>JetBrains</td>
			<td>PyCharm 3.0</td>
			<td>PyCharm is a Python IDE (Integrated Development Environment) 
			a list of required python: 
			modulesmatplotlib 
			skimage 
			torch 
			os 
			time 
			pydensecrf 
			opencv 
			glob 
			PIL 
			torchvision 
			numpy 
			tkinter</td>
		</tr>
		<tr>
			<td>PyTorch&#160;</td>
			<td>Facebook</td>
			<td>PyTorch 1.4&#160;</td>
			<td>PyTorch is an open source Python machine learning library , based on Torch , used for natural language processing and other applications.PyTorch can be viewed both as the addition of GPU support numpy , but also can be viewed as a powerful deep neural network with automatic derivatives .</td>
		</tr>
	</tbody>
</table>

end-to-end deep neural network for salient object detection in complex environments

Salient object detection has emerged as a burgeoning area of interest within the realm of computer vision. However, prevailing algorithms exhibit diminished precision when tasked with detecting salient objects within intricate and multifaceted environments. In light of this pressing concern, this article presents an end-to-end deep neural network that aims to detect salient objects within complex environments. The study introduces an end-to-end deep neural network that aims to detect salient objects within complex environments. Comprising two interrelated components, namely a pixel-level multiscale full convolutional network and a deep encoder-decoder network, the proposed network integrates contextual semantics to produce visual contrast across multiscale feature maps while employing deep and shallow image features to improve the accuracy of object boundary identification. The integration of a fully connected conditional random field (CRF) model further enhances the spatial coherence and contour delineation of salient maps. The proposed algorithm is extensively evaluated against 10 contemporary algorithms on the SOD and ECSSD databases. The evaluation results demonstrate that the proposed algorithm outperforms other approaches in terms of precision and accuracy, thereby establishing its efficacy in salient object detection within complex environments.

Author Spotlight: Enhancement of Salient Object Detection for Smart Grid Applications

End-To-End Deep Neural Network for Salient Object Detection in Complex Environments

Our research range from salient object detection, computer vision, image processing and more. We want to respond to the application of salient object detection such as its use in smart grid for inspecting insulators. The next research advancement involves transferring salient object algorithms for learning in the context of smart grids.The convolutional neural networks can more effectively act as salient information from images, enhancing the processing of salient object detection. Therefore, they advance the research in salient object detection. Our research outcome improves the accuracy of salient object detection, and allow for the transfer of algorithms such as their application in smart grids for effectively segmenting and detecting insulators.

This study introduces an end-to-end deep neural network comprising two complementary networks: a pixel-level multi-scale fully convolutional network and a deep encoder-decoder network. The first network integrates contextual semantics to derive visual contrasts from multi-scale feature maps, addressing the challenge of fixed receptive fields in deep neural networks across different layers. The second network utilizes both deep and shallow image features to mitigate the issue of blurred boundaries in target objects. Final...

Watch this Scientific Journal Video about End-To-End Deep Neural Network for Salient Object Detection in Complex Environments at JoVE.com

The present protocol describes a novel end-to-end salient object detection algorithm. It leverages deep neural networks to enhance the precision of salient object detection within intricate environmental contexts.

Salient object detection has emerged as a burgeoning area of interest within the realm of computer vision. However, prevailing algorithms exhibit ...

end-to-end-deep-neural-network-for-salient-object-detection-complex

Research

JoVE Journal

Engineering

285 Views.  Zhengzhou University of Economics and Business. Our research range from salient object detection, computer vision, image processing and more. We want to respond to the application of salient object detection such as its use in smart grid for inspecting insulators. The next research advancement involves transferring salient object algorithms for learning in the context of smart grids.The convolutional neural networks can more effectively act as salient information from images, enhancing the processing of salient object detection. The...