2.1

The proposed scheme

In order to achieve precise segmentation of offshore farms in high-resolution SAR images, we propose a segmentation scheme based on improved UNet++. The overall segmentation flowchart is shown in Figure 1. From Figure 1, the scheme can be divided into model training and model application. In the model training stage, firstly, the training samples are resized to 256×256 pixels. Secondly, performing data augmentation operations on the samples to expand the sample set and reduce the overfitting of the training model¹⁸. Then, the enhanced data is input into the improved UNet++, and the simulated annealing strategy is used to constrain the learning rate of the network. Finally, the best trained model is chose by the validation samples. In the model application stage, the test set is resized to 256×256 pixels and input into the trained model for testing. Then, resizing the segmentation image to the size of the original image, which is the required binary segmentation image.

Figure 1.

Overall scheme. “Resize(256)” denotes to resize the image to 256×256 pixels. “Resize(original)” denotes to resize the image to the original size.

2.2

Simulated annealing strategy

It is very easy for the network training to fall into the local optimal during the training. Therefore, we introduce a simulated annealing strategy¹⁹ for the update of the learning rate, as shown in equations (1) and (2).

where η_t denotes the current learning rate. η_max and η_min respectively denote the set initial learning rate and minimum learning rate. T₀ is the epoch of the first restart. ß denotes the multiple of epochs compared to the previous restart interval, which is a constant. T_cur is the number of epochs since the last restart and T_i is the number of epochs between two restarts.

As shown in Figure 2, it shows the learning rate changes with epochs when η_max = 0.0003, η_min = 0.00001, T₀ = 3, ß = 2. The change in the learning rate is rippled, and the number of epochs between restarts every two times increases multiply. When the weights generated by epoch training fall into the local optimum, initializing the learning rate will help it jump out of the local optimum. And the epochs between the next two restarts are getting larger and larger, which provides a sufficient number of iterations for the weight to converge to the minimum.

Figure 2.

Using simulated annealing strategy, the change of learning rate during network training.

2.3

Improved UNet++

In order to enhance the feature extraction capability of the original UNet++ network, we integrate SE-Net as a feature extraction network structure into UNet++, and propose an improved UNet++. As shown in Figure 3, our improved UNet++ uses SE-Resnet101 (SE-Net) as the feature extraction network module, and replaces the encoding part of the original UNet++ with the feature map output from the five stages of SE-Resnet101. SE-Resnet101 is formed by adding an SE attention module that distributes weights to each channel on the Resnet101 network structure, which can pay more attention to the feature information of the target. At the same time, our proposed network is based on UNet++. UNet++ is an improvement based on U-Net. It has excellent performance in segmentation tasks and is widely used.

Figure 3.

the network structure of Improve UNet++.

2.4

Loss function and loss optimization

For the research task, simply using the binary cross-entropy loss function will cause the loss value to be low but the segmentation effect is not accurate enough. Therefore, this research uses a combination of binary cross-entropy loss and dice loss²⁰, and its expression is as shown in equation (3).

where p_n,c ∈ P and y_n,c ∈ Y denote the target labels and predicted probabilities of the c-th classes and n-th pixels in the batch processing, respectively. Y and P denote the true value and prediction result of the test image, respectively. C and N denote the number of classes and pixels, respectively.

Using cross-entropy loss and dice loss integrated loss function can promote the rapid convergence of the model. ε(Y, P) will be continuously updated and gradually approach 0 during the network training.

3.1

Data set

The high-resolution SAR offshore farm data set is derived from the SAR data of Haisi No. 1 and Gaofen No. 3 with a resolution of 1-3m. The entire data set has a total of 4000 images with sizes ranging from 512 to 2048 pixels, and most of the samples are 512×512 pixels. The scene covers the coastal areas of southeastern China, and the target includes large-scale seafood farms commonly found in the coastal waters of China. Figure 4 shows part of the sample images and annotated images. We can find that the SAR image has a lot of noise and the features are not obvious.

Figure 4.

Some sample samples from the high-resolution SAR offshore farm data set.

3.2

Experimental environment and parameter settings

The experimental environment of this research is ubuntu18.04 operating system. GPU is RTX 2080Ti 11G. The epochs are 100, the batch sizes are 8. The learning rate is 0.0003, the momentum is 0.9, and the weight decay is 0.0005. The initial number of rounds of simulated annealing is 3. The number of epochs between two restarts increases by multiples.

The optimization of the weights in the deep learning network requires a large number of training samples. In the experiment, we randomly use rotation transformation and flip transformation to enhance the data of the training samples to improve the robustness of the generated model.

In view of the evaluation metric for the local test set, we choose FWIoU²¹ as the evaluation metric, as shown in equation (4)

where P_ii denotes it was originally the i-th class, and it is predicted to be the i-th class; P_ij denotes it is originally the i-th class, but it is predicted to be the j-th class; P_ji denotes it is originally the j-th class, but it is predicted to be the i-th class. k denotes the total number of classes.

3.3

Effect verification of simulated annealing strategy

In order to better verify the effectiveness of the simulated annealing strategy, both U-Net and UNet++ have been verified in the experiment. The experimental results are shown in Table 1. The images in the experiment are all resized to 256×256 pixels for experiment.

Table 1.

Comparison of segmentation accuracy of whether to use simulated annealing strategy under U-Net and UNet++ networks.

It can be seen from Table 1 that using the simulated annealing strategy, the segmentation performance of U-Net and UNet++ has been significantly improved. The U-Net and UNet++ respectively increased by 0.0224 and 0.0191 in FWIoU. At the same time, by observing the segmentation performance of U-Net and UNet++, it is found that the segmentation performance of the UNet++ network is better than that of U-Net.

3.4

Effect verification of resize strategy

For the high-resolution SAR offshore farm data set, we consider that the segmentation performance of uniformly resizing the image to 256×256 pixels for training is better than resizing the image to 512×512 pixels. In order to better verify this idea, we use different resize strategies on U-Net and UNet++, and compare the final segmentation performance. The experimental results are shown in Table 2. The simulated annealing strategy is used in the experiments.

Table 2.

The impact of different resize strategies on segmentation accuracy under U-Net and UNet++ networks.

Note: “512” denotes to resize the image to 512×512 pixels. “256” denotes to resize the image to 256×256 pixels.

From Table 2, the segmentation accuracy of resizing to 256×256 pixels is better than that of 512×512 pixels under U-Net and UNet++. The U-Net and UNet++ effect respectively increase by 0.0034 and 0.0031 in FWIoU. On the one hand, by resizing the image to 256×256 pixels, the receptive field of convolution becomes relatively larger. On the other hand, the target area of data set is all in blocks, and the edges are relatively flat. Using a smaller size image input can increase the robustness of the input image to some small disturbances.

In addition to the comparative experiments of segmentation accuracy, we also compare the test time. As shown in Table 3, it will take less time to resize the image to 256×256 pixels. The speed of U-Net has increased by 20% (from 0.02 to 0.016), and the speed of UNet++ has increased by 33% (from 0.027 to 0.018).

Table 3.

The impact of different resize strategies on test time under U-Net and UNet++ networks.

Note: “512” denotes to resize the image to 512×512 pixels. “256” denotes to resize the image to 256×256 pixels.

3.5

Effect verification of improved U-Net++

In order to verify the segmentation performance of the proposed improved U-Net++, we compare it with multiple methods. The experimental results are shown in Table 4.

Table 4.

Comparison of different segmentation methods.

From Table 4, the proposed improved UNet++ has achieved relatively optimal segmentation accuracy. And compared to the original UNet++, the proposed network has an increase of 0.0034 in FWIoU. In order to show the segmentation effect of the proposed network more intuitively, the mask generated after the experimental segmentation is drawn to the original image as shown in Figure 5.

Figure 5.

The segmentation effect of the test set under the improved UNet++.

From Figure 5, although there is a small part of the mis-segmentation area between two adjacent farm areas in some images, the segmentation boundaries of most images are relatively accurate. It shows that the overall segmentation effect of the proposed scheme is very good.