2.1

Backbone

MobileNet¹³ uses the reverse residual module of linear bottleneck to improve feature extraction based on adopting deeply separable convolution. The images are fed into the backbone feature extraction network, which uses the bneck structure. SE¹⁴ indicates that the attention mechanism is added to this layer. Bneck structure is used to up-dimension the input feature map first and then perform deep separable convolution, while squeeze-and-excite attention module is added to balance the weights of each channel of the feature map.

In the backbone network, the h-swish activation function modified by the swish activation function was used. Equation (1) is the swish activation function.

where x is input, к is the hyper-parameter used to adjust the slope of the activation function, σ is the sigmoid function. h-swish uses the ReLU6 activation function to optimize the σ(κx) in swish.

The use of ReLU6 activation allows limiting the input x to between 0 and 1, thus replacing the function of the sigmoid function. At the same time, h-swish reduces the number of activation functions in the bneck structure to 16 while maintaining the same accuracy of 32 activation functions using swish, reducing the complexity of the network.

2.2

Lite fusion feature pyramid network

Since the pests have different scales of targets, the single-scale convolution kernel cannot adapt to multi-angle and multi-scale changing pictures. Thus we need the feature pyramid network architecture¹⁵. The FPN shallow layer has a larger resolution and contains clearer location information, the deep layer features contain rich semantic information, and the feature layers at different scales contain different feature information and are more adaptable to objects of different sizes.

To solve the problem that multi-scale pests reduce model accuracy, a lightweight multi-layer fusion module is constructed for the feature pyramid network, which is shown in Figure 2. The feature map of size 52×52 is first downsampled using a 2×2 averaging pooling layer. This allows feature fusion operations to provide shallow visual information and preserve more detailed features. Second, the feature map of size 13×13 is upsampled to size 26×26. This feature map has high-level semantic information and contains global object information. In the end, three feature maps of size 26×26 are concat into one feature map. The size of 13×13 and 26×26 feature maps are upsampled to generate 52×52 and 13×13 additional feature maps then combined them to feature pyramids.

Figure 2.

The architecture of the proposed lite fusion module.

3.1

Experiment platform and dataset

Experiment platform. In this paper, the model is trained on Ubuntu 18.04 operating system, using PyTorch framework, Intel Core I7-10700 CPU, NVIDIA TITAN RTX GPU (24GB), CUDA10.0, CuDNN7.6, Python3.7 software environment. The input images are resized to 640×640, the training batch is set to 16, the initial learning rate is 0.001, the IoU threshold is set to 0.5, and all models are trained for 100 epochs according to these parameters.

Dataset. We collected 11,130 pest images with the resolution of 1944×2592 using under-light pest image acquisition equipment. The images are annotated by agricultural experts with the pest bounding boxes and classes and the labeling open source software is LabelImg. Based on this, we built a dataset named Croppest12. Table 1 shows the pest names and their corresponding instance numbers, the average height and width of the pests in that category. As can be seen in Table 1, the pest instances range from 198 to 18,463. There are 8 categories of pests with instances less than 1,000. And the width and height of the pest boxes are mostly less than 100 pixels, which is still small compared to the 1944×2592 resolution image.

Table 1.

Statistics of Croppest12.

3.2

Evaluation metrics

To evaluate the performance of the algorithm in this paper, some evaluation metrics such as Precision (P) and Recall (R) are used to quantitatively evaluate the model, which is calculated in the form shown in equation (5). TP, TN, and FP denote the number of targets that are correct, targets that are incorrect, and undetected, respectively.

Average Precision (AP) is used to evaluate the performance of the model on the test set. The multi-category detection results are usually measured using mean Average Precision (mAP), it calculates based on the shape of the PR curve. In equation (5), C is the number of classes. In addition, we measure the speed of the detection algorithm in terms of the number of images processed per second (FPS).

3.3

Experiment results

As shown in Table 2, we compare the number of model parameters, mAP, and FPS of Faster R-CNN, SSD, and YOLOv3 while keeping the training parameters consistent. Compared with Faster RCNN, the average precision of YOLO-pest is 5 mAP higher, but in terms of inference speed, YOLO-pest is 40 FPS faster than Faster R-CNN, which meets the real-time detection requirement. Table 3 shows the scientific names of each pest category, as well as the number of instances. There is also the AP of different methods for each category of pests, and it can be seen that our method surpasses other methods for almost all pest categories.

Table 2.

Performances of different models.

Table 3.

Performances of single class pest.

Table 4 shows the results of the ablation experiments performed on YOLO-pest. Mobilenetv3 in the table represents replacing the CSPDarknet53 backbone network in YOLOv4 with the Mobilenetv3 structure. Lite-fusion FPN represents replacing the PANet FPN in YOLOv4 with the Lite-fusion FPN structure fusion network. The ablation experiments compare the module parameters, FPS, and mAP under various structure combinations, respectively. It can be seen that the number of parameters of the original YOLOv4 is 245.7M, and the model size further decreases to only 47.6M after replacing CSPDarknet53 with Mobilenetv3 on this model, but the mAP also decreases to 68.6%, and it can be concluded that if only PANet FPN is replaced with Lite-fusion FPN structure, the model size is almost unchanged, but the mAP is increased by 2.6 mAP, indicating that Lite-fusion FPN can indeed improve the module with almost no effect on the model size. Finally, by combing both Mobilenetv3 and Bi-FPN-Lite structures, the mAP of the algorithm increases to 70.1%, and the model size decreases to only 46.9M, which ensures a high mAP even though the model size and number of parameters are significantly reduced. We report the detection result in Figure 3. Our method can accurately detect most pest targets, which meets the needs practical application requirements.

Figure 3.

Some detection results of our model on Croppest12.

Table 4.

Ablation experiment.