This is a list of reviewers who served the Journal of Electronic Imaging in 2020.

Person reidentification (ReID) requires the discriminative features of an entire pedestrian image to handle the problems of inaccurate person bounding box detection, background confusion, and occlusion. Many recent person ReID methods have attempted to learn the feature information of an entire pedestrian image through parts feature representations, but it is often too time consuming. Person ReID relies on discriminative pedestrian features, and different spatial scales can distinguish features by differing degrees. We propose an innovative and effective adaptive spatial scale person ReID network model based on the residual neural network (ResNet) of an instance batch normalization. Through experimental visualizations, pedestrian features extracted by ResNet from four layers are analyzed, and two layers with discriminative features are selected. Using an adaptive dimension adjustment module, different spatial scale features are aggregated and merged by the aggregation layer. To effectively learn spatial channel correlations and prevent overfitting, a multilayer distribution normalization processing module is designed to implement end-to-end training and evaluate the person ReID networks. Compared with other methods, this network model showed excellent performance on four public person ReID datasets and is superior to most current methods.

We propose a robust license plate detection and recognition (LPDR) framework with automatic rectification. We explore the YOLOv2 object detector based on deep learning and train it to detect license plates (LPs) effectively. The LPs in natural scene images tend to be tilted and distorted because of the shooting angle or the geometric deformation of LPs. To solve the problem in which the LP tilt and distortion affect recognition accuracy, we introduce a spatial transformation network with thin-plate-spline transformation and propose a neural network called inverse compositional spatial transformer network-hierarchical spatial transformer network (ICSTN-CRNN). ICSTN-CRNN can automatically rectify and recognize LPs. Furthermore, we manually supplement the LP character annotations in PKUData. Our LPDR method achieves satisfactory results on three datasets, including Chinese City Parking Dataset, PKUData, and application-oriented license plate. Through a series of comparative experiments, we prove that our method is more accurate than other advanced methods.

We propose to rely on the wave equation for the detection of repeatable keypoints invariant up to image scale and rotation and robust to viewpoint variations, blur, and lighting changes. The algorithm exploits the properties of local spatial–temporal extrema of the evolution of image intensities under the wave propagation to highlight salient symmetries at different scales. Although the image structures found by most state-of-the-art detectors, such as blobs and corners, occur typically on highly textured surfaces, salient symmetries are widespread in diverse kinds of images, including those related to poorly textured objects, which are hardly dealt with by current pipelines based on local invariant features. The impact on the overall algorithm of different numerical wave simulation schemes and their parameters is discussed, and a pyramidal approximation to speed-up the simulation is proposed and validated. Experiments on publicly available datasets show that the proposed algorithm offers state-of-the-art repeatability on a broad set of different images while detecting regions that can be distinctively described and robustly matched.

With modern science and technology development, vehicles are equipped with intelligent driver assistant systems, of which lane detection is a key function. These complex detection structures (either wide or deep) are investigated to boost the accuracy and overcome the challenges in complicated scenarios. However, the computation and memory storage cost will increase sharply, and the response time will also increase. For resource-constrained devices, lane detection networks with a low cost and short inference time should be implemented. To get more accurate lane detection results, the large (deep and wide) detection structure is framed for high-dimensional and highly robust features, and deep supervision loss is applied on different resolutions and stages. Despite the high-precision advantage, the large detection network cannot be used for embedded devices directly because of the demand for memory and computation. To make the network thinner and lighter, a general training strategy, called self-knowledge distillation (SKD), is proposed. It is different from classical knowledge distillation; there are no independent teacher-student networks, and the knowledge is distilled itself. To evaluate more comprehensively and precisely, a new lane data set is collected. The Caltech Lane date set and TuSimple lane data set are also used for evaluation. Experiments further prove that a small student network and large teacher network have a similar detection accuracy via SKD, and the student network has a shorter inference time and lower memory usage. Thus it can be applied for resource-limited devices flexibly.

Traditional super-resolution techniques are generally presented as optimization problems with variations in the choice of optimization methods and cost functions. Even for the overdetermined cases, the problem is ill-conditioned. The situation is worsened when considering underdetermined cases with unknown regions due to occlusions or lack of data. Deep learning-based methods have shown promise in solving a similar problem. One recent advancement has come in the form of partial convolutions, which were developed to perform infilling of holes in images. When used in an appropriate deep neural network, this particular variant of the convolutional filter has shown great promise in approximating missing spatial information. The method described is formulated as a two-stage process. Lower resolution images are first registered and placed on a high-resolution grid. The problem is then treated as an in-painting task where the missing regions are reconstructed using a deep neural network with partial convolutional filters. We compare our method against deep learning-based single image super-resolution methods and classical multi-image super-resolution techniques using two similarity metrics and show that our method is more robust to occlusions and errors in registration while also producing higher quality outputs.

We propose the fractional-order total variation (TV) algorithm with nonlocal self-similarity for image reconstruction in compressed sensing to alleviate texture details deterioration and eliminate staircase artifacts, which results from the TV algorithms. The Grünwald–Letnikov fractional-order differential operators, which consider more neighboring image pixels and use four different directions to handle fractional-order gradients, are used to replace the integer-order differential operators. To suppress the staircase artifacts, modified nonlocal means operators are introduced into our method, which can contain prior image structural information and update the Lagrangian multipliers. An efficient augmented Lagrangian algorithm is used to solve the TV problem. Numerical results show that the algorithm achieves remarkable performance improvements at various sampling ratios. Compared with fractional-order TV-based projections onto convex sets, the maximum gains of peak signal-to-noise ratio and structural similarity index with all images are up to 2.52 dB and 0.0178, respectively, and the algorithm performs the better for preserving details and eliminating the staircase effect at the cost of taking more time.

Geospatial object detection in remote sensing images is a challenging subject since objects in remote sensing images are dense, multioriented, and multiscale. We present an attention network for object detection in remote sensing images. Through channel attention and spatial attention, the framework pays more attention to important channels and emphasizes position information of objects. Meanwhile, saliency learning is proposed to enhance objects information. Furthermore, saliency loss is added to the loss function to guide network learning in the training stage. In addition, multiscale feature module is added into the network to capture scale variations. Experimental results on public remote sensing image datasets validate the effectiveness of the proposed method.

Image encryption plays an essential role in the community of image security. Most existing image encryption approaches adopt a permutation–diffusion scheme to permute pixel positions and change pixel values separately. One limitation of this scheme is that it has a high risk of being cracked. To solve this problem, we propose an approach that jointly permutates and diffuses (JPD) the pixels in a color image for encryption. First, a 4D hyperchaotic system with two positive Lyapunov exponents is used to generate a sequence for almost all encryption procedures. To enhance security, the plain image’s information is introduced to the hyperchaotic system’s initial parameters. Then, the hyperchaotic sequence is used to permute and diffuse the pixels in images jointly. More specifically, two index matrices that determine which pixels will be permuted and diffused and one mask matrix that determines how the pixels will be diffused are generated by the hyperchaotic sequence. We test the proposed JPD with several popular images. Experimental results and security analysis demonstrate that the JPD is capable of resisting various types of attacks. Moreover, the JPD can accelerate the encryption process. All these indicate that the JPD is effective and efficient for color image encryption.

Aiming at the problems of color distortion, nonuniform illumination, and low contrast caused by degradation of underwater images, an underwater image enhancement method (MSFF-GAN) based on generative adversarial network was proposed. A multiscale featured fusion generator is designed, which improves the ability to use different scale features of the model and ensures that the generated image retains more detailed information. The residual dense module is constructed to solve the problem of generator characteristics extracted slower. In the discriminator, to achieve the extraction of local image features, the output matrix is discriminating so that the generated image is closer to the real image. Compared with the existing underwater image enhancement methods qualitatively and quantitatively, the proposed method has better enhancement effect on EUVP and RUIE datasets. The proposed method is superior to the contrast method of three evaluation indexes: PSNR, SSIM, and UIQM.

Multispectral image compression can considerably reduce the volume of data and promote their application. However, conventional single-scale compression schemes, such as JPEG2000 and three-dimensional set partitioning in hierarchical tree (3D-SPIHT), do not accurately preserve the features of images due to the complex features of multispectral images. A compression framework based on adaptive multiscale feature extraction with a convolutional neural network is proposed. First, an adaptive multiscale feature extraction module, which is the basic component of the compression framework, is designed to extract the multiscale spatial–spectral features of the multispectral images and adaptively adjust the weights of the features according to the content of the images. Second, the encoder, which is composed of multiscale feature extraction modules, extracts the multiscale spatial–spectral features of the multispectral images, and the extracted features are quantized and encoded by the quantizer and the entropy coder to generate a compressed bitstream. Third, the decoder, which is structurally similar to the encoder, is utilized to recover the images. The rate-distortion optimizer is embedded in the encoder to control the trade-off between the rate loss and the distortion. The results of these experiments on multispectral images of the Landsat 8 satellite and the WorldView-3 satellite validate the better performance of our compression framework compared with the performances of conventional schemes, including JPEG2000 and 3D-SPIHT. In order to further verify the effectiveness of multiscale features, the framework is compared with a single-scale compression algorithm based on deep learning, the experimental results validate that the performance of the single-scale compression algorithm superior to the conventional schemes but inferior to our multiscale algorithm, which indicates that the multiscale features can significantly improve the performance of the compression algorithm.

In recent years, a number of detectors have been proposed to improve the accuracy and speed of object detection tasks. However, poor detection performances for small objects and difficulties in optimizing deep networks remain critical challenges for object detection. We try to tackle these problems in two ways. First, we propose an innovative cross-fusion block (CFB) module that can enhance the representational power of features for instances of small objects. In CFBs, high-level features with rich semantic information and low-level features from different layers at the same scale are cross-fused together. Second, we propose a periodic oscillation attenuation learning rate (POA_lr) that can effectively skip some purely locally optimal solutions in the training process to obtain better detection accuracy. Extensive experiments on PASCAL VOC and MS COCO datasets show that CFB and POA_lr can achieve higher detection accuracy while maintaining real-time processing speeds. The code will be made publicly available.

Lung cancer has caused the most cancer deaths in the past several years. Benign–malignant lung nodule classification is vital in lung nodule detection, which can help early diagnosis of lung cancer. Most existing works extract the features of chest CT images using the well-designed networks, which require substantial effort of experts. To automate the manual process of network design, we propose an attention-guided differentiable architecture search (A-DARTS) method, which directly searches for the optimal network on chest CT images. In addition, A-DARTS utilizes an attention mechanism to alleviate the effect of the initialization-sensitive nature of the searched network while enhancing the feature presentation ability. Extensive experiments on the Lung Image Database Consortium image collection (LIDC-IDRI) benchmark dataset show that the proposed method achieves a lung nodule classification accuracy of 92.93%, which is superior to the state-of-the-art methods.

Guided (joint) image filters play an important role in many computer vision and image processing applications. The main principle behind these filters is transferring the structural information from a guidance image to an input one. However, in practice, the structures between the two images are not always consistent. As a result, the filtering outputs become sensitive to outliers, which easily leads to texture-copying artifacts. Most recently, by relaxing the dependence on the guidance, static/dynamic (SD) filter overcomes the drawback effectively. With the SD strategy, this filter can jointly leverage structural information from the guidance and input. However, due to the locality of its regularizer, SD is prone to another deficiency, i.e., edge blurring. To tackle this problem, in our work, we extend SD filter to a nonlocal version [nonlocal static/dynamic (NSD)]. Particularly, a nonlocal regularizer is first established in a subspace transformed by partial least squares, which can better respect the unequal roles of the images. Then, to efficiently formulate the structural consistency between the two images (guidance and input), a novel joint term is plugged into the regularizer. Finally, an acceleration approach is designed to reduce the computational complexity induced by the nonlocal extension, which makes NSD achieve a comparable running time in practice. Thorough experimental results demonstrate that the proposed filter not only can avoid texture copy effectively but also can preserve edges powerfully.

Color-texture image segmentation remains a challenging problem due to extensive color-texture variability. Thus, the limited prior knowledge that is expressed by pairwise constraints can be exploited to guide the segmentation process. We propose a new semisupervised method by combining constrained feature selection and spectral clustering (SC) to perform color-texture image segmentation. The pairwise constraints are used by the constraint feature selection to choose the most relevant features among an available set of color and texture features. For this purpose, an innovative constraint score is developed to evaluate a subset of features at one time. A specific constrained SC algorithm involving the pairwise constraints is then applied to regroup the pixels into clusters. Experimental results on four benchmark datasets show that the proposed constraint score outperforms the main state-of-the-art constraint scores and that our semisupervised segmentation method is competitive compared with supervised, semisupervised, and unsupervised state-of-the-art segmentation methods.

Pedestrian occlusion, variations in the cross-view angle, and the appearances of pedestrians significantly hinder person reidentification (ReID). A dual attention and part drop network (DAPD-Net) for person ReID is proposed. The dual attention module enables the deep neural network to focus on the pedestrian in the foreground of a given image and weakens background perturbance. It can speed up learning and improve network performance. Feature maps in the part drop branch that we proposed are divided into multiple parts, one of them is randomly dropped, and the remainder are learned to obtain a feature that is robust against occlusion. Through part drop training, the antiocclusion ability of the network is effectively improved. The middle-layer branch is used, which help our network to learn mid-level semantic feature and promote capability of the system. These innovative modules can help deep neural network to extract discriminative feature representations. We conduct extensive experiments on multiple public datasets of person ReID. The results show that our method outperforms many state-of-the-art methods.

Transfer learning provides a useful solution to learn a new conceptual domain from few examples, which exploits prior knowledge from a related domain. We proposed a simple and yet effective transfer learning method for image classification that constructs an activation ensemble generative adversarial net (AE-GAN) to transfer knowledge from one dataset to another. The AE-GAN is mainly composed of three convolutional layers and adopts an ensemble of multiple activation functions. Experimental results on five benchmark datasets show that when only a few samples are available for training a target task, leveraging datasets from other related datasets by AE-GAN can significantly improve the performance for image classification with a small set of samples.

Visual target tracking is an important function in real-time video monitoring application, whose performance determines the implementation of many advanced tasks. At present, Siamese-network trackers based on template matching show great potential. It has the advantage of balance between accuracy and speed, due to the pre-trained convolutional network to extract deep features for target representation and off-line tracking of each frame. During tracking, however, the target template feature is only obtained from the first frame of the video in the existing algorithms. The tracking performance is completely depending on the framework of template matching, resulting in the independence of frames and ignoring the feature of inter-frame connection of video sequence. Therefore, the existing algorithms do not perform well in the face of large deformation and severe occlusion. We propose a long short-term memory (LSTM) improved Siamese network (LSiam) model, which takes advantages of both time-domain regression capability of the LSTM and the balanced ability in tracking accuracy and speed of Siamese network. It focus on the temporal and spatial correlation information between video sequences to improve the traditional Siamese-network trackers with an LSTM prediction module. In addition, an improved template updating module is constructed to combine the original template with the changed appearance. The proposed model is verified in two types of difficult scenarios: deformation challenge and occlusion challenge. Experimental results show that our proposed approach can get better performance in terms of tracking accuracy.

Image steganography has emerged as a method of hiding secret data within an image file to ensure the security of the transmitted data. In this study, we propose an architecture named MDRSteg to unobtrusively hide a large-size image in another image based on a residual neural network with dilated convolution and multi-scale fusion. The architecture consists of an embedding network to hide the secret image in the cover-image and a revealing network to reveal the secret image from the stego-image, both networks are made up of fully convolutional residual modules. The networks are jointly trained with a loss function which is the combination of chi-square distance (CSD) and mean-square error. The proposed MDRSteg are trained and tested on three datasets, Labeled Faces in the Wild, Pascal visual object classes, and ImageNet. Extensive experiments have been done and the experimental results suggest that the proposed model can not only hide a large size image in another image with good invisibility and large hiding capacity (24 bits-per-pixel), but also exhibits good generalization ability. The experimental results also show that dilated convolution, multi-scale fusion, and combined CSD loss function have positive effects on the delicate image steganography results and proves that the model is practically useful for many applications.

Road accidents continue to increase and cause intense fatalities. Studies based on manual and/or semiautomatic methods remain unable to reliably track the random behavior of pedestrians. Hence, we fill this gap by developing an automatic Bayesian and cognitive model (ABC model) for pedestrian behavior analysis. We propose a full and complete computer vision process composed of two images processing levels: (i) low-level for pedestrian and traffic features extraction to characterize crossing the street scenarios. (ii) High-level for data correlation and behavior recognition based on Bayesian model. Since computer vision sensors are usually unsure, we propose an innovative metric to introduce the detection uncertainty as Bayesian evidence in the Bayesian network decision level. Results highlight computer vision techniques’ potential to collect random and reliable road user’s data at a degree of automation and accuracy that cannot be feasibly achieved by manual or semiautomated techniques during the first time. During the second time, the Bayesian network structure provides very reliable decisions that can more completely characterize a person’s random behavior. In addition to that, quantifying the uncertainty of computer vision’s sensors as Bayesian evidence is an important contribution. In fact, the proposed tool will be an important contribution to deal with pedestrian behavior. Hence, the proposed process reliably addresses a person’s Bayes behavior and illustrates the pedestrian/environment causal relationship. The proposed ABC model is validated based on cross-street sequences integrating different scenarios and pedestrians’ behaviors every 1  /  10 time slice.