This PDF file contains the front matter associated with SPIE Proceedings Volume XXXXX, including the Title Page, Copyright information, Table of Contents, Author and Conference Committee lists.

Scene text recognition has attracted the attention of many researchers owing to its widely application. Numerous methods have been proposed in this field and achieved unprecedented success. However, most of the datasets and algorithms are designed for English scene text recognition while only a few researches focus on the Chinese scene text recognition. Furthermore, despite the higher processing speed and ability of working as language model, attention-based mechanism suffers from the misalignment problem. Triggered attention model is proposed to tackle these problems. Taking character image as input, A triggered attention-based encoder-decoder network that outputs character sequence is proposed in this paper. Furthermore, an encoder network that takes colorful images as input is implemented to extract deep representations of input images. Experiments on various datasets show that the Triggered attention method substantially outperforms the existing attention based methods.

Stacked hourglass (HG) networks have been successfully applied to face alignment. However, due to the complex geometry of the facial appearance, the HG model still lacks the robustness of aligning faces in large poses. In this paper, a two-step method is proposed for robust face alignment. First, by using a convolutional neural network (CNN) to directly output the transformation parameters, the conventional procedure of normalizing the face region by performing Procrustes analysis based on the detected landmarks and the mean shape is simplified. In this way, faces with different poses can be converted to a canonical state, which is more advantageous for subsequent face alignment. Second, motivated by recent deformable convolutional networks, we propose a modulated deformable residual block and replace the plain counterparts in the HG model, resulting in deformable hourglass networks (DHNs). The DHN yields large performance improvements over original HG model while having the almost same amount of parameters and bringing minor additional computation costs. Depending on the synergistic effect of two innovations, the proposed method achieves better performance in comparison to the state-of-the-art methods on challenging benchmark datasets.

Image set annotation is an important task in the supervised training of the deep neural network. Manual and data-driven dataset annotation methods are commonly used approaches. Both of them have shortcomings, especially in the case of the dataset requiring professional knowledge, which leads to high cost with manual annotation methods and poorly diversified annotation samples with data-driven annotation methods. Although the recommendation annotation method based on cosine similarity using deep neural network features takes advantages of manual annotation and data-driven method, there are still problems such as low accuracy and click-through rate. In order to improve the recommendation accuracy and click-through rate, we propose a confusion graph recommendation annotation method, which builds a confusion graph based on the Largest Margin Nearest Neighbor (LMNN) distance among deep neural network features, to recommend the most confusing images to annotators. In this paper, we made ablation studies on the self-built child face dataset in terms of Precision, mAP (mean Average Precision), and CTR (click-through-rate). The experimental results show that the proposed method achieves superior performance, compared with the cosine similarity recommendation annotation method and the manual annotation method.

In this research, we are studying on image-based identification of trees species that we can see everywhere. In our previous study, we showed that convolutional neural network (CNN) can recognize tree species by using a region of interest (ROI) image of bark. However, the bark region is manually extracted from a natural bark image. This paper solves this problem using semantic segmentation, and proposes an automatic tree species identification from natural bark image. The proposed method was evaluated with the bark image dataset collected independently. We confirmed the effectiveness of the proposed method.

With the rapid development of information technology, text recognition in natural scenes has become a hot topic of current research. In order to accurately and quickly identify the box number in the container image in the natural scene, this paper proposes a deep learning-based image text recognition model (Faster-RCNN and CNN with Attention (FRCA)), which consists of two stages: box number area detection and box number character recognition. We use the improved Faster-RCNN network to detect the location of the box number, which increase the attention mechanism in the area generation network (RPN) to speed up the detection speed while ensuring the accuracy. And we use the improved CNN to recognize the box number characters. The experiments on the benchmark dataset and the real dataset prove that compared with the connected region detection method, the Faster-RCNN and VGG-16 combination method, the FasterRCNN and ResNet-101 combined detection method, the accuracy of FRCA model in this paper is better than the former two schemes, and the speed of detection of FRCA network is faster than that of the second and third scheme due to the increase of attention mechanism.

In this paper, we propose a deep neural network based approach for the group-level emotion recognition in 6th Emotion Recognition in the Wild Challenge (EmotiW 2018). The task of this challenge is to classify a group’s perceived emotion as Positive, Neutral or Negative. Like the most of current researchers on visual emotion recognition, we mainly focus on facial, scene and body clues in images. We treat each clue as mono-model feature and apply early fusion method to combine them together. Experimental results show that our proposed method has outperformed the baseline techniques with the overall test accuracy of 62.90%.

This paper presents a non-iterative fuzzy neural classifier to improve the efficiency and training speed of the automatic face recognition system. Main issue of face recognition is that its performance deteriorates if there are variations like illumination, pose and expressions. In the proposed classifier, principal component analysis (PCA) is used for feature extraction that projects the face image into eigen space that best describes the data with reduce size of the database. These features are fed to non-iterative fuzzy neural classifier in which hidden neurons are randomly generated and output weights are calculated analytically in non-iterative manner. Fuzzy neural classifier offers fuzzy activation function which provides various normalization in face images. Experimental results on Yale Face database demonstrate that the proposed classifier performs well compared to the state-of-art recognition techniques in terms of recognition error rate and learning speed.

The Juxtapleural nodule regions are often missed in the result of lung segmentation algorithms. To tackle this problem, an identification method basing on the SIFT information and elliptic Fourier descriptor is proposed. Firstly, the SIFT information is used to locate the position of key points in the lung mask. Then with the help of distance relationship, the support borderlines of key points are calculated. Thirdly, the elliptic Fourier descriptor is introduced to describe a support line. Finally, an adaptive threshold is designed to decide whether the current support line is corresponding to an under-segmented region. Experiments on real CT images demonstrate that the proposed model provides an efficient way to perform under-segmented region identification task.

Image recognition is of great significance currently, especially the one based on RGB images. Recently, as a substantial amount of RGB based CNNs showing great performance on image recognition problems, the initialization problem sees an increasingly importance as is highly associated with the variance as well as the convergence effect. Even though there exists several prevailing initialization methods, they didn’t consider the difference of contributions from different RGB channels on image recognition. Thus, in this paper, we put forward a proper approach to take the RGB contribution into account. Firstly, we conduct experiments to prove that RGB channels have respectively different contributions upon RGB based image recognition by the means that training one general CNN and three CNNs under datasets seperated from CIFAR-10 by RGB channel. Then, the ensemble of those CNNs shows how similar performance as the general one and also their respective accuracies are recorded as the influence proportion, namely their contribution ratio. As long as the influence proportion is obtained, according to that, we can initialize the weights of convolution filters in order to improve the effect of initialization.

In previous works, the channel attention mechanism has been widely used in person re-identification. However, the channel attention mechanism completely compresses the spatial dimension during calculation, which harms the diversity of the channel information over different pixels. In this paper, a channel convolution residual block is proposed for more detailed inter-channel correlation modeling. First, we preserve spatial context information when introducing the channel dependency, which enables pixel-wise inter-channel correlation modeling. At the same time, a bottleneck strategy is used to reduce parameters in the spatial dimension. Second, the channel convolution instead of the fully connected layer is employed to reduce the parameters in the channel dimension. In addition, the inter-channel correlation is merged into the backbone network directly in the form of residual, and thus the block can be embedded in any deep neural networks. Experiments on Market1501 and DukeMTMC-ReID datasets demonstrate that the channel convolution residual block improves the accuracy of person re-identification task effectively.

Spatial attention mechanism is widely used to extract local feature in person re-identification. However, some existing multi-stage spatial attention structures lack flexibility and require complicated training process. In this paper, a plug-and-play LSTM-based Attention Module(LAM) is proposed to enhance flexibility of the multi-attention mechanism. First, we employ the single-stage multi-attention structure to replace the traditional multi-stage multi-attention structure. Our structure encapsulates multiple attention machines in single module and thus the module can be added to any backbone networks without any modification directly. Then, correlation is introduced to spatial attention machines through LSTM. Correlation between different attention machines preserves diversity of the local feature and exploit the capacity of multi-attention mechanism. Moreover, the LAM is added to the backbone network in the form of residual, which enables the LAM to be trained with the backbone network synchronously. Therefore, the training process is simplified effectively. Experiments on CUHK03, Market-1501 and DukeMTMC-ReID datasets demonstrate the advantage of the proposed method.

In recent years, face recognition has achieved high accuracy and has been widely used in many fields. However, the performance of face recognition is not satisfied in some unrestricted situations such as in monitoring scenarios. The light field camera, as an image acquisition equipment, can capture images before focusing them and allows us to achieve clearer images. However, the existing face recognition algorithms based on light field camera need to search the optimal face manually. In this paper, an automatic algorithm for optimal face search is proposed. With this search algorithm, a new face recognition framework based on light field camera is proposed. Two face recognition algorithms are used to test the proposed framework. The experimental results show that the proposed methods combined with the light field camera can achieve better results than with conventional camera and the whole framework does not need extra manual operation.

The purpose of image activity recognition is the understanding of human visual behavior presented in images, and marking the identified activity picture with a pre-defined category label. Still Image activity recognition plays an important role in the field of image recognition. But now, the performance of activity recognition has not reached satisfactory results. In this paper, we construct a hybrid recognition model combined with Region-CNN (RCNN), AlexNet and SVM (or Random Forest model), which can effectively improve the performance compared with each single method. Firstly, the main object regions in an image are extracted using RCNN, then for each object region, the AlexNet is implemented to extract features, finally, all these object features are concatenated together and construct a SVM classifier (or Random Forest classifier) for still image activity recognition. The experimental results on still image dataset including 40 activity categories show that the hybrid model achieves better performance compared with single CNN model and other traditional methods. The AlexNet model only achieves an accuracy of 69.48%, the hybrid model of RCNN, AlexNet and SVM achieves an accuracy of 75.48%, and the hybrid model of RCNN, AlexNet and RF even reaches 78.15%, which verifies the effectiveness of our method.

This paper aims to study the problem of text recognition in complex natural scenes. Now the rapid development of technology makes most of the industry need fast and efficient text recognition tools, such as the logistics industry and publishers. Although traditional OCR technology can solve most simple background text recognition problems, the performance of complex scenes is somewhat unsatisfactory. In this paper, the focus is to extract the text in complex scenes accurately. The algorithm of stroke width transform and Convolution Neural Network is proposed. Using the stroke width transform algorithm for text positioning; And then separate the text line, end to the classifier to identify, through the recognition of the correct rate determine the effectiveness of the algorithm. We evaluate the proposed approach on a widely used dataset. Results show that our method achieves desired result compared with the state-of-the-art methods of the participant team which clearly demonstrate its competency.

Saliency detection model has been widely used in many fields of computer vision. Currently, most models are not applicable to the foggy scenario. Because these models are highly dependent upon high-level features extracted by deep learning and handcrafted features, these features cannot effectively highlight the significant targets in foggy images. In this paper, we present a saliency detection model for foggy images. This model extracts non-local feature, i.e., jointly learned with local features under a unified deep learning framework. The key idea of the proposed model is to hierarchically introduce non-local module with local contrast processing blocks, aiming to provide robust representation of saliency information towards foggy images with low signal-to-noise ratio property. Experiments have been conducted on three challenging datasets and our foggy image dataset consisting of dynamic object images. By comparing with the state-of-the-art models, our model gets better performance.

In the past few years, Generative Adversarial Network (GAN) became a prevalent research topic. GAN has ability to generate good quality images that look like natural images from a random vector. In this paper, we follow the basic idea of GAN and propose a novel model for image saliency detection, which is called Supervised Adversarial Networks (SAN). However, different from GAN, the proposed method uses fully supervised learning to learn both G-Network and D-Network by applying class labels of the training set. Moreover, a novel kind of layer call conv-comparison layer is introduced into the D-Network to further improve the saliency performance. Experimental results on Pascal VOC 2012 database show that the SAN model can generate high quality saliency maps for many complicate natural images.

Targets such as plastic bags can absorb a lot of high-energy lasers which fired from the laser in a long distance. By aligning the laser center with the optical center of the high definition camera, we can determine that the laser emission point is located at the center of video. We can adjust the angle and position of the camera to make the video center coincide with the target, and finally accurately clean up the target from a long distance. In the process of video, this paper uses the target tracking algorithm based on template and the abnormal target detection method based on inter-frame difference. After we lock the target to be cleaned in a frame, the system can automatically calculate the angle and distance if the target swings in the wind, and send the corresponding motion command to the rotating device which can drive the camera to move, finally realize the target tracking and cleaning. Abnormal target detection algorithm is used to monitor the designated area near the target while emitting high-energy laser. When abnormal target such as human is found to enter, protection mechanism is triggered in time to turn off laser which can prevent dangerous accidents.

Due to environmental illumination and other factors, actual shearing interference image used to measure the rotation angle contains high-frequency speckle noises. According to the characteristics of shearing interference images, a low-pass filter based on homomorphic filtering principle is selected to filter these images. The desired filtering result can be achieved by adjusting the filter parameters such as the cutoff frequency radius and the order of the filter. The Hilditch thinning algorithm is applied to extract the center of the fringes to calculate the spacing between neighbor fringes. The accuracy of the algorithm is verified by the actual and simulation experiments with the error of fringe spacing detection less than 0.01%.

In order to ensure the safety of rail transit, detecting the flaws on the rail surface is vitally important. Instead of present manual inspections, detecting defects on rail surface by an automatic approach enables the work more efficient and safe currently. In this paper, we propose a novel two-stage pipeline method for defect detection on rail surface by localizing rails and sliding a deep convolutional neural network (DCNN) on rail surface. Specifically, in the first stage, we use an anchor-free detector to locate the tracks in original images and get the cropped images which focus on rail part. In the second stage, a trained deep convolutional neural network slide on the cropped images to detect defects and we can finally get the types and approximate locations of the defects on rail surface. The experimental results show that the proposed method has robustness and achieves practical performance in defect detection precision.

Object detection is one of the most basic and central task in computer vision. Its task is to find all the interested objects in the image, and determine the category and location of the objects. Object detection is widely used and has strong practical value and research prospects. Applications include face detection, pedestrian detection and vehicle detection. In recent years, with the development of convolutional neural network, significant breakthroughs have been made in object detection. This paper describes in detail the classification of object detection algorithms based on deep learning. The algorithms are mainly divided into one-stage object algorithm and two-stage object algorithm, and the general data sets and performance indicators of object detection.

Cloud cover is one of the main factors affecting the quality of remote sensing images, and cloud detection is the first problem to be solved in remote sensing data production. Therefore, cloud judgment becomes one of the primary tasks and key technologies of remote sensing image processing. The main detection objects include thin cloud, thick cloud and cloud shadow. The main interference factors include snow, Gobi, desert, reflective buildings and other highlights. Firstly, the high reflectance characteristics of clouds are used to analyze the distribution of near-infrared and red bands in multispectral remote sensing images to realize preliminary cloud detection. And then using the detected suspected cloud region, different morphological features to distinguish the cloud from other highlight features, and the higher cloud detection accuracy in high-resolution satellite images is achieved.

In the production process of hot rolling strip, wave-shaped defects appear at the edge of strip, which affects the quality of steel and the interests of steel enterprises. In this paper, the method of identifying and calculating the wave-shaped defects on the working side of strip steel based on the convex hull detection algorithm is discussed. Firstly, the classic Graham's Scan algorithm is used to detect the processed strip images. Then, the accuracy of the algorithm is discussed, and the error judgment of wave shape defects is analyzed. Finally, the improved Graham's Scan algorithm was used to detect the wave-shaped defects of strip steel again. The experimental results show that the improved algorithm can significantly reduce the misjudgment and has high utilization value in the actual production process. The actual problem is solved by processing the image.

L₁ loss function and Intersection over Union (IoU) are commonly used in object detection. However, minimizing the loss function through training process does not necessarily amount to maximizing IoUs. L₁ loss simply assigns equal weights to difference of the width, height, and center point between a prediction box and a ground truth box but pays less attention to the contribution of each shape property. Observing this, we propose scaling loss which can be easily embedded in convolutional neural networks for mitigating the gap between IoU and loss function. The key insight is to add in the loss function the adaptive weights for width, height, and center point that encode the shape properties of the bounding box. The contribution of each shape property will be adaptively adjusted according to the difference between a prediction box and a ground truth box, i.e. increasing the weight assigned to the bad-regressed shape property. By this means, the scaling loss is able to obtain more accurate prediction box. The proposed scaling loss was embedded in Faster R-CNN and SSD, and was validated on PASCAL VOC 2007. Experimental results verify that the proposed scaling loss can improve the detection accuracy over the smooth L₁ loss and Softer-NMS.

Research on characteristics of helicopter rotor blade tip vortex (BTV) is one of the key elements for helicopter rotor aerodynamic characteristics research. The existing traditional computational fluid dynamics (CFD) based detection methods of vortex core area in the flow field mainly use points or lines in the flow field for calculation. However, the traditional CFD-based model is complex, huge computational cost and without effective vortex core model. So the manual analysis will be necessary in some scenarios to simplify the work of vortex detection, such as vortex region detection for helicopter rotor BTV in domestic. In order to decrease the workload of manual analysis, we draw on the advanced research results in the field of computer vision and machine learning, especially target detection, and firstly propose a vortex region detection method in blade tip vortex based on You Only Look Once (YOLO) network. First of all, the vortex region is marked in flow field images under the guidance of domain experts to construct the vortex data set. Secondly, we propose an improved model based on yolo v3-tiny. Finally, the self-built vortex data set is used to train models. Experiments show that the CNN-based method has better result than traditional methods.

A method based on ultrasound B-mode images clustered with the Gaussian Mixture Model (GMM) was proposed to improve the detection accuracy of intima and media thicknesses in carotid arteries. Firstly, the B-mode images were clustered based on the GMM, and the boundary between the intima and media of the vessel wall is then detected by the gray threshold algorithm to estimate the thicknesses of the inner and middle membranes. With 28 healthy subjects, the thicknesses of intima and media based on GMM cluster processing were (0.1675±0.0164) mm and (0.2278±0.0208) mm, respectively. Compared with the results based on original B-mode images of (0.1742±0.0208) mm and (0.2211±0.022) mm, the standard deviations based on GMM images were reduced by 21% and 5%, respectively. Compared with the results finely measured by two professional doctors, the mean error of intima and media thickness based on GMM images decreased by 7.5 % and 8.7%, respectively. The proposed method is helpful to provide the accurate diagnosis information for cardiovascular diseases in early stage.

In this paper, we present a new one stage detector for object detection. In order to meet the requirements of real-time detection, we use MobileNetV2-FPN as backbone for feature extraction. The lightweight depthwise separable convolutions can improve the speed of detection and make the model smaller. In order to improve the accuracy of our small detection model, we add Stem Block into backbone and we add SEnet in front of two task-specific subnets. The stem block can reduce the information loss from raw input images. The SEnet can enhance useful features from backbone network and suppress features that are little use to two-specific tasks. Inspired by RetinaNet, we also use Focal Loss as our classification loss function. We measure our performance on PASCAL VOC2007 and PASCAL VOC2012. Our detector with 300300 input achieves 73.8% mAP on VOC2007 test, 71.4% mAP on VOC2012 test. And our detector can run at 97FPS and the number of parameters is only 7.7M that meets the requirements of real-time detection. The accuracy of our detector is close to SSD, our detector uses about only 1/3 parameters to SSD. Keywords: One-

Multi-temporal Earth remote sensing images of the same territory may include random scene-distortions coming from the different natural phenomena, for example, clouds or shadows. These distortions are time dependent and may appear only in several images of the analyzed image set. Thus, they define irrelevant image parts that should be eliminated in the further data fusion process. In this article, we suggest an algorithm for detecting such scene distortions using a series of multi-temporal remote sensing images. The algorithm is based on super-pixel segmentation and anomaly detection methods. The algorithm produces a mask of random scene-distortions for each of the images in the analyzed series. The resulting mask could be used to take into account only the scene-relevant parts in the data fusion methods. The proposed approach allows processing images with different spectral and spatial sampling simultaneously that is very useful for multi-sensor data fusion. We tested an algorithm quality by modeling a series of multispectral images with different parameters of spectral and spatial sampling under the different conditions of cloudiness and cloud shadows as an example of random distortions in the scene. As a result, it was shown that the algorithm provides the accuracy of scene distortion detection about 90% and false detection about 10%.

For satellite remote sensing image, ship detection is affected by factors such as cloud, weather and sea clutter, and there are problems such as high false alarm rate and missed detection rate. A deep neural network (DT-YOLO) for real-time detection of surface ships in unmanned aerial vehicle (UAV) aerial photography is proposed. DT-YOLO firstly improves the traditional YOLOv3 algorithm by constructing a deep neural network using densely connected modules and transition modules to extract ship features and designs five different scale convolution feature maps. After upsampling, they are merged with the corresponding scale feature maps to form a multi-scale feature pyramid to perform ship prediction tasks. Then the k-means algorithm is used to cluster the target frame dimensions, determine the target frame parameters, and improve the positioning accuracy of the model; In the detection network, the non-maximum suppression algorithm (NMS) is optimized by attenuating the confidence score linearly, which effectively mitigates the problem of mutual occlusion detection of ships. Finally, the self-constructed sea surface ship dataset is established to test the performance of the algorithm in different scenes by using multi-scale training and data enhancement strategies. The results show that the proposed method improves the detection accuracy under the premise of ensuring real-time performance, especially in the detection of larger targets and mutually occluded ships.

Linear discriminant analysis (LDA) is one of the most popular methods for dimensionality reduction, and there have many different variants based on LDA. However, conventional LDA have some drawbacks: 1) The projection matrix offers the disadvantage of interpretability; 2) LDA assumes that every class data are drawn from Gaussian distribution which may not be applicable to many real-world data. Aiming to solve these problems, we propose a robust feature selection method, namely Self-weighted Locality Discriminative Feature Selection (SLD-FS), by combining row sparsity l_2,1-norm regularization and self-weighted locality discriminant analysis strategy. Additionally, an effective iterative algorithm is developed to optimise this objective function, and the algorithm is proved to convergence. Extensive experiments conducted on various data sets demonstrate the effectiveness of SLD-FS when compared with some state-of-the-art supervised feature selection methods.

In this paper, a dilated convolutional neural network is proposed for hyperspectral image classification. Compared with other methods, 2-dimension dilated convolution is used for the first time to extract and classify the spatial-spectral features in hyperspectral image processing fields. Firstly, 1-dimension convolution is extended to 2-dimension convolution for spatial-spectral features extraction. Secondly, a dilated convolutional structure is utilized to fuse the multi-scale information, which is used to extract the multi-scale information without loss of resolution. The experiments of University of Pavia were repeated with the method proposed in this paper, and some better results are obtained, which proved the effectiveness of the proposed model.

Owing to the great adaptability to the weak light, the infrared camera equipped on the unmanned aerial vehicle (UAV) is more and more utilized to capture the aerial images. Therein, the registration of the aerial infrared images play a vital role in the infrared image processing, such as object tracking, information collecting, and behavior analysis. However, due to the low contrast, low-resolution, and few texture features of the infrared images, it is really difficult to implement the image registration in the aerial infrared images. In our work, an accurate and efficient image registration algorithm in the aerial infrared images is put forward. To improve the registration accuracy and efficiency, we construct multiple screening mechanisms to screen out the incorrect and redundant feature points. Besides, to further improve the registration accuracy, the improved SIFT and convolution neural network (CNN) descriptors are combined for the feature description. Finally, the coarse-to-fine feature matching strategy is put forward via FLANN and RANSAC algorithms. Experiments and comprehensive analyses demonstrate that the proposed algorithm generates satisfactory and competitive registration results.

Objective of camera calibration is to provide a transformation between object space and image space in practical computer vision tasks. Thus, in order to acquire the accurate and reliable information on visual image, intrinsic and extrinsic parameters of camera should be precisely calibrated. Compared with traditional methods and active vision based methods, self-calibration methods are not rely on prior information of visual scenes, which have more widely application prospects. However, in spite of self-calibration has been studied for years, and multitudes of ideas have been proposed, current research difficulties on self-calibration methods still lies in calibration precision. This paper put forward a camera self-calibration method only using linear features from targets, which could calibrate camera parameters in a highly accuracy. Moreover, an experiment is implemented to verify the effectiveness of our method, simultaneously, the experimental result illustrate that the proposed method has a highly calibration precision.

In view of the fact that multiple complementary feature representation can effectively improve the performance of image retrieval, this paper proposes a new texture image retrieval method based on statistical distribution feature fusion in dual-tree complex wavelet transform domain. Firstly, the statistical distribution energy of the coefficients is calculated in the low frequency subband. Then, in the high frequency complex subbands, the magnitude coefficients are modeled as the Weibull distribution and the relative phase coefficients are modeled as the von Mises distribution. Furthermore, the distribution energy and the estimated model parameters are fused into new features. Finally, the similarity measurement adopting optimal weighted sum is used to retrieve the texture images in the VisTex database. The experimental results show that, compared with the existing texture image retrieval approaches, the proposed method has a higher average retrieval rate.

Automatically generating the description of an image is a task that connects computer vision and natural language processing. It has gained more and more attention in the field of artificial intelligence. In this paper, we present a model that generates description for images based on RNN (recurrent neural network) with multi-feature weighted by object attention to represent images. We use LSTM (long short term memory), which is a RNN model, to translate multi-feature of images to text. Most existing methods use single CNN (convolution neural network) trained on ImageNet to extract image features which mainly focuses on objects in images. However, the context in the scene is also informative to image captioning. So we incorporate the scene feature extracted with CNN trained on Places205. We evaluate our model on MSCOCO dataset based on standard metrics. Experiments show that multi-feature performs better than single feature. In addition, the saliency weight on images emphasizes the salient objects in images as the subject in image descriptions. The results show that our model performs better than several state-of-the-art methods on image captioning.

The fully convolution network is a very powerful visual model that can be used to extract features in an image. We improved a network model that can be used for end-to-end, pixel-to-pixel training to extract target motion trajectories in infrared images. The dataset used in our training comes from the simulation dataset produced by the public infrared dataset combined with the simulation trajectory. In order to enhance the model’s robustness, we add the pepper and salt noise and white noise to the simulated image, and use image augmentation to increase the number of the image. We achieved highly train and test accuracy in our simulation dataset.

This paper proposes a deep convolutional neural network-based low-light image enhancement method. In order to adaptively enhance the image brightness, a convolutional neural network with convolutional modules is designed. Lowlight image is firstly down-sampled into sub-images. Then an illumination map is obtained from the input image to provide additional information to the network. The network works on a tensor that consists of sub-images and illumination map, achieving a good performance in brightness increasing and structure preservation. The enhanced result is reconstructed from the output sub-images. Experimental results demonstrate the effectiveness of the proposed method in low-light image enhancement.

Narrowband image extraction is an essential preprocessing for multispectral imaging of filter array. This paper proposes an automatic method to extract narrowband images from a multispectral image. During offline phase, the method first employs horizontal and vertical projections to obtain initial positions of subsampling points. Then, integerpixel searching and half-pixel searching optimizes subsampling location. During on-line phase, 1:4 or 1:16 interpolation generates interpolation grids. Finally, four narrowband images are obtained according to the optimized subsampling points and interpolation grids. The experimental result indicates that this method effectively extracts narrowband images from originally mosaic images of multiple spectrums.

Compared with grayscale images, color images with color information have more advantages in target recognition, detection and tracking, but the color of the image can be easily affected to be distorted by light. Aiming at solving the problem of color distortion, a method based on wavelet transform is proposed to eliminate the influence of illumination. Firstly, the V-channel components in HSV space are decomposed by wavelet and the illumination component in the image scene is estimated by reconstructing the approximate coefficients of wavelet decomposition. Secondly, according to the imaging principle model, the illumination component is removed from the original image, and the reflection component that characterizes the color information is preserved. Finally, the normalized distribution function is used to normalize the reflection components, and the image color recovery is achieved in RGB space. The experimental results show that the proposed method can reduce the influence of non-uniform illumination on the image to restore the color and improve the image quality.

In this paper, the AMBTC image compression method is proposed which employs three alternative weights to predict the initial center. AMBTC is a simple but efficient compression algorithm, whereas in terms of the image with salt and pepper noise, the visual quality is not satisfying provided that the mean value is used as the threshold in each 4×4 block. The possible reason accounting for this drawback is that image pixels are modified while the mean value can only best represent the original image data rather than noise data. On the contrary, the median value is more proper to represent noise data alone, which indicates that the mean value alone is unable to properly represent the whole image added with noise. That is to say, when the weighted sum of the mean value and the median value is taken as the initial center, the deviation of image added with salt and pepper noise will be reduced. What’s more, noises with various intensities are added to each image in the standard database, which intends to search for the optimal center in each block, and make a mapping between intensities and weights. It is feasible for us to directly use three weights and analyze the noise intensity to predict the initial center. PSNR of an AMBTC image is improved with 1 dB on average. In order to further improve the visual quality, we search for the optimal center in the confidence interval around the initial center so that PSNR can be improved by 3 dB. The experimental results reveal that the proposed fixed-point prediction compression algorithm enhances the image quality and reduces the computational cost to a large extent.

Due to the limitation of the communication system resources, the compression is required to reduce the complexity, the required storage, and the processing time. In traditional compression techniques the signal is sampled according to Shannon-Nyquist theory. Consequently, the complexity of signal processing and the encoder/ decoder increases. In this paper, we study the utilization of Compressive Sensing (CS) to solve this challenge. Compressive sensing considers as an under sampling signal processing technique which can sample the signal with sampling rate under the ShannonNyquist rate and grantee the proper recovery of the signal. The performance of CS is evaluated under different compression ratio. Additionally, we utilize reconstruction compressive sensing techniques including -minimization ( ), 2-minimization ( ), Orthogonal Matching Pursuit (OMP), Compressive Sampling Matching Pursuit (CoSaMP), and Generalized Orthogonal Matching Pursuit (GOMP). The performance is evaluated in terms of peak signal to noise ratio (PSNR), Correlation (CORR), Mean Square Error (MSE), Energy ratio (ER), Structural Similarity Index (SSIM), Dissimilarity Index (DSSIM) and Recovery Tim (RT). That is to stand on the best recovery technique that suits image transmission over communication systems.

SAR image registration is a key step in SAR data processing, and it plays an important role in change detection, image mosaic fusion and target recognition. The traditional rigid registration model can only realize the rotation and translation transformation of a whole image. However, for SAR images with coherent speckle noise and complex deformation, the local transformation result can’t meet the accuracy requirement of the later data processing. In this paper, the existing SAR image registration method is improved. After finishing SAR image feature matching by means of SURF and RANSAC algorithms, we construct an image conversion model with thin plate spline, which can decompose the space transformation of SAR image into global affine transformation and local non affine transformation. In this way, the whole and local mapping transformation of images can be realized simultaneously without the local distortion. The experimental results show that the improved method can achieve better transformation and obtain higher registration accuracy in the whole and local processing of SAR images.

Steganography is an art of hiding secret information by embedding it in a medium like an image, thus it secures the information from unauthorized access of confidential information. This paper presents an image steganography technique using a Novel Puzzle. The solution of the puzzle is expanded and used as the reference matrix in determining the embedding location of the secret message in the cover image. Experimental results show that the resemblance between the stego-images and cover images is high. Having an average PSNR value of 48.64db and average SSIM value of 0.99965 the chances that the stego image will be recognized with the use of Human Visual System (HVS) is small. The proposed technique is robust against Salt and Pepper Noise attack with the simulation result of 95.87% accuracy.

Subspace clustering, which aims at yielding a low-dimensional structure of high-dimensional data, is a fundamental clustering problem. Sparse subspace clustering (SSC) achieves state-of-the-art clustering performances by imposing sparse constraint on the coefficient matrix. However, most SSCs do not exploit the intrinsic relationship or prior information embedded in data. In this paper, we propose Laplacian embedded sparse subspace clustering, in which the intrinsic relationship of data is enforced by introducing a graph Laplacian regularization term into the clustering model. Then a symmetric constraint is imposed on the sparse representation to guarantee weight consistency for each pair of data points. To further offset the instability and control smoothness, a consistency penalty term is utilized to encourage the sequential property of data. Finally, the inexact augmented Lagrange multipliers (ALM) technique is adopted to solve the optimization problem. Experimental results on real-world data sets demonstrate the superior performance of the proposed algorithm over state-of-the-art methods.

Single image dehazing has become a hot issue in image processing. Almost all the dehazing algorithms suffer from two problems: bringing out the halo artifacts or high computational complexity. In this paper, we propose a novel single image dehazing algorithm with a modified criterion based segmenting dark channel prior (SDCP), which is simple and effective. Based on the estimated global atmospheric light, the transmission map (TM) is obtained by a novel region segmenting logic. Then, the TM is smoothed by guided image filtering (GF) with the edge information being preserved. Experiments reveal that the proposed algorithm eliminates the halo artifacts more effectively than SDCP or GF based dark channel prior (DCP) both subjectively and objectively. Also, it is faster than the soft matting based DCP.

Retinal diseases are recurrent these days and there are many ways to scan the fundus of the eye in order to diagnose the disease. The most common type of retinal disease are Diabetic Maculopathy and Pediatric Malarial Retinopathy. Some previous studies showed that the leakage detected in retinal angiography can point to the existence of the disease. Hence, in this report, a saliency-based technique is proposed in order to detect the leakage regions in a fluorescein angiography. First, the image of a fundus is divided into intensity-based clusters of pixels using super pixels and then a saliency map is estimated. For each level of the super pixel, a compactness and intensity-based saliency cues are computed. An averaging operator is used to combine all the saliency maps obtained for each cue. Later, the final saliency map of each cue is combined using a multiplication operator to give an output of a single saliency map image. The leakage regions are finally segmented using thresholding and graph-cut segmentation techniques. It can be seen from the final result that compared to the previous techniques in leakage detection, the method proposed in this paper yields better results. The principle target of this paper is to make and approve programming to consequently segment leakage area in true clinical Fundus fluorescein angiography (FFA) images of subjects with diabetic macular edema (DME). Our segmentation strategy can reproducibly and precisely evaluate the area of leakage of clinical evaluation FA images and is compatible with master manual segmentation. Of an estimated 285 million individuals worldwide with diabetes mellitus, approximately one third have signs of Diabetic Retinopathy (DR) and of these, a further one third of DR is vision-threatening DR, including DME.

Brain tumor segmentation is an essential task in image based brain tumor analysis. Many deep learning based (DL) brain tumor segmentation methods have been proposed. They usually segment the brain tumor into three different sub-regions, i.e., the edema (ED), the enhancing tumor (ET) and the necrotic (NCR) from input multi-modal brain images. Some of them treat the brain tumor segmentation as a multi-label classification task, where voxels in the input brain images are classified into one of the four classes (i.e., 3 sub-regions + 1 background) simultaneously. Currently proposed methods decompose the multi-label classification task into separate binary classification tasks, each of which is responsible for classifying one of the three sub-regions. Since binary classification is easier to be optimized than multi-label classification, more accurate segmentation can be achieved by binary classification. However, most of binary classification based methods are not end-to-end, i.e., each DL model is trained to segment one tumor sub-regions separately, and correlations among them are ignored. To solve this issue, in this paper, we propose an end-to-end cascaded bootstrapping DL model, which considers the correlations among different binary classification tasks. Our model contains three levels, through which the whole tumor (WT), the tumor core (TC) and ET are segmented in a cascaded way. More important, each level estimates the corresponding segmentation using the features learnt from the previous level. Moreover, a multi-view fusion is adapted to further boost the segmentation performance. We evaluate our method using BRATS 2017 dataset, and the experimental results show excellent performance on the BRATS 2017 validation set.

Recently, semantic segmentation which requires to recovering all detailed information of the original image has achieved significant improvement. In this work, we utilize skip-layer, multi-scale context module, and encoder-decoder structure to perform the task of semantic image segmentation. To handle the problem that objects with different scales, we adopt the multi-scale context information module with different convolution filters to capture diverse range information in the encoder network. Furthermore, we utilize the skip-layer to fuse semantic information produced by a coarse layer and appearance information generated by a fine layer to recover more precise and detailed results. In order to prove the effect of the proposed model, we explain the implementation details. Finally, our model attains the test set performance of 71.8% mIoU on the PASCAL VOC 2012 dataset.

Cardiac pathologies are generally associated with regional ventricular dysfunction. The left ventricular myocardium (LVM) segmentation is essential to clinical quantification and diagnosis of cardiac images. In this paper, a two stage registration-based segmentation method is proposed to segment the LVM from the cardiac 4DCT. In the first stage, a coarse global registration procedure is implemented in between the first time phase and other phases to spatially map the manually extracted ROI including the LVM in the first time phase to those in other phases. In the second stage, a fine local registration procedure is performed with respect to ROIs in between the first time phase and other phases to spatially map the manually obtained segmentation labels of the LVM in the first time phase to other phases to be segmented. In this way, the whole LVM segmentation results of 4DCT datasets can be obtained by propagating the labels from the 3D dataset in one phase to that in other phases. Experimental results showed that our proposed two stage registration-based segmentation method outperformed the global automatic registration-based segmentation method in effectiveness.

Camouflage pattern design is supposed to simulate natural background with using several colors. Therefore, in order to design camouflage patterns automatically, texture feature of background image is extracted by image segmentation technologies which is the hot topic in camouflage research field recently. Image mean-clustering segmentation is the most common method of image segmentation; however it is easily to lose image details due to using of pixel-value differences directly. To solve the deficiency of the mean clustering segmentation in existence, the image segmentation method based on SOFM is proposed in this paper, combined with the technology of artificial intelligence network learning, it can retain the more details of the image, and make up the deficiencies of mean clustering method.

This paper introduces a new interactive image segmentation approach based on global pairwise relationship. Many conventional interactive image segmentation methods only consider local relationship of neighboring pixels or unary probability of pixels, which results in the sensitivity to seeds. To overcome this drawback, we utilizes the pixel-pairwise relationship to obtain the global pairwise relationship of pixels. The constructed global binary probability is used to estimate the labels of pixels. In order to improve the computational efficiency, we further replace pixels with superpixels and use the binary global relationship of superpixels for image segmentation. Our method makes full use of global binary information and has stronger robustness to limited seeds information. The superior performances of our method are demonstrated in the experiments on the Berkeley segmentation dataset and Microsoft GrabCut database.

Convolutional Neural Network (CNN) has been introduced to in-loop filtering in video coding for further performance improvement. For intra frame coding, a CNN model can be directly trained through learning the correlation between the reconstructed and the original frames, and the obtained model can then be applied to every single reconstructed frame to help improve the video quality. In contrast, for inter frame coding, intertwined reference dependency exists across frames. If a similar procedure of model training and deployment is adopted for inter as that for intra coding, over-smoothed reconstructed frames may be generated, which may further seriously deteriorate the overall coding performance. To address such an issue, state-of-the-art work resorts to the Rate Distortion Optimization (RDO) to determine whether to adopt the conventional or the CNN-based scheme for in-loop filtering, however leading to high computational complexity. In this paper, we propose a Coding-unit (CU) level Adaptive Decision approach (CAD) which employs an early decision for each CU, based on their coding parameters. Experimental results show that the proposed scheme achieves comparable performance with that of the RDO scheme while effectively reduces the encoding time complexity.

Spatial relationship and structure models play an important role in various computer vision tasks. Many spatial models focus on global relationship modeling and ignore the detail of the internal structure. This will cause a poor performance in capturing discriminative features for a complex multi-object image. In this paper, we present a novel spatial structure model for complex regions with explicit similar measure. Three aspects of our method should be noted in comparison to the previous approaches. First, we propose a detailed layered graph model for representing complex regions, which has complete description performance and even the slight differences among the local objects can be also recognized. We further provide similar matching algorithms for topological structures based on the detailed layered graph model. Finally, an effective retrieval framework is proposed and applied to multi-object image retrieval. Rigorous experimental evaluations on two datasets demonstrate that the proposed method outperforms other prevailing approaches.

There are affordable houses that violation of construction like construction disorder and less construction in China, which will cause wasting and misappropriating the government investment. Thus, it needs to be verified. In view of the large number and wide distribution of residential housing, as well as traditional artificial verification methods are too time-consuming and laborious. In this paper, based on the characteristics of high-width and wide-width of GF-1 satellite, a fast verification method of affordable house from GF-1 panchromatic image by geographic constraint is proposed. First, use the Morphological Building Index (MBI) method to extract the building features of the entire study region, and crop out image blocks by using the affordable house’s vector data that measured by GPS for geographic constraints. Secondly, for the local features of each image block, combined with the Canny operator and the adaptive Mean-shift image segmentation algorithm to extract the buildings within the image block. Finally, based on the overlap rate between the building extraction result and the vector data, it is judged whether the location of the affordable house exists. Experiments show that the building extraction module of this paper can effectively extract the buildings in the image blocks on the GF-1 panchromatic image and is better than the MBI method, which can effectively realize the verification of the affordable houses.

Image classification plays a vital role in the field of computer vision. Many existing image classification methods with high accuracy are based on supervised learning, which requires a great number of labeled images. However, the labeling of images requires a lot of human and material resources. In this paper, we focus on semi-supervised image classification, which can build a classifier using a few labeled images and plenty of unlabeled images. We propose an attention-based generative adversarial network (GAN) for semi-supervised image classification, which can capture global dependencies and adaptively extract important information. Furthermore, we apply spectral normalization, which can stabilize the training of attention-based GAN. The experimental results obtained with the CIFAR-10 dataset show that the proposed method is comparable with the state-of-the-art GAN-based semi-supervised image classification methods.

Crop seeds sorting is an important task for industrial production. In the sunflower seed production processing, a large number of impurities are mixed into the seeds. Besides, the image of sunflower seeds has the characteristics of random distribution, various kinds of bad sunflower seeds. In addition, the distinction between the bad seeds and normal seeds is not obvious. the recognition rate of traditional methods suffer from a low accuracy is not meeting the actual requirements. In the past few years, many methods based on convolutional neural network (CNN) has made great success in object detection and recognition. However, these networks have a large model size. In this paper, we developed a CNN model with eight convolutional layers to extract the image feature and a skip connection is used to increase the learning ability of the model. Compared with the classical convolution network, it has smaller size without reducing the accuracy. However, because of the CNN features obtained by the convolutional layers are seldom investigated due to their high dimensionality and lack of global representation. Therefore, we introduced a channel attention mechanism adaptive to recalibrate the channel-wise features by considering dependencies among feature channels to strengthen the image features that are import to the classification tasks. Extensive experiments show that our model with attention mechanism achieves better accuracy on the sunflower seed images dataset compared with several classical networks.

We proposed a model framework which was based on generative adversarial network for video conversion. Our goal is that two different target videos can synchronize the movements (such as the head displacement and facial movements of the person), and the movements was not existed in the original video. Our key observation is that a video prediction model is added to the original framework of the generative adversarial network, so that the generated video can get the time sequence characteristics of the target video to improve the action consistency and time synchronization stability. In the training process, we obtained and aligned the spatial position of the action in video through landmark points detection, to ensure that the generated samples would not appear the phenomenon of spatial dislocation. In the training process, we will generate sample t and obtain t+1 sample through pre-trained time predictor, calculating the generate sample loss feedback pre-trained generative model. Using this framework, we can: (1) obtain more convenient to make available training samples and improve the available range of the model; (2) improve the accuracy of target generate video.

We proposed a model framework which was inspired by generative adversarial network for video conversion. Our goal is that two different target videos can synchronize the movements (such as the head displacement and facial movements of the person), and the movements were not existed in the original video. Our key observation is that a video prediction model is added to the original framework of the generative adversarial network, so that the generated video can get the time sequence characteristics of the target video to improve the action consistency and time synchronization stability. In the training process, we obtained and aligned the spatial position of the action in video through landmark points detection, to ensure that the generated samples would not appear the phenomenon of spatial dislocation. In the training process, we will generate sample t and obtain sample t + 1 through pre-trained time predictor, calculating the generate sample loss feedback pre-trained generative model. Using this framework, we can: (1) obtain more convenient to make available training samples and improve the available range of the model; (2) improve the accuracy of target generates video.

Deep neural networks have been applied to video compressive sensing (VCS) task recently. The existing DNN-based VCS methods compress and reconstruct the scene video only in space or time dimensions, which ignores the spatial-temporal correlation of the video. And they generally utilize pixel-wise loss as the loss function, which causes the results to be over-smoothed. In this paper, we propose a perceptual spatial-temporal VCS network. The spatial-temporal VCS network, which compresses and recovers the video in both space and time dimensions, can preserve the spatial-temporal correlation of the video. Besides, we refine the perceptual loss by selecting specific feature-wise loss terms and adding a pixel-wise loss term. The refined perceptual loss can guide the spatial-temporal network to retain more textures and structures. Experimental results show the proposed method can achieve better visual effect with less recovery time than the state-of-the-art.

Estimating the number of people in images is an important task in computer vision, and has a wide range of applications, such as video surveillance, traffic monitoring, public safety and urban planning. The task of crowd counting faces many challenges, e.g. extremely scale variations in extremely dense crowd scenarios, severe occlusion, and perspective distortion. We have studied a new type of multi-granularity full convolutional network as an effective solution for crowd counting. We not only designed the parallel multi-receptive field module to learn the mapping from the crowd images to the density maps, but also introduced the skip-connection mechanism to better train the mapping to improve the quality of the estimated density map, which is critical for accurate crowd counting. The experimental results upon ShanghaiTech public dataset showed that the proposed method can obtain more accurate and more robust results on crowd counting than the most advanced method.

Based on the pictures of rural housing buildings, the characteristics of housing for the poor are studied, and the appearance of wall is classified by the deep learning method. The degree of poverty is determined by the classification of wall characteristics. Using the transfer learning method, the ResNet101 network is combined with the AdaptNet network to train the house image set. The house pictures are classified using the trained model. Experiments show that the classification accuracy in the recognition of wooden walls and tile walls is improved.

The purpose of convective storms nowcasting is to predict whether a region will experience strong convective weather in the next 30 minutes. Unlike traditional prediction, we study the problem of convective storm nowcasting from the perspective of machine learning. First, this paper divides the research area into a number of position-fixed small cells, then transforms the nowcasting problem into a question: Is there a radar echo > 35 dBZ in a cell within 30 minutes? Secondly, the problem of nowcasting is formulated as a kind of spatiotemporal classification learning. From this point of view, this paper introduces a sliding oversampling method to mitigate the class imbalance issue of convective storm nowcasting. A spatiotemporal convolutional network nowcasting method is proposed which is in less computational cost and easier to train than recurrent neural network. In experiments, this spatiotemporal convolution network can better captures temporal and spatial correlations, compared with previous studies, therefore results in better predictive performance. Although no sophisticated tracking algorithms are used, storm movement trends and storm growth can be predicted with reasonable skill.

A new infrared small target tracking method is proposed to track objects. First, the two-scale flux density calculating method based on the infrared orientation gradient feature of IR image is presented to extract feature of tracker, which is an effective method to solve the problem in feature extraction of tracking infrared small targets. Second, the least-square trajectory prediction algorithm is applied to deal with the difficulty of target loss in the tracking process. This algorithm makes full use of the continuity and direction of the target motion, avoids the interference of noise and achieves the prediction and discrimination of the target trajectory. The experimental results indicate that the proposed tracking algorithm is superior in precision and has less processing time compared with the contrast algorithms, so it is an efficient method for IR small target tracking in a complex background.

The application of convolutional neural network (CNN) in hyperspectral image (HSI) classification has aroused widespread concern, especially spectral 1D CNN and spatial 2D CNN. Due to intense requirements of calculations and memories, 3D CNN, which is able to process jointly spectral and spatial features, has not yet been widely adopted. Recently, researchers have proposed a hybrid CNN for HSI classification, which obtained better performance than 3D CNN alone. Nevertheless, such a hybrid network has excessive parameters and limited capacity for feature utilization, where smaller training samples are prone to lower accuracy. This paper proposes an improved hybrid CNN to enhance the classification performance, which involves global average pooling, skip connection and appropriate adjustments of the convolution kernels and overall structure. Experimental results from benchmark HSI datasets suggest the effectiveness of our CNN for HSI classification in the situation of limited training set.

Aiming at the shortcoming of deep neural network in crop disease diagnosis, a lightweight compressed depth neural network for tomato disease diagnosis is proposed. Multi-scale convolution is used to increase receptive field, extract more abundant features, reduce model parameters and realize lightweight of the network by adopting the strategies of group convolution, depth separable convolution, pointwise convolution, channel shuffle, etc. For the lightweight model that has been initially trained, pruning operation is used to cut filter weight that is not important to reduce redundancy of the model. Experiments show that the accuracy of tomato disease diagnosis using the lightweight model is 98.61% after training only 10 epochs, it meets the needs of tomato disease diagnosis in agricultural production due to the small calculation and fast detection speed, and when cutting about 50% filter weight, the accuracy has only dropped 0.70%, which has a good effect.

The second generation Audio Video Standard (AVS2) adopts the flexible partitioning structure, which recursively divides the coding tree unit (CTU). The prediction models (PM) in inter prediction include PSKIP, P2Nx2N, P2NxN, PNx2N, PHOR_UP, PHOR_DOWN, PVER_LEFT and PVER_RIGHT. Each PM needs to perform integer motion estimation (IME) and fractional motion estimation (FME) in the motion estimation (ME) process. These methods adopted by AVS2 contribute significant coding efficiency while generating great encoding complexity. In order to reduce the encoding complexity, an adaptive inter mode decision fast algorithm is proposed in this paper from two aspects. Firstly, we statistically analyze the PMs in both spatially and temporally adjacent positions and establish a mode complexity (MC) measure metric. Through experiments we know that the complexity of inter mode decision decreases with the reduction of MC. Therefore, MC can be utilized to guide skipping some PMs with low occurrence probability. Secondly, a fast algorithm based on the parent prediction unit (PU) is proposed. FME in child PUs is skipped when the parent PU’s MV from IME is equaled to FME. Experimental results show that, compared to the AVS2 reference software RD17.0, the proposed fast algorithm reduces total encoding time by an average of 21.2% while the performance loss is negligible.

Because of the advantages of high sampling rate, high-speed image acquisition system is widely used in military, sports, biological and other fields. Unfortunately, just due to its high frame rate, the design risk and design cycle are greatly increased, and the back end image processing is still a challenging problem. In order to solve this problem, an image acquisition system architecture and protocol is proposed in this paper, which divides the image acquisition system into control plane and processing plane. The processing plane is divided into sensor independent layer, cache layer, processing layer and application layer. These layers are physically connected by AXI bus, and the logical relationship between them is as small as possible. Our proposed architecture and protocol effectively reduces the design complexity and design risk, also the design efficiency is greatly improved.

In practice of clinical, two kinds of medical image information should be considered comprehensively to judge the lesion, so it is necessary to register these images. A method for registration based on pure geometric transformation is presented in this paper. Generally, a group of transverse two-dimensional skull slices are obtained. Firstly, using canny operator to extract the edge of these images, three-dimensional modelling and unifying the spatial resolution. Obtain the maximum common imaging area in the z direction to unify imaging region. Calculating the centroid of the region, shifting the centroids to the origin of coordinates respectively. Secondly, using the bridge of the nose which is the obvious feature of skull to construct the first unit vector. Using the rotation operator of Clifford geometric algebra space to rotate the first unit vector to coincide with the y axis. Thirdly, by using the principle that the sum of the distances from the contour points on the xoy plane to the vector which pass through the origin is the smallest to construct the second unit vector, this vector of the floating mode is rotated by affine transformation to coincide with the reference one. The data of cranial images are derived from the "Retrospective Image Registration Assessment" project database of Vanderbilt University, and the results are evaluated using the gold standard of this database. The results of this experiment show that this method has the advantages of simple calculation, intuitive geometric meaning, high registration accuracy and high execution efficiency.

Optic disc is one of the main features in retinal images. Its location plays an important role for automated screening systems. A novel approach for automatic optic disc location is proposed in this paper. It consists of the following three stages, in the first stage, preprocessing is applied for correcting the uneven illumination and improving the low contrast. In the second stage, a series of key points can be extracted using alternative sequential filters and regional maxima. In the third stage, the key point that has the maximum correlation coefficient calculated by adaptive multi-scale template matching method is located as the center of the optic disc. The proposed algorithm is tested on two publicly available databases including DRIVE and DIRATEDB1. And our experiment results indicate that the proposed algorithm gives better performance than the state-of-the-art approaches.

Lung juxtapleural nodule regions are often excluded from the extracted lung region by the intensity information-based methods. In order to solve this problem, an adaptive morphology template method is proposed. First of all, the SIFT information is used to extract some feature points in the borderline. Then, the Fourier descriptor is introduced to identify those juxtapleural nodule regions from all borderline sections. Finally, adaptive morphology templates are used to correct the recognized region. Through the experiments on real CT slices, perfect correction effect proves that the proposed model has a good power of re-correction for CT images.

The precise stitching of microscopic images of large-scale biological sequence slices is of great significance for the study of biological structure and function, but the slight scale changes of microscopic images and the blank areas in the images seriously affect the accuracy of mosaic. In this paper, we propose a electron microscope sequence image stitching based on belief propagation algorithm, which basically solves this problem. Firstly, the relative scale of adjacent images is calculated by extracting the sift feature points of the images. Then the global optimization model is used to obtain the absolute scale of each image, and the image is scaled to obtain the microscopic image with consistent scale. Secondly, obtain the relative displacement relationship of adjacent images by template matching method, and then the global positions of all images are optimized by Belief Propagation (BP) algorithm to eliminate the influence of blank regions and repetitive structures on the stitching results. In the case study, the proposed method demonstrates high quality.

In order to improve the diagnostic effect of MRI images, a multiparametric magnetic resonance imaging (MRI) based classification method was proposed in this paper. The study included 85 patients. The radiomics method was used to extract morphological and texture features, while Apparent diffusion coefficient (ADC) was used as functional feature.Three classification methods, including Linear Discriminate Analysis (LDA), Support Vector Machine (SVM) and Random Forest (RF), were used to distinguish benign and malignant of pulmonary lesions. The performance of multiparametric MRI sequences and single sequences were compared. The experimental results shown that multiparametric MRI classification with SVM classifier had best performence (AUC=0.82±0.03), indicating that multiparametric MR diagnosis has great potential.

Graph classification is a challenging problem that assigns the structured data into several categories. The success of kernel methods in graph classification has aroused many designs of novel graph kernel. In this paper, we present a multiview graph kernel, a new method to combine the advantages of multiple kernel functions (Graphlet kernel and Weisfeiler-Lehman kernel). Experiments on several benchmark datasets show that multiview graph kernel could achieve significant improvements compared with the original graph kernels.

The airborne cockpit display system realizes airborne data visualization and human-computer interaction, and is one of the core systems of modern integrated avionics systems. Aiming at the defects of low reuse rate, poor inheritance and inability to reconstruct, the simulation software of airborne cockpit display system proposed a graph generation modeling theory of display and control system based on abstract factory pattern. By classifying and classifying the graphics library, it was established. Display processing model for text, basic graphics, and complex graphics. The graphics model development library and graphics generation component development architecture are implemented by OpenGL technology. This method is used to quickly build a certain aircraft cockpit display system. Experiments show that the method realizes the general development technology of display system and graphic display reconstruction technology.

Multi-view super-resolution refers to the process of reconstructing a high-resolution image from a set of low-resolution images captured from different viewpoints typically by different cameras. These multi-view images are usually obtained by an array of the same color cameras. However, the color cameras have color filter array to acquire color information, which reduces the quality of obtained images. To avoid color camera, and obtain higher resolution color images, we do research on a camera array which consists of interlaced different monochrome cameras and propose a new super-resolution method based on the camera array. Given that MVSR is an ill-posed problem and is typically computationally costly, we super-resolve multi-view monochrome images of the original scene via solve a regularization optimization problem consisting of a data-fitting term and three regularization terms on image, blur and cross-channel priors. The resulting optimization problems with respect to the desired image and with respect to the unknown blur are efficiently addressed by the alternating direction method of multiplier. Corresponding experimental results, conducted on a series of datasets captured by our own camera array system, demonstrate the effectiveness of the proposed method.

In terms of methods for measuring the distance between two points in space, field measurements have limitations due to time-space constraints. Moreover, methods using 3D scanners require high costs and long working hours. In order to improve such limitations, this study proposes an algorithm of automated pixel correspondence searching method using CNN (Convolutional Neural Network) and Z-NCC (Zero-Mean Normalized Cross-Correlation). The proposed algorithm applies a method where the user selects one pixel from a photo among multiple photos from the field after aligning the photos by position using SfM (Structure from Motion) and then automatically calculates the corresponding pixel from another photo using CNN and Z-NCC to triangulate the depth of the selected pixel.

With the development of domestic express business in China, there is a potential and huge need for automatic information extraction from express mail images, which however is challenging due to the skewness/folds of the captured express mail images. This paper presents a robust approach to text extraction from camera captured express mail images. Firstly, the proposed approach use a deep direct regression neural network to predict both the express bill region and type; then the form frame lines and corners are detected, with which the Coherent Point Drift (CPD) based point matching is adopted to obtain the mapping between the template and the input image. Finally, based on the mapping, accurate writing fields can be extracted. The Experiments on realistic express mail images demonstrate the effectiveness of the proposed approach.

Due to the low-cost, lightweight and widely available, approaches that use a single monocular camera to relocalize pose in 3D point-cloud dense map has gained significant interests in a wide range of applications. Relocalizing camera in a given map is of crucial importance for vision-based autonomous navigation. When the robot run in a new map or tracks lost, the robot's global localization cannot be obtained. We propose a novel approach, which relocalizes the pose of monocular camera with respect to a prebuilt dense map. A common monocular visual odometry system is employed to reconstruct a sparse set of 3D points based on local bundle adjustment. These reconstructed points are matched against the dense map to get the camera global pose with the particle filter (PF) algorithm and iterative closest point (ICP) algorithm. The particles state represents the potential initial pose. The ICP alignment result is used to update particles pose and important weight. Meanwhile, adaptive resampling is used to approximate the distribution of pose. Our monocular camera relocalization approach has several advantages. The improved PF algorithm is utilized to tackle the local convergence of ICP alignment. And our approach only relies on matching geometry between local reconstruction and dense map, it is robust to photometric appearance variance in the environment. In addition, the approach also can estimate the metric scale, which cannot be recovered from monocular visual odometry. We present real-world experiments demonstrating that our approach can relocalize the monocular camera and accurately estimate the metric scale in a given dense map. The results verify the effectiveness and robustness of this algorithm.

Visual localization determines the position of the viewer in a scene based on his or her viewpoint picture. The problem is challenging because we must handle various view-points in addition to picture quality. In this paper, we present a model that regresses the 6-DOF camera pose from a single RGB image. We use a spatial grid that splits the target space into cells and apply position regression after classifying the viewpoint into a cell. Combining with the adaptive loss and spatial LSTMs, our method outperforms existing approaches by a large margin in both indoor and outdoor scenarios. Furthermore, we present a new integrated indoor and outdoor localization dataset. Results on both public and our datasets show that our method can improve both positional and orientational precision, especially for large scenes.

With the development of intelligent vehicle technology, the environment perception of intelligent vehicles is becoming more and more important. The detection of pavement roughness is an important component of environmental perception. In this paper, binocular stereo vision technology is used to detect the roughness of pavement on which vehicles drive, and a square segmentation method is used to calculate the value of the roughness. The binocular vision sensor is used to acquire the image and process it to get the depth image. Then square segmentation method is used to calculate the roughness of the pavement. By comparing scanning each point individually and calculating the difference of each point, square segmentation method will greatly reduce the calculating time. Through the algorithm verification of the actual pavement detection, the experiment shows the effectiveness of the method.

Advanced volumetric data three-dimensional (3D) display can provide depth cues than conventional two-dimensional. Here, an algorithm of view frustum shear transformation and ray casting integral to realize real-time volume data 3D display is presented. It is compatible with the existing volume drawing pipeline and robust for other types of 3D displays. Experimental results demonstrate that the volume data is clearly represented, and real-time frame rate is more than 30 fps under the condition of subpixel-level at 4K (3840×2160) resolution. Besides, its performance does not rely on the viewpoint number and viewing angle, therefore, it is very suitable for 3D display of large viewing angles and dense viewpoints.

For the visualization of a 4-D object, information in 4-D space is obtained by projecting a 4-D space onto a 3-D space. Previous researches mainly paid attention to the geometrical information of a single 4-D object in 4-D space. Our research aims to develop a visualization system for physical phenomena in 4-D space and to study the human spatial cognition of the 4-D space through the projected 3-D space. In this research, we focus on a collision of two 4-D objects in 4-D space and develop an algorithm for the collision detection between two 4-D objects to be displayed in 3-D space. In our developed algorithm, a 4-D object is defined as a mesh data of tetrahedrons, and a tetrahedron-based collision detection algorithm is performed by a combination of half-space tests with the use of 5-D homogeneous processing to enhance the calculation accuracy of the collision detection. The results of the collision detection between the 4-D objects are presented.

In order to solve the problem of inadequate visualization and intuitive description of space environment, a method of space environment visualization based on data field was proposed. The space environment mainly includes neutral atmosphere, ionospheric plasma and high-energetic charged particles. Firstly, by analyzing the calculation model of space environment, a unified multi-dimensional dynamic space environment data model was designed, and the space environment data field is constructed. Secondly, because of the large difference in magnitude and uneven distribution of elements in spatial environment data field, it cannot be directly used for display. Considering the requirements of data processing speed and computer visualization, an adaptive normalized data preprocessing method was proposed. Based on the physical characteristics of spatial environment layer elements, a mathematical model for layer representation of data field, and an adaptive optimization layer algorithm for data field were proposed. Finally, the adaptive layer grid data was tracked by contours, and the layer rendering principle was used to realize the visualization of spatial environment data field. This method was used to visualize neutral atmosphere, ionospheric plasma and high-energetic charged particles. Experiments show that this method can meet the visualization requirements of space environment data field and effectively improve the visualization expression of space environment information.

Tikhonov regularization and total variation (TV) are two famous smoothing techniques used in variational image processing problems and in particular for optical flow computation. We consider a new method that combines these two approaches in order to reconstruct piecewise-smooth optical flow. More precisely, we split the flow vector into the sum of its smooth and piecewise constant components, and then regularize the smooth part by quadratic Tikhonov regularization and the piecewise constant part by total variation. We solve the new variational optical flow problem through a discretize-optimize approach by applying a fast multilevel truncated Newton method. Experiments are performed on images from the Middlebury training benchmark to show the performance of our proposed method.

Registration of electron microscopy (EM) images is one of the most important steps in reconstructing neurons. Image registration algorithm based on SIFT have been widely used in the EM image registration. But SIFT matching procedure both costs a lot of time and introduce massive false matches. In this paper, we propose an improved EM image registration method using the scale information of SIFT keypoints. In the feature matching procedure, our method saves up to 45.8% of the computation time compared to SIFT. We also added a preprocessing procedure for RANSAC to eliminate false matches in small-scale matches sets. Experimental results show that the method improves the accuracy of results on every test EM image set while highly reducing the registration time.

Fasteners are the important components of railway system, which can be used to fix the tracks to the sleepers and reduce the likelihood of derailment. Nowadays, the extensively used approaches for the automatic detection of defective fasteners are vision-based approaches. However, they are not robust and efficient enough to be applied in reality. To solve this problem, this paper applies deep convolutional networks for the automatic detection of fastener defect and proposes a two-stage fastener defect detection framework. The framework is composed of a CenterNet-based fastener localization module and a VGG-based defect classification module. Besides,we innovatively introduce an attention mechanism named CBAM into localization network and an adaptive weighted softmax loss in classification network training procedure to elevate the accuracy of both modules. The experiment result shows that both methods have obviously improved the performance of the fastener defect detection system. The proposed localization network has a better accuracy-speed trade-off with 99.94% AP at 63 FPS on the test set. In addition, the proposed defect classification network has the best accuracy (up to 98.10%) on the test set and can be used to classify up to 5 categories of defects.

In the production process of strip steel, detecting wave edge in real time is quite important, otherwise it will contribute to the abandonment of strip steel materials. At this stage, an automatic identification system based on machine vision which aims to figure out the wave edge of strip steel is gradually being put into use. However, it is shown that the complicated environment of the factory makes it difficult to reach its goals. In order to solve this problem, this paper designs a motion strip steel target detection algorithm based on single image rapid defog and morphological interframe difference. Firstly, based on the physical model of foggy image degradation, using a simple mean filtering to estimate the environmental light and global atmospheric light, thus the removal of water mist in video screen is realized. Then, using interframe differential method to extract the motion strip steel in the image, and the noise is filtered by mask segmentation and morphological operator. At last, the experimental results show that the optimization algorithm is more accurate and effective compared with the traditional motion target detection algorithm.

Word embedding have been used in numerous Natural Language Processing and Machine Learning tasks. However, it is a high-dimensional vector field that propagate stereotypes to software applications. Its current debiasing frameworks do not completely capture its embedded patterns. In this paper, we propose deb2viz, a visual debiasing approach that explores and manipulates the high-dimensional patterns of word embedding field. First, we partition this vector field into interrelated low-dimensional subspaces to equalize and neutralize distances between gender-definitional and gender-neutral words. To further reduce gender bias, we update the distances of appropriate nearest neighbors for gender-neutral words to be arbitrarily close. Experimental results on several benchmark standards show the competitiveness of our proposed method in mitigating bias within pre-trained word2vec embedding model.

This paper explores the methods and applications of intuitive visualization and interactive analysis of medical domain knowledge to help medical experts editing it autonomously. SissiExpression system is developed using C# and WPF technology, in which, medical knowledge is modeled as expression. The application of knowledge and execution process is clear and visible, the work of debugging and maintenance is convenient, so the result can be understood and gain recognition by experts. Expressions and knowledge are saved in XML file format, decoupled from program framework, subsequent maintenance, adding new knowledge and functions, can be achieved agilely, conveniently, with better cost control. Appropriate knowledge edition can be chosen flexibly according to demand, medical experts can possess their personal executable knowledge base. This program can help medical experts to mitigate human-computer interaction obstacles to some extent, both medical and information experts can focus on their own profession and tasks.

High dynamic range (HDR) imaging expands the capabilities of a camera by synthesizing a sequence of different exposure images. However, due to camera and object motion, ghosts exist in the synthesized HDR image. The low-rank matrix completion (LRMC) model has achieved some success in ghost-free HDR imaging, but leads to artifacts around the observation region edges for neglecting local image structure. In this paper, a locality-constrained LRMC (LcLRMC) model is proposed, in which we iteratively update the background irradiance and the observation region based on the result from previous iteration. Specifically, the proposed method incorporates global and local structures. Experimental results show that compared to the conventional LRMC model, the proposed method effectively eliminates artifacts around the observation region edges.

The modeling of hair is too difficult to simulate because of its number and shape, as well as the texture of the hair itself. The traditional way of constructing hair based on physics and geometry requires complex calculations and various parameters. In recent years, hair modeling methods based on single images, based on multiple images, and based on videos have begun to develop. The advantage is that modeling is fast. At present, the geometry of the hair is mainly represented by lines of three-dimensional points. In this paper, a three-dimensional multi-strip is used to represent the geometry of the hair. Through deep learning, the position and type of the hair in a single image are obtained, and the similar hairstyle model in the database is matched. The selected hair model and head model are connected and fixed by further fitting. Then we simulate dynamic hair by setting gravity, friction, collision detection, and other more. The model preserves the image appearance of the image as much as possible and can be used to simulate common hair geometry.

In computer vision, object reconstruction is the task of inferring the 3D shape of an object based on a single or multiple 2D images. For such purpose, most common frameworks use voxel grids and point clouds. However, both of these approaches have strong limitations. On one hand, the computational cost of using voxels grows cubically as the resolution of the voxels increases. Therefore, 3D object reconstructions are usually set to low resolution. On the other hand, point clouds are unstructured in nature and the proper definition of surfaces and contours is complex. In this study, 3D object reconstruction is carried out applying free-form deformations on pre-existent 3D meshes, through two basic learning processes: template selection and template deformation. From this approach, it is possible to generate high-quality 3D object reconstructions with a lower computational cost. Concretely, two novel lightweight CNNs models are developed and tested: a multi-target learner (Model A) and depth information learner (Model B). According to the results, the performance of the multi-target learner regarding the template selection was around three times better (lower error) than in the baseline architecture, which improved the quality of the 3D reconstructions, whereas the depth-information learner showed promising results in the reconstruction of objects with complex geometry. The inherent issue of using chamfer distance as a loss measure is also examined.

A road scene model contributes a critical infrastructure in many computer applications. However, few methods exist for the automatic generation of 3D realistic road networks, such as driving simulations require a real and immersed 3D road network models. An effective method including three major components (namely: 2D road centerline extraction, 3D road network dataset integration and 3D real roadway view model reconstruction) was put forward here for a mixedreality road reconstruction, where the road view model system represents a hybrid view integrating a real geography scene with the geometric model. The image semantic segmentation was exploited by means of SegNet to extract the 2D road network from Google earth, and overcome the specific defect of the custom image processing method; moreover, Unity 3D was selected to show how roadway geometrical model for a simulator system can be visualized by means of remote sensing images. Results show that this method can realize a high-fidelity road view with little human costs.

In the three-dimensional reconstruction based on camera pair, the angle between two cameras should have a great influence on the reconstruction accuracy. When we shoot around the scene, when the angle between the camera axis is small, it is easy to match the images, but it is not conducive to restoring the three-dimensional coordinates. But when the angle between the two cameras axis is large, it is conducive to restoring the three-dimensional points, but it is not conducive to matching between the images, so there is an optimal position relationship. In this paper, the best position relationship between cameras is found through experiments and analysis. In addition, the matching algorithm is improved to resolve the contradiction mentioned above to a certain extent. The measures adopted are window adaptive matching and rotation matching. Experiments show that both methods are effective.

Laser triangulation is often the best solution for the 3D (three-dimensional) image reconstruction of an object. And its features, such as simple structure, fast reconstruction speed and flexible use make it widely used in 3D reconstruction. However, due to its high image frame rate, huge data, and requiring a large amount of computation for image processing algorithm, it is hard to be applied in embedded system. This paper presents a hardware solution for high-speed image acquisition system which is implemented on Xilinx FPGA. The experimental results indicate that this acquisition system works normally and its performance is steady and reliable, under the extreme conditions with an image resolution of 1280*384 and a frame rate of 9,000. It is still a challenging problem to extract the position information of the light bar in real time. In order to solve this problem, a parallel gray center gravity method based on FPGA is proposed to detect the position of light bar in this paper. Test results show that the proposed method can correctly extract the position information of the light bar in real time when the image frame rate reaches 9,000 frames. The power consumption is only about 4 watts.

Benchmarking multi-view 3D reconstruction using synthetic images is a supplement for the traditional ways. The ground-truth data of synthetic benchmarks can be easily recorded in simulation and have a high accuracy. However, synthetic images from artificial scenes are usually lack of geometry and texture details. We propose a framework to build realistic synthetic benchmarks for real-world scenes. Real photos are also taken around the scenes. The key step of our approach is to transfer the image properties of the real photos to corresponding synthetic images, followed by a validation step to verify their similarity. Based on the proposed framework, we construct the Yungang Grotto dataset for large-scale multi-view 3D reconstruction, in which the largest scene is up to four thousand images, and the benchmarking results of a prevalent reconstruction pipeline on our dataset are also presented.

The research of the technology of modulation identification of digital communication signals is one of the key technologies of receivers in non-cooperative communication systems, and it also has important application value in civil and military fields. In the actual wireless communication environment, the phenomenon of multipath transmission exists all the time. However, most of the existing modulation identification methods are based on the ideal environment, which ignore the influence of multipath interference, and the performance of these algorithms is seriously degraded in multipath fading channels. In order to solve this problem, in this paper we proposed a new algorithm for modulation identification in multipath channels, which based on wavelet transform, higher order cyclic cumulant and high-order cumulant. The simulation results show that the algorithm proposed can effectively eliminate the influence of multipath channel and performs well for the identification of signal modulation.

A new integer ambiguity estimation method which can improve computational speed and accuracy of ambiguity estimation significantly is proposed in this paper. Firstly, the carrier-phase double-difference equation is used to solve the float solution of the double-difference integer ambiguity. Then, the float solution is decorrelated by the inverse integer Cholesky algorithm. Finally, the adaptive differential evolution algorithm (ADE) is applied to search for the optimum solution of integer ambiguity. Simulation results show that ADE has a faster convergence speed and is more robust than other intelligent algorithms, and ADE has a higher success rate of estimation than those traditional integer ambiguity estimation algorithms.

In the field of modern wireless communication, electromagnetic spectrum resources are increasingly tight, and it is becoming more and more common for one single antenna to receive multiple communication signals. High-order cumulant can suppress noise well, it has been widely used for modulation recognition of single signal in single-channel but seldom used for time-frequency aliasing signals in single-channel. This is because we need to use estimated carrier frequency for down-conversion to obtain the baseband signal before calculating high-order cumulant. But there will be more than one carrier frequencies when they are time-frequency aliasing signals. So it focuses to select the suitable carrier frequency to improve the calculation accuracy of high-order cumulant and recognition performance. In this paper, an algorithm of selecting suitable carrier frequency which uses the nature of high-order cumulant is proposed. And also two high-order features have been analyzed and constructed. Then they are used to classify seven kinds of timefrequency aliasing signals randomly combined by the digital modulated signals including BPSK, QPSK, 8PSK and 4ASK. The simulation results show that the algorithm has better recognition performance.

At present, the satellite signal resource allocation strategy does not consider the coupling relationship between power and bandwidth. At the same time, due to the use of multi-beam antenna and frequency reuse techniques in satellite communication systems, the co-channel interference of inter-beam signals is a factor that should be considered. And Satellite systems have to deal with many real-time signal transmissions, and the limitation of transmission requirements on delay is also a consideration. Considering these factors and different signal transmission conditions between beams, a new satellite signal resource allocation algorithm is proposed by reconstructing the satellite system model. The simulation results show that the new algorithm improves the system capacity and reduces the second-order service rejection of the system while comparing with the previous resource allocation algorithm.

Voice is the main way of communication and information sharing with others, It brings great convenience to human life. The existing speech recognition classification has the problem of considerable performance attenuation to environment noise and accent. Most of these problems can be mitigated by training on large amounts of data. However, collecting large Numbers of high-quality datasets in real life is time-consuming and expensive. In order to solve this problem, this paper proposes a data enhancement method,which is suitable for the speech image extension of small samples. S-GAN is used to generate datasets that conform to the real distribution of samples, and GAN-train and GAN-test methods are used to evaluate the quality and diversity of network generated images. Meanwhile, the spatial transformation network (STN) and CNN framework are combined to get the useful information part of the data for data classification. The results show that this method can significantly improve the classification accuracy of speech recognition and lay a foundation for small sample data enhancement.

No-contact heart rate monitoring like Fig.1 based on remote Photoplethysmography (rPPG) via common camera has drawn more and more attention because of its promising use in patient nursing, telemedicine, fitness, trial. Many traditional signal processing methods (FFT, ICA, PCA) have been proposed to solve this problem, but still subject to interference of motion and lighting conditions. In facial RGB images, the signal-to-noise ratio of green channel is higher than that of the other two channels, and the heart rate can be measured more accurately by assigning different weights to three channels. In this paper we propose a novel deep convolution neural network model based on channel-attention mechanism to extract the heart rate information from each frame of the video. To get more accurate result of the heart rate in the condition of face moving, light change and other interference factors, the model was trained on the newly introduced public challenge ECG-Fitness dataset and the model’s robustness was tested on this dataset. Testing results show that the model outperforms previous methods.

This paper designs a new communication jamming signal waveform generation model and the novel signal waveform separation method respectively based on blind source separation mathematical model and algorithm. The proposed signal waveforms have the characteristic of diversity which can ensure the security of communication. Simultaneously, the new form of jamming signal can be applied in the field of communication jamming to complete both deceptive jamming and repressive jamming. Simulation results of jamming on 2ASK, 4QAM and 16QAM communication systems show that the new jamming signal has larger jamming to signal ratio (JSR) range than the spread spectrum jamming signal and the monophonic jamming signal when effective interference is accomplish. This confirms its innovation and practicability in the field of communication jamming.

Help-seeking students are those who seek academic help during the assignment or course. Classifying help-seeking students in a virtual learning environment (VLE) is a challenging task for the instructor because the student is not physically present. In this study, machine learning techniques and statistical methods were used to detect the help-seeking student by analyzing the student logs data in an e-learning system. We determined that which factors are associated with help-seeking behavior of the students. We found that late submitted, and low assessment score students need more help in solving the course assignment. Also, the result shows that Decision Tree (DT), and Fast Large Margin (FLM) is high accuracy predictive machine learning models as compared to Support Vector Machine (SVM), and Logistic Regression (LR) finding the help-seeking students in a course and instructors can easily categorize the students who seek help, disseminate personalized feedback to those students accordingly, and also embrace the sustainable environment for education.

The fifth generation (5G) cellular network is mainly used to meet the needs of mobile communication after 2020. Orthogonal Frequency Division Multiplexing (OFDM) is widely used in existing wireless communication systems, and filtered orthogonal frequency division multiplexing has been proposed. (F-OFDM) as a new waveform that meets the requirements of the fifth generation cellular network. High-order subband filters are commonly used in F-OFDM systems, which results in high computational complexity when filtering is performed in the time domain. Therefore, a frequency domain filtering method based on overlapping preservation method is proposed to reduce the high computational complexity of time domain filtering caused by high-order subband filters in F-OFDM. The complexity analysis and simulation results show that the proposed low complexity filtering scheme can significantly reduce the computational complexity while achieving similar bit error rate performance compared to traditional OFDM and FOFDM systems using time domain filtering.

Low-density parity-check code (LDPC) is a kind of better error correction code which is played an important role in the reliability transmission of communication systems. In the aspect of decoding algorithm, the NMS algorithm becomes the mainstream algorithm dues to its lower complexity. what’s more, the Semi-serial decoding algorithm was put forward to reduce decoding iterations. However, the compensation factor of NMS algorithm is a fixed value and the Semi-serial decoding algorithm only considers one aspect of reducing iterations. In the actual decoding process, this value varies with the degree of the check node and the number of iterations. in order to solve the problems of these algorithms, this paper will propose a new improved algorithm. Firstly, we will adopt a dynamic compensation factor to improve performance of NMS algorithm on regular LDPC code. Then combined with the idea of Semi-serial decoding algorithm to reduce the number of iterations. The dynamic factor is firstly analyzed theoretically by density evolution theory. Then through the Monte Carlo experiment and the true LLR, the suitable dynamic compensation factors can be obtained which is changes with the number of iterations.

The quantity of radio communication service is growing up with the development of wireless communication technology. While limited spectrum allocation, inefficient utilization of frequency band and plenty of unused spectrum have led to spectrum scarcity. To solve this issue, we study dynamic spectrum access technique to maximize the utilization of multi-channel wireless network. we assume that N users sharing K channels, and the users can choose any channel to transmit. each user selects a channel and transmits a packet with a certain attempt probability. After each time slot, every user receives a binary observation which indicates the result of their transmission. We aim to find an optimal strategy for spectrum access, which maximizes channel utilization in a partial observation scheme without message exchange or online coordination between users. Because of the larger scale of state space and incomplete partial observation, acquiring the optimal solution is computationally expensive in general. To solve this issue, we develop a partial observation dynamic spectrum access algorithm based on deep reinforcement leaning. We take experiments to show the strong performance of the algorithm which can increase the channel utilization to 90% in partial observation scheme.

In high dynamic environments, signals in low earth orbit (LEO) satellites and terrestrial communications have large Doppler shifts. Traditional carrier tracking loops based on multi-stage frequency-locked loops and phase-locked loops perform poorly. In recent years, particle tracking based carrier tracking loops have been brought into consideration. In this paper, the standard particle filter (PF) and unscented particle filter (UPF) are analyzed in principle, and based on this, an improved unscented particle filter algorithm is derived, and the carriers based on these three algorithms are derived. The tracking loop is simulated by MATLAB. The new algorithm reduces the complexity of the UPF algorithm under the premise of ensuring accuracy.

The max/min filters are the most basic operator in gray-scale morphology. van Herk proposed an efficient O(1) rectangular kernel max/min filters algorithm which is independent of the size of filter window. This paper mainly studies the parallel optimization of van Herk’s max/min filters algorithm by using the single-instruction, multiple data (SIMD) technology and OpenMP parallel optimization technology for multi-core processors. First, the Intel Sreaming SIMD Extensions 2 (SSE2) is used to perform data-level parallel optimization for van Herk’s max/min filters algorithm in column direction, and a method based on histogram statistics is proposed to accelerate the calculation in row direction, the parallel speed-up ratio can reach 2. Then, the OpenMP parallel optimization technology is used to further perform thread-level parallel optimization for the SSE2 optimized version. The total speedup ratio under 4 cores and 4 threads CPU is about 7. Finally, compared with the rectangular kernel gray morphological operators in the powerful machine vision software MVTec HALCON17.12, the optimized algorithm in this paper has a great advantage in large window filtering size.

As a new type of modern modulation technology, much attention has been attached to binary offset carrier (BOC) modulation due to its characteristics of band sharing and spectrum separation for military and civilian signals. Basically, it is essential to investigate BOC modulation with jamming to defend various types of jamming which guarantee the safety and reliability of the satellite navigation system. In this paper, the transmitter and receiver of BOC modulation is given, and we propose a mathematical model for BOC modulation and demodulation. In addition, the error probability is derived for BOC modulation with jamming in AWGN channel. Furthermore, the mathematical analysis is verified by simulation results for non-jamming situations. In jamming cases, the single-tone jamming and the multi-tone jamming are considered.

Compared with the superheterodyne transceivers, zero-intermediate frequency (IF) transceivers have simpler structure and lower power consumption. Moreover, communication signals can be processed digitally with a sampling rate equaling to the signal bandwidth before transmission or reception. Binary frequency shift keying (2FSK), as a type of binary single-carrier modulation with excellent anti-interference performance, can also be transmitted or received via a zero-IF structure. Therefore, broadband or frequency hopping 2FSK transceivers can be realized easily in baseband. The traditional zero-IF 2FSK signal demodulation-phase demodulation has a simple structure, but its anti-noise performance is poor due to the unused priori information. In this paper, both coherent and non-coherent demodulation schemes are proposed for zero-IF 2FSK. Mathematical derivations and numerical simulation results show the bit error rate (BER) performance of coherent demodulation of zero-IF 2FSK is 3dB better than that of bandpass 2FSK.

Most of existing methods to identify the Reed-solomon (RS) codes, commonly applied as channel codes, mainly focus on non-shortened RS codes. In this paper, a method to identify the shortened RS codes with short codeword length is proposed. Firstly, based on Gauss-Jordan elimination through pivoting (GJETP), the codeword length of shortened RS codes is identified. The shortened RS codewords are zeroized by estimating the order of finite field and the number of shortened symbols. Then, the Chinese remainder theorem(CRT) based algorithm is adopted to identify the primitive polynomial. Finally, the generator polynomial is obtained based on Galois field Fourier transform (GFFT). The simulation experiments are carried out showing that the probability of recognition of shortened RS codes is higher than 90% with bit error rate (BER) less than 5×10^-3 , which demonstrates the viability of the proposed recognition approach.

Recent researches on learning local descriptor are mainly based on Convolutional Neural Network (CNN), including loss function, network architecture, sample mining, etc. In this paper, we investigated the existing datasets and difficult sample mining methods for learning local feature descriptor. And we find that the samples in the train set are often not enough for various situations. Because the changes in illumination and view are continuous, but the samples are discrete. In the meanwhile, current sample mining ways only excavate existing data of train set, which is not optimal. So we proposed a new sample mining fashion, called latent hard sample mining, which can utilize potential samples for learning. This method can mitigate the influence of inadequate training samples. We compare our approach to recently introduced convolutional local feature descriptors on Photo-tour and Hpatches dataset with different losses, and demonstrate the advantages of the proposed methods in terms of performance.

The non-binary low density parity check coded continuous phase modulation (LDPC-CPM) system improves the reliability and effectiveness of the digital signal transmission by relying on the error-correcting performance of nonbinary LDPC code and the spectral efficiency of high-order CPM. However, due to the channel interference during transmission and the phase jitter of local oscillator at the receiving end, the introduced carrier phase offset will drastically deteriorate the system performance. In order to solve the above problem, combining the output characteristic of LDPC iterative decoding, this paper studies the code-aided carrier phase synchronization algorithm suitable for nonbinary LDPC-CPM system, the basic idea is to use the decoding soft information to aid the synchronization parameter estimation, and carrier phase correction of the system is accomplished by joint iteration of synchronizer, demodulator and decoder. Theoretical research and simulation results show that under low signal-to-noise ratio (SNR) conditions, the code-aided synchronization algorithm for non-binary LDPC-CPM system solves the problem that the traditional codeaided synchronization algorithm cannot be applied to M-ary CPM signal, so that it can achieve effective carrier phase synchronization in the large phase offset range and obtain approximate ideal system performance at the cost of low system complexity.

In the outdoor scene, most users’ localization requirements can be fulfilled through the Beidou system (BDS). However, when facing some challenging environments with serious shadowing and reflection, such as the positioning of seaport containers, the number of observations of the BDS will decrease and the confidence of the observed pseudorange will degrade. In this case, auxiliary data must be added in order to achieve high-precision positioning. Ultra-wide band (UWB) has strong penetrating power and nanosecond temporal resolution, meantime it has the characteristics of highprecision positioning, so this paper adopts the combined positioning method of BDS and UWB to locate targets in the challenging environment. In this paper, appropriate UWB layout location is selected according to the principle of minimum geometric dilution of precision (GDOP). For BDS/UWB co-location system, this paper proposes a new adaptive weighted method based on pseudorange residual detection for weighted least squares (WLS) localization algorithm. Based on the WLS localization algorithm, the Kalman filter (KF) is introduced to improve the WLS performance, so this paper adopts a WLS and KF positioning algorithm (WLS-KF) for BDS/UWB co-location system. The simulation results show that this positioning method can achieve the positioning accuracy below m-level, which can meet the demand of high precision positioning in the complex environment such as seaport containers.

For uplink massive MIMO systems with hundreds of antennas at the base station, the Linear Minimum Mean Square Error (MMSE) signal detection algorithm is near optimal but involves matrix inversion with high complexity. In this paper, we proposed a low complexity detection algorithm in uplink large-scale MIMO based on Reactive Tabu Search (RTS) algorithm by using SOR iterative algorithm as the initial solution vector algorithm. The simulation result shows that it can reduce the computational complexity from ο(K³) to ο(K²), where K is the number of users. Under the premise of BER performance of the original algorithm, the simulation result shows that the performance of SOR-RTS method is always close to the original RTS algorithm.

This paper invests three methods to calculate satellite downlink power flux density (PFD) mask for the new VHF data exchange system (VDES), which aims at providing a wide-bandwidth two-way communications for vessels. Considering Rec. ITU-R M.1808, different protection criteria combined with performance degradation to the land mobile service incumbent is invested by the author. One method is considered on interference to noise ratio, and second method is seminar to method 1 besides the noise floor calculated considering the environment noise floor defined in Rec. ITU-R P.372. And the third method is derived to calculate the carrier to interference and noise ratio. Based on theoretical calculation and simulation, the PFD mask derived from the Carrier-to-Noise plus interference constraint is a more adequate PFD mask for VDES satellite components, which can protect both analog and digital land mobile systems in the same band.