This PDF file contains the front matter associated with SPIE Proceedings Volume 12083 including the Title Page, Copyright information and Table of Contents.

The fault tree(FT) analysis method plays an important role in system safety analysis and has been widely used in large and complex engineering systems. Therefore, this paper expounded the basic principles of FT analysis, using object-oriented programming technology, developed FT analysis and calculation software in the VC environment, and realized the drawing, qualitative and quantitative analysis of the FT. Finally, the effectiveness and usability of the software were verified according to the application analysis of an engineering example, and it has a certain engineering application value.

In order to solve the problem of slow convergence rate and premature stagnation of standard firefly algorithm (FA), an optimized firefly algorithm based on Levy flight and Gaussian function inertia weight (LFAGDIW) was proposed. This algorithm designs a decreasing inertia weight based on Gaussian function and adds it to the position updating formula to improve the convergence speed of the algorithm. Levy flight strategy is introduced to realize the random movement of individual fireflies and make the optimal search result jump out of the local optimal. Simulation test shows that the improved algorithm has a better convergence effect and avoids falling into local optimum.

Subspace clustering aims at grouping data into a number of partitions, which has become one of the most powerful tools to analyze and interpret big data, particularly for clustering high-dimensional data. Although a series of related approaches have been proposed, real data are often corrupted leading to the learned graph is inexact or unreliable. Meanwhile, constructing an informative graph is one of the most important steps, which can greatly affect the final performance of clustering approaches. To learn a robust graph, this paper presents a novel data representation approach called Robust Adaptive Graph Learning with Manifold Constraints (RAGLMC) for subspace clustering. RAGLMC introduces the l2,1-norm constraint on sparse coding coefficients for ensuring the learned coding coefficients would be optimal as adaptive graph weights. Moreover, in order to reduce the influence of noise points on the local structure in graph construction, we combine l2,1-norm with manifold constraints on the coding coefficients to learn a locality and smoothness representation. Therefore, the proposed approach can estimate the graph from data alone by using manifold constraint on coding coefficients, which is independent of a prior affinity matrix. Extensive experiments verify the effectiveness of the proposed approach over other state-of-the-art approaches.

China's coastal waters have significant alternating winter and summer monsoons, various meteorological activities are frequent, and marine resources are rich, and the development prospects are broad. Energy such as heat, function, and potential energy has a huge impact on human production and life. Therefore, accurate wind field data is useful for studying the coastal waters of China. This article uses multi-source satellite data, combined with the ERA-5 data provided by the European Weather Forecast Center (ECMWF), and uses classic machine learning methods and traditional interpolation methods to propose a feasible wind field interpolation method. The experiment generated seamless fined wind field data, and the wind field data has good accuracy, which can provide good data support for the construction of offshore areas in China.

Recently, Visual Entailment is proposed as a new task in the multi-modal field. Its main focus is to reason about entailment relations between a real-world image as a premise and a natural language as a hypothesis. Some papers have proposed models to obtain more accurate entailment relation judgments. However, these models do not consider the semantics of text at both global and local granularity, and the refining of the two modalities is not sufficient. In this paper, a new stacked multi-modal refining and fusion network is proposed. For cross-sniffing and key information activation between global and local features of hypothesis-sentence, a Global & Local Textual Features Fusion block is introduced. Secondly, a Refining and Affine Fusion block is proposed to achieve efficient multi-modal attention and fusion between image and text features. Finally, this paper presents a stacked structured network which embedded an adaptive hypothesis-preserving mechanism to enriching the grounds for semantic implication judgements. The experiments demonstrate that our model has a certain improvement in the accuracy of visual entailment classification compared with some existing methods in this field.

Recently, lots of works try to capture contextual information to benefit semantic segmentation problems. However, most approaches adopt the uniform method to obtain context information, which means each pixel gets its context from the same region. We argue that for each pixel, contextual information aggregated from the region it belongs to can benefit the dense prediction, while those from other irrelevant regions possibly mislead the prediction. In this work, we propose a Region Context Module (RCM) that aggregates context for each pixel only from its object region. Furthermore, we design a Region Context Network (RCNet) embedded in the ASPP Module and Region Context Module. We conduct experiments on three datasets: Cityscapes, Vaihingen and Potsdam datasets. Extensive quantitative and qualitative evaluations demonstrate our model achieves favorable performance against state-of-the-art approaches.

In China, college students have a high incidence of depression. The assessment and early warning of depression are conducive to the management of colleges and universities and the development of college students. In this paper, we designed a rapid assessment and early warning system of college students' depression based on sensors and daily health data. Logistic Regression Model and Multilayer Perceptron Model were used in the system. The system has the function of depression early warning, which can provide students with timely psychological treatment and intervention in the early stage of depression, so as to prevent the tragedy caused by mental health problems. Through the simple sensor data and health data collected daily, the system can make rapid judgment.

In this paper, we propose a simple but very effective approach in the presence of noisy labels. The memorization effects of Deep Neural Networks (DNNs) manifests that they first memorize training data with clean labels and memorize data with noisy labels gradually. Based on this phenomenon, we build Class Prediction Distributions(CPD) for each sample in the initial stage of network training. On the basis of CPD, we use our clean data selection strategy to divide training data into confidently clean data and noisy data. In this selection strategy, we rank the maximum value of CPD. Top-ranked samples are more likely to be clean samples. Finally, noisy labels classification is successfully achieved by using semi-supervised learning. Experiments on benchmark datasets including MNIST, Cifar-10, Cifar-100 and Clothing1M demonstrate that our approach can achieve a competitive performance.

The length change of some equipment or buildings has a marked impact on the performance. Therefore, the measurement of length is of great significance. Aiming at the problem of small measuring range with traditional binocular vision method, a measurement method of the length of long line based on epipolar constraint is established. Compared with traditional binocular vision method, this method has the advantages of small common field of view, large measuring range and can calculate virtual imaging point coordinates. In addition, according to the three-dimensional coordinates of two camera optical centers and two endpoints of the long line, we establish a matrix and get the relationship between virtual imaging position error and the rank of the matrix. The simulation result shows that the relative error of measurement is less than 0.2%.

Non-line-of-sight (NLOS) imaging, which utilizes weak photons that diffusely reflect from the visible surfaces (e.g., diffuse walls), can reconstruct hidden objects around the corner. Recently, lots of non-line-of-sight imaging methods have been proposed, such as time-of-flight (ToF)-based methods, coherence-based methods, and intensity-based methods. However, most of these methods are time-consuming for data acquisition and have poor robustness in the reconstruction process. In this paper, the novel application of Generative Adversarial Network is introduced to NLOS imaging. A robust, real-time NLOS imaging method based on autocorrelation mapping Generative Adversarial Network (AMGAN) is proposed, which reconstructs hidden scenes by learning the autocorrelation mapping from speckle-autocorrelation to the hidden target. In order to train the proposed AMGAN, we also analyze the principles of speckle-autocorrelation NLOS imaging and the noise model of the imaging process. Then a speckle-autocorrelation NLOS imaging dataset SANLOS is synthesized in this paper. Finally, our method is compared with other methods based on deep learning quantitatively and qualitatively. The experimental results demonstrate that the proposed approach achieves better NLOS reconstruction quality and is more robust under different exposure times compared with state-of-art methods

Arbitrary image style transfer aims to mimic the artistic features of a randomly given style image while maintaining the semantic characters of a reference content image. Existing algorithms have achieved astonishing style transfer results. However, they are insufficient to capture the global information of the content image due to the locality in convolutional neural networks. As a result, the content structures of the stylized images are disrupted by scattered textures. To address this issue, we propose to embed additional global spatial information of the reference content image into image style transfer algorithms with the adaptive instance normalization trick. We train a light-weighted network to distill the global structural information provided by the depth map of the reference content image. We then utilize the refined results to enhance the transformations between deep features of the inputs. Experimental results show that our proposed method can generate stylized outputs with more consistent structures and less unwanted textures, achieving impressive visual effects.

Optical measurement mechanics is a technique studying structural behaviors by processing optical information. It is significant to measure time accurately in high-speed processes. Optical time codes can be used to represent and measure time. However, current time codes have limitations, which may affect accuracy and stability of time measurement. Herein, a high-speed time measurement system is demonstrated, including the system architecture and the data processing approach. A binary optical model is developed to enhance anti-noise capability and increase accuracy of time measurement. Variablecontrolled experiments are carried out and analyzed to verify the effectiveness of the proposed processing model. With the model, requirements of hardware system and operation are reduced. The accuracy of time measurement can be stabilized at microsecond level in the presence of noises. The proposed binary optical time measurement has wide application prospect. It may contribute to various fields in photogrammetry, photo-mechanics, communications, and measurements.

Limited by the imagery sensors, hyperspectral images (HSIs) are characterized by their rich spectral information but poor spatial information. With the guidance of the panchromatic (PAN) images, hyperspectral pansharpening aims at achieving a HSI with both the fine spatial detail and the high spectral discrimination ability. Although many deep learning-based methods have gained great attention in recent years, it is still challenging for obtaining an appealing performance. In this paper, we propose a novel detail injection network for the hyperspectral pansharpening, which fully exploits the hierarchical features in both the low resolution HSI and the high-resolution PAN. Specifically, the lowresolution HSIs are firstly upsampled to the desired size, and make a concatenation with the PAN image to formulate a new HSI. The new HSI is sent into a residual dense network, in which residual dense block are designed to extract the abundant local features. Finally, details are injected in hierarchical levels for achieving the acceptable performance. Experimental results and data analysis on two datasets which include both indoor and outdoor scenarios have demonstrated the effectiveness of the proposed method.

For increasing speed of camera external calibration, ensuring the same size of two control point sets and increasing matching rate of camera images after projection transformation, a fast extraction method of control points is established and a point set matching method based on Delaunay triangulation is proposed. Experimental result shows the extraction method can extract all the control points correctly without spurious and lost extraction. In addition, we obtain simulation images with projection transformation according to camera imaging principle. Through simulation we can see the point set matching method can adapt to the projection transformation and improve matching rate in the limited angle range. Lastly, the experimental result shows the external calibration relative error based on our method is below 0.3% compared with manual external calibration.

Hyperspectral image (HSI) super-resolution attracts the great interest in remote sensing, since its effectiveness in obtaining the HSI with rich spatial information while preserving the high spectral discriminative ability, without modifying the imagery equipment. This paper proposes a novel HSI super-resolution method via gradient guided residual dense network (G-RDN), in which the spatial gradient is utilized to guide the super-resolution process. Specifically, there are three modules in the super-resolving process. Firstly, the spatial mapping between the low-resolution HSI and the desired high resolution HSI is learnt via a residual dense network. The residual dense network (RDN) is exploited to fully exploit the hierarchical features learnt from all the convolutional layers. Meanwhile, the gradient detail is extracted via a residual net (ResNet), which is further utilized to guide the super-resolution process. Finally, the fully obtained global hierarchical features is merged with the gradient details via an empirical weight. Experimental results and data analysis on three benchmark datasets show that our method achieves favorable performance.

Multi-view subspace clustering aims to divide a set of multi-source data into several groups according to their underlying subspace structure. However, the similarity matrix which is learned by most existing methods, can not well characterize data themselves more comprehensively in original data space and the global similarity the multi-view data. In this paper, we propose to jointly learn the shared latent representation by different views and the similarity matrix in a unified model. Our model can learn the shared latent representation of multi-view data from the latent space and the similarity matrix from the shared latent representation simultaneously. Experimental results on four benchmark datasets demonstrate that our method outperforms other existing competitive multi-view clustering methods.

At present, human pose estimation with depth images faces some challenges. Methods based on deep learning perform well but rely on massive amounts of data, while traditional machine learning methods are simple to implement but depend on feature extraction and have low accuracy. To deal with them, this paper proposes a novel method based on the Manifold Gaussian Process, which combines tomographic image denoising and feature fusion to solve human pose estimation with depth images. The experimental prediction accuracy on ITOP datasets outperforms other machine learning methods, achieving 83.3% and 77.9% for full body from the front view and top view respectively, which proves the effectiveness of Manifold Gaussian Process on human pose estimation with depth images.

redicting a convincing depth map from a monocular RGB image is a daunting task in the field of computer vision. An efficient detail preserving depth prediction algorithm on the basis of multi-scale deep network and gradient network is presented in this paper. Specifically, the proposed method leverage the designed multi-scale deep network to obtain the global depth image from training datasets. Moreover, a depth gradient generation strategy on the basis of gradient network, which permits us to obtain the local depth detail information of the scene is developed. In the end, the reliable depth map with better details could be reconstructed via merging the depth gradient and depth information on the basis of the optimization algorithm. Experimental results evaluated from the qualitative and quantitative perspective on the Make 3D and NYUv2 datasets indicate that the designed depth prediction scheme is superior to several depth estimation approaches, and can reconstruct plausible depths

Multi-View Stereo (MVS) is a technology that reconstructs the three-dimensional structure of objects or scenes through 2D images and camera parameters. PatchMatch based MVS methods are widely used nowadays. However, these algorithms have two weaknesses: cross-correlation based similarity measurement methods becoming ineffective on regions that are texture-less (e.g. white wall) or have stochastic textures (e.g. grass), as the similarity measurement method such as NCC could not perform well on these regions; Sometimes, the reconstructed result may be stuck into global optimal which heavily drifts from the ground truth because of noise, occlusion and etc. To tackle these issues, we present an MVS pipeline called Adaptive Pixelwise Inference Multi-View Stereo (API-MVS). First, a strategy is proposed to adaptively infer the matching window sizes of each pixel, making the reconstructed results on texture-less or stochastic texture regions have a better trade-off between accuracy and completeness. Second, a new cost function is proposed to integrate the matching cost values computed using different neighboring images, and experiments confirmed that the cost function we used can make locally optimized results closer to the ground truth. We have tested our algorithm on ETH3D benchmarks. The result shows the effectiveness of our method, and it is comparable to state-of-the-art methods.

Based on the papers in IEEE Transactions on Image Processing from 2005 to 2020，using knowledge mapping visualization software Citespace, this article analyzed the knowledge map structure of the research of digital image processing. Furthermore, the research makes a comparative study of digital image processing at home and abroad.

Style transfer is the rendering of the content image into the style of another image, such that the content image achieves the stylization of the style image. In this paper, we propose the cartoonlization style transfer method by using multiple discriminators in the Generative Adversarial Network (GAN). Cartoon images have specific characteristics which include black outlines, smooth surfaces, blurred effects that separate them from real images. We preprocessed the input images by Felzenszwalb segmentation algorithm and Gaussian kernel to extract the individual features from the input cartoon image dataset. In this work, we have introduced the usage of three discriminators for each such feature to help the generator in learning and generalizing the input data distribution. The generator network used in this work contains only 0.8M trainable parameters, which leads to less inference time. This approach shows that our method with the small size generator network can learn to generate superior results as compared to other methods. Qualitative and quantitative comparison with other methods demonstrated the superiority of our method.

Siamese trackers have attracted great attention on visual object tracking due to their real-time speed and high accuracy. In this paper, we propose a dual path aggregation network (SiamDPAN) for high-performance tracking. First, we build a multi-level similarity maps aggregation (MSA) structure, which predicts and fuses the similarity maps from multi-level features. Second, we propose a mask path aggregation module (MPA) for better capturing the appearance changes of objects by propagating maps in low-layers. We conduct sufficient ablation studies to demonstrate the effectiveness of our proposed tracker. We only train our network with two datasets, achieving 0.436 EAO and 0.351 EAO on VOT2016 and VOT2018.

The image acquisition equipment influenced by the dense fog, dust, and other weather factors produced low-contrast and low-quality images, which are unfriendly to smart city applications. Various algorithms, models, and networks have been proposed to achieve the dehazing work, all of which have behaved well to the uniform noise. However, in the real scene, due to the nonhomogeneous distribution of the fog, the performances of the dehazing method still need to be improved. In this paper, we proposed a Feature Attention Dehazing Network based on U-Net and Dense Connection, which is FAD-U-Net, to minimize the influence of the nonhomogeneous haze. The FAD-U-Net employed the U-Net as the base structure used the feature attention model to fit the contracting path of U-Net; in addition, the dense connection technique has been added between each block on both contracting path and expansive path, which aims to enhance the feature information. Finally, a group of 18 residual blocks has been added to the networks to improve the image quality. Experiment results show that the proposed dehazing network outperforms most of the existing dehazing methods under nonhomogeneous fogging environments. Regarding the four state-of-art methods we compared in this paper, the PSNR value is increased by 3.21 on average, and SSIM is increased by 0.14. The FAD-U-Net is suitable for most of the application scene in the smart city.

One of the main methods for dealing with low-rank problems is the matrix completion, which attempts to recover incomplete values in an incomplete matrix. It has been employed in a large number of real-world applications like recommender systems and image recovery. When the data is highly sparse, many practical problems of recommender systems and image recovery are hard to solve. Therefore, we use the fully connected neural networks for the matrix completion to avoid problems caused by sparse matrices. In this paper, we utilize the local density loss function to measure the difference of the matrix completion results, where trainable parameters are updated via calculating the derivatives according to the influence function. The local density loss function effectively measures the deviation between the predicted value and the real value, and the convergence of the model is guaranteed. So as to validate the effectiveness of the proposed method, we conduct substantial experiments in terms of image recovery and recommender systems. In addition, we employ three metrics, including root mean square error and peak signal-to-noise ratio and structure similarity, to measure the recovery accuracy of missing matrix. Experimental results demonstrate that this framework is superior to other state-of-the-art methods in running time and learning performance.

Star tracker is a high-precision attitude sensor that obtains carrier attitude information by processing star images, and star centroid extraction is an important step in the work of star tracker. This paper proposes a star centroid extraction algorithm based on Field Programmable Gate Array (FPGA), which consists of six parts: threshold segmentation, 3X3 matrix generation, pixel weight calculation, connected component labeling, star centroid weight summation, and divider IP core. In order to verify the accuracy of FPGA centroid extraction algorithm, three sets of experiments were conducted. Firstly, generate a noise-free simulated star image, and compare the extraction result which was done on the ModelSim platform with the setting real values to analyze the FPGA centroid extraction accuracy under ideal situation. Secondly, Gaussian noise was added to the simulated star image, and analyzed the FPGA extraction accuracy under noisy situation. Finally, a real star image is extracted on Matlab and ModelSim, respectively, and take the Matlab extraction result as a reference value to evaluate the FPGA extraction accuracy. Simulation and experimental results show that the proposed algorithm has high extraction accuracy ,which can meet the working requirements of star tracker.

At present, the surface defect detection task of mobile phone lens still suffers from low detection accuracy and slow detection speed. To solve these problems, this paper proposes a real-time and effective algorithm based on YOLOv4. Firstly, we combine the cross stage partial block of YOLOv4 and convolutional block attention module, introducing channel attention and spatial attention to learn discriminative features of defects. Secondly, due to the limited differential characteristics of small defects, a novel feature fusion network is designed to further integrate the shallow details with deep semantics. Finally, in order to further boost the detection speed without reducing in accuracy, the proposed model is refined by using the structure tailoring strategies. Compared with YOLOv4 algorithm, our algorithm improves average precision (AP) of linear defect by 2.11%, reduces model size by 13.3% and parameters by 14.14%. Besides, our algorithm improves frames per second (FPS) by 50% and achieves the real-time performance for industrial production. Compared with other algorithms, our algorithm has superior performance in both AP and FPS.

Radar performs target detection by transmitting electromagnetic waves and reflecting them on obstacles. It is widely utilized in military, rescue and other occasions because of the advantages beyond visual range and not restricted by lighting conditions. Since electromagnetic waves have different penetrating capabilities in different frequency bands, it is possible to penetrate the wall to image the life target on the other side of the wall by selecting the appropriate frequency band. When multiple micro-moving targets are detected by life-detection radar, the targets are lost in the image due to the mutual occlusion of the targets. In this paper, a multi-target imaging method based on multi-station radar is proposed. Two stepped-frequency continuous waves (SFCW) MIMO radars detect targets by forming multiple-stations to obtain multi-directional scattering information of targets. A target image is obtained based on a back-projection (BP) image algorithm, and three image fusion methods of information weighted image fusion, fuzzy logic image fusion and M-N-K detector image fusion are adopted to generate the images of multiple micro-moving targets, including occluded targets. In the experiment, the three image fusion methods are analyzed separately, and the experimental results show that the three methods can effectively recognition the occluded targets.

Visual sentiment analysis intends to understand the sentiment evoked by images, which is an important yet challenging field. Existing methods focus on learning single-scale sentiment features from the whole image. However, it has been proved that image emotions are evoked by the sentiment-specific regions of the image, which are related to high-level to low-level sentiment features. To exploit multi-scale sentiment features from both holistic and localized information, we propose a novel end-to-end multi-task framework for joint sentiment- specific regions detection and sentiment classification. Our method contains three components: Multi-Scale Features Extractor, Sentiment-Specific Regions Detection Branch and Sentiment Classification Branch. In the Multi-Scale Features Extractor, by fusing multi-scale features extracted from convolutional neural networks by means of Feature Pyramid Network (FPN) and Adaptively Spatial Feature Fusion (ASFF), the proposed approach first generates a holistic feature carrying more sentiment-related information. Then, the semantic map is automatically discovered from the enhanced holistic feature by adopting attention mechanism in the Sentiment- Specific Regions Detection Branch, which does not require any manual annotations. Finally, the localized and holistic information are adaptively integrated for final sentiment classification in the Sentiment Classification Branch. Extensive experiments on public benchmark datasets demonstrate the robustness and effectiveness of our method in visual sentiment analysis.

In the complex tracking environment, most of existing correlation filter-based tracking algorithms are often unable to track the target stably for a long time. To solve this challenging problem, in this paper, we propose a long-term correlation filter tracking algorithm based on adaptive feature fusion. Firstly, we normalize the response peaks of different features to dynamically assign feature weights for the purpose of combining HOG features and color histogram features adaptively. Secondly, a detection filter is learned to detect the tracking results, and we judge the confidence of detection results by the detection response peak and the normalized value of average peak correlation energy. If the target is in a low confidence state, the re-detection module is employed to relocate the target to achieve long-term tracking. Our experimental results on the OTB-2013 and OTB-2015 benchmark datasets demonstrate that the proposed method performs favorably against some state-of-the-art methods in terms of accuracy and robustness. Furthermore, the proposed method satisfies the accuracy and real-time requirements of long-term tracking in complex environments.

Automatic road defect detection plays a significant role in road upkeep and transportation safety. However, existing approaches still have some shortcomings in detection accuracy, real-time, and hardware requirement. In this paper, we propose a novel anchor-free road defect detection method based on multi-scale hybrid feature fusion. First, we design a lightweight first-order detector to keep more semantic features. Then, we employ a depth separable convolutional layer to reduce the computational complexity. Finally, we propose a hybrid feature fusion framework to improve the feature description capability. Rigorous experimental evaluations on road benchmark data sets demonstrate that our method achieves the highest accuracy and outperforms the YOLO series models. Furthermore, our method has a short inference time of 32ms, which makes it an excellent model in real-time defect detection tasks.

Recently, object trackers based on Siamese deep network have achieved great progress and attract much attention. It can be helpful to improve the performance of a tracker to mining more effective information from feature maps. In this paper, we propose a Siamese network tracker with channel attention mechanism based on the SiamCAR tracker. Firstly, in each residual unit of the ResNet backbone network, each channel feature of the feature map has different importance for discriminating an object. Channel attention mechanism can be used to calculate the importance of each channel feature. Secondly, deep features and lower-level features play different roles for tracking too, and attention mechanisms can be used to fuse the features of each residual stage. On the benchmark datasets, OTB50 and OTB100, the experiments show that our proposed tracker achieves better tracking performance in AUC and Precision and achieves the real-time speed of 46 FPS.

Nuclear Magnetic Resonance Imaging(MRI) is the mainstream way to predict Alzheimer's disease, but the accuracy of traditional machine learning method based on MRI to predict Alzheimer's disease is low. Although Convolutional Neural Network(CNN) can automatically extract image features, convolution operations only focus on local regions and lose global connections. The attention mechanism can focus on local and global information at the same time, and improve the performance of the model by strengthening the key information to suppress invalid information.Therefore, this paper constructs a deep CNN based on multiple attention mechanisms for Alzheimer's disease prediction. Firstly, the MRI image is enhanced by cyclic convolution to enhance the feature information of the original image, so as to improve the prediction accuracy and stability. Secondly, multiple attention mechanisms are introduced to re-calibrate features and adaptively learn feature weights to identify brain regions that are particularly relevant for disease diagnosis. Finally, an improved VGG model is proposed as the backbone network. The maximum pooling is adjusted to average pooling to retain more image information and the network efficiency is improved by reducing the number of neurons in the fully connected layer to suppress over-fitting merging. The experimental results show that the prediction accuracy, sensitivity and specificity of Alzheimer's disease prediction method based on multiple attention mechanism are 99.8%, 99.9% and 99.8%, respectively, which is superior to the existing mainstream methods.

Catenary is an important infrastructure along electric railways, which should be periodically checked to guarantee the normal working status of power supply for the trains. Technologies of ultraviolet detection has been introduced into catenary inspection, and it can be used to detect the electrical state for insulators that are the isolation components between high voltage and low voltage in catenary. However, due to the dynamic detection condition with the running of catenary inspection car, the performance of multi-channel image fusion becomes a key issue in the application on electric railways. In this paper, we proposed a multi-channel image fusion method for the ultraviolet and visible image in dynamic detection on electric railways. The ultraviolet detection module is designed with dual optical coupling alignment, which guarantees the consistency of multi-channel in spatial domain. The post detection is adopted and used as an external trigger signal for both the ultraviolet and visible cameras, which are working in triggered acquisition mode with good time synchronization. Finally, insulator localization algorithms based on deep learning are utilized in the automatic detection of insulator area in visible image, and the ultraviolet signals are successfully extracted from the image fusion results. The performance of multi-channel image fusion method proposed in the paper has been validated, in both lab test and dynamic test on electric railways

Due to the light scattering and absorption phenomenon in underwater medium, underwater images always suffer from contrast degradation, detail loss, and color distortion. Color correction is an important process in dehazing underwater images, while current techniques could not remove color bias accurately. This paper introduces an effective color correction method for underwater image, which mainly contains three steps. First, a novel algorithm is proposed to choose the dominant and degraded channel(s) of input. Then a parameter adaptive-adjusted color compensation algorithm is proposed to compensate the degraded channels locally. Last, the traditional gray-world assumption and the famous histogram equalization operation is conducted on the color-compensated image to get the final result. Our experimental results demonstrate the superior performance of the proposed method in both subjective and objective evaluations.

Aiming at the problem of imperfect utilization of subband statistical information in multi-scale texture image retrieval, a texture image retrieval method based on statistical modeling of all subband coefficients in the dual-tree complex wavelet transform domain is proposed. Firstly, the low-frequency subband coefficients, the high-frequency subband amplitude coefficients, and the high-frequency subband relative phase coefficients are modeled as mixed Gaussian distribution, Weibull distribution, and wrapped Cauchy distribution, respectively. Then, the model parameters are estimated by the expectation-maximization algorithm and the maximum likelihood method respectively, and the closed-form Cauchy-Schwarz divergence and Kullback-Leibler divergence are used as similarity measures for retrieval. Finally, the simulation experiment was carried out in two standard texture image libraries of Vistex and Brodatz. Experimental results show that the method proposed in this paper has better retrieval performance than existing methods.

This article presents a dust image enhancement method based on color correction and haze removal. For color correction, a novel image prior---histogram similarity prior is proposed. With this image prior, image color is recovered by histogram matching. For contrast enhancement, a haze removal method based on boundary condition constraints and Retinex theory is developed, where boundary condition constrains is used for initial transmission map estimation, and Retinex is employed for reducing the effect of the incident light on output images. Considerable evaluations on both synthetic and real dust images show that this method performs favorably against other existing similar methods.

This paper considers how to restore an image with large stained areas. An end to end model is proposed, which contains two networks, an edge generation adversarial network and a content generative adversarial network. First, the edge generation adversarial network is deployed to infer the missing boundaries of the image. Then the second network with a designed edge information channel is employed to restore the missing or stained areas of the image with the guidance of the inferred boundaries. Experiments were performed on ImageNet. The results show that the proposed model can better understand the semantic information of the stained area by introducing additional object contour channels and greatly improve the inpainting capability of the model. Quantitative evaluation indexes show that the proposed model is 4.5% better than the DeepFill V2 model in structural similarity and 7.1% better than the DeepFill V2 model in Peak Signal-to-Noise Ratio.

Due to the absorption and scattering effect of the atmosphere, it is difficult to extract effective target information from the image sensor under severe hazy conditions. We propose an end-to-end convolutional neural network designed to solve the problem of image restoration in scattering imaging. And we explicitly consider the atmospheric scattering model of the hazing process in the network design. The encoder and decoder modules are used for feature extraction, and the pyramid pooling network is used to preserve the multi-scale features of the image. The attention mechanism is introduced in the network. Encoder module is adopted for BoTNet that incorporates self-attention for multiple computer vision tasks. Due to the lack of real hazy images, we collected scattered images under low visibility conditions and corresponding haze-free images. We examine the proposed method by the challenge datasets, the experiments demonstrate that the proposed method can effectively extract and recover the feature information of the target. The results show that, compared with the traditional signal processing method, our model achieves significant practical performance gains and restores the detailed information of the image.

Blind image restoration has always been a hot and difficult problem in the field of image processing. As the vertical motion blur image has obvious characteristics in the frequency domain, the distance between the center point and the nearest horizontal black strip is automatically estimated by setting thresholds, and the estimated vertical motion point spread function is obtained; the image is blindly restored by the constrained least squares algorithm. Experimental results show that the proposed vertical motion point spread function estimation algorithm has high accuracy, and the restored image has better visual effects and higher PSNR.

Aiming at the problem of inaccurate extraction of feature points of unevenly illuminated images and poor matching effect by traditional feature matching algorithms, this paper proposes an AKAZE image matching algorithm combined with single-parameter homomorphic filtering. The algorithm first compensates for the illumination information of the image through single-parameter homomorphic filtering; secondly, it uses the AKAZE algorithm to extract feature points of the image after the single-parameter homomorphic filtering, and then uses Hamming distance as the similarity measure to perform brute force matching on the feature points; Finally, the RANSAC algorithm based on the homography matrix is used for precise matching, and the wrong matching point pairs are eliminated. The experimental results show that the algorithm can not only effectively improve the brightness and contrast of the image, detect more feature points, but also improve the accuracy of matching, which has strong applicability.

Inspired by biological vision, neuromorphic sensors attempt to mimic the sensing and visual processing in the retina. Its working principle is quite different from the conventional camera. Among these revolutionary sensors, the spike camera can report an accumulation of per-pixel brightness intensity by emitting high-speed spikes. The spike camera has enormous potential for computer vision and robotics in all kinds of scenarios. However, the fixed threshold used in the sampling method of the spike camera limits the dynamic range of the scene. Inspired by the HDR of conventional cameras, this paper designs a multi-scale threshold spiking mechanism based on the spike camera and the corresponding reconstruction algorithm, which increases the dynamic range. Experimental results show that it can significantly improve the quality of reconstructed frames.

There are different degrees of “pepper γ salt” noise in most acquired images. To overcome the shortcomings of traditional median filtering algorithm, an adaptive median filtering algorithm is proposed. Firstly, the appropriate filter window size is automatically selected according to the intensity of noise in the image. Secondly, whether each pixel is a noise point is judged. The noise point outputs the median value and the non-noise point outputs the original value. Thirdly, the denoising effect of the two algorithms was compared by simulation experiments, and the influence of different “pepper & salt” noise density on denoising effect was tested. From the perspective of subjective visual effect and objective PSNR value, the denoising effect of adaptive median filtering is better than that of traditional median filtering.

In recent years, image set classification has attracted the attention of many scholars. It is mainly used in complex real scene classification, such as network short video classification, monitoring record classification, and multi-view camera network classification. Pairwise linear regression classification (PLRC) creatively introduces the unrelated subspace to increase the discriminative information, and it achieves a demonstrated better performance on image set classification. However, in the face of large scale image set classification, PLRC is not competent. One possible reason is that it is affected by too many outliers. In order to solve the problem of large scale image set classification, this paper propose two methods. Specifically, one using sparse subspace clustering to mine discriminative features, the other using the self-expressive function to extract exemplar from one image set, both of two methods using PLRC to finish the final classification task, which is not only conductive to reduce the impact of outliers, but also cuts down the computational burden of PLRC. Extensive experiments on two well-known databases prove that the performance of proposed algorithms is better than that of PLRC and several state-of-the-art classifiers.

Upon progression of how to detect a quality of a potato whether it may be a normal or a diseased potato. Potatoes are a tuber crop that populated 60% of the total crop production. Having potatoes that are green on its skin or even thumbnail marks on it is a sign of it not being healthy to consume. This paper presents a research and development to the quality detection of a potato with image processing and support vector machine. The prototype should be able to detect the skin of a potato if it has thumbnail marks and green skin with the support of Microcomputer Raspberry Pi 4 B and an integrated Raspberry Pi Camera 1.3 for image processing and machine learning. With Python as a language used in the creation of the system a series of function such as OpenCV, SKLearn, Image Acquisition, Image processing, Canny Edge, Gaussian Blur, area, and length plotting, and SVM. Fifteen samples of varying potatoes from green skin, thumbnail crack, and normal to know the effectiveness of the design giving a total of ninety photos per sample that gives a result of 96.67% detection accuracy. Hence the system is an effective method on detecting the said defects on the skin of the potato.

In recent times, deep learning methods have been employed to learn anatomical and functional brain changes from high discriminative features extracted from neuroimaging data such as Magnetic Resonance Imaging (MRI) which can enhance the performance in the classification and early diagnosis of neurodegenerative diseases. However, features that exist between brain regions that are farther apart are usually not captured by most state-of-the-art deep learning methods. Thus, an effective and robust model for the extraction of high-dimensional descriptive features especially from brain MRI remains an open challenge. In this paper, we investigate the applicability of an enhanced 3D Residual Network (ResNet) for the extraction of high-dimensional descriptive features for an improved classification of neurodegenerative disease using MRI scans. In particular, we enhanced the ResNet-18 by using a dilated convolutional layer instead of the typical convolution layer to expand the receptive field for effective feature extraction and an attention mechanism to the residual blocks to help focus on the relevant extracted features for improved classification. Our proposed method was evaluated on MRI images from the Alzheimer’s Disease Neuroimaging Initiative (ADNI) database. Three MRI scan groups were considered: Alzheimer’s Disease (AD), Mild Cognitive Impairment (MCI), and Normal Cognitive (NC). Meanwhile, a three-binary classification task was developed (AD vs. NC, AD vs. MCI, and NC vs. MCI) to test the efficacy of our proposed model. The accuracy of our proposed model for each binary task is 92.12%, 74.07%, and 87.16%, respectively. We further compared the robustness of our proposed model to two state-of-the-art architectures and our model performed better due to its ability to extract discriminative features from the MRI data relevant for the classification tasks. Thus, revealing the effectiveness of our proposed method on the MRI scans.

Image classification is an important research branch in the field of computer vision. Ordinary image classification focuses on the discrimination of object differences, while fine-grained image classification focuses on the distinction between different sub-categories. Considering the small differences between sub-categories and the larger differences within sub-categories due to factors such as shooting angle, background, and posture, fine-grained image recognition becomes a challenging task. In order to solve the problem of fine-grained object classification, this paper proposes a finegrained object classification method based on block diagonal feature and ensemble learning. The Gabor feature of the object image is block diagonalized by using low-rank recovery technology, and the low-rank feature representation of the object is constructed by introducing the block diagonal sparse regular term to increase the discriminability of features between object sub-categories. On this basis, the stacking ensemble learning method is used to classify objects, and a fine-grained real animal image and non-real animal image (toy animal image) classification database is constructed. And the algorithm design and experiment are completed. Through the proposed algorithm, accuracy of fine-grained classification of objects is improved

In this paper, we propose a Height-Aware Graph Convolution Network (HA-GCN) to solve the challenging problem of Airborne laser scanning (ALS) point cloud classification. For samples with uneven distribution and large differences in scale, classification using local features is unstable and easily affected by noise. Therefore, we use a multi-layer stacked Edge Convolution (EdgeConv) operators to extract local and global information at the same time. In addition, in view of the characteristics of the height distribution of airborne LiDAR point cloud, we introduce height attention weights as a supplement to feature extraction. First, the original point cloud is divided into sub-blocks and sampled to a fixed number of points. Then, the EdgeConv operator is used to extract local-global features. At the same time, the Height-Aware (HA) module is used to generate attention weights. Finally, the height attention weights are applied to the feature extraction network and the classification is completed after post-processing. The experimental results on the Vaihingen dataset show that the proposed method achieves the effect of the state-of-the-art methods in overall accuracy, as well as impressive results in single-category classification accuracy.

Semantic segmentation is a key step of image prehension. Single use of convolutional networks for semantic segmentation makes it difficult to distinguish the same class of objects with large contour deviations, while the higherlevel features will lose some the detailed information. Currently, Networks such as ACFNet and DANet have introduced attention mechanism to improve scene classification by obtaining rich contextual information through self-controlled system, but they do not combine both global scope and class feature relationships in local space to further advance intraclass consistency and inter-class divisibility of features. In terms of this problem, semantic segmentation network of categorical attention with spatial constraints has been proposed, which contains two submodules, one using the category spatial distribution to introduce local spatial location information of features, and the other using the global category average strength to introduce global strength information of category features. By selecting a kind of appropriate backbone network, this network model obtains the feature map from the backbone network and stacks the features into the original features after two submodules of global category strength and category local space processing, finally, performing classification processing by the classification layer and up-samples to the input image size to complete the pixel-level label prediction. The experiment result demonstrates that this proposed segmentation network has higher accuracy than existing segmentation networks.

Mobile Laser System and Airborne Laser System can quickly collect a large quantity of urban scene point cloud data in real time, the collected point cloud data is the main source of road area extraction. However, the obtained point cloud data are redundant and unordered discrete, which are challenging for efficient classification and extraction. In order to solve these problems, we propose a road extraction method based on the difference of normal vector: 1) preprocess the data for simplifying the point cloud, making the subsequent operations more efficiently; 2) use the progressive morphological filter to obtain the ground point cloud data, and then calculate the difference of normal vector for the clusters of the point cloud to get the preliminary road area; 3) leverage the random sample consensus plane fitting method to optimize the road area. The experimental results show that the proposed method can extract the road area accurately and quickly from the urban 3D point cloud data.

3D point cloud simplification is an important pre-processing task in geometric reconstruction to save computer resources and increase reconstruction speed. Compared with traditional methods, the existing point cloud simplification algorithm based on k-means has a better effect on the extraction and retention of feature points, while there will be many larger holes near the feature points. In this paper, we propose a scattered point cloud simplification algorithm (SPSA). The SPSA strikes a balance between retaining enough feature points and keeping the sample points evenly distributed. The SPSA uses boundary search and the fast point feature histogram to extract feature points. Further, we apply the improved farthest point sampling algorithm to sample non-feature regions to ensure the integrity of the simplified model. Extensive experiment results on standardized information entropy evaluation show that the SPSA outperforms other simplification algorithms. In addition, the visualization results indicate that the proposed method can ensure the integrity of the simplified model and retain the most of feature information of the point cloud.

3D reconstruction from a single RGB image is making some progress especially with the advent of deep learning in recent years. In this paper, we focus on 3D reconstruction of indoor scenes taking as input a single image, without point clouds, multi-view images, depth or masks. Our mesh reconstruction is based on an encoder-decoder framework of mesh deformation. Features extracted by the encoder from the input image will be used as supervision for mesh deformation. We propose an inter-point interaction attention in the decoder to exploit the vertexes’ influence on each other. We also use a smooth loss to generate smoother surface for objects. We have evaluated the proposed framework on the Pix3D dataset, and state-of-the-art performance has been achieved with visually appealing 3D geometry.

The survival time of high-grade and low-grade gliomas is different, so it is very important to accurately identify the grade of gliomas and develop personalized treatment plans for patients. We aim to non-invasively differentiate glioma grades based on the deep learning radiology (DLR) model of multiplanar reconstruction of contrast-enhanced T1-weighted (CE-T1W) images. First, we included 122 and 52 patients with gliomas diagnosed by pathology in the two institutions, respectively, and made sure that these cases had MPR axial, coronal, and sagittal CE-T1W images. Then, we extract the radiomics features from CE-T1W images and the deep learning features of the VGG6 pre-training model respectively. Spearman and recursive feature elimination (RFE) feature selection methods are used to select important features, and support vector machine (SVM) and logical regression (LR) modeling are used to distinguish high-grade and low-grade gliomas. Finally, the area under receiver operating curve (AUC), sensitivity, specificity, and accuracy were evaluated in an independent test set. In SVM and LR models that use radiomics features, the result of the MPR three-phase merge model is better than that of the MPR single-phase model. In LR, the merged model of deep learning and radiomics is better than the model of using both alone. Therefore, the CE-T1W image MPR three-phase merge feature is superior to the singlephase feature in differentiating high-grade and low-grade gliomas. The model combined with radiomics features and deep learning features can improve the prediction accuracy of glioma grading.

This paper proposes a 3D reconstruction method for the cross-scale narrow space containing thin structure. This method takes monocular image sequences as input and respectively constructs large-scale scene point cloud of main part based on incremental SfM and small-scale thin structure point cloud based on the curve method. The small-scale thin structure point cloud reconstructed individually alleviates the defect of incomplete point cloud from the large-scale scene. By introducing PCA for initial registration, alternatively iterating the closest point algorithm and the global optimization algorithm, we achieve a more precise registration for the thin structure point cloud. Then, a replacement algorithm is used to fuse it with the point cloud of the main part. The experiment containing two scenes verifies the effectiveness of our method, which reconstructs point cloud of the thin structure with high accuracy and retains the overall 3D information of the scene at the same time.

Image super-resolution is to restore a high-resolution image from a low-resolution image or image sequence. High resolution means that the image has a high pixel density and can provide more details, which often play a key role in the application. Aiming at the application of single-frame low-resolution reconstruction and super-resolution, this paper proposes a method based on image fusion. This method combines two or more methods of super-resolution image reconstruction using generative adversarial neural networks. The reconstructed images are fused. Image fusion uses the integration of two or more images into a new image. Fusion can make use of the temporal and spatial correlation and information complementarity of two or more images, which can make the image obtained after fusion have a more comprehensive and clear description of the scene, which is more conducive to human eye recognition. This paper draws on the idea of ensemble learning, and uses the super-resolution images generated by the three super-resolution reconstruction algorithms of BasicSR, SRGAN and ESRGAN to carry out two-by-two cross fusion for simulation experiments. The experimental results show that this kind of reconstruction using different generation adversarial networks to generate the super-resolution image by fusion is simple and effective. The super-resolution image quality after fusion is generally better than the image quality before fusion in terms of PSNR and SSIM.

Estimating a six-degree-of-freedom pose from a set of correspondences remains a popular solution for 3D point cloud registration. The random sample consensus (RANSAC) method is a typical pose estimator for this task. However, RANSAC still suffers from several limitations including low efficiency and the sensitivity to high outlier ratios. To tackle these problems, we propose a 1-point sample consensus method. It first constructs a local reference frame for the keypoint based on multi-scale normal vectors, which allows our method to exhibit a linear time complexity. Then, we propose a novel hypothesis evaluation method that concentrates on accurate inliers and is more reliable for hypothesis evaluation. With comparisons with two RANSAC-like methods, our method manages to achieve more accurate and efficient registrations, making it a good gift for practical applications.

Despite that 3D human body reconstruction from a single image has obtained rapid progress in recent years, most methods aim at the body without the hands and face. However, hand gestures and facial expressions are also important for delivering human intentions or emotions. This paper proposes a method for holistic 3D reconstruction of the human body from a single RGB image, including hands, body, and face. Our approach is based on the SMPL eXpressive (SMPL-X), a unified 3D parametric human body model of body, hands, and face. Since it is difficult to exactly regress the model's parameters of different body parts by a single framework, we use a divide-and-conquer strategy for the whole human body reconstruction. We exploit different deep neural networks to predict the hand, body, and head model's parameters, then integrate them into an entire 3D model to realize a holistic and expressive 3D human body reconstruction. Simulation results demonstrate that our method has obtained state-of-the-art performance with better facial expression.

In this paper, we propose a cross-modal unsupervised domain adaptive cardiac substructure automatic segmentation method to address the problem of a sharp decline in performance of a neural network trained in a specific imaging domain to migrate to another target domain that lacks annotations. The proposed multiple generative adversarial networks guided by self-attention framework is called GANSA for short. GANSA eliminates the differences in image appearance and features between different domains (MRI and CT) through GANs, and uses cyclic consistency loss to ensure that the features of the image itself remain unchanged. We use the self-attention mechanism to control GANs, which ensures the details of the generated image, and pays attention to the connection of long-distance information of the image, in order to complete the cross-modal cardiac segmentation task. The proposed framework is trained and tested on the Multi-Modality Whole Heart Segmentation (MM-WHS) Challenge 2017 dataset. Experimental results show that our method outperforms the state-of-the-art method.

Based on the requirements of large-scale and high-resolution remote sensing image data processing, this paper proposes a distributed parallel processing model based on sea and land segmentation tasks. Based on the trained DeepUnet model, the mpi4py function library is used for parallel algorithm design to realize multi-process synchronization processing. Increase the number of processors and reduce the processing time of large-scale and high-resolution remote sensing image data. The experimental results show that on the basis of ensuring the detection accuracy, the parallel sea-land segmentation technology can significantly shorten the image processing time compared with the traditional serial sea-land segmentation technology, and has strong scalability.

Segmenting the pelvic region accurately and quickly from T2-weighted MR images is beneficial for the treatment of pelvic-related diseases. Nevertheless, there are great challenges in segmenting the pelvic region due to its varying scale range, similar tissue strength, and blurred edges. Although the well-known encoder-decoder convolutional neural network has achieved great success in the field of medical image segmentation, its ordinary convolutional layer is insufficient to adequately extract and convey features. Moreover, this architecture does not make the full utilization of different scale contextual information and sometimes fails to capture long-range dependencies. To address these challenges, this paper proposes a DFM-Net which applies Dense Block as the basic module to extract features and enhance the transfer of features. Aiming at the problem of tissue strength similarity, the non-local operation Feature Similarity Module (FSM) was introduced to capture long-range dependencies of the pelvis structure. To enrich the extraction of global and local context information, a new multi-scale method is applied to optimize the model during the up-sampling process of the decoder. Finally, experiments and evaluations were performed on the T2-weighted MR image data set of the pelvis. The encouraging results show that our proposed method is superior to the other five different methods of segmentation.

Intensity inhomogeneity is widely present in real images. When there is intensity inhomogeneity in the image, the curve evolution will be guided by the wrong target edge, which seriously affects the efficiency and accuracy of image segmentation. Aiming at the intensity inhomogeneity distribution in the medical image, this paper proposes an image segmentation algorithm that combines moving grid method and level set method. The grid is encrypted at the uneven intensity level to make the image uniform and applied to the image segmentation. By calculating a monitor function about image intensity gradient, this paper makes grid encryption automatically implemented with the change of image gradient. This makes it more accurate to calculate image segmentation at uneven intensity. In order to improve efficiency, this paper will reduce the grid to 1/4. Then the level set method is applied to achieve computational acceleration. The final results show that the model in this paper has high segmentation accuracy and efficiency for intensity inhomogeneity images.

3D Medical image segmentation is a basic and key task in computer-aided diagnosis, and glioma has a high degree of non-uniformity and irregular shape in multimodal MRI images. Therefore, accurate and reliable segmentation of brain tumors is still a challenging work in medical image analysis. U-Net has become the de facto standard in various medical image segmentation tasks and has achieved great success. However, due to the inherent locality of convolution operations, U-Net is generally limited in terms of explicitly modeling long-term dependencies. Although this problem is solved in Transformer, it has extreme complexity in terms of calculation and space when processing high-resolution 3D feature maps. In this paper, we propose the Trans-coder, which is embedded in the end of the U-Net encoder to improve the segmentation performance while reducing the amount of calculation. The Trans-coder takes the feature map from U-Net as the input sequence, and extracts the relative position information of the feature map, so as to get more detailed information of the image, and input it into the decoder to obtain good segmentation performance. At the same time, a variational autoencoder is used for regularization to prevent over-fitting problems. our method achieves superior performances to certain competing methods on the Multimodal Brain Tumor Segmentation Challenge (Brats) dataset.

With the large number of face recognition devices deployed in real application scenarios, face anti-spoofing has become a hot topic nowadays. Previous methods are mostly based on handcrafted features, while recent methods are mostly based on neural networks. However, both the traditional hand-crafted based method and the deep learning methods are still faced with the problem of insufficient generalization ability. In traditional deep learning methods for classification tasks, the label of samples is often a code of the category name. Recent studies have also shown that besides color and distortion, the depth information of face is also considered as an important clue to distinguish real and fake face. In order to combine prior knowledge of face depth information with deep learning method, it is a way worth exploring to expand the label information by using estimated depth image labels instead of coding labels. In this paper, we proposed an auxiliary supervised method to extend label information by using estimated depth information of face. A SA-UNet model which combined spatial attention modules with classic UNet is proposed to generate the depth estimation image for face anti-spoofing. Moreover, contrast depth loss is introduced to focus on the neighborhood information of the pixels, and a scoring method based on the proportion of non-background area is proposed to do the classification. In order to measure the generalization ability of our method, we choose CASIA FASD dataset, Idiap Replay Attack dataset and OULUNPU dataset for experimental verification. Experimental results show that our method is effective for face anti-spoofing task.

Visual question answering task requires utilizing the content of the question to locate the corresponding regions of the image. But the traditional attention-based VQA methods can not accurately match the regions of the image that is relevant to the question, which results in less satisfactory performance. In this paper, an Efficient Multi-step Reasoning Attention Network (EMRA), which is mainly composed of the multi-step reasoning attention module and the G-LReLU non-linear layers, is proposed to address this problem. Specifically, the multi-step reasoning attention module combines the initial visual features, question features and the jointed features to generate more effective attended features, which can precisely represent the regions information of an image related to question. Then, the attended visual features generated by multi-step reasoning attention model and the question features are fed into the G-LReLU non-linear layers executing non-linear transformation to better fusion for answer prediction. In addition, considering the relationship between the scaling and the reasoning steps, as the number of inference steps increases, increasing the model width will improve the accuracy of our model. Experimental results on the VQA v2.0 dataset demonstrate that our model significantly outperforms the Bottom up and Top-down Attention based methods, and can be competitive with state-of-the-art models.

Cryptography based on the properties such as unpredictability and initial state sensitivity of chaotic systems is a strategy to improve the security of image encryption algorithms. In this paper, a Logistic Coupled with Sine (LCS) chaotic map is proposed, and it turns out that the proposed map has more complex behaviors than existing chaotic maps in terms of performance analyses. Furthermore, as a potential application, an efficient and effective image encryption algorithm is proposed based on the keystreams generated by this LCS map. In the encryption algorithm, for a given plain image, it is split by the parity of indices of image rows first, and then a new confusion-diffusion strategy is proposed to scramble and diffuse thoroughly the elements of these two divided parts of the image. The security analyses indicate that the proposed encryption algorithm has excellent statistical properties, high security and efficiency for image encryption.

This paper proposes a joint trajectory prediction algorithm based on graph neural networks and map scene information assistance. And proposed a kernel function to calculate the influence between different targets in the scene.

Aiming at the problems of limited parameter range of traditional one-dimensional mapping, narrow chaotic range, and insufficient complexity, this paper proposes a compound mapping based on R-order Chebyshev. The Lyapunov exponent, uniformity and complexity analysis of the R-order compound Chebyshev map are performed, and the NIST randomness test is performed on it to verify its good randomness. The R-order Chebyshev composite map is applied to the scrambling and diffusion stages of the image encryption system. Through comparative research with other encryption algorithms, the image encryption algorithm proposed in this paper has high security and can effectively resist common attacks

As a pivotal technique of fashion visual analysis, fashion landmark detection has attracted extensive attention in recent years. However, prior works often ignore the importance of structural layout information for fashion landmark detection, which leads to ambiguous detection results of hard landmarks. In this paper, we propose a Layout-Aware Bidirectional Transfer Network(LBTNet) which first combines the layout features learning with a powerful human pose estimation backbone - HRNet. The LBTNet can learn the layout information by our proposed Group-wise Layout Embedded Module(GLEM) which can model the dependency among fashion landmarks on the convolutional layer and perform information passing among the adjacent landmarks. We also design a novel head structure called Bidirectional Transfer Module(BTM) to capture global semantic information of fashion landmarks through a bidirectional transmission path. Therefore, the LBTNet can accurately detect these hard landmarks (e.g. occluded landmarks and invisible landmarks). And the experimental results on two large-scale fashion datasets show that our LBTNet outperforms the state-of-the-art methods by a large margin.

The band design of satellite payload is bound up with its application. Generally speaking, visible and near-infrared (NIR) wavelength bands are included. Panchromatic and four multispectral bands, i.e., blue, green, red, and NIR wavelength bands have become the standard configuration of satellite optical payloads. In this study, based on the band settings, in-orbit operation status, imaging conditions and surface features, a color-enhanced remote sensing image processing method for is proposed for earth observation satellites in view of the application requirements. What’s more, the reasonable loading of the NIR band information lays a foundation for subsequent image visual interpretation and application display. Based on the above color restoration and enhancement processing, the brightness and sharpness of the image are enhanced to meet the visual effect by scientifically setting parameters, especially by improving the greenness of vegetation.

Pedestrian target recognition in multiple videos is a hot issue in video image processing. Many existing recognition algorithms have problems in extracting pedestrian image features, such as multiple dimensions, slow recognition speed and so on. In this paper, a pedestrian recognition method with multiple angles in different videos based on the fusion of HOG, LBP, HSV features and XGBoost classify is proposed. First, use the coefficient variation to reduce the HOG feature dimension; then use LBP to extract the strong expressive features as the local features of the image; then fuse the HOG, LBP and HSV to enhance the ability of pedestrian target features; finally, use the XGBoost algorithm to classify the fusion features, use Cosine similarity to calculate the distance between target and candidates. Experiments show that the proposed method has greatly improved the recognition speed and recognition accuracy in public data sets and selfmade data sets.

Deep learning has developed rapidly and made unprecedented achievements especially in the field of image cognition, since Alex et al. proposed AlexNet in 2012. This paper focuses on the mainstream research methods of traffic sign detection and recognition, including the traditional feature-based image processing method and the target detection method based on deep learning. The traffic sign detection and recognition method based on deep learning is divided into two categories for discussion and analysis, namely, the Anchor based and Anchor-Free neural network architectures. Finally, the paper makes a brief summary and prospects the future development.

Natural scene text recognition is one of the most challenging tasks in recent years. Compared with traditional document text, natural scene text has the characteristics of various shapes and different directions, so the accuracy of scene text recognition still needs to be improved. In order to locate the text region better and identify the text content more accurate, we present a multi-scale deformable convolution network model for text recognition. The initial image is irregularly corrected through the rectified network, and the ResNet with FPN structure is used as the backbone network to achieve multi-scale feature extraction. In addition, the feature fusion method of Add is adopted to reduce feature information losing and increase the strength of feature extraction in the text area. The deformable convolution block is introduced in the deep convolution to improve the deformation modeling ability of convolution and expand the receptive field. The prediction module adopts the Transformer and abandons the inherent pre and post attributes of RNN to realize parallel operation and solve the problem of path length between remote dependencies. In order to evaluate the effectiveness of the proposed method, we trained our model on two mixed data sets, MJSynth and SynthText, and tested it on some regular and irregular data sets. The experiment results demonstrate that this method performs well in irregular scene text recognition, especially in CUTE80.

Images can convey rich semantic information and arouse strong emotions in the viewer. With the growing trend of online images and videos to express opinions, evaluating emotions from visual content has attracted considerable attention. Image emotion recognition aims to classify the emotions conveyed by images automatically. The existing image sentiment classification studies using manual features or deep models mainly focus on low-level visual features or high-level semantic representation without considering all factors. In this paper, we adopt visualization to study the working principle of deep representation in emotion recognition. Research shows that the deep model mainly relies on deep semantic information while ignoring the features of shallow visual details, which are essential to evoke emotions. To form a more discriminative representation for emotion recognition, we propose a multi-level representation model with side branches that learns and integrates different depth representations of the backbone for sentiment analysis. Unlike the hierarchy CNN structure, our model provides a description from the deep semantic representation to shallow visual representation. Additionally, several feature fusion approaches are analyzed and discussed to optimize the deep model. Extensive experiments on several image emotion recognition datasets show that our model outperforms various existing methods.

Most state-of-art deep learning based no reference image quality assessment (NR IQA) methods usually have a complex network structure which are memory-consuming, hard to train and so not applicable in practical scenarios. Aim at these problems, a RepVGG based NR-IQA algorithm with transfer learning is proposed. The method uses ImageNet dataset to pre-train RepVGG network to get network parameters, and then uses the trained network to extract image features of image quality assessment data set. Finally, a simple fully connected network is trained to get the quality score of the image based on these features. The experimental results show that on the KADID-10K, LIVE、TID2013 and CSID datasets, the overall objective assessment obtained by the method is better than the state-of-art deep learning-based methods with good consistency with the subjective assessment.

There is a tendency to deal with a speaker-independent recognition task in the lip-reading field by collecting speech scenes from many speakers. The data collection task is time-consuming. This paper proposes a method to solve this problem. According to a driving video, First Order Motion Model (FOMM) is a deep generative model that generates a video sequence from a source image. Our idea is to apply FOMM to all speech scenes in the dataset to generate the speech scenes recording from one speaker. We propose a preprocessing method to replace the speaker-independent recognition task with the speaker-dependent recognition task by applying FOMM. We applied the proposed method to two publicly available databases: OuluVS and CUAVE, and confirmed that the recognition accuracy was improved by applying the proposed method to both databases.

Personality analysis defines the task of computing human personality based on their behavior. Traditionally psychological personality research experiment depending on the subjective answers of participants has many inherent challenges. In this paper, we propose a multimodal personality analysis model based on audio features, facial features and environmental features to accurately analyze Big Five traits of people personality. In the feature extraction stage, MFCC features are extracted as the representation of audio data, for facial and environmental characteristics, ResNet and LSTM are used to extract features and integrate time information respectively. We then evaluate our model using the ChaLearn First Impressions V2 challenge dataset and finally reached an accuracy of 89.12%. We demonstrate that multimodal method is more effective for personality analysis, comparing with most of the methods using single modal. Finally, a representational personality analysis system is presented based on pyqt5. It has the function of taking photos or uploading videos to portray the character's personality.

Endometrial receptivity assessment based on the ultrasound image is a common and non-invasive way in clinician practice. Clinicians consider that the thickness of the endometrium is one of the most important assessment markers, which can be calculated with the endometrial region in ultrasound images. Suffering from low contrast of the boundaries in ultrasound images, it’s a challenge that makes accurate segmentation of endometrial for thickness calculation. An automated assessment framework with a multi-task learning segmentation network is proposed in this paper. The VGGbased U-net is trained with an auxiliary pattern classification task, the losses of different tasks are combined by weighted sum based on uncertainty in the training phase. Experiment shows that the network has a more accurate prediction than single-task learning and the framework does a better thickness calculation.

CT image technology has been widely used in recent research on dental imaging. High-quality CT images can be accessed by adopting a high-dose radiation dose that unfortunately exerts a detrimental influence on the human body. Reducing the radiation dose is a solution to eliminate the negative effect, while it will put noise into the CT images. In this paper, a dental CT image denoising method is proposed based on the generative adversarial network. A total of 6144 pairs in which each contained a noise dental CT image and a corresponding original dental CT image were trained through the model of the generative adversarial network. The test was carried out on the original untrained image. Comparing with the image without denoising, the test results indicate a significant improvement on the results. For example, the PSNR is increased by 21.52% and the SSIM is increased by 52.95%. In the comparison with the professional artificial vision performance, the result has also been significantly improved, which proves the effectiveness of the method in this paper. Moreover, the method proposed in this paper merely needs little training data. It is foreseeable that with the increasing of the training data, this method will have a better performance in the aspect of noise reduction in dental CT images.

Mixed reality (MR) is the merging of real and virtual worlds to produce new environments and visualizations, where virtual and physical objects can interact with each other in real time. The traditional fixed single camera desktop AR platform provides a new display and interaction mode for experimental teaching in middle schools. However, to observe the experimental phenomenon, we have to use a fixed-position screen with a fixed viewing angle which makes the experiment operation unnatural. The use of traditional Hamming code mark can not let people know what kind of AR object it is. In this paper, we propose a virtual experiment platform that can conduct various experiments in middle school. In this virtual platform, users observe the experimental phenomena with a pair of movable MR glasses. Meanwhile, the mark images required for interaction are customized and indicates what the AR object is. In this way, users can select the AR objects they need easily during the experiment instead of memorizing the AR objects corresponding to different marks in advance. However, the value and importance of different modules are different. What criteria do we use to judge the value of different modules? Kano's Theory provides the guidance of requirements classification attributes for the research of virtual experimental platform. Therefore, we evaluate five requirements classification attributes, including the function of camera capture, custom mark, visualization of magnetic line, voice interaction and multi-camera tracking. Nineteen pedagogical volunteers participated in our experiment. Experimental results prove that our platform stimulates the users’ interest in learning. Further, most participants believe that it would promote the development of experiment education effectively in middle schools.

Multimodal medical image fusion can provide comprehensive and rich information for doctors' diagnosis. Aiming at the problems of traditional fusion algorithms, such as PET/SPECT color distortion, insufficient MR texture and timeconsuming, this paper proposes a new multi-modal medical image fusion algorithm. Firstly, a nonsubsampled shearlet transform (NSST) was introduced to perform multi-scale decomposition of the source image to obtain low frequency and high frequency subbands. Then, since the low frequency subband image contains most of the intensity energy of the source image, which is divided into high energy region and low energy region according to the maximum between-class variance method, and the adaptive weighted fusion rule is proposed, which is beneficial to the high fidelity of the fused image and the visual effect is better. High-frequency subband have strong sparsity characteristics, adopting the maximum value fusion rule, and the image texture after fusion is clear. Finally, inverse NSST is performed on the fused low-frequency and high-frequency subbands to obtain the fused image. Compared with the representative medical image fusion algorithms in recent years, good results have been obtained in evaluation and computational efficiency.

Research travel have gradually been applied and studied by many schools in teaching practice, but under many interference factors, the vigorous implementation of research travel has not been carried out successfully. And virtual reality technology can effectively solve this problem, help students conduct practical learning, and improve students' inquiry ability. This research developed a virtual research program which is taking Suzhou Shantang Street as an example to explore the application of virtual reality technology in the middle school research travel course.

An automatic matching interactive and visualization system for virtual martial arts is proposed in this paper. The system is developed to improve the user's interest in the intangible cultural heritage of martial arts. Four basic actions in Wing Chun are designed as an example. The body joint coordinate information retrieval is employed to estimate the matching degree between the player's and template actions. We visualize the user's hand movement data according to the speed. The results of user study show that the system is helpful for the beginner to understand martial arts, and it also attracts the user's curiosity and indirectly improves the user's interest in martial arts. Moreover, the system is relatively comfortable in usability, because the movement is captured by the Kinect, which has little restraints on the user's body.

In recent years, machine learning methods have been extensively studied in Alzheimer's disease (AD) prediction. Most existing methods extract the handcraft features from images and then train a classifier for prediction. Although it has good performance, it has some deficiencies in essence, such as relying too much on image preprocessing, easily ignoring the latent lesion features. This paper proposes a deep learning network model based on the attention mechanism to learn the latent features of PET images for AD prediction. Firstly, we design a novel backbone network based on ResNet18 to capture the potential features of the lesion and avoid the problems of gradient disappearance and gradient explosion. Secondly, we add the channel attention mechanism so that the model can learn to use global information to selectively emphasize information features and suppress low-value features, which is conducive to the extraction of semantic features. Finally, we expand the data by horizontal flipping and random flipping, which reduces the over-fitting phenomenon caused by the limited medical data set and improves the generalization ability of the model. This method is evaluated on 238 brain PET images collected in the ADNI database, and the prediction accuracy is 94.2%, which is better than most mainstream algorithms.

With the development and popularization of virtual reality (VR) technology, it has been widely used in the field of online experimental education. Due to the large volume of VR digital assets for online experimental education, the existing VR digital asset management systems have problems such as low bandwidth utilization and high latency. In this research, in order to alleviate these problems, we designed a novel VR digital asset distributed management system, which has the ability to integrate centralized cloud storage and distributed storage at the same time, has secondary development interfaces, has multiple customer service terminals, supports OSS, IPFS and other storage modes, and has developed a complete set of Restful API interfaces to facilitate upper-level application calls. Finally, experiments demonstrate that the system has the characteristics of decentralization, high bandwidth utilization and low latency.

Human-computer interaction has always been a hot research direction in the field of computers, and augmented reality technology can combine the virtual world and the real world, which has a wide application prospect. In recent years, augmented reality based on gesture recognition interaction has been greatly developed. This paper proposed three modules: Gesture recognition module based on MediaPipe framework, Special effect generation module based on the posture estimation model mixed with feature points and straight lines, Intelligent media interactive application development module. Through the above modules, an intelligent media interactive application based on gesture recognition to generate 2D/3D interactive effects in real-time has been designed and developed.

Human action recognition is a basic problem in video tasks. The deep learning method has profound theoretical research significance and strong application value in intelligent monitoring, automatic driving, medical care and other aspects. Firstly, the paper summarizes the video preprocessing and the improvement of network structure, and concludes the latest and hottest deep learning methods in the field of human action recognition. Then two kinds of datasets related to human action recognition are listed and introduced in detail.

Identifying the property of sandstone slice has important directive to exploration of oil and gas resources. The traditional sandstone slice identification method based on manual observation, which is time-consuming and laborintensive. In recent years, the Convolutional Neural Networks (CNNs) have achieved excellent results in the field of image recognition and a CNN-supported recognition algorithm has practical value in various application. However, there are multiple challenges for using the classic CNN for the sandstone slice image recognition: the industrialized recognition needs efficiency, which means the sandstone slice image recognition should be finished accurately and quickly; the sandstone image recognition requires expert annotations, resulting in lack of data; some mineral particles in sandstone are uncommon, which caused the sandstone image dataset has unbalanced data distribution problem. We invited several geologists to annotate sandstone slice image and start experiments. For sandstone slice image recognition, in order to ensure recognition speed and accuracy of sandstone slice, we built the Res2DwNet model. The model is optimized for the characteristics of the sandstone slice image. In this model, we proposed the Res2Dw module, which combines deep separable convolution, dense connection and residual learning ideas. Combining geology expertise, the sandstone slice image in our model is divided into two types of single-polarized and cross-polarized images for processing, which helps the recognition accuracy improves significantly. To solve the problem of small amount of data, a unique data enhancement method is designed on the basis of the experience of sandstone slice identification. To handle the uneven data distribution problem, the class-balanced softmax loss function is used during training. Compared with other classic Convolutional Neural Network models, the Res2DwNet model has a deeper network structure, which can achieve higher recognition accuracy than models of the same magnitude while maintaining lightweight parameters, and is suitable for sandstone slice image recognition tasks.

Skeleton-based human action recognition has achieved a great interest in recent years. The strong robustness of skeleton data to scene and camera interference makes the recognition algorithm follow with interest robust features of actions. Recent works has proved the effectiveness of skeleton modeling based on graph and learning spatio-temporal modes by Graph Convolutional Network (GCN). Although GCNs have excellent ability of neighborhood feature learning, it is not good at capturing long-distance dependence between joints. In particular, linear temporal skeleton sequences contain a great quantity of joints., which makes the process of learning advanced temporal cues unduly slow. In this paper, we propose a temporal feature enhancer using Temporal Kernel Attention (TKA). And guided by TKA, we design a performance-oriented network TKA-GCN and a lightweight network Mini-TKA-GCN for skeleton-based action recognition. Finally, on NTU-RGBD 60 and Kinetics-Skeleton 400 datasets, TKA-GCN and Mini-TKA-GCN proposed by this work, outperform most advanced works.

Patients with intractable neuropathy may use a mouth-shape character (MSC)-based communication as an alternative to communication when speech, writing, and PC are unavailable. MSC-based communication requires not only a speaker but also a supporter. However, it may not be easy to read the mouth shape due to the skill of the supporter. The communication support system that automatically recognizes the mouth shape of a speaker is expected. This study aims to develop the whole support system, and this paper works on mouth shape recognition. We introduce 3DCNN-based mouth shape recognition. As for the input data of CNN, not only the color image but also the flow image obtained by applying the optical flow is used, and two outputs of two CNN models are integrated. We collected speech scenes of eight patients with intractable neurological diseases and conducted recognition experiments. As a result, an average recognition rate of 77.1% was obtained. Excluding the two patients, one had difficulty recognizing the mouth shape even by a human due to little movement of the mouth, and the other had a problem shooting, and the average recognition rate of 86.6% was obtained. We demonstrated the effectiveness of the proposed method.

Human action recognition task has gradually become one of the most popular research topics in the field of computer vision. In this task, the action recognition based on human bone data is the most attractive. The human skeleton data contains a lot of correlation information and hidden information, so this kind of task model can well extract the difference characteristics and human movement trajectory, etc., which plays a key role in improving the accuracy of the task. At the same time, the skeleton-based action recognition algorithm based on CNN, RNN, GCN, LSTM and other basic models improves the task capability from the aspects of accuracy, computational complexity and so on. From this perspective, this paper reviews the deep learning models and variants of action recognition based on skeleton data, and also summarizes the bone information datasets used for such tasks.

Attention mechanism is one of the most basic and core tasks in computer vision. Its essence is to locate the information in the region of interest and suppress useless information. The results are usually displayed in the form of probability graph or probability eigenvector. Attention mechanism has become an important concept in convolutional neural network, which has been widely studied in different application fields and has strong practical value. This paper introduces the classification of attention mechanism and its application in fine-grained image recognition. The classification is mainly divided into channel attention mechanism, spatial attention mechanism and channel spatial mixed attention mechanism. Finally, the future research direction of attention mechanism in fine-grained images is discussed.

Object detection is one of the basic problems in computer vision. Currently, the detection models based on fully supervised learning which demand fine labeled data such as bounding box annotated images are the mainstream of this research field. However, obtaining high-precision tagged images usually costs huge time and human labor. To lighten the restriction for training data of detection models, we propose an attention-based weakly supervised object detection model which can be trained only using image-level annotated images. The weakly supervised object detection model consists of two stages. In the first stage, an attention-based convolutional neural network (CNN) is designed to enhance the localization ability of CNN and generate coarse detection results. In the second stage, a neural network for edge detection is utilized to get the fine results based on the coarse results in stage one. Tested on PascalVOC 2007 and 2012, the proposed weakly supervised learning detection model achieves 53.4mAP and 48.9mAP in these two datasets, respectively, which is competitive with the state-of-the-art weakly supervised learning detection models and reduces the gap with the fully supervised learning detection models.

In order to solve the problem that it is difficult to control the abuse of high beam at night, a high beam detection algorithm YOLO-CAU based on YOLOv4 is designed and implemented. Then, aiming at the problem that it is difficult deployed on devices with limited computing power with a complex neural network, we apply the model compression algorithm of knowledge distillation to the image object detection task, and conducts distillation training on the YOLOCAU model with the high beam data set, transferring the knowledge learned by YOLO-CAU to a simple network (YOLO-CAU-S) with less computation and faster detection speed. Compared with the existing knowledge distillation algorithm, the algorithm designed in this paper not only restricts the output of the student network to be close to the output of the teacher, but also proposes a multi-scale pixel attention module to construct the attention feature maps of the teacher model and student model, which further transfers the strong attention of the teacher network to the student. The experimental results show that the knowledge distillation algorithm designed in this paper can greatly improve the detection efficiency of the model on the premise of ensuring the detection accuracy, and the compressed model parameters are only 6% of the teacher network.

Small ship detection in remote sensing images is an increasingly important area in computer vision. The main greatest challenges that face it are a small size and different aspect ratios of ships. Recently, convolution neural network (CNN) achieve outstanding performance in object detection. CNN consists of several consecutive layers categorized to shallow and deep according to their positions. The shallow layers contribute in the detection of tiny targets significantly. However, the semantic features of these layers are weak to classify them correctly. In this paper, an aggregation context network is proposed to enhance the semantic features of the shallow layers for the state-of-the-art RetinaNet detector. This network is located before the feature pyramid network of the RetinaNet. It consists of aggregation and context modules. The aggregation module aggregates the different features to improve the semantic features at the shallow layers. The context module is to increase the receptive fields at each layer of the feature pyramid network. The experiments are carried on a proposed ship dataset. The dataset is carefully picked up where most of its instances relative area is less than 1% of the input image. The proposed network achieves 89.39%, which outperforms other state-of-the-art detectors.

Infrared target detection is a key technology in maritime distress target search and tracking systems. Particularly, detecting small targets overwhelmed in heavy waves is an important and challenging task. In order to effectively enhance small infrared maritime target saliency and suppress heavy wave clutter, a small infrared maritime target detection method based on gradient amplitude difference and multidimensional dissimilarity measure is proposed in this paper. Firstly, we employ Sobel operator to measure the gradient amplitude difference (GAD) by calculating minimum component of gradient between horizontal and vertical directions. Meanwhile, we use facet kernel filtering followed by adaptive threshold segmentation to extract the sizes, shapes, and locations of candidate targets; then, multidimensional dissimilarity information, i.e. multi-direction and multi-scale, is constructed based on original infrared image and locations of candidate targets. Multidimensional dissimilarity measure (MDM) achieves target enhancement and background clutter suppression. The final saliency map is obtained by multiplying GAD and MDM. Finally, an adaptive threshold is used to segment targets from residual interferences. Experimental results on three real infrared maritime image sequences show that, the proposed method achieves better performance in terms of local contrast gain, background suppression factor, and detection probability with low false alarm. Our method performs more satisfactorily and robustly than the state-of-the-art methods.

Pedestrian detection is a particular issue in both academia and industry. However, most existing pedestrian detection methods usually fail to detect small-scale pedestrians due to the introduction of feeble contrast and motion blur in images and videos. In this paper, we propose a multi-level feature fusion strategy to detect multi-scale pedestrians, which works particularly well with small-scale pedestrians that are relatively far from the camera. We propose a multi-level feature fusion strategy to make the shallow feature maps encode more semantic and global information to detect small-scale pedestrians. In addition, we redesign the aspect ratio of anchors to make it more robust for pedestrian detection task. The extensive experiments on both Caltech and CityPersons datasets demonstrate that our method outperforms the state-of-the-art pedestrian detection algorithms. Our proposed approach achieves a MR⁻² of 0.84%, 23.91% and 62.19% under the “Near”, Medium” and “Far” settings respectively on Caltech dataset, and also leads a better speed-accuracy trade-off with 0.28 second per image of 1024×2048 pixel compared with others on CityPersons dataset.

The traditional gray correlation analysis method extracted edge exists edge discontinuities, noise sensitivity and other issues, this paper combines Krisch operator with the thought of gray correlation analysis to detect edge, and proposes an improved image edge detection algorithm based on deng’s correlation degree, which combines with Krisch eight direction operators technology, uses median method to achieve unit consistency for the reference sequence and comparison sequence of gray correlation degree at the same time, and makes improvements for deng’s correlation degree model , with which obtained correlation degree to extract image edges. The simulation experiments show that the edges extracted by this paper algorithm are continuous and clear, which refines edges of the image, suppresses noise interference to a certain extent, and improves the efficiency of the algorithm.

There are many deep based semantic segmentation algorithms to strengthen robot perception. However, most existing methods hardly emphasis on the apply of deep neural networks. Therefore, this paper proposes a new Semantic Simultaneous Localization and Mapping system based on DeeplabV3+. The system can accurately reconstruct the 3D dense model of observed environment. Firstly, the Deeplabv3+ semantic extractor with the best visual effect is trained supervisedly, and the robot motion estimation is completed by the ORBSLAM2 framework. Subsequently, the semantic images, the depth images and the pose transformation matrix are sent to an efficient mapping module to fuse an accurate semantic model. Experimental results show that the constructed map can reflect the real distribution of objects in the scene, and the system perform well in the standard TUM datasets. The proposed method with loose coupling of novel segmentation networks can efficiently reduce the complexity of semantic SLAM system. Moreover, this method can improve the performance in the evolution of semantic segmentation network and SLAM framework.

The special points of the pulse signal, especially the position of the second peak tidal wave, are sometimes not easy to identify. For this reason, this paper designs and develops a tool for identifying the special points of the pulse signal based on the fourth derivative. We use MATLAB to develop a gadget to calculate higher derivatives of each pulse signal. In this process, the S-G filter is taken to smooth the pulse signal and its derivative. With the gadget, we can manually adjust the S-G filter parameters to get the final results. The positions of the first three peaks of each cycle of the fourth derivative are used as estimates of the locations of the special points of the original pulse signal. We use artificial method and the developed tool to find the special points of pulse signal respectively, and compare the difference of the two methods by analysis of variance (ANOVA). The experimental results show that there is no significant difference between the two methods (P<0.05). The tool developed in this paper is able to accurately locate the special points of the pulse signal, which provides support for extracting time domain characteristics of the pulse signal.

Different from traditional methods, this paper uses machine learning methods to solve the nowcasting problem of severe convective weather. Therefore, we propose a spatiotemporal convolutional block for nowcasting. The network adopts the encoder-decoder structure and proposes a new spatiotemporal causal convolution for timing feature extraction. Our network inputs 5 frames of images to predict the weather conditions for the next 30 minutes. The experimental results show that the results of our network structure are better and the training time is shorter. Compared with other networks, it can better capture the temporal and spatial correlation.

H.264 is an advanced video coding standard with wide application prospects. It has excellent compression performance and excellent image coding quality. It is widely used in many common fields such as video transmission, video storage and video editing. H. 264 uses a lot of complex computation, in which motion estimation takes up the largest amount of computation. Efficient motion estimation algorithm is of great significance to improve the efficiency of video coding. In this paper, the motion estimation and its typical algorithms of H.264 video coding standard are studied, focusing on UMHexagons (hereinafter referred to as UMH), the initial prediction order of motion vector and 5×5 spiral full search template is optimized, greatly improving the search efficiency.

The new breakthroughs of high dynamic range (HDR) technology have boosted the development of ultra-high-definition televisions (Ultra-HD TVs), and the evaluation standard of HDR video plays a significant role. But owing to the imperfect technique and standards, the current high-end products can hardly reach the requirements of Ultra-HD TVs. To improve the HDR assessment technique, a grading method for consumer-oriented products which balances the subjective and objective assessment is proposed to evaluate the HDR performance of Ultra-HD TVs. The subjective assessment is based on existing HDR standards, and the peak brightness, dynamic range, color gamut coverage and the absolute deviation of the electro-optical transfer function (EOTF) constitute the objective evaluation. With the proposed method, the videos which quality between the SDR and HDR standards can be divided into 3 or 4 grades by 4 metrics. The statistical results expound that many Ultra-HD TVs in the consumer-oriented market performs better than the HDR access standard, but still far from the professional-level, and the experiment results certify that the proposed method is an effective standard and can facilitate the standardized development on HDR performance of Ultra-HD TVs.

The Gaofen-4 satellite (GF-4) is the first Chinese remote sensing satellite to be placed in a geosynchronous orbit, which has a wide range of application prospects. The image sharpness evaluation method is a key step to quickly, accurately and automatically obtain the quality of a large number of GF-4 remote sensing data and understand the changes of the in-orbit satellite. Moreover, the assessment of image sharpness changes is the verification of the stability of the satellite operation, and can also provide reference for the follow-up satellite planning. Taking the time-series GF-4 panchromatic images of Zhengzhou and Chifeng from May 2016 to July 2020 as the data source, we use the block standard deviation sharpness evaluation method based on spatial domain and the power spectrum sharpness evaluation method based on frequency domain to calculate four kinds image sharpness evaluation values and carry out the correlation analysis from the same day and the same month, considering whether to remove the influence of image brightness. We analyze the influence of image brightness on the sharpness evaluation, and explores the applicability of these two methods for GF-4 image, so as to select the best sharpness evaluation method. The results show that the image power spectrum that has been normalized by the average image brightness is the most suitable image sharpness evaluation method for GF-4 image. This method is used to evaluate the image quality of 208 GF-4 panchromatic images in Zhengzhou and Chifeng. The results show that: since the delivery of GF-4 in orbit test satellite, the image sharpness is stable. At the same time, some suggestions are given for the planning of subsequent geosynchronous orbit satellites. It is necessary to improve the anti-interference ability of the camera to the atmospheric conditions, platform jitter and other external conditions.