Recently, color filter array patterns based on optimal design by reducing interchannel aliasing have been developed. Although these optimally subsampled patterns produce less degradation of resolution for the images in daylight about 6500 K, aliasing and grid effect become more prominent under the different lighting conditions. Moreover, the color interpolation (CI) for Bayer pattern cannot be utilized for these patterns. Since it is impossible to change coated patterns according to various conditions, CI algorithms have to compensate for light variations. In this letter, a method for generating intermediate quincuncial pattern is proposed for two patterns. Also, edge adaptive interpolation for the quincuncial pattern is carried out. Experimental results show that the proposed method reduces aliasing and grid effect.

An algorithm for multiview images captured by a parallel multicamera array is presented. An ergodic method is proposed for tracing the ideal rotation matrix and the preferred locations of the optical centers. Moreover, the camera’s intrinsic parameters are optimized in order to obtain the ideal projection matrix from which the image rectifying transform matrix (IRTM) is further derived. Then, the multiview image is rectified based on the IRTM. Experimental results show that this proposed algorithm dramatically improves accuracy. Moreover, the robustness of this algorithm is superior to the robustness of the previous method.

Most existing superresolution (SR) techniques focus primarily on improving the quality in the luminance component of SR images, while paying less attention to the chrominance component. We present an edge and color preserving image SR approach. First, for the luminance channel, a heavy-tailed gradient distribution of natural images is investigated as an image prior. Then, an efficient optimization algorithm is developed to recover the latent high-resolution (HR) luminance component. Second, for the chrominance channels, we propose a two-stage framework for luminance-guided chrominance SR. In the first stage, since most of the shape and structural information is contained in the luminance channel, a simple Markov random field formulation is introduced to search the optimal direction for color local interpolation guided by HR luminance components. To further improve the quality of the chrominance channels, in the second stage, a nonlocal auto regression model is utilized to refine the initial HR chrominance. Finally, we combine the SR reconstructed luminance components with the generated HR chrominance maps to get the final SR color image. Systematic experimental results demonstrated that our method outperforms some state-of-the-art methods in terms of the peak signal-to-noise ratio, structural similarity, feature similarity, and the mean color errors.

A tamper detection method to protect the integrity of indexed color images is presented. This method generates authentication codes of the indices based on the pseudorandom number generator. The length of authentication codes can be adaptively chosen according to the user’s requirement. The authentication codes are embedded into the index table of the indexed color image. In the tamper detection procedure, the embedded authentication codes are extracted. Besides, the pseudorandom number generator with the specific random seed is used to generate another set of authentication codes. By means of comparing the two sets of authentication codes, the tampered areas of the indexed color images can be found. Experimental results show that the proposed method efficiently detects the tampered areas while keeping good image qualities of embedded images.

We propose a robust photometric stereo method by using structural arrangement of light sources. In the arrangement, light sources are positioned on a planar grid and form a set of collinear combinations. The shadow pixels are detected by adaptive thresholding. The specular highlight and diffuse pixels are distinguished according to their intensity deviations of the collinear combinations, thanks to the special arrangement of light sources. The highlight detection problem is cast as a pattern classification problem and is solved using support vector machine classifiers. Considering the possible misclassification of highlight pixels, the ℓ ₁ regularization is further employed in normal map estimation. Experimental results on both synthetic and real-world scenes verify that the proposed method can robustly recover the surface normal maps in the case of heavy specular reflection and outperforms the state-of-the-art techniques.

This paper provides a comprehensive study on the effect of the biprism angle and position on the field-of-view (FOV) of a single-lens biprism-based stereovision system. The image captured by this system consists of subimages that are regarded as virtual images taken by two virtual cameras. The overlapping area in a stereo-image pair constitutes the common FOV. Stereo-correspondence is only possible for objects placed in this common FOV region. A geometrical analysis of this system by tracing the rays enables an enhanced understanding of the system, both qualitatively and quantitatively. First of all, a simple geometrical approach is proposed to predict the type of FOV produced given a fixed biprism angle, α . A dimensionless coefficient, ε , is proposed to describe the different types of FOV. Second, the effect of the translation of biprism in the z - and x -axes on the system’s FOV can also be determined using geometrical analysis. Experiments are conducted to verify the above predictions. Although there are some degrees of quantitative error between the experimental results and theory, the general theoretical trends are largely supported by the results.

We propose a progressive transmission approach of an image authenticated using an overlapping subimage that can be removed to restore the original image. Our approach is different from most visible watermarking approaches that allow one to later remove the watermark, because the mark is not directly introduced in the two-dimensional image space. Instead, it is rather applied to an equivalent monovariate representation of the image. Precisely, the approach is based on our progressive transmission approach that relies on a modified Kolmogorov spline network, and therefore inherits its advantages: resilience to packet losses during transmission and support of heterogeneous display environments. The marked image can be accessed at any intermediate resolution, and a key is needed to remove the mark to fully recover the original image without loss. Moreover, the key can be different for every resolution, and the images can be globally restored in case of packet losses during the transmission. Our contributions lie in the proposition of decomposing a mark (an overlapping image) and an image into monovariate functions following the Kolmogorov superposition theorem; and in the combination of these monovariate functions to provide a removable visible “watermarking” of images with the ability to restore the original image using a key.

Multiclass classification is an important problem in pattern recognition. Various classification methods have been proposed in the past few decades. However, most of these classification methods neglect the errors or the noises that exist in samples. As a result, classification accuracy is badly influenced by the errors or noises. In this paper, we propose a joint sparsity matrix learning method, which exploits l _2,1 -norm minimization to perform multiclass classification. In order to overcome the influence of the errors or noises, we introduce a sparse matrix to explicitly model the errors or noises and apply an iterative procedure to solve the l _2,1 -norm regularized problem. We perform experiments on four face databases to verify the effectiveness of the proposed method.

Intra prediction is a very important tool in current video coding standards. High-efficiency video coding (HEVC) intra prediction presents relevant gains in encoding efficiency when compared to previous standards, but with a very important increase in the computational complexity since 33 directional angular modes must be evaluated. Motivated by this high complexity, this article presents a complexity reduction algorithm developed to reduce the HEVC intra mode decision complexity targeting multiview videos. The proposed algorithm presents an efficient fast intra prediction compliant with singleview and multiview video encoding. This fast solution defines a reduced subset of intra directions according to the video texture and it exploits the relationship between prediction units (PUs) of neighbor depth levels of the coding tree. This fast intra coding procedure is used to develop an inter-view prediction method, which exploits the relationship between the intra mode directions of adjacent views to further accelerate the intra prediction process in multiview video encoding applications. When compared to HEVC simulcast, our method achieves a complexity reduction of up to 47.77%, at the cost of an average BD-PSNR loss of 0.08 dB.

This paper proposes a new watermarking scheme for color images, in which all pixels of the image are used for embedding watermark bits in order to achieve the highest amount of embedding. For watermark embedding, the S component in the hue-saturation-value (HSV) color space is used to carry the watermark bits, while the Vcomponent is used in accordance with a human visual system model to determine the proper watermark strength. In the proposed scheme, the number of watermark bits equals the number of pixels in the host image. Watermark extraction is accomplished blindly based on the use of a 3×3 spatial domain Wiener filter. The efficiency of our proposed image watermarking scheme depends mainly on the accuracy of the estimate of the original S component. The experimental results show that the performance of the proposed scheme, under no attacks and against various types of attacks, was superior to the previous existing watermarking schemes.

A composition of the viscous opening and the lower leveling is introduced to extract brain in magnetic resonance imaging T1. The innovative transformation disconnects chained components and has better control on the reconstruction process of the marker inside of the original image. However, the sequential operator requires setting several parameters, making its application difficult. Due to this situation, a simplification is carried out on it to obtain a more practical method. The proposed morphological transformations were tested with the Internet Brain Segmentation Repository (IBSR) database, which is used as a benchmark among the community. The results are compared using the Jaccard and Dice indices with respect to (i) manual segmentations obtained from the IBSR, (ii) mean indices reported in the current literature, and (iii) segmentations obtained from the Brain Extraction Tool, since this is one of the most popular algorithms used for brain segmentation. The average indices of Jaccard and Dice indicate that the reduced transformation produces similar results to the other methods reported in the literature while the sequential operator presents a better performance.

A new switching-based trimmed median filter to remove high-density salt-and-pepper noise in digital images is proposed. Initially, a 3×3 sliding window is applied on each pixel in the noisy image. The minimum- and maximum-intensity values are trimmed, and the noisy pixels are detected based on the predefined threshold value. In the filtering stage, the noisy pixels are replaced by median value of uncorrupted pixels in the trimmed array. At very high noise density, if all the pixels in the sliding window are corrupted, then the proposed algorithm replaces noisy pixels by the midpoint of recently processed pixels. The experimental results for various test images show that the performance of the proposed algorithm is superior to the existing algorithms, namely SMF, WMF, CWMF, AMF, DBA, and MDBUTMF in terms of visual quality and edge preservation, even at noise levels as high as 95%.

It has proved that fractional differentiation can enhance the edge information and nonlinearly preserve textural detailed information in an image. This paper investigates its ability for face recognition and presents a local descriptor called histograms of fractional differential gradients (HFDG) to extract facial visual features. HFDG encodes a face image into gradient patterns using multiorientation fractional differential masks, from which histograms of gradient directions are computed as the face representation. Experimental results on Yale, face recognition technology (FERET), Carnegie Mellon University pose, illumination, and expression (CMU PIE), and A. Martinez and R. Benavente (AR) databases validate the feasibility of the proposed method and show that HFDG outperforms local binary patterns (LBP), histograms of oriented gradients (HOG), enhanced local directional patterns (ELDP), and Gabor feature-based methods.

Computer vision-based crack detection methods for large-scale civil structures are mainly developed within the framework of image enhancement and segmentation. In contrast, we propose an idea that converts the images into parametric surfaces and then detects the crack surfaces using shape recognition techniques. The shape variations among noncrack surfaces are caused by approximately isometric deformations, but the dissimilarities between crack and noncrack surfaces are produced by nonisometric deformations. Therefore, the two classes of surfaces can be discriminated by their geodesic distance maps. To tackle the disturbances caused by the gaps in cracks, we develop a dedicated method, steady marching method, for the computation of the distance maps. In the subsequent quantitative comparison, we first construct distance difference matrices from the distance maps. Next, sub-block contrast ratios of these matrices are calculated and used as shape descriptors, which can be directly compared for the surface classification. Experimental results demonstrate that our method achieves a better performance than some typical methods. The extension of our method for crack localization is also presented.

The challenge of learning-based superresolution (SR) is to predict the relationships between low-resolution (LR) patches and their corresponding high-resolution (HR) patches. By learning such relationships from external training images, the existing learning-based SR approaches are often affected by the relevance between the training data and the LR input image. Therefore, we propose a single-image SR method that learns the LR-HR relations from the given LR image itself instead of any external images. Both the local regression model and nonlocal patch redundancy are exploited in the proposed method. The local regression model is employed to derive the mapping functions between self-LR-HR example patches, and the nonlocal self-similarity gives rise to a high-order derivative estimation of the derived mapping function. Moreover, to fully exploit the multiscale similarities inside the LR input image, we accumulate the previous reconstruction results and their corresponding LR versions as additional example patches for the subsequent estimation process, and adopt a gradual magnification scheme to achieve the desired zooming size step by step. Extensive experiments on benchmark images have validated the effectiveness of the proposed method. Compared to other state-of-the-art SR approaches, the proposed method provides photorealistic HR images with sharp edges.

Vision-based road detection is one of the key techniques of autonomous driving, intelligent vehicles, and visual navigation. At present, methods based on vanishing point perform best with general roads. However, it is difficult for them to meet the needs of a real-time system due to high time consumption. This paper presents a fast detection method, namely simple road detection, which achieves high efficiency by employing sky segmentation and two new optimization schemes-sample convolution and fast voting. The optimizations are based on lookup tables, sample computing, and computing simplification. The interval sampling in sample convolution makes the proposed method flexible to meet various efficiency and accuracy demands by different sample-step values. Mean filter and vote orientation limitation are also proposed to help improve detection accuracy. Experiments have been conducted with a large number of road images under different environmental conditions, and the results demonstrate that our proposed method is efficient and effective in detecting both structured and unstructured roads.

Automatic recognition of human activities and behaviors is still a challenging problem for many reasons, including limited accuracy of the data acquired by sensing devices, high variability of human behaviors, and gap between visual appearance and scene semantics. Symbolic approaches can significantly simplify the analysis and turn raw data into chains of meaningful patterns. This allows getting rid of most of the clutter produced by low-level processing operations, embedding significant contextual information into the data, as well as using simple syntactic approaches to perform the matching between incoming sequences and models. We propose a symbolic approach to learn and detect complex activities through the sequences of atomic actions. Compared to previous methods based on context-free grammars, we introduce several important novelties, such as the capability to learn actions based on both positive and negative samples, the possibility of efficiently retraining the system in the presence of misclassified or unrecognized events, and the use of a parsing procedure that allows correct detection of the activities also when they are concatenated and/or nested one with each other. An experimental validation on three datasets with different characteristics demonstrates the robustness of the approach in classifying complex human behaviors.

Robust local descriptors usually consist of high-dimensional feature vectors to describe distinctive characteristics of images. The high dimensionality of a feature vector incurs considerable costs in terms of computational time and storage. It also results in the curse of dimensionality that affects the performance of several tasks that use feature vectors, such as matching, retrieval, and classification of images. To address these problems, it is possible to employ some dimensionality reduction techniques, leading frequently to information lost and, consequently, accuracy reduction. This work aims at applying linear dimensionality reduction to the scale invariant feature transformation and speeded up robust feature descriptors. The objective is to demonstrate that even risking the decrease of the accuracy of the feature vectors, it results in a satisfactory trade-off between computational time and storage requirements. We perform linear dimensionality reduction through random projections, principal component analysis, linear discriminant analysis, and partial least squares in order to create lower dimensional feature vectors. These new reduced descriptors lead us to less computational time and memory storage requirements, even improving accuracy in some cases. We evaluate reduced feature vectors in a matching application, as well as their distinctiveness in image retrieval. Finally, we assess the computational time and storage requirements by comparing the original and the reduced feature vectors.

We propose an efficient and easy-to-implement method to settle the inpainting problem for low-rank images following the recent studies about low-rank matrix completion. In general, our method has three steps: first, corresponding to the three channels of RGB color space, an incomplete image is split into three incomplete matrices; second, each matrix is restored by solving a convex problem derived from the nuclear norm relaxation; at last, the three recovered matrices are merged to produce the final output. During the process, in order to efficiently solve the nuclear norm minimization problem, we employ the alternating direction method. Except for the basic image inpainting problem, we also enable our method to handle cases where corrupted images not only have missing values but also have noisy entries. Our experiments show that our method outperforms the existing inpainting techniques both quantitatively and qualitatively. We also demonstrate that our method is capable of processing many other situations, including block-wise low-rank image completion, large-scale image restoration, and object removal.

A multimodal biometric system has been considered a promising technique to overcome the defects of unimodal biometric systems. We have introduced a fusion scheme to gain a better understanding and fusion method for a face-iris-fingerprint multimodal biometric system. In our case, we use particle swarm optimization to train a set of adaptive Gabor filters in order to achieve the proper Gabor basic functions for each modality. For a closer analysis of texture information, two different local Gabor features for each modality are produced by the corresponding Gabor coefficients. Next, all matching scores of the two Gabor features for each modality are projected to a single-scalar score via a trained, supported, vector regression model for a final decision. A large-scale dataset is formed to validate the proposed scheme using the Facial Recognition Technology database-fafb and CASIA-V3-Interval together with FVC2004-DB2a datasets. The experimental results demonstrate that as well as achieving further powerful local Gabor features of multimodalities and obtaining better recognition performance by their fusion strategy, our architecture also outperforms some state-of-the-art individual methods and other fusion approaches for face-iris-fingerprint multimodal biometric systems.

This paper presents a computationally efficient solution to three-dimensional point cloud reconstruction from video image sequences that are captured by a hand-held camera. Our solution starts with a frame selection step to remove frames that cause physically nonrealizable reconstruction outcomes. Then, a computationally efficient approach for obtaining the absolute camera pose is introduced based on pairwise relative camera poses. This is followed by a computationally efficient rotation registration to update the absolute camera pose. The reconstruction results obtained based on actual video sequences indicate lower computation times and lower reprojection errors of the introduced approach compared to the conventional approach.