Modern cardiac x-ray imaging systems regulate their radiation output based on the thickness of the patient to maintain an acceptable signal at the input of the x-ray detector. This approach does not account for the context of the examination or the content of the image displayed. We have developed a machine vision algorithm that detects iodine-filled blood vessels and fits an idealized vessel model with the key parameters of contrast, diameter, and linear attenuation coefficient. The spatio-temporal distribution of the linear attenuation coefficient samples, when appropriately arranged, can be described by a simple linear relationship, despite the complexity of scene information. The algorithm was tested on static anthropomorphic chest phantom images under different radiographic factors and 60 dynamic clinical image sequences. It was found to be robust and sensitive to changes in vessel contrast resulting from variations in system parameters. The machine vision algorithm has the potential of extracting real-time context sensitive information that may be used for augmenting existing dose control strategies.

Markerless human motion capture has a wide variety of applications, but recovering a pose from multiple-view calibrated images remains challenging. In general, the problem of pose estimation is an optimization problem, but cost functions easily trapped in a local minimum based on extrinsic similarities (silhouette, edge, etc.) and pose estimation fail if the body is largely self-occluded. We define an effective cost function that combines extrinsic and intrinsic similarities, that can be translated by projecting a mesh model onto camera views to best match the silhouettes from an extrinsic point of view, and that attempts to preserve the intrinsic geometry of the shape using predefined mesh models based on heat diffusion. The results of experiment show that our proposed method is more robust for sequence motion capture than pure extrinsic similarity or pure intrinsic similarity.

An efficient view invariant framework for the recognition of human activities from an input video sequence is presented. The proposed framework is composed of three consecutive modules: (i) detect and locate people by background subtraction, (ii) view invariant spatiotemporal template creation for different activities, (iii) and finally, template matching is performed for view invariant activity recognition. The foreground objects present in a scene are extracted using change detection and background modeling. The view invariant templates are constructed using the motion history images and object shape information for different human activities in a video sequence. For matching the spatiotemporal templates for various activities, the moment invariants and Mahalanobis distance are used. The proposed approach is tested successfully on our own viewpoint dataset, KTH action recognition dataset, i3DPost multiview dataset, MSR viewpoint action dataset, VideoWeb multiview dataset, and WVU multiview human action recognition dataset. From the experimental results and analysis over the chosen datasets, it is observed that the proposed framework is robust, flexible, and efficient with respect to multiple views activity recognition, scale, and phase variations.

Moving object segmentation using change detection in wavelet domain under continuous variations of lighting condition is a challenging problem in video surveillance systems. There are several methods proposed in the literature for change detection in wavelet domain for moving object segmentation having static backgrounds, but it has not been addressed effectively for dynamic background changes. The methods proposed in the literature suffer from various problems, such as ghostlike appearance, object shadows, and noise. To deal with these issues, a framework for dynamic background modeling and shadow suppression under rapidly changing illumination conditions for moving object segmentation in complex wavelet domain is proposed. The proposed method consists of eight steps applied on given video frames, which include wavelet decomposition of frame using complex wavelet transform; use of change detection on detail coefficients (LH, HL, and HH), use of improved Gaussian mixture-based dynamic background modeling on approximate coefficient (LL subband); cast shadow suppression; use of soft thresholding for noise removal; strong edge detection; inverse wavelet transformation for reconstruction; and finally using closing morphology operator. A comparative analysis of the proposed method is presented both qualitatively and quantitatively with other standard methods available in the literature for six datasets in terms of various performance measures. Experimental results demonstrate the efficacy of the proposed method.

Cardiologists use x-ray image sequences of the moving heart acquired in real-time to diagnose and treat cardiac patients. The amount of radiation used is proportional to image quality; however, exposure to radiation is damaging to patients and personnel. The amount by which radiation dose can be reduced without compromising patient care was determined. For five patient image sequences, increments of computer-generated quantum noise (white + colored) were added to the images, frame by frame using pixel-to-pixel addition, to simulate corresponding increments of dose reduction. The noise adding software was calibrated for settings used in cardiac procedures, and validated using standard objective and subjective image quality measurements. The degraded images were viewed next to corresponding original (not degraded) images in a two-alternative-forced-choice staircase psychophysics experiment. Seven cardiologists and five radiographers selected their preferred image based on visualization of the coronary arteries. The point of subjective equality, i.e., level of degradation where the observer could not perceive a difference between the original and degraded images, was calculated; for all patients the median was 33%±15% dose reduction. This demonstrates that a 33%±15% increase in image noise is feasible without being perceived, indicating potential for 33%±15% dose reduction without compromising patient care.

The person reidentification task applied in a real-world scenario is addressed. Finding people in a network of cameras is challenging due to significant variations in lighting conditions, different color responses, and different camera viewpoints. State-of-the-art algorithms are likely to fail due to serious perspective and pose changes. Most of the existing approaches try to cope with all these changes by applying metric learning tools to find a transfer function between a camera pair while ignoring the body alignment issue. Additionally, this transfer function usually depends on the camera pair and requires labeled training data for each camera. This might be unattainable in a large camera network. We employ three-dimensional scene information for minimizing perspective distortions and estimating the target pose. The estimated pose is further used for splitting a target trajectory into reliable chunks, each one with a uniform pose. These chunks are matched through a network of cameras using a previously learned metric pool. However, instead of learning transfer functions that cope with all appearance variations, we propose to learn a generic metric pool that only focuses on pose changes. This pool consists of metrics, each one learned to match a specific pair of poses and not being limited to a specific camera pair. Automatically estimated poses determine the proper metric, thus improving matching. We show that metrics learned using only a single camera can significantly improve the matching across the whole camera network, providing a scalable solution. We validated our approach on publicly available datasets, demonstrating increase in the reidentification performance.

Real-time execution of processing algorithms for handling depth images in a three-dimensional (3-D) data framework is a major challenge. More specifically, considering depth images as point clouds and performing planar segmentation requires heavy computation, because available planar segmentation algorithms are mostly based on surface normals and/or curvatures, and, consequently, do not provide real-time performance. Aiming at the reconstruction of indoor environments, the spaces mainly consist of planar surfaces, so that a possible 3-D application would strongly benefit from a real-time algorithm. We introduce a real-time planar segmentation method for depth images avoiding any surface normal calculation. First, we detect 3-D edges in a depth image and generate line segments between the identified edges. Second, we fuse all the points on each pair of intersecting line segments into a plane candidate. Third and finally, we implement a validation phase to select planes from the candidates. Furthermore, various enhancements are applied to improve the segmentation quality. The GPU implementation of the proposed algorithm segments depth images into planes at the rate of 58 fps. Our pipeline-interleaving technique increases this rate up to 100 fps. With this throughput rate improvement, the application benefit of our algorithm may be further exploited in terms of quality and enhancing the localization.

Video quality assessment plays an important role in video processing and communication applications. We propose a full reference video quality metric by combining a content-weighted spatial pooling strategy with a temporal pooling strategy. All pixels in a frame are classified into edge, texture, and smooth regions, and their structural similarity image index (SSIM) maps are divided into increasing and saturated regions by the curve of their SSIM values, then a content weight method is applied to increasing regions to get the score of an image frame. Finally, a temporal pooling method is used to get the overall video quality. Experimental results on the LIVE and IVP video quality databases show our proposed method works well in matching subjective scores.

A vehicle detection algorithm based on the multilevel knowledge base is proposed to overcome the problem of poor robustness as well as the difficulty of identifying weak vehicle targets. The multilevel task-driven method is adopted in the algorithm by building three different classes of knowledge bases to achieve the accuracy recognition of vehicle targets. First, a simple knowledge base is constructed via choosing Haar-like features to detect the vehicle region of interest in the traffic scene image; second, the optimal structure symmetry decision function is obtained by establishing the structure characteristics knowledge base, which is used to determine the region of the potential vehicle; finally, the property feature knowledge base is built to precisely identify vehicle targets via calculating maximum similarity. Then the relevant knowledge base will be continuously updated to achieve the method adaptive adjustment when satisfying the criterion. Experimental results illustrate that the recognition rate is more than 95% in different traffic scenarios, while the recognition rate for weak contrast vehicle targets is in excess of 71% and the false-alarm rate is simultaneously under 5%.

A large collection of reproductions of calligraphy on paper was scanned into images to enable web access for both the academic community and the public. Calligraphic paper digitization technology is mature, but technology for segmentation, character coding, style classification, and identification of calligraphy are lacking. Therefore, computational tools for classification and quantification of calligraphic style are proposed and demonstrated on a statistically characterized corpus. A subset of 259 historical page images is segmented into 8719 individual character images. Calligraphic style is revealed and quantified by visual attributes (i.e., appearance features) of character images sampled from historical works. A style space is defined with the features of five main classical styles as basis vectors. Cross-validated error rates of 10% to 40% are reported on conventional and conservative sampling into training/test sets and on same-work voting with a range of voter participation. Beyond its immediate applicability to education and scholarship, this research lays the foundation for style-based calligraphic forgery detection and for discovery of latent calligraphic groups induced by mentor-student relationships.

We propose a numerical strategy to define a multiscale analysis for color and multicomponent images based on the representation of data on a graph. Our approach consists of computing the graph of an image using the psychovisual information and analyzing it by using the spectral graph wavelet transform. We suggest introducing color dimension into the computation of the weights of the graph and using the geodesic distance as a mean of distance measurement. We thus have defined a wavelet transform based on a graph with perceptual information by using the CIELab color distance. This new representation is illustrated with denoising and inpainting applications. Overall, by introducing psychovisual information in the graph computation for the graph wavelet transform, we obtain very promising results. Thus, results in image restoration highlight the interest of the appropriate use of color information.

Tone-mapping operators are the typical algorithms designed to produce visibility and the overall impression of brightness, contrast, and color of high dynamic range (HDR) images on low dynamic range (LDR) display devices. Although several new tone-mapping operators have been proposed in recent years, the results of these operators have not matched those of the psychophysical experiments based on the human visual system. A color-rendering model that is a combination of tone-mapping and cone-response functions using an XYZ tristimulus color space is presented. In the proposed method, the tone-mapping operator produces visibility and the overall impression of brightness, contrast, and color in HDR images when mapped onto relatively LDR devices. The tone-mapping resultant image is obtained using chromatic and achromatic colors to avoid well-known color distortions shown in the conventional methods. The resulting image is then processed with a cone-response function wherein emphasis is placed on human visual perception (HVP). The proposed method covers the mismatch between the actual scene and the rendered image based on HVP. The experimental results show that the proposed method yields an improved color-rendering performance compared to conventional methods.

We present a straightforward brightness preserving image enhancement technique. The proposed method is based on an original gradient and intensity histogram (GIH) which contains both gradient and intensity information of the image. This character enables GIH to avoid high peaks in the traditional intensity histogram and, thus alleviate overenhancement in our enhancement method, i.e., gradient and intensity histogram equalization (GIHE). GIHE can also enhance the gradient strength of an image, which is good for improving the subjective quality since the human vision system is more sensitive to the gradient than the absolute intensity of image. Considering that brightness preservation and dynamic range compression are highly demanded in consumer electronics, we manipulate the intensity of the enhanced image appropriately by amplifying the small intensities and attenuating the large intensities, respectively, using a brightness preserving function (BPF). The BPF is straightforward and universal and can be used in other image enhancement techniques. We demonstrate that the proposed method can effectively improve the subjective quality as well as preserve the brightness of the input image.

Three-dimensional (3-D) face recognition provides a potential to handle challenges caused by illumination and pose variations. However, extreme expression variations still complicate the task of recognition. An accurate and robust method for expression-invariant 3-D face recognition is proposed. A 3-D face is partitioned into a set of isogeodesic stripes and the spatial relationships of the stripes are described by 3-D weighted walkthrough and the centroid distance. Moreover, the method of the similarity measure is given. Experiments are performed on the CASIA dataset and the FRGC v2.0 dataset. The results show that our method has advantages for recognition performance despite large expression variations.

We propose an impulse noise removal algorithm which can effectively realize the elimination of the impulse noise in an image even though these noise pixels are on image edges or details. The algorithm introduces a statistic, spatial global outlier measurement to identify the impulse noise in an image. Once the noise pixels are identified, noise candidates are restored by a nonlocal edge-preserving regularized filter. Extensive experimental results indicate that our algorithm provides a significant improvement over many other existing techniques.

Contrast enhancement plays a key role in a wide range of applications including consumer electronic applications, such as video surveillance, digital cameras, and televisions. The main goal of contrast enhancement is to increase the quality of images. However, most state-of-the-art methods induce different types of distortion such as intensity shift, wash-out, noise, intensity burn-out, and intensity saturation. In addition, in consumer electronics, simple and fast methods are required in order to be implemented in real time. A bihistogram equalization method based on adaptive sigmoid functions is proposed. It consists of splitting the image histogram into two parts that are equalized independently by using adaptive sigmoid functions. In order to preserve the mean brightness of the input image, the parameter of the sigmoid functions is chosen to minimize the absolute mean brightness metric. Experiments on the Berkeley database have shown that the proposed method improves the quality of images and preserves their mean brightness. An application to improve the colorfulness of images is also presented.

This paper proposes a progressive sparse representation-based classification algorithm using local discrete cosine transform (DCT) evaluation to perform face recognition. Specifically, the sum of the contributions of all training samples of each subject is first taken as the contribution of this subject, then the redundant subject with the smallest contribution to the test sample is iteratively eliminated. Second, the progressive method aims at representing the test sample as a linear combination of all the remaining training samples, by which the representation capability of each training sample is exploited to determine the optimal “nearest neighbors” for the test sample. Third, the transformed DCT evaluation is constructed to measure the similarity between the test sample and each local training sample using cosine distance metrics in the DCT domain. The final goal of the proposed method is to determine an optimal weighted sum of nearest neighbors that are obtained under the local correlative degree evaluation, which is approximately equal to the test sample, and we can use this weighted linear combination to perform robust classification. Experimental results conducted on the ORL database of faces (created by the Olivetti Research Laboratory in Cambridge), the FERET face database (managed by the Defense Advanced Research Projects Agency and the National Institute of Standards and Technology), AR face database (created by Aleix Martinez and Robert Benavente in the Computer Vision Center at U.A.B), and USPS handwritten digit database (gathered at the Center of Excellence in Document Analysis and Recognition at SUNY Buffalo) demonstrate the effectiveness of the proposed method.

Within intelligent transportation systems, fast and robust license plate localization (LPL) in complex scenes is still a challenging task. Real-world scenes introduce complexities such as variation in license plate size and orientation, uneven illumination, background clutter, and nonplate objects. These complexities lead to poor performance using traditional LPL features, such as color, edge, and texture. Recently, state-of-the-art performance in LPL has been achieved by applying the scale invariant feature transform (SIFT) descriptor to LPL for visual matching. However, for applications that require fast processing, such as mobile phones, SIFT does not meet the efficiency requirement due to its relatively slow computational speed. To address this problem, a new approach for LPL, which uses the oriented FAST and rotated BRIEF (ORB) feature detector, is proposed. The feature extraction in ORB is much more efficient than in SIFT and is invariant to scale and grayscale as well as rotation changes, and hence is able to provide superior performance for LPL. The potential regions of a license plate are detected by considering spatial and color information simultaneously, which is different from previous approaches. The experimental results on a challenging dataset demonstrate the effectiveness and efficiency of the proposed method.

A single sensor camera can capture scenes by means of a color filter array. Each pixel samples only one of the three primary colors. We use a color demosaicking (CDM) technique to produce full color images and propose a robust adaptive sparse representation model for high quality CDM. The data fidelity term is characterized by l₁ norm to suppress the heavy-tailed visual artifacts with an adaptively learned dictionary, while the regularization term is encouraged to seek sparsity by forcing sparse coding close to its nonlocal means to reduce coding errors. Based on the classical quadratic penalty function technique in optimization and an operator splitting method in convex analysis, we further present an effective iterative algorithm to solve the variational problem. The efficiency of the proposed method is demonstrated by experimental results with simulated and real camera data.

We present a simple yet effective scene annotation framework based on a combination of bag-of-visual words (BoVW), three-dimensional scene structure estimation, scene context, and cognitive theory. From a macroperspective, the proposed cognition-based hybrid motivation framework divides the annotation problem into empirical inference and real-time classification. Inspired by the inference ability of human beings, common objects of indoor scenes are defined for experience-based inference, while in the real-time classification stage, an improved BoVW-based multilayer abstract semantics labeling method is proposed by introducing abstract semantic hierarchies to narrow the semantic gap and improve the performance of object categorization. The proposed framework was evaluated on a variety of common data sets and experimental results proved its effectiveness.

The Log-Gabor transform, which is suitable for analyzing gradually changing data such as in iris and face images, has been widely used in image processing, pattern recognition, and computer vision. In most cases, only the magnitude or phase information of the Log-Gabor transform is considered. However, the complementary effect taken by combining magnitude and phase information simultaneously for an image-feature extraction problem has not been systematically explored in the existing works. We propose a local image descriptor for face recognition, called Log-Gabor Weber descriptor (LGWD). The novelty of our LGWD is twofold: (1) to fully utilize the information from the magnitude or phase feature of multiscale and orientation Log-Gabor transform, we apply the Weber local binary pattern operator to each transform response. (2) The encoded Log-Gabor magnitude and phase information are fused at the feature level by utilizing kernel canonical correlation analysis strategy, considering that feature level information fusion is effective when the modalities are correlated. Experimental results on the AR, Extended Yale B, and UMIST face databases, compared with those available from recent experiments reported in the literature, show that our descriptor yields a better performance than state-of-the art methods.

In the field of image classification, it has been a trend that in order to deliver a reliable classification performance, the feature extraction model becomes increasingly more complicated, leading to a high dimensionality of image representations. This, in turn, demands greater computation resources for image classification. Thus, it is desirable to apply dimensionality reduction (DR) methods for image classification. It is necessary to apply DR methods to relieve the computational burden as well as to improve the classification accuracy. However, traditional DR methods are not compatible with modern feature extraction methods. A framework that combines manifold learning based DR and feature extraction in a deeper way for image classification is proposed. A multiscale cell representation is extracted from the spatial pyramid to satisfy the locality constraints for a manifold learning method. A spectral weighted mean filtering is proposed to eliminate noise in the feature space. A hierarchical discriminant manifold learning is proposed which incorporates both category label and image scale information to guide the DR process. Finally, the image representation is generated by concatenating dimensionality reduced cell representations from the same image. Extensive experiments are conducted to test the proposed algorithm on both scene and object recognition datasets in comparison with several well-established and state-of-the-art methods with respect to classification precision and computational time. The results verify the effectiveness of incorporating manifold learning in the feature extraction procedure and imply that the multiscale cell representations may be distributed on a manifold.

Most of the tracking methods attempt to build up feature spaces to represent the appearance of a target. However, limited by the complex structure of the distribution of features, the feature spaces constructed in a linear manner cannot characterize the nonlinear structure well. We propose an appearance model based on kernel ridge regression for visual tracking. Dense sampling is fulfilled around the target image patches to collect the training samples. In order to obtain a kernel space in favor of describing the target appearance, multiple kernel learning is introduced into the selection of kernels. Under the framework, instead of a single kernel, a linear combination of kernels is learned from the training samples to create a kernel space. Resorting to the circulant property of a kernel matrix, a fast interpolate iterative algorithm is developed to seek coefficients that are assigned to these kernels so as to give an optimal combination. After the regression function is learned, all candidate image patches gathered are taken as the input of the function, and the candidate with the maximal response is regarded as the object image patch. Extensive experimental results demonstrate that the proposed method outperforms other state-of-the-art tracking methods.

In order to improve the precision of visible and infrared (VIS/IR) image registration, an image registration method based on visual salient (VS) features is presented. First, a VS feature detector based on the modified visual attention model is presented to extract VS points. Because the iterative, within-feature competition method used in visual attention models is time consuming, an alternative fast visual salient (FVS) feature detector is proposed to make VS features more efficient. Then, a descriptor-rearranging (DR) strategy is adopted to describe feature points. This strategy combines information of both IR image and its negative image to overcome the contrast reverse problem between VIS and IR images, making it easier to find the corresponding points on VIS/IR images. Experiments show that both VS and FVS detectors have higher repeatability scores than scale invariant feature transform in the cases of blurring, brightness change, JPEG compression, noise, and viewpoint, except big scale change. The combination of VS detector and DR registration strategy can achieve precise image registration, but it is time-consuming. The combination of FVS detector and DR registration strategy can also reach a good registration of VIS/IR images but in a shorter time.

Spatially varying motion blur in video results from the relative motion of a camera and the scene. How to estimate accurate optical flow in the presence of spatially varying motion blur has received little attention so far. We extend the classical warping-based variational optical flow method to deal with this issue. First, we modify the data term by matching the identified nonuniform motion blur between the input images according to a fast blur detection and deblurring technique. Importantly, a downsample-interpolation technique is proposed to improve the blur detection efficiency, which saves 75% or more running time. Second, we improve the edge-preserving regularization term at blurry motion boundaries to reduce boundary errors that are caused by blur. The proposed method is evaluated on both synthetic and real sequences, and yields improved overall performance compared to the state-of-the-art in handling motion blur.

We present a robust method of detecting text in natural scenes. The work consists of four parts. First, automatically partition the images into different layers based on conditional clustering. The clustering operates in two sequential ways. One has a constrained clustering center and conditional determined cluster numbers, which generate small-size subregions. The other has fixed cluster numbers, which generate full-size subregions. After the clustering, we obtain a bunch of connected components (CCs) in each subregion. In the second step, the convolutional neural network (CNN) is used to classify those CCs to character components or noncharacter ones. The output score of the CNN can be transferred to the postprobability of characters. Then we group the candidate characters into text strings based on the probability and location. Finally, we use a verification step. We choose a multichannel strategy to evaluate the performance on the public datasets: ICDAR2011 and ICDAR2013. The experimental results demonstrate that our algorithm achieves a superior performance compared with the state-of-the-art text detection algorithms.

Automatic detection of road boundaries in traffic surveillance imagery can greatly aid subsequent traffic analysis tasks, such as vehicle flow, erratic driving, and stranded vehicles. This paper develops an online technique for identifying the dominant road boundary in video sequences captured by traffic cameras under challenging environmental and lighting conditions, e.g., unlit highways captured at night. The proposed method works in real time of up to 20  frames/s and generates a ranked list of road regions that identify road and lane boundaries. Our method begins by segmenting each frame into a set of superpixels. An adaptive sampling step approximates superpixel contours to a collection of edge segments. Next, we show how online hierarchical clustering can be efficiently used to organize edges into clusters of colinearly similar sets. Promising clusters are paired with each other to form cluster pairs. Then we present and prove a statistical ranking measure that is used along with road-activity and perspective cues to find the dominant road boundaries. We evaluate the proposed approach on two real-world datasets to test our method under camera viewpoint changes and extreme environmental and lighting conditions. Results show that our method outperforms two state-of-the-art techniques in precision, recall, and runtime.

We propose optimal bilateral filtering techniques for Gaussian noise suppression in images. To achieve maximum denoising performance via optimal filter parameter selection, we adopt Stein’s unbiased risk estimate (SURE)—an unbiased estimate of the mean-squared error (MSE). Unlike MSE, SURE is independent of the ground truth and can be used in practical scenarios where the ground truth is unavailable. In our recent work, we derived SURE expressions in the context of the bilateral filter and proposed SURE-optimal bilateral filter (SOBF). We selected the optimal parameters of SOBF using the SURE criterion. To further improve the denoising performance of SOBF, we propose variants of SOBF, namely, SURE-optimal multiresolution bilateral filter (SMBF), which involves optimal bilateral filtering in a wavelet framework, and SURE-optimal patch-based bilateral filter (SPBF), where the bilateral filter parameters are optimized on small image patches. Using SURE guarantees automated parameter selection. The multiresolution and localized denoising in SMBF and SPBF, respectively, yield superior denoising performance when compared with the globally optimal SOBF. Experimental validations and comparisons show that the proposed denoisers perform on par with some state-of-the-art denoising techniques.

Semantic gap limits the performance of bag-of-visual-words. To deal with this problem, a hierarchical abstract semantics method that builds abstract semantic layers, generates semantic visual vocabularies, measures semantic gap, and constructs classifiers using the Adaboost strategy is proposed. First, abstract semantic layers are proposed to narrow the semantic gap between visual features and their interpretation. Then semantic visual words are extracted as features to train semantic classifiers. One popular form of measurement is used to quantify the semantic gap. The Adaboost training strategy is used to combine weak classifiers into strong ones to further improve performance. For a testing image, the category is estimated layer-by-layer. Corresponding abstract hierarchical structures for popular datasets, including Caltech-101 and MSRC, are proposed for evaluation. The experimental results show that the proposed method is capable of narrowing semantic gaps effectively and performs better than other categorization methods.

Lip-reading techniques have shown bright prospects for speech recognition under noisy environments and for hearing-impaired listeners. We aim to solve two important issues regarding lip reading: (1) how to extract discriminative lip motion features and (2) how to establish a classifier that can provide promising recognition accuracy for lip reading. For the first issue, a projection local spatiotemporal descriptor, which considers the lip appearance and motion information at the same time, is utilized to provide an efficient representation of a video sequence. For the second issue, a kernel extreme learning machine (KELM) based on the single-hidden-layer feedforward neural network is presented to distinguish all kinds of utterances. In general, this method has fast learning speed and great robustness to nonlinear data. Furthermore, quantum-behaved particle swarm optimization with binary encoding is introduced to select the appropriate feature subset and parameters for KELM training. Experiments conducted on the AVLetters and OuluVS databases show that the proposed lip-reading method achieves a superior recognition accuracy compared with two previous methods.

We present a quick method to detect dissimilar binary images. The method is based on a “probabilistic matching model” for image matching. The matching model is used to predict the probability of occurrence of distinct-dissimilar image pairs (completely different images) when matching one image to another. Based on this model, distinct-dissimilar images can be detected by matching only a few points between two images with high confidence, namely 11 points for a 99.9% successful detection rate. For image pairs that are dissimilar but not distinct-dissimilar, more points need to be mapped. The number of points required to attain a certain successful detection rate or confidence depends on the amount of similarity between the compared images. As this similarity increases, more points are required. For example, images that differ by 1% can be detected by mapping fewer than 70 points on average. More importantly, the model is image size invariant; so, images of any sizes will produce high confidence levels with a limited number of matched points. As a result, this method does not suffer from the image size handicap that impedes current methods. We report on extensive tests conducted on real images of different sizes.

This paper presents a method for automatic exposure time adjustment for multispectral and hyperspectral cameras. The method presented here is based upon a spectral power image. Here, we use the photopic response function due to its widespread usage in photography and psychophysics. Note that, however, the method presented here is quite general in nature and can employ a number of spectral sensitivity functions for the computation of the spectral power image. Making use of this spectral power image, the exposure time is then computed via iterative updates so as to minimize the squared error between a target image and the current spectral power yielded by the imager. This target image is recovered in a straightforward manner using histogram equalization and the Commission Internationale de l’Éclairage (CIE) photopic function. This, in turn, yields an automatic method devoid of calibration targets or additional inputs. We perform a stability and controllability analysis of our method using a state-space representation. We also show the applicability of the method for exposure time calculation on staring array and multicharge coupled device architecture cameras on real-world scenes.

This paper proposes a visual cryptography scheme (VCS) based on quantum signal processing (QSP). VCS is an image encryption technique that is very simple in formulation and is secure. In (k,n)-VCS, a secret binary image is encoded into n share images and minimum k shares are needed to decrypt the secret image. The efforts to encrypt a grayscale image are few in number and the majority are related to grayscale to binary conversion. Thus, a generalized approach of encryption for all types of images, i.e., binary, gray, and color is needed. Here, a generic VCS is proposed based on QSP where all types of images can be encrypted without pixel expansion along with a smoothing technique to enhance the quality of the decrypted image. The proposed scheme is tested and compared for benchmark images, and the result shows the effectiveness of the scheme.

We present an algorithm that simultaneously computes optical flow and estimates illumination change from an image sequence in a unified framework. We propose an energy functional consisting of conventional optical flow energy based on Horn–Schunck method and an additional constraint that is designed to compensate for illumination changes. Any undesirable illumination change that occurs in the imaging procedure in a sequence while the optical flow is being computed is considered a nuisance factor. In contrast to the conventional optical flow algorithm based on Horn–Schunck functional, which assumes the brightness constancy constraint, our algorithm is shown to be robust with respect to temporal illumination changes in the computation of optical flows. An efficient conjugate gradient descent technique is used in the optimization procedure as a numerical scheme. The experimental results obtained from the Middlebury benchmark dataset demonstrate the robustness and the effectiveness of our algorithm. In addition, comparative analysis of our algorithm and Horn–Schunck algorithm is performed on the additional test dataset that is constructed by applying a variety of synthetic bias fields to the original image sequences in the Middlebury benchmark dataset in order to demonstrate that our algorithm outperforms the Horn–Schunck algorithm. The superior performance of the proposed method is observed in terms of both qualitative visualizations and quantitative accuracy errors when compared to Horn–Schunck optical flow algorithm that easily yields poor results in the presence of small illumination changes leading to violation of the brightness constancy constraint.

A study for feature extraction is proposed to handle the problem of facial appearance changes including facial makeup and plastic surgery in face recognition. To extend a face recognition method robust to facial appearance changes, features are individually extracted from facial depth on which facial makeup and plastic surgery have no effect. Then facial depth features are added to facial texture features to perform feature extraction. Accordingly, a three-dimensional (3-D) face is reconstructed from only a single two-dimensional (2-D) frontal image in real-world scenarios. Then the facial depth is extracted from the reconstructed model. Afterward, the dual-tree complex wavelet transform (DT-CWT) is applied to both texture and reconstructed depth images to extract the feature vectors. Finally, the final feature vectors are generated by combining 2-D and 3-D feature vectors, and are then classified by adopting the support vector machine. Promising results have been achieved for makeup-invariant face recognition on two available image databases including YouTube makeup and virtual makeup, and plastic surgery-invariant face recognition on a plastic surgery face database is compared to several state-of-the-art feature extraction methods. Several real-world scenarios are also planned to evaluate the performance of the proposed method on a combination of these three databases with 1102 subjects.

Three-dimensional (3-D) reconstruction is an indispensable tool in areas such as biology, chemistry, medicine, material sciences, etc. The sample can be reconstructed using confocal or nonconfocal mode of a microscope. The limitation of the confocal approach is the sample size. Currently used devices work mostly with sample surface area up to 1  cm². We suggest a three-step method that creates 3-D reconstruction from multifocal images in nonconfocal mode that is qualitatively comparable to the confocal results. Our method, thus, takes advantage of both microscope modes—high-quality results without sample size limitation. The preprocessing step eliminates the additive noise with Linderberg-Lévi theorem. The main focus criterion is based on adjusted Fourier transform. In the final step, we eliminate the defective clusters using the adaptive pixel neighborhood algorithm. We proved the effectiveness of our noise reduction and 3-D reconstruction method by the statistical comparisons; the correlation coefficients average 0.987 for all types of Fourier transforms.