Recent advances in computing and communications are leading to the creation of large-scale databases of visual and multimedia information. Such databases are finding ready applications in a wide range of fields such as advertising and marketing, education and training, entertainment, medicine, and remote sensing.

Content-based image retrieval (CBIR) has become one of the most active research areas in the past few years. Most of the attention from the research has been focused on indexing techniques based on global feature distributions. However, these global distributions have limited discriminating power because they are unable to capture local image information. The use of interest points in content-based image retrieval allow image index to represent local properties of the image. Classic corner detectors can be used for this purpose. However, they have drawbacks when applied to various natural images for image retrieval, because visual features need not be corners and corners may gather in small regions. In this paper, we present a salient point detector. The detector is based on wavelet transform to detect global variations as well as local ones. The wavelet-based salient points are evaluated for image retrieval with a retrieval system using color and texture features. The results show that salient points with Gabor feature perform better than the other point detectors from the literature and the randomly chosen points. Significant improvements are achieved in terms of retrieval accuracy, computational complexity when compared to the global feature approaches.

Relevance feedback is a powerful and widely used technique in content-based image retrieval systems. However, most relevance feedback approaches use only weighted feature sum of the feedback images to optimize the query for refining image similarity assessment. Such approaches do not work very well in most cases, especially when the user wants to express an ‘‘OR’’ relationship among the queries. In this paper, we propose three methods, weighted distance sum, minimal distance (MD), and minimal distance rank (MDR), to measure the similarity between images in database and the feedback images in query refinement. After experimental comparisons we propose a relevance feedback scheme using the MDR method and the MD method to describe the user’s multiple intentions. Based on this scheme, an image retrieval and semiautomatic annotation system, iFind [Y. Lu, C. H. Hu, X. Q. Zhu, H. J. Zhang, and Q. Yang, ‘‘A unified framework for semantics and feature based relevance feedback in image retrieval systems,’’ in Proc. 8th ACM International Conference on Multimedia, pp. 31–37 (2000) and X. Q. Zhu, W. Y. Liu, H. J. Zhang, and L. D. Wu, ‘‘An image retrieval and semi-automatic annotation scheme for large image databases on the web,’’ in Proc. SPIE 4311, 168–177 (2001)], which integrates query refinement and semantic information, is presented. Experiments show that the proposed methods can result in substantial improvement in retrieval accuracy and can be especially useful for retrieval or annotating large image databases.

A necessary capability for content-based retrieval is to support the paradigm of query by example. In the past, there have been several attempts to use low-level features for video retrieval. However, most of the approaches support queries using image sequences only. We present an algorithm for matching multimodal (audio-visual) patterns for the purpose of content-based video retrieval. The novel ability of our approach to use the information content in multiple media coupled with a strong emphasis on temporal similarity differentiates it from the state of the art in content-based retrieval. At the core of the pattern matching scheme is a dynamic programming algorithm, which leads to a significant improvement in performance. Coupling the use of audio with video this algorithm can be applied to grouping of shots based on audio-visual similarity. We also support relevance feedback. The user can provide feedback to the system, by choosing clips, which are closer to the user’s desired target. The system then automatically adjusts the relative weights or relevance of the media and fetches different sets of target clips accordingly. It is our observation that a few iterations of such feedback are generally sufficient, for retrieving the desired video clips.

In this article we present DANCERS, a system for automated indexing and retrieval of TV broadcast news. The system output, a topic-based report organization structure, is obtained in three major steps. In the first step, the visual and the speech streams of a news program are analyzed in order to partition the program into report segments. In the second step, the transcribed speech of each of the report segments is matched with the content of a large prespecified textual topic database. This database covers a large number of topics and can be updated anytime, e.g., according to the user’s interests. The result of the matching procedure is a list of most probable topics per report segment that are used in the last step to merge related neighboring segments into reports. The most probable topics that serve as the base for the segmentmerging procedure are at the same time the indexes of the newly created reports, using which the reports can be retrieved or classified. A relative simplicity of the indexing approach, minimized user involvement and highly satisfying indexing performance are the major characteristics of our system.

Color features are reviewed and their effectiveness assessed in the application framework of key-frame clustering for abstracting unconstrained video. Existing color spaces and associated quantization schemes are first studied. Description of global color distribution by means of histograms is then detailed. In our work, 12 combinations of color space and quantization were selected, together with 12 histogram metrics. Their respective effectiveness with respect to picture similarity measurement was evaluated through a query-by-example scenario. For that purpose, a set of still-picture databases was built by extracting key frames from several video clips, including news, documentaries, sports and cartoons. Classical retrieval performance evaluation criteria were adapted to the specificity of our testing methodology.

In this article, several practical algorithms are proposed to support content-based video analysis, modeling, representation, summarization, indexing, and access. First, a multilevel video database model is given. One advantage of this model is that it provides a reasonable approach to bridging the gap between low-level representative features and high-level semantic concepts from a human point of view. Second, several model-based video analysis techniques are proposed. In order to detect the video shots, we present a novel technique, which can adapt the threshold for scene cut detection to the activities of variant videos or even different video shots. A seeded region aggregation and temporal tracking technique is proposed for generating the semantic video objects. The semantic video scenes can then be generated from these extracted video access units (e.g., shots and objects) according to some domain knowledge. Third, in order to categorize video contents into a set of semantic clusters, an integrated video classification technique is developed to support more efficient multilevel video representation, summarization, indexing, and access techniques.

We describe a video summarization technique that uses motion descriptors computed in the compressed domain. It can either speed up conventional color-based video summarization techniques, or rapidly generate a key-frame based summary by itself. The basic hypothesis of the work is that the intensity of motion activity of a video segment is a direct indication of its ‘‘summarizability,’’ which we experimentally verify using the MPEG-7 [MPEG-7 Visual Committee Draft URL: http://www.cselt.it/mpeg/working documents. htm official MPEG site] motion activity descriptor and the fidelity measure proposed in H. S. Chang, S. Sull, and S. U. Lee, ‘‘Efficient video indexing scheme for content-based retrieval,’’ IEEE Trans. Circuits Syst. Video Technol. 9(8), (1999). Note that the compressed domain extraction of motion activity intensity is much simpler than the color-based calculations. We are thus able to quickly identify easy to summarize segments of a video sequence since they have a low intensity of motion activity. We are able to easily summarize these segments by simply choosing their first frames. We can then apply conventional color-based summarization techniques to the remaining segments. We thus speed up color-based summarization by reducing the number of segments processed. Our results also motivate a simple and novel key-frame extraction technique that relies on a motion activity based nonuniform sampling of the frames. Our results indicate that it can either be used by itself or to speed up color-based techniques as explained earlier.

In this paper we present a fuzzy framework for domaindependent analysis of video sequences. Fuzzy clustering and cluster validation methods are first employed to determine the number of distinct shot patterns and construct a reference model for a program or video domain of interest, using an appropriate training set. This model is subsequently utilized to assign new input data to the available classes by a fuzzy minimum-distance classifier. Additional domain-specific information can be introduced after classification to further enhance the annotations associated with every shot. The main advantage of the approach is that it builds a model for the input video automatically from training data, and thus eliminates the need for extensive user supervision. The fuzzy representation method improves the interpretability of the results, and reduces the number of erroneous classifications, since the continuous class affiliations of each input sample provide a confidence measure for the final assignments. The proposed approach presents a computationally efficient, unsupervised method for building browsable semantic descriptions of video sequences. Specifically, the algorithm can be used to generate various components of an MPEG-7-compliant description.

In this article, we report a study on the problem of person identification in TV programs, such as situation comedy shows. A person identification system is constructed based on the joint use of visual and audio information. The system consists of two modules, namely, the analysis and the fusion modules. The analysis module contains a visual analysis component responsible for detection, tracking, and recognition of faces in video, and the audio analysis component, which operates by speaker identification. Both components have their advantages under different circumstances and we studied how to exploit the interaction between them for improved performance. Two fusion strategies are compared in our research. In the first strategy, the audio-verify-visual fusion strategy, speaker identification is used to verify the face recognition result. The second strategy, the visual-aid-audio fusion strategy, consists of using face recognition and tracking to supplement speaker identification results. By comparing the output from our system with our ground truth database, we evaluate the performance of each individual analysis component and their fusion. The results show that while the audio-verify-visual fusion strategy has slightly lower recall than the original face recognition system, it achieves the best identification precision among different algorithms. This suggests that such a strategy is suitable for applications where precision is much more critical than recall (e.g., security systems). The visual-aid-audio fusion strategy, on the other hand, generates the best overall identification performance. It outperforms either of the individual analysis components greatly in both precision and recall. This strategy is suitable to more general applications, such as, in our case, person identification in TV programs.

The general problem of developing algorithms for the automated or computer-assisted indexing of images by structural contents is a significant research challenge. This is particularly so in the case of biomedical images, where the structures of interest are commonly irregular, overlapping, and partially occluded. Examples are the images created by digitizing film x-rays of the human cervical and lumbar spines. We have begun work toward the indexing of 17 000 such spine images for features of interest in the osteoarthritis and vertebral morphometry research communities. This work requires the segmentation of the images into vertebral structures with sufficient accuracy to distinguish pathology on the basis of shape, labeling of the segmented structures by proper anatomical name, and classification of the segmented, labeled structures into groups corresponding to high level semantic features of interest, using training data provided by biomedical experts. In this paper, we provide a technical characterization of the cervical spine images and the biomedical features of interest, describe the evolving technical approach for the segmentation and indexing problem, and provide results of algorithms to acquire basic landmark data and localization of spine regions in the images.

In this article we present research work in the field of content-based image retrieval in large databases applied to the paleontology image database of the Universite´ de Bourgogne, Dijon, France, called ‘‘TRANS’TYFIPAL.’’ Our indexing method is based on multiresolution decomposition of database images using wavelets. For each family of paleontology images we try to find a model image that represents it. The K-means automatic classification algorithm divides the space of parameters into several clusters. A model image for each cluster is computed from the wavelet transform of each image of the cluster. Then a search tree is built to offer users a graphic interface for retrieving images. So users have to navigate through this tree of model images to find an image similar to that they are requesting. Our contribution in the field is the building of the model and of the search tree to make user access easier and faster. At the end of this article we give experimental results and a description of future work that will be done to enhance our indexing and retrieval method.

Methods of lossless compression of medical image data are considered. Selected class of efficient algorithms have been constructed, examined, and optimized to conclude the most useful tools for medical image archiving and transmission. Image data scanning, 2D context-based prediction and interpolation, and statistical models of entropy coder have been optimized to compress effectively ultrasound (US), magnetic resonance (MR), and computed tomography (CT) images. The SSM technique of suitable data decomposing scanning method followed by probabilistic modeling of the context in arithmetic encoding have occurred the most useful in our experiments. Context order, shape, and alphabet have been fitted to local data characteristics to decrease image data correlation and dilution of statistical model. Average bit rate value over test images is equal to 2.53 bpp for SSM coder and significantly overcomes 2.92 bpp of CALIC bit rate. Moreover, optimization of lossless wavelet coder by thinking of efficient subband decomposition schemes, and integer-to-integer transforms is reported. Efficient hybrid coding method (SHEC) as a complete tool for medical image archiving and transmission is proposed. SHEC develops SSM by including CALIC-like coder to compress the highest quality images and JPEG2000 wavelet coder for progressive delivering of high and middle quality images in telemedicine systems.

An object-to-object color mapping method based on image segmentation is proposed. The pictorial color image is segmented into different object areas with clustered color distributions. Euclidian or Mahalanobis color distance measures, and Bayesian decision rule are introduced to the image segmentation. After the image segmentation, each segmented pixel is projected onto principal component space by Hotelling transform and the color mappings are performed for the principal components to be matched in between the individual objects of original and printed or displayed images. Experiments on the automatic color correction for inkjet prints and color mapping for preferred skin color reproduction are reported. The paper further discusses how the setting of initial seeds points affects the image segmentation results.

Like data encryption, secret sharing is an important method of protecting secret messages. However, the basic ideas of secret sharing and data encryption are radically different. In this paper, a new method for secret sharing is proposed that focuses on image data. It is called the virtual image sharing method (VISM). It is basically derived from the virtual image cryptosystem. According to our experiments and security analyses, VISM can divide a secret image into a large number of different shadows which are guaranteed to be significant images. Illegal users cannot detect whether they are real images or not. VISM also guarantees the security of the secret image because no one is able to reconstruct any piece of the secret image without possessing all of the shadows. Besides camouflage and security, VISM has two other benefits. One is its compression ability. The total size of shadows can be smaller than that of the secret image. The other benefit is its efficiency. VISM avoids the painstaking task of applying complex operations or techniques.

A novel theorem linking reflectional symmetry and rotational symmetry has been established in this paper. This theorem constrains that, for a rotationally symmetric image with K?1 fold(s) (K-RSI), its number of reflection-symmetric axes must be either K or zero. The novel constraint theorem provides the guidance for the procedure of symmetry detection. It has been demonstrated by the experiments included in this paper.

In recent work, we introduced some topological notions for grayscale images based on a cross-section topology. In particular, the notion of destructible point, which corresponds to the classical notion of simple point, allows us to build operators that simplify a grayscale image while preserving its topology. In this paper, we introduce new notions and operators in the framework of the crosssection topology. In particular, the notion of the l-destructible point allows us to selectively modify the topology, based on a local contrast parameter l. By combining homotopic and nonhomotopic operators, we introduce new methods for filtering, thinning, segmenting, and enhancing grayscale images.

Among the techniques to limit the effect of error propagation in low bitrate video coding, the best performance is achieved through the joint use of forward error correction (FEC) and automatic repeat request strategies. However, retransmission of corrupted data introduces additional delay which may be critical in some practical applications. In such cases, only a FEC strategy is feasible at the expense of a significant increase of the overall transmission bit rate. In this paper, a strategy is proposed which instead of protecting the whole H.263 video stream, uses all the redundancy to protect the most important parts of the bit stream, i.e., group-ofblock (GOB) start codes. More specifically, the 22-bit long start code used in the H.263 standard is replaced with longer Gold sequences which ensure a higher protection against noise. A different sequence is used for each GOB so that additional information can be obtained which can be used to improve the quality of the decoded sequence. The new technique is derived by assuming a slow-fading channel such as those encountered in pedestrian applications. Simulation results prove the effectiveness of the proposed technique with respect to classical FEC schemes in terms of PSNR and overall bit rate.

This PDF file contains the editorial “Erratum” for JEI Vol. 10 Issue 04