A program of evaluation of compression algorithms proposed for use in a digital cinema application is described and the results presented in general form. The work was intended to aid in the selection of a compression system to be used for the digital cinema release of Star Wars: Episode II, in May 2002. An additional goal was to provide feedback to the algorithm proponents on what parameters and performance levels the feature film industry is looking for in digital cinema compression. The primary conclusion of the test program is that any of the current digital cinema compression proponents will work for digital cinema distribution to today's theaters.

Film has been used for movies for over 100 years. D-cinema will soon produce images that equal or surpass film. When that happens, movies will take their basis from computer technology, leading to higher quality, lower cost, and greater flexibility for all aspects of the industry. For example, d-cinema will allow higher frame rates, flexible subtitles, alternative content, and resolutions and color spaces beyond film. Given these opportunities, we must not simply emulate the mechanical past. Insisting, for instance, on a single type of compression or security scheme is misguided. Both these areas are evolving, and we should take advantage of that evolution, while promoting standards that allow that evolution to happen in an orderly way. Mechanical projectors can play back only one kind of film; computer servers can play back any number of formats. It would be wrong to select a single format at this time, only to have the technology become more capable in the near future. We must allow the continuously evolving technology that characterizes computers, not the frozen technology that has characterized mechanical systems. We must work toward an environment that allows interoperability of d-cinema technologies-not systems that limit us to a single technology.

The proponents of digital cinema seek picture quality exceeding that of the best film-based presentation. Quantifying the performance of systems for the presentation of high quality imagery presents several challenges. One is that the dynamic range and the resolution may not be simply related to the nominal characteristics of bit-depth and pixel counts. We review some of the measurement methods that have been applied to determining these characteristics. One of the presumed advantages of high bit depth systems is to reduce the visibility of image banding. Non-uniformity of the display can be compensated in test pattern design to enable the measurement of banding contrast. The subjective assessment of banding is compared to a contrast-weighted model of just noticeable image differences. Applied to a class of image banding test patterns, the metric relates dynamic range to contouring. The model produces an estimate of the visibility threshold for image contouring in a 10-bit system, superior to a simple Weber model. These measurement issues will continue to be challenges as d-cinema systems improve.

The process of distributing and exhibiting a motion picture has changed little since the Lumière brothers presented the first motion picture to an audience in 1895. While this analog photochemical process is capable of producing screen images of great beauty and expressive power, more often the consumer experience is diminished by third generation prints and by the wear and tear of the mechanical process. Furthermore, the film industry globally spends approximately $1B annually manufacturing and shipping prints. Alternatively, distributing digital files would theoretically yield great benefits in terms of image clarity and quality, lower cost, greater security, and more flexibility in the cinema (e.g., multiple language versions). In order to understand the components of the digital cinema chain and evaluate the proposed technical solutions, the Entertainment Technology Center at USC in 2000 established the Digital Cinema Laboratory as a critical viewing environment, with the highest quality film and digital projection equipment. The presentation describes the infrastructure of the Lab, test materials, and testing methodologies developed for compression evaluation, and lessons learned up to the present. In addition to compression, the Digital Cinema Laboratory plans to evaluate other components of the digital cinema process as well.

The Motion Picture Industry began a transition from film based distribution and projection to digital distribution and projection several years ago. Digital delivery and presentation offers the prospect to increase the quality of the theatrical experience for the audience, reduce distribution costs to the distributors, and create new business opportunities for the theater owners and the studios. Digital Cinema also presents an opportunity to provide increased flexibility and security of the movies for the content owners and the theater operators. Distribution of content via electronic means to theaters is unlike any of the traditional applications for video compression. The transition from film-based media to electronic media represents a paradigm shift in video compression techniques and applications that will be discussed in this paper.

We have developed a prototype digital cinema system that can store, transmit and display extra high quality movies of 8-million pixel resolution, using JPEG2000 coding algorithm. The image quality is 4 times better than HDTV in resolution, and enables us to replace conventional films with digital cinema archives. Using wide-area optical gigabit IP networks, cinema contents are distributed and played back as a video-on-demand (VoD) system. The system consists of three main devices, a video server, a real-time JPEG2000 decoder, and a large-venue LCD projector. All digital movie data are compressed by JPEG2000 and stored in advance. The coded streams of 300~500 Mbps can be continuously transmitted from the PC server using TCP/IP. The decoder can perform the real-time decompression at 24/48 frames per second, using 120 parallel JPEG2000 processing elements. The received streams are expanded into 4.5Gbps raw video signals. The prototype LCD projector uses 3 pieces of 3840×2048 pixel reflective LCD panels (D-ILA) to show RGB 30-bit color movies fed by the decoder. The brightness exceeds 3000 ANSI lumens for a 300-inch screen. The refresh rate is chosen to 96Hz to thoroughly eliminate flickers, while preserving compatibility to cinema movies of 24 frames per second.

In this paper, a new concept is introduced for economy image segmentation applicable in an earlier designed object based motion estimation algorithm. The image segmentation is based on simple features, like average grayscale within a segment, and uses spatial-temporal predictions in order to economize the segmentation procedure. Focus is on the segmentation process and the robust application of a non-perfect segmentation mask in the object based motion estimator. In this application, the new image segmentation method helps to improve the motion segmentation, while reducing the operations count. The paper describes both the object-based motion estimation and the block-based image segmentation. Experimental results are described in order to proof the validity of the concept.

In this paper the motion estimation between two consecutive color frames is analyzed by means of the Laguerre Gauss (LG) Wavelet Transform and the Mean Field theory. This contribution extends some previous work of the authors in this field. By decomposing the frames using the LG functions, a local image representation is obtained; using the self-steerability property of the LG it is possible to design a maximum likelihood estimation procedure to identify both the displacement and the rotation fields. To regularize the motion field taking in account the possible occlusions, the mean field theory has been used.

This paper deals with the problem of estimating multiple transparent motions that can occur in computer vision applications, e.g. in the case of semi-transparencies and occlusions, and also in medical imaging when different layers of tissue move independently. Methods based on the known optical-flow equation for two motions are extended in three ways. Firstly, we include a regularization term to cope with sparse flow fields. We obtain an Euler-Lagrange system of differential equations that becomes linear due to the use of the mixed motion parameters. The system of equations is solved for the mixed-motion parameters in analogy to the case of only one motion. To extract the motion parameters, the velocity vectors are treated as complex numbers and are obtained as the roots of a complex polynomial of a degree that is equal to the number of overlaid motions. Secondly, we extend a Fourier-Transform based method proposed by Vernon such as to obtain analytic solutions for more than two motions. Thirdly, we not only solve for the overlaid motions but also separate the moving layers. Performance is demonstrated by using synthetic and real sequences.

Motion compensated wavelet video coding provides very high coding efficiency while enabling spatio-temporal-SNR-complexity scalability. Besides the high degree of adaptabitility, the inherent data prioritization leads to increased robustness in conjunction with unequal error protection (UEP) schemes, and improved error concealment. Hence, motion compensated wavelet video coding schemes are generating great interest for wireless video streaming. Such schemes use motion compensated temporal filtering (MCTF) to remove temporal redundancy. Many extensions to conventional MCTF schemes, that increase the flexibility and the coding efficiency, have been proposed. However these extensions require the coding of additional sets of motion vectors. In this paper, we first define a redundancy factor to identify the additional number of motion vectors that need to be coded with such schemes. We then propose to exploit the temporal correlations between motion vectors to code and estimate them efficiently. We use prediction to reduce the bits needed to code motion vectors. We describe two prediction methods, and highlight the advantages of each scheme. We also use MV prediction during motion estimation, i.e. change the search center and the search range based on the prediction, and describe the tradeoffs to be made between rate, distortion, and complexity. We perform several experiments to illustrate the gains of using temporal prediction, and identify the content dependent nature of results.

Global motion estimation is a powerful tool used in computer vision and video processing. In this paper, we propose an efficient approach of estimating general global motion parameters from coarsely sampled motion vector fields. The approach is based on gradient descent with outlier rejection. Experimental results show the effectiveness of the method in terms of robustness and accuracy.

Despite the fact that the existing region-wise motion compensation techniques (RWMC) are more efficient than the conventional variable size block motion compensation technique, they still do not use the visual characteristic of a frame in terms of brightness, contrast and sharpness in order to determine the motion information of the partitioned regions in a most efficient manner. The objective of this paper is to present a quad-tree structured region-wise motion compensation technique using enhanced motion vectors. The scheme described in this paper uses the brightness information of a frame in order to reach the objective of having motion vectors with a higher accuracy. Even though using enhanced motion vectors reduces the distortion, it causes the bit rate to increase. This poses the challenge of optimizing the bit rate-distortion performance. The proposed method partitions a frame based on a fine to coarse resolution strategy through the merging and combining processes. The merging process permits 4-to-1, 3-to-1 and 2-to-1 merges. For such merges, the main idea is to minimize the distortion subject to the constraint that the bit rate should not exceed a pre-defined value. The combining process further reduces the total number of partitioned regions by combining some of the regions that have the same enhanced motion vectors. The proposed technique uses a 2-bit code for coding the quad-tree structure while two different code lengths encode the enhanced motion vectors of the partitioned regions depending on the differential brightness threshold value. The proposed method is applied to a number of video sequences and compared with the other existing methods. The test results show that the new method can significantly improve the rate-distortion performance.

Layered coding (LC) and multiple description coding (MDC) have been proposed as two different kinds of 'quality adaptation' schemes for video delivery over the current Internet or wireless networks. To combine the advantages of LC and MDC, we present a new approach -- Multiple Description Layered Coding (MDLC), to provide reliable video communication over a wider range of network scenarios and application requirements. MDLC improves LC in that it introduces redundancy in each layer so that the chance of receiving at least one description of base layer is greatly enhanced. Though LC and MDC are each good in limit cases (e.g., long end-to-end delay for LC vs. short delay for MDC), the proposed MDLC system can address intermediate cases as well. Same as a LC system with retransmission, the MDLC system can have a feedback channel to indicate which descriptions have been correctly received. Thus a low redundancy MDLC system can be implemented with our proposed runtime packet scheduling system based on the feedback information. The goal of our scheduling algorithm is to find a proper on-line packet scheduling policy to maximize the playback quality at the decoder. Previous work on scheduling algorithms has not considered multiple decoding choices due to the redundancy between data units, because of the increase in complexity involved in considering alternate decoding paths. In this paper, we introduce a new model of Directed Acyclic HyperGraph (DAHG) to represent the data dependencies among frames and layers, as well as the data correlation between descriptions. The impact of each data unit to others is represented by messages passing along the graph with updates based on new information received. Experimental results show that the proposed system provides more robust and efficient video communication for real-time applications over lossy packet networks.

This paper first introduces an asymmetric tree structure for utilization with an error resilient form 3-D SPIHT, called ERC-SPIHT. Then, we present a fast error concealment scheme, borrowed from Rane et al.'s work with images, for embedded video bitstream using ERC-SPIHT. In addition to using simple averaging method in the root subband, we detect the presence or absence of edges in the lost block of every image. Then we use an interpolation scheme to recover the lost edge information.

The most recent literature indicates Multiple Description (MD) as a new coding approach to handle the problem of video transmission over unreliable networks. This technique describes the source information in equally important and independent streams named multiple descriptions, by introducing some kind of redundancy into each description. Recent works have shown the success of using motion compensation prediction also in an MD video coding scheme. Even though these MD systems have proved to be robust in an error-prone environment, they are in practice forced to use only two descriptions. This is a very strong constraint when more than two levels of reconstruction are required. For this reason this paper proposes two architectures of MD video coders; the first, that we call DC-MDVC, is based on the classical MD video codec architecture and the principle contribution is the use of a new MD algorithm based on a polyphase down-sampling technique. Since this architecture is suitable only in case of two descriptions, then a second architecture, called IF-MDVC, is given, where multi-level scalability is introduced by generating the multiple descriptions before the prediction loop. Results show that both architectures are appealing, and the second architecture seems to perform impressively well.

This paper proposes a novel robust framework for scalable video over Internet. The main contribute of our work is that a simplified rate-distortion theory is specially developed for scalable bit-stream over the network and the corresponding bit allocation is presented to determine the sets of the channel rate for each video layer. Compared with the traditional iterative optimal bit allocation with complexity O(n^L) time, simulations show that our scheme achieves high quality video transmission with much less complexity O(L x n) time, only no more than 0.2dB under different network conditions (different bandwidth and different packet loss case). Besides, our error control scheme can be naturally combined with congestion control and error resilient techniques to enhance the performance of the overall system.

Two types of scalabilities exist in current scalable video streaming schemes: (1) nested scalability, in which different representations (i.e., descriptions) of each frame are generated using layered scalable coding and have to be decoded in a fixed sequential order, and (2) parallel scalability, which is used in multiple description coding (MDC) where different descriptions are mutually refinable and independently decodable. In this paper, we present a general framework that includes both scalabilities and demonstrate the similarity between the leaky prediction based layered coding and an MDC scheme that uses motion compensation. Based on this framework, we introduce nested scalability into each description of the MDC stream and propose a fine granularity scalability (FGS) based MDC approach. We also develop a scalable video coding structure that is characterized by the dual-leaky prediction to balance the trade-off between coding efficiency and the error resilience performance of the coded bit stream.

In this paper, a scheme for robust image coding and transmission over noisy channels is presented. First, the spatial-orientation trees in wavelet domain are independently encoded using the algorithm of set partitioning in hierarchical trees (SPIHT). We generate multiple bitstreams in order to limit the error propagation. Because the self-similarity across subbands is preserved in each wavelet tree, a high coding efficiency is also obtained. However, when these bitstreams with variable length are transmitted sequentially, error impact may still be propagated across the boundaries between two consecutive bitstreams and may cause catastrophic decoding. Therefore, we apply error resilient entropy coding (EREC) to reorganize these bitstreams into fixed length slots so that synchronization of the start of each bitstream is automatically achieved at the receiver. This is particularly useful for the progressive data where bits located at the beginning of bitstream are always more important than those at the end. Finally, to further alleviate the impact of errors in the beginning of the bitstreams, parity bits are added and exploited by an error concealment technique conducted only in the lowest frequency subband. Experimental results demonstrate an excellent error resilient performance of the proposed scheme.

Several algorithms have been proposed to enhance the resolution of a reference image from multiple still images or video that may be captured/stored in raw or compressed format. This paper provides a thorough study of how the performance of the projections onto convex sets (POCS) method is affected by camera parameters, quantization of the pixel-values, motion estimation errors, and quantization in the DCT-domain (when compressed data is used). Experimental results are provided to evaluate the practical applicability of super-resolution reconstruction in various scenarios. It has been observed that the quality of the resolution enhancement depends on the quantization of pixel intensity values in the RGB (uncompressed) or YUV (compressed) domains by the video capture device, as well as the accuracy of the estimated motion parameters between successive frames.

An efficient MPEG-2 to MPEG-4 video transcoder is presented in this paper. We consider the transcoding from high quality and bit rate MPEG-2 video with larger image size (e.g. 4CIF/4SIF, CIF) to lower quality and bit rate MPEG-4 video with smaller image size (e.g. CIF, QCIF). First, the transcoder needs to down-sample the input MPEG-2 video. Since the motion vectors carried by the MPEG-2 stream will be reused in the transcoding process, they are sub-sampled besides the frame pixels, and the coding mode for each down-sampled macroblock is examined. A new rate control method is proposed to convert the high bit rate MPEG-2 video to the low bit rate MPEG-4 counterpart. The proposed rate control scheme adjusts the frame rate and the frame quantization step size simultaneously according to the channel bandwidth to achieve a good temporal-spatial quality tradeoff. Due to the reuse of motion vectors, key frames (i.e. I and P frames) cannot be skipped to maintain the prediction sequential order, while some B frames containing less temporal information may be skipped in transcoding to save the bit rate. Skipped B frames can be reconstructed at the decoder to ensure the full frame rate playback. The TM7 quadratic Rate-Qtz model is adopted in the proposed rate control scheme to calculate the re-quantization step size from the given target bit. Simulations show that the proposed MPEG-2 to MPEG-4 video transcoder with rate control can out-perform the basic MPEG transcoder that adjusts the re-quantization step size at a constant frame rate. The complexity of the proposed transcoder is low so that it can be used in real-time applications.

This paper considers the transmission of QCIF resolution MPEG-4 video signals over hostile channels like wireless and mobile at transmission rate of 64 kbps. Although MPEG-4 incorporates some error resilience tools, this standard is still fragile for video transmission over time-variant deep fading channels. We investigated the significance of MPEG-4 video syntactical elements considering MPEG-4 encoded streams, QPSK modulated and delivered through Binary, AWGN and Rayleigh (50 Km/h) channels with SNR range between 0 and 40 dB. We have used four QCIF video test sequences, Foreman, News, Container and Coastguard, and enabled slice resynchronization and data partitioning modes in MPEG-4 video coding. The simulation results show that despite significance of particular elements varies with the channel SNR and with the particular test sequence, we could graphically identify two classes of significance at low channel SNRs. The most significant class includes elements such as VOP header, Video Packet header, DC Marker and DC Coefficients of INTRA-VOP & MB header. At high Rayleigh channel SNR, above 15 dB, AC coefficients of INTRA-VOPs and Motion Vectors were found to be the most significant elements. Any unequal error protection, data partitioning and grouping strategies covering a wide range of SNRs can then be designed based on the significance results presented in this paper.

One of the well-known weaknesses of block-based video compression schemes such as MPEG-4, at low bit-rates, is blocking artifacts. They can be well removed by post-processing solutions offering an efficient deblocking filter, unfortunately at the cost of a significant increase of the decoding process complexity. In this paper, we propose a new method to reduce blocking artifacts (thus increasing the visual quality of sequences or the compression rate) at the encoding stage to get rid of, or to help, the post-processing step. We consider a filtering operation in the DCT domain, applied on blocks before quantization to leave the decoding process unmodified. The solution is adaptive and focuses on regions which would be a priori blocky regarding some texture features and encoding parameters. It is based on the mix of quantization step reduction at the frame level, with DCT coefficients low-pass filtering at the block level, on top of any bit-rate regulation module. Tests showed the efficiency of the method: the sequences encoded with our DCT filter are cleaner, with less ringing and, above all, fewer blocking artifacts than the ones without any processing. Visual quality is therefore better for a constant bit consumption. Depending on cases, post-processing is either useless or improved by our process.

Selective enhancement mechanism of Fine-Granular-Scalability (FGS) In MPEG-4 is able to enhance specific objects under bandwidth variation. A novel technique for self-adaptive enhancement of interested regions based on Motion Vectors (MVs) of the base layer is proposed, which is suitable for those video sequences having still background and what we are interested in is only the moving objects in the scene, such as news broadcasting, video surveillance, Internet education, etc. Motion vectors generated during base layer encoding are obtained and analyzed. A Gaussian model is introduced to describe non-moving macroblocks which may have non-zero MVs caused by random noise or luminance variation. MVs of these macroblocks are set to zero to prevent them from being enhanced. A segmentation algorithm, region growth, based on MV values is exploited to separate foreground from background. Post-process is needed to reduce the influence of burst noise so that only the interested moving regions are left. Applying the result in selective enhancement during enhancement layer encoding can significantly improves the visual quality of interested regions within an aforementioned video transmitted at different bit-rate in our experiments.

As emerging video coding standards, e.g. H.264, aim at high-quality video contents at low bit-rates, the encoding and decoding processes require much more computation than most existing standards do. This paper analyzes software implementation of a real-time H.264 decoder on general-purpose processors with media instructions. Specifically, we discuss how to optimize the speed of H.264 decoders on Intel Pentium 4 processors. This paper first analyzes the reference implementation to identify the time-consuming modules. Our study shows that a number of components, e.g., motion compensation and inverse integer transform, are the most time-consuming modules in the H.264 decoder. Second, we present a list of performance optimization methods using media instructions to improve the efficiency of these modules. After appropriate optimizations, the decoder speed improved by more than 3x---it can decode a 720×480 resolution video sequence at 48 frames per second on 2.4GHz Intel Pentium 4 processors compared to reference software’s 12 frames per second. The optimization techniques demonstrated in this paper can also be applied to other video/image processing applications. Additionally, after presenting detailed application behavior on general-purpose processors, this paper discusses a few recommendations on how to design future efficient/powerful video/image applications/standards with given hardware implications.

This paper proposes a fast multi-resolution block-matching algorithm (MRBMA) for MPEG-2 video encoding, which satisfies high estimation performance and efficient LSI implementation. MRBMA is based on the characteristic that field motion vector’s (MV’s) are very close to its corresponding frame MV. Firstly, MRBMA performs frame-based motion estimation (ME) as follows: At the coarsest level, two MV candidates are found on the basis of minimum matching error for the next level search. The two MV candidates from the coarsest level search and the other one based on spatial MV correlation are used as center points for three local searches at the middle level. At the finest level, a frame MV is obtained from a local search around a single candidate from the middle level search. Field MV’s are estimated with the single MV candidate from the middle level search of frame ME as initial estimates at the finest level, without any coarser level searches. This paper also describes a VLSI architecture based on MRBMA. This architecture is optimized to provide a good tradeoff between on-chip memory size and I/O bandwidth with high throughput. We implemented this architecture with about 140K gates and 25K bytes SRAM for a large search range of [-192.0, +191.5] by using a synthesizable Verilog HDL.

This paper presents FaDA (Fast Deblocking Algorithm), a low complexity post-processing algorithm that improves the visual quality of MPEG-4 low bit rate videos displayed on mobile devices. Most of the algorithms proposed up to now to enhance the visual quality after the decoder are far too complex for this kind of devices, and unfortunately deblocking algorithms efficiency is tightly related to their complexity. FaDA achieves complexity reduction while maintaining quality thanks to two principles: 1- The artifact correction is driven by Human Visual System properties, 2- A particular attention has been paid on the design of the most consuming parts of the algorithm, following implementation guidelines. We present a complete complexity evaluation that considers not only computational charge, but also number and locality of memory accesses, data dependency, and internal data size requirements. Critical steps of the algorithm were implemented in C and optimized on a TriMedia DSP. The complexity evaluation reveals a reduction by a factor ranging between 2 and 3.5, compared to the reference MPEG-4 informative deblocking, depending on architecture and implementation. Extensive subjective tests performed on both mobile LCD screen and high resolution monitor show that FaDA average visual quality is equivalent to the reference algorithm.

A new high-performance scalable systolic array processor architecture module is presented which can simultaneously convolute k different (n x n) Filter Coefficient (FC) planes with a single (i x j) pixel Input Image Plane (IP). The architecture will have the capability to simultaneously perform convolution of k different (n x n) FC planes on 600dpi (dot per inch) IPs of size 8½” x 11” at a rate such that k convoluted Output Image (OI) plane pixels are output each system clock cycle for a system clock cycle time of less than 10 nanoseconds. Bit-parallel arithmetic is used and each IP pixel is 8-bits in length and each FC plane coefficient is 6-bits in length. A new pipelined systolic type architecture module is first developed which can generate one convoluted OI plane pixel per system clock cycle using a level of 'r' hardware resources for the case of (n = 5). The architecture is then extended in a scalable and deeper pipelined manner to allow simultaneous convolution of a single IP pixel, with k different (n×n) FC planes for the case of (n = 5), within one system clock cycle, utilizing less than (k × r) hardware resources. Synthesis and post-implementation VHDL simulation results are shown for an experimental model of the architecture which validates the scalability and functionality of the architecture. Simulation results demonstrate the performance of the architecture to be directly proportional to pipeline depth.

Monolithic integration of photodetectors, analog-to-digital converters, digital processing, and data storage can improve the performance and efficiency of next-generation portable image products. Our approach combines these components into a single processing element, which is tiled to form a SIMD focal plane processor array with the capability to execute early image applications such as median filtering (noise removal), convolution (smoothing), and inside edge detection (segmentation). Digitizing and processing a pixel at the detection site presents new design challenges, including the allocation of silicon resources. This research investigates the area-time (A•T²) efficiency by adjusting the number of Pixels-per-Processing Element (PPE). Area calculations are based upon hardware implementations of components scaled for 250nm or 120nm technology. The total execution time is calculated from the sequential execution of each application on a generic focal plane architectural simulator. For a Quad-CIF system resolution (176×144), results show that 1 PPE provides the optimal area-time efficiency (5.7 μs² x mm² for 250nm, 1.7 μs² x mm² for 120nm) but requires a large silicon chip (2072mm² for 250nm, 614mm² for 120nm). Increasing the PPE to 4 or 16 can reduce silicon area by 48% and 60% respectively (120nm technology) while maintaining performance within real-time constraints.

A simple scheme for fast Rate Distortion optimization based on Lagrange multipliers is presented in this paper. It is based on the observation that for accurate target rates, the change of the multiplier alone may not have an effect for small step sizes. However, a combination of small step sizes alogn with appropriate quantizer loop parameterization may solve this problem thus result in in significant computaiotnal savings since only a portion of the Rate distortion characteristics need to be considered for optimization. Extensive experiments using quantizer loop parameterization reveal savings between 60.14 and 92.94% for the rate bounding case and between 56.48 and 91.01% for the bisection search for a wide variety of video sequences exhibiting different characteristics.

n this paper, we consider the transmission of video over wireless direct-sequence code-division multiple access (DS-CDMA) channels. A layered (scalable) video source codec is used and each layer is transmitted over a different CDMA channel. Spreading codes with different lengths are allowed for each CDMA channel (multirate CDMA). Thus, a different number of chips per bit can be used for the transmission of each scalable layer. For a given fixed energy value per chip and chip rate, the selection of a spreading code length affects the transmitted energy per bit and bit rate for each scalable layer. An MPEG-4 source encoder is used to provide a two-layer SNR scalable bitstream. Each of the two layers is channel-coded using Rate-Compatible Punctured Convolutional (RCPC) codes. Then, the data are interleaved, spread, carrier-modulated and transmitted over the wireless channel. A multipath Rayleigh fading channel is assumed. At the other end, we assume the presence of an antenna array receiver. After carrier demodulation, multiple-access-interference suppressing despreading is performed using space-time auxiliary vector (AV) filtering. The choice of the AV receiver is dictated by realistic channel fading rates that limit the data record available for receiver adaptation and redesign. Indeed, AV filter short-data-record estimators have been shown to exhibit superior bit-error-rate performance in comparison with LMS, RLS, SMI, or 'multistage nested Wiener' adaptive filter implementations. Our experimental results demonstrate the effectiveness of multirate DS-CDMA systems for wireless video transmission.

Multiple Description Coding is a forward error correction scheme where two or more descriptions of the source are sent to the receiver over different channels. If only one channel is received the signal can be reconstructed with distortion D₁ or D₂. On the other hand, if both channels rae received the combined information is used to achieve a lower distortion D₀. Our approach is based on the Multiple State Video Coding with the novelty that we achieve a flexible unbalance rate of the two streams by varying the quantization step size while keeping the original frame rate constant. The total bitrate Rτ is fixed which is to be allocated between the two streams. If the assigned bitratres are not balanced there will be PSNR variations between neighboring frames after reconstruction. Our goal is to find the optimal rate allocation while maximizing the average reconstructed frame PSNR and minimizing the PSNR variations given the total bitrate Rτ and the packet loss probabilities p₁ and p₂ over the two paths. The reconstruction algorithm is also taken into account in the optimization process. The paper will report results presenting optimal system designs for balanced but also for unbalanced path conditions.

In this paper, we present effective quality-of-service renegotiating schemes for streaming video. The conventional network supporting quality-of-service generally allows a negotiation at call setup. However, it is not efficient for the video application since the compressed video traffic is statistically non-stationary. Thus, we consider the network supporting quality-of-service renegotiations during the data transmission, and study effective quality-of-service renegotiating schemes for streaming video. Simple token bucket model, whose parameters are token filling rate and token bucket size, is adopted for the video traffic model. The renegotiating time instants and the parameters are determined by analyzing the statistical information of compressed video traffic. In this paper, two renegotiating approaches, i.e. fixed renegotiating interval case and variable renegotiating interval case, are examined. Finally, the experimental results are provided to show the performance of the proposed schemes.

Traffic specification plays a crucial role in resource reservation for multimedia services over packet networks. The current development of QoS-guaranteed services still leaves a wide room for the selection of traffic specification. In this paper, we propose a new method to estimate the traffic specification of deterministic service. Usually, the estimation of traffic specification is independent of the QoS requirements, e.g. the delay bound. The proposed method is based on the point-of-view of the call admission control that takes the delay bound requirement into the estimation process. The solution is efficient in the sense that the resulting traffic specification is simple while still providing the excellent network utilization. Additionally, traffic smoothing is now a popular technique used to flatten the bursty video traffic, so reducing the potential congestion in the networks. We consider the traffic specification of video traffic in two cases: with traffic smoothing and without traffic smoothing. Initial experiments show that traffic smoothing does not result in significant changes in values of traffic specification. This highlights the importance of global burstiness of video traffic to the traffic parameters. The results of this research can be directly applied to some existing applications such as the integrated services of ATM and IETF.

In this paper, we present a novel scheme to deliver the scalable video over priority network. Firstly, we describe the background on the scalable video transmission over the network and the motivation of this research. Secondly, a new scheme is proposed, covering bitstream classification, prioritization and packetization. Thirdly, we present a simple and effective mechanism of rate control and adaptation by selectively dropping packet aiming to minimize the end-to-end distortion. We also describe a framework of transmission. Simulations show that our scheme is also effective to video multicast scenario.

There have been increasing market demands to develop more effective schemes for applying MPEG video coding standard to meet the challenges from the emerging applications. We present a real-time multi-pass MPEG video coding sys tem for constant bit rate (CBR) applications, such as Video-on-Demand (VoD) over ADSL. This scheme employs two MPEG encoders. Both encoders operate at the same CBR. The second encoder has a buffer to delay the input by an amount of time relative to the first encoder to create a look-ahead window. In encoding, the first encoder collects statistical and rate-quality characteristic information. An on-line processor then uses the collected information to derive the best coding strategy for the second encoder to encode the incoming frames in the look-ahead window. The second encoder uses the encoding parameters from the processor as the coding guide to execute the coding strategy and produce the compressed bitstream. Results on an IBM MPEG-2 encoder testing system with a 15 frame look-ahead window have shown that the scheme can obtain 0.6 to 1.6dB improvement in PSNR over the conventional single encoder on MPEG test video sources. Moreover, measured by the Tektronix Inc.'s Picture Quality Analysis (PQA) System, this scheme achieved 0.87 ~ 1.35 reduction in PQA index. The visual quality improvment is also very evident.

Video applications over the Internet are getting increasingly popular because of the explosive growth of the Internet. However, video packets loss due to network congestions can degrade the video quality substantially. In this paper, we propose a transmission scheme for Motion-JPEG2000 video sequences with an active networking approach. Our scheme utilizes the progression modes in Motion-JPEG2000. It can be implemented in an active network environment efficiently. Our simulation shows that the proposed scheme gracefully adapts to network congestion and improves the quality of video transmission in congested IP networks.

This article introduces a novel approach for scalable video coding based on an analysis-synthesis scheme. Active meshes are used to represent motion model, this permits to exploit temporal redundancy along motion trajectories in a video sequence using temporal wavelet transform. The use of 3D wavelets in the coding strategy provides natural scalability functionalities to the video coder. Furthermore, the analysis-synthesis scheme allows to decouple motion and texture and to code these informations separately. Motion can then be lossy coded, bitrates gain can be reported to texture coding. Because motion is lossy coded, a new quality criterion measured in the texture domain is then proposed. Finally, the proposed analysis-synthesis video scheme overcomes some of the limitations of existing video coding schemes using 3D wavelets, limitations due for the most part to the use of block-based motion model. Our video coding scheme performs as well as fully optimized H26Lv8, while providing a scalable bitstream.

Detecting and tracking objects in spatio-temporal datasets is an active research area with applications in many domains. A common approach is to segment the 2D frames in order to separate the objects of interest from the background, then estimate the motion of the objects and track them over time. Most existing algorithms assume that the objects to be tracked are rigid. In many scientific simulations, however, the objects of interest evolve over time and thus pose additional challenges for the segmentation and tracking tasks. We investigate efficient segmentation methods in the context of scientific simulation data. Instead of segmenting each frame separately, we propose an incremental approach which incorporates the segmentation result from the previous time frame when segmenting the data at the current time frame. We start with the simple K-means method, then we study more complicated segmentation techniques based on Markov random fields. We compare the incremental methods to the corresponding sequential ones both in terms of the quality of the results, as well as computational complexity.

Spatial segmentation of image sequences is usually performed based on motion between two frames, and then followed by tracking. Some recent approaches extend this to joint segmentation in space-time; the resulting 3-D segmentation (in x-y-t space) can be interpreted as a volume 'carved out by a moving object in the image sequence domain. We call such volumes 'object tunnels'. In this paper, we propose a new approach to occlusion analysis and characterization that is based on object tunnels. It results from the observation that object-tunnel wall for a fully visible object has different shape than that for an object undergoing occlusion or exposure. Walls of tunnels associated with moving objects have tangent planes that are, in general, non-parallel to the time axis. When an object gets occluded or exposed by a static feature, part of the object tunnel wall stops evolving freely; its spatial coordinates remain fixed (static occlusion boundary) while the temporal coordinate increases linearly (time evolution). This forces part of the wall to be comprised of lines parallel to the time axis, each line defined by a single point on the occlusion boundary. In case this boundary is a straight-line edge, the occluding part of the wall becomes planar. We propose to detect occlusions by searching for such characteristic surfaces of object tunnel walls. We formulate the problem for planar occlusion walls based on a robust distance metric, and we show experimental results for various occlusion types on synthetic and camera-acquired image sequences.

Shadow segmentation is a critical issue for systems aiming at extracting, tracking or recognizing objects in a given scene. Shadows can in fact modify the shape and colour of objects and therefore affect scene analysis and interpretation systems in many applications, such as video database search and retrieval, as well as video analysis in applications such as video surveillance. We present a shadow segmentation algorithm which includes two stages. The first stage extracts moving cast shadows in each frame of the sequence. The second stage tracks the extracted shadows in the subsequent frames. Tentative moving shadow regions are first identified based on spectral and geometrical properties of shadows. In order to confirm this tentative identification, shadow regions are then tracked over time. This second stage aims at exploiting the prior knowledge of a shadow detected in previous frames by evaluating its temporal behaviour. Shadow tracking is a difficult task, since colour, texture, and motion features in shadow regions cannot be used for solving the correspondence problem. Colour and texture change according to changes in the background's characteristics. The measurement of motion cannot be reliably computed for shadows. Therefore shadows may be described only by a limited amount of information. The proposed tracking algorithm makes use of this information and provides a reliability estimation of shadow recognition results of the first stage over time. This temporal analysis eliminates the possible ambiguities of the first stage and improves the efficiency of the overall shadow detection algorithm. The benefit of the proposed shadow segmentation and tracking algorithm is evaluated on both indoor and outdoor scenes. The obtained results are validated based on subjective as well as objective comparisons.

The radio plays a song that you like but that you do not recognize. How do you find the title and the artist? Previous approaches to finding a song in a database are based on pattern recognition. In some of the previous work features are extracted from a hummed song and decision rules are used to retrieve probable candidates from the database. Feature matching has not resulted in reliable searches from microphone samples. In this work, to find the song, we process a short, microphone recorded sample from it. Both a feature vector and a signal are precomputed for each song in a database and also extracted from the recording. The database songs are first sorted by feature distance to the recording. Then, normalized cross-correlation, even though nonlinear, is applied using overlap-save FFT convolution. A decision rule presents likely matches to the user for confirmation but controls the number of false alarms shown. This system, tested using hundreds of recordings, is reliable because signals are matched. The addition of the feature-ordered search and the decision rule result in database searches five times faster than signal matching alone.

The paper presents research on a robust technique for texture-based image retrieval in multimedia museum collections. The aim is to be able to use a query image patch containing a single texture to retrieve images containing some area with similar texture to that in the query. A retrieval technique without the need for segmentation is presented. The algorithm uses a multiscale sub-image matching method together with an appropriate texture feature extractor. The multiscale sub-image matching is achieved by first decomposing each database image into a set of 64×64 pixel patches covering the entire image. The resolution of the database image is then rescaled to create sub-images corresponding to a larger scale. The process continues until the final resolution of the image is equal to some pre-determined value. Finally, a collection of sub-images corresponding to different image regions and scales is obtained. The final image feature vector consists of a collection of feature vectors corresponding to each sub-image. Several wavelet-based feature extractors are tested with the multiscale technique. From the experiments, it is found that the multiscale sub-image matching method is an efficient way to achieve effective texture retrieval without any need for segmentation.

This paper addresses the problem of image content characterization in the compressed domain for the purpose of facilitating similarity matching in a multimedia database. Specifically, given the disparity of the content characterization power of compressed domain approaches and those based on pixel-domain features, with the latter being usually considered as the more superior one, our objective is to transform the selected set of compressed domain feature histograms in such a way that the retrieval result based on these features is compatible with their spatial domain counterparts. Since there are a large number of possible transformations, we adopt a genetic algorithm approach to search for the optimal one, where each of the binary strings in the population represents a candidate transformation. The fitness of each transformation is defined as a function of the discrepancies between the spatial-domain and compressed-domain retrieval results. In this way, the GA mechanism ensures that transformations which best approximate the performance of spatial domain retrieval will survive into the next generation and are allowed through the operations of crossover and mutation to generate variations of themselves to further improve their performances.

In this paper, we consider the problem of rate control for video transmission. We focus on finding off-line optimal rate control for constant bit-rate (CBR) transmission, where the size of the encoder buffer and the channel rate are the constraints. To ensure a maximum minimum quality is obtained over all data units (e.g., macro blocks, video frames or group-of-pictures), we use a minimum maximum distortion (MMAX) criterion for this buffer-constrained problem. We show that, due to the buffer constraints, a MMAX solution leads to a relatively low average distortion, because the total rate budget is not completely utilized. Therefore, after finding a MMAX solution, an additional minimization of average distortion criterion is proposed to increase overall quality of the data sequence by using remaining resources. The proposed algorithm (denoted MMAX+ as it incorporates both MMAX and the additional average quality optimization stage) leads to an increase in average quality with respect to the MMAX solution, while providing a much more constant quality than MMSE solutions. Moreover we show how the MMAX+ approach can be implemented with low complexity.

In this work, we introduce the concept of low-pass filtering of rate-distortion (R-D) functions and develop a one-pass encoding with smoothed rate control (SRC) which is able to maximize the video presentation quality, especially the temporal perceptual quality smoothness, in real-time video recording applications. In addition, theoretically, we prove that SRC is able to achieve the target bit budget determined by the storage space or network bandwidth. Without look ahead and pre-processing, the one-pass SRC algorithm has a similar performance as the offline two-pass encoding with optimum bit allocation. Our extensive simulation results demonstrate that SRC significantly reduces picture quality fluctuation and motion jerkiness, and improves the perceptual video presentation quality.

Wireless access to encoded video requires sophisticated error resilient mechanisms. Although some video coding standards describe error resilience algorithms, they are inefficient at deep fading wireless and mobile channels. In this paper, we present a novel scheme of error protection for narrowband wireless channels which is independent of any video coding standard. Being, however, this scheme common to all standards, it provides some acceptable video quality degradation. We have evaluated the quality degradation of the Common solution in comparison to the optimal scheme of protection corresponding to the existence of a finite number of channel codes. Besides, we also present the quality degradation for both video coding standards, H.263 and MPEG-4, obtained from a finite number of codes in comparison with the ideal scheme corresponding to the existence of an infinite number of channel codes. Additionally, we proposed a protection solution tailored to H.263/MPEG-4 source coding with an average PSNR improvement of about 0.2 dB relatively to the above mentioned Common solution. Thus, in the framework of joint source-channel coding, we present an adaptive scheme of protection according to the channel bit error rate (BER) and with an overall transmission rate of 64 kbps. In our investigation, we used four QCIF video test sequences and rate compatible punctured convolutional (RCPC) codes. For instance, a PSNR gain of about 16.7 dB is obtained at BER=10^-2 for 'Foreman' video sequence, encoded either by H.263 or by MPEG-4 and protected using the Common scheme in comparison to the unprotected case.

We address the bit allocation problem in scenarios where there exist two-dimensional (2D) dependencies in the bit allocation, i.e., where the allocation involves a 2D set of coding units (e.g., DCT blocks in standard MPEG coding) and where the rate-distortion (RD) characteristics of each coding unit depend on one or more of the other coding units. These coding units can be located anywhere in 2D space. As an example we consider MPEG-4 intra-coding where, in order to further reduce the redundancy between coefficients, both the DC and certain of the AC coefficients of each block are predicted from the corresponding coefficients in either the previous block in the same line (to the left) or the one above the current block. To find the optimal solution may be a time-consuming problem, given that the RD characteristics of each block depend on those of the neighbors. Greedy search approaches are popular due to their low complexity and low memory consumption, but they may be far from optimal due to the dependencies in the coding. In this work, we propose an iterative message-passing technique to solve 2D dependent bit allocation problems. This technique is based on (i) Soft-in/Soft-out (SISO) algorithms first used in the context of Turbo codes, (ii) a grid model, and (iii) Lagrangian optimization techniques. In order to solve this problem our approach is to iteratively compute the soft information of a current DCT block (intrinsic information) and pass the soft decision (extrinsic information) to other nearby DCT block(s). Since the computational complexity is also dominated by the data generation phase, i.e., in the Rate-Distortion (RD) data population process, we introduce an approximation method to eliminate the need to generate the entire set of RD points. Experimental studies reveal that the system that uses the proposed message-passing algorithm is able to outperform the greedy search approach by 0.57 dB on average. We also show that the proposed algorithm requires slightly more memory and higher complexity than the greedy search approach.

We present a new scalable video codec which features a computationally efficient decoder as well as an operational rate-distortion optimal encoder. The SNR scalability is accomplished by splitting the Discrete Cosine Transform (DCT) coefficients of the Displaced Frame Difference (DFD) into groups that correspond to the scalable layers. Our Operational Rate-Distortion optimal scalable codec partitions the DCT coefficients of the DFD (or the intensity for intra blocks) into a base layer and one or more enhancement layers. The base layer is constructed by subtracting a value from each quantized DCT coefficient. The subtracted values are then sent as enhancement. If more than two scalable layers are required, the values subtracted for the creation of the base layer are further broken into other values. The partitioning of the DCT coefficients into layers is accomplished by formulating a constrained optimization problem which is then solved using lagrangian optimization. A Dynamic Programming (DP) solution is proposed in order to minimize the Lagrangian cost in a computationally efficient manner. We introduce a new method for finding the required Lagrangian multiplier λ which will meet our target bitrate. The method further reduces the computational complexity of the encoder by minimizing the need for an iterative calculation of the Lagrangian multiplier. Our experimental results show that the proposed codec typically outperforms H.263+ SNR scalability in terms of PSNR while exhibiting a low-complexity decoder. Due to the use of Dynamic Programming and a new method for the estimation of λ, the computational complexity of the operational rate-distortion optimal encoder is competitive with H.263+.

This paper proposes a modified Fixed-length Entropy Coding(FLC) algorithm suitable for MPEG-like hybrid-based video compression. In such applications, the alphabets of symbols sending to entropy coding are of heavy-tail distributions. Previously proposed algorithms either have poor compression efficiency or are computationally complex in this situation. The technique proposed in this paper extends the previous work for the important case of large alphabets by introducing a new alphabet segmentation and splitting algorithm. Simulation results show that for sources with relative large, skewed and heavy-tail distributed alphabets, the proposed approach has faster alphabet shrinking and higher compression efficiency compared with previous FLC approaches. A hybrid video codec using the proposed FLC is implemented and compared to a MPEG-2 video codec in both noise-free and noisy environments. The results demonstrate that the proposed FLC has similar compression gain as common VLC, and FLC codes provide more robustness to video streams than VLC codes. The proposed technique can be used to compress video sequences transmitted over channels with random bit errors.

This paper addresses the problem of selecting a single transcoding method from multiple transcoding possibilities that satisfy a specified delivery constraint. We first discus the rate-distortion modeling of DCT coefficients. Then, we quantify the distortion of the video transcoding output by examining the rate-distortion relationship of the popular transcoding techniques, including requantization, spatial downsampling and temporal downsampling. The use of our model is illustrated in some typical applications.

For the majority of object tracking scenario, the emphasis has been put on achieving robust motion estimation under different situations. In this paper, besides introducing a multiple frame based 3D motion estimation to handle the partial occlusion problem, we present an object shape delineation scheme by dual region growing. As it is well-known, the problem of shape extraction of a moving object undergoing self-occlusion is highly ill-posed. Approaches have been proposed by assuming the similarity of object pixels in the vicinities of the boundaries between the current frame and the previous one. Such an assumption is usually broken down when occlusion occurs; instead, our implementation is based on a stronger assumption. The system consists of (i) a multi-frame motion estimation, (ii) dual-region growing, and (iii) boundary points arbitration. This method is shown to recover favorable object shape during the tracking process.

In the context of content-oriented applications such as video surveillance and video retrieval this paper proposes a stable object tracking method based on both object segmentation and motion estimation. The method focuses on the issues of speed of execution and reliability in the presence of noise, coding artifacts, shadows, occlusion, and object split. Objects are tracked based on the similarity of their features in successive images. This is done in three steps: object segmentation and motion estimation, object matching, and feature monitoring and correction. In the first step, objects are segmented and their spatial and temporal features are computed. In the second step, using a non-linear voting strategy, each object of the previous image is matched with an object of the current image creating a unique correspondence. In the third step, object segmentation errors, such as when objects occlude or split, are detected and corrected. These new data are then used to update the results of previous steps, i.e., object segmentation and motion estimation. The contributions in this paper are the multi-voting strategy and the monitoring and correction of segmentation errors. Extensive experiments on indoor and outdoor video shots containing over 6000 images, including images with multi-object occlusion, noise, and coding artifacts have demonstrated the reliability and real-time response of the proposed method.

A highly challenging object detection task is the recognition of relevant events in outdoor applications, such as it is the case in sport broadcasts. Changing illumination, different weather conditions, and noise in the imaging process are the most important issues that require a truly robust detection system. The original contribution of this work is to take advantage of a dynamic integration of object beliefs from different evidences of spatial and temporal context to receive a recursively updated object hypothesis, with the aim to render object detection more robust. The object representation is outlined in a probabilistic framework to enable reasoning on multiple instances of detection results and decision making based on statistical evaluations. The representation is based on the local appearances of the objects, and therefore makes the interpretation more robust to occlusion by enabling reasoning based on spatial context between the appearances of individual object parts. Reasoning is driven by the evaluation of Bayesian decision fusion of the single probabilistic local image interpretations. The detection system is evaluated on the detection of company logos in extensive video material from Formula One broadcasts. The experimental results demonstrate that fusion is crucial to improve robustness and accuracy of the outdoor detection system.

In this paper, we present a new method for tracking objects with shadows. Traditional motion-based tracking schemes cannot usually distinguish the shadow from the object itself, and this results in a falsely captured object shape. If we want to utilize the object's shape information for a pattern recognition task, this poses a severe difficulty. In this paper we present a color processing scheme to project the image into an illumination invariant space such that the shadow's effect is greatly attenuated. The optical flow in this projected image together with the original image is used as a reference for object tracking so that we can extract the real object shape in the tracking process. We present a modified snake model for general video object tracking. A new external force is introduced into the snake equation based on the predictive contour such that the active contour is attracted to a shape similar to the one in the previous video frame. The proposed method can deal with the problem of an object's ceasing movement temporarily, and can also avoid the problem of the snake tracking into the object interior. Global affine motion estimation is applied to eliminate the effect of camera motion and hence the method can be applied in a general video environment. Experimental results show that the proposed method can track the real object even if there is strong shadow influence.

The steadily increasing need for video content accessibility necessitates the development of stable systems to represent video sequences based on their high-level (semantic) content. The core of such systems is the automatic extraction of video content. In this paper, a computational layered framework to effectively extract multiple high-level features of a video shot is presented. The objective with this framework is to extract rich high-level video descriptions of real world scenes. In our framework, high-level descriptions are related to moving objects which are represented by their spatio-temporal low-level features. High-level features are represented by generic high-level object features such as events. To achieve higher applicability, descriptions are extracted independently of the video context. Our framework is based on four interacting video processing layers: enhancement to estimate and reduce noise, stabilization to compensate for global changes, analysis to extract meaningful objects, and interpretation to extract context-independent semantic features. The effectiveness and real-time response of the our framework are demonstrated by extensive experimentation on indoor and outdoor video shots in the presence of multi-object occlusion, noise, and artifacts.

In this paper, we propose a new system for video object detection based on user-defined models. Object models are described by 'model graphs' in which nodes represent image regions and edges denote spatial proximity. Each node is attributed with color and shape information about the corresponding image region. Model graphs are specified manually based on a sample image of the object. Object recognition starts with automatic color segmentation of the input image. For each region, the same features are extracted as specified in the model graph. Recognition is based on finding a subgraph in the image graph that matches the model graph. Evidently, it is not possible to find an isomorph subgraph, since node and edge attributes will not match exactly. Furthermore, the automatic segmentation step leads to an oversegmented image. For this reason, we employ inexact graph matching, where several nodes of the image graph may be mapped onto a single node in the model graph. We have applied our object recognition algorithm to cartoon sequences. This class of sequences is difficult to handle with current automatic segmentation algorithms because the motion estimation has difficulties arising from large homogeneous regions and because the object appearance is typically highly variable. Experiments show that our algorithm can robustly detect the specified objects and also accurately find the object boundary.

Camera sensor network as a new advent of technology is a network that each sensor node can capture video signals, process and communicate them with other nodes. The processing task in this network is to generate arbitrary view, which can be requested from central node or user. To avoid unnecessary communication between nodes in camera sensor network and speed up the processing time, we have distributed the processing tasks between nodes. In this method, each sensor node processes part of interpolation algorithm to generate the interpolated image with local communication between nodes. The processing task in camera sensor network is ray-space interpolation, which is an object independent method and based on MSE minimization by using adaptive filtering. Two methods were proposed for distributing processing tasks, which are Fully Image Shared Decentralized Processing (FIS-DP), and Partially Image Shared Decentralized Processing (PIS-DP), to share image data locally. Comparison of the proposed methods with Centralized Processing (CP) method shows that PIS-DP has the highest processing speed after FIS-DP, and CP has the lowest processing speed. Communication rate of CP and PIS-DP is almost same and better than FIS-DP. So, PIS-DP is recommended because of its better performance than CP and FIS-DP.

Sky is among the semantic object classes frequently seen in photographs and useful for image understanding, processing, and retrieval. We propose a novel hybrid approach to sky detection; based on color and texture classification, region extraction, and physics motivated sky signature validation. Sky can be of many different types; clear blue sky, cloudy/overcast sky, mixed sky, and twilight sky, etc. A single model cannot correctly characterize all the various types of skies due to the large difference in physics and appearance associated with different sky types. We have developed a set of physics-motivated sky models to identify clear blue-sky regions and cloudy/overcast sky regions. An exemplar-based approach is to generate the initial set of candidate sky regions. Another data-derived model is subsequently used to combine the results for different sky types to form a more complete sky map. Extensive testing using more than 3000 (randomly oriented) natural images shows that our comprehensive sky detector is able to accurately recall approximately 96% of all sky regions in the image set, with a precision of about 92%. Assuming correct image orientation, the precision on the same set of images increases to about 96%.

Context-based arithmetic coding has been widely adopted in image and video compression and is a key component of the new JPEG2000 image compression standard. In this paper, the contexts used in JPEG2000 are analyzed using the mutual information, which has a direct link with the compression performance. We first show that, when combining the contexts, the mutual information between the contexts and the encoded data will decrease unless the conditional probability distributions of the combined contexts are the same. Given I, the initial number of contexts, and F, the final desired number of contexts, there are S(I, F) possible context classification schemes where S(I, F) is called the Stirling number of the second kind. The optimal classification scheme is the one that gives the maximum mutual information. Instead of exhaustive search, the optimal classification scheme can be obtained through a modified Generalized Lloyd algorithm with the relative entropy as the distortion metric. For binary arithmetic coding, the search complexity can be reduced by using the dynamic programming. Our experimental results show that the JPEG2000 contexts capture very well the correlations among the wavelet coefficients. At the same time, the number of contexts used as part of the standard can be reduced without loss in the coding performance.

In practice, interlaced video sequences are typically coded with either a frame-only or field-only structure, irrespective of the content. However, coding in this way will not provide the best coding efficiency. This paper proposes an adaptive picture-level field/frame coding scheme with corresponding rate control. First, a two-pass field/frame decision scheme is proposed. In this scheme, we formulate the field/frame decision as a constrained optimization problem. The actual rate and distortion data are collected and the optimal picture-level coding decision is determined based on this data. An effective rate control for the proposed two-pass algorithm is also presented. However, since the complexity of the two-pass scheme is relatively large since motion estimation must be performed for both the frame-based picture and the field-based picture, we also propose a one-pass field/frame decision scheme. This one-pass scheme calculates the variance of each macroblock in a field and estimates the correlation between two fields. Based on the correlation, a decision to code the picture as a frame or as fields is made. A rate control method for the proposed one-pass scheme is also presented. Simulation results demonstrate that our scheme outperforms frame-only and field-only coding for several sequences coded at a wide range of bit-rates, and the proposed one-pass scheme obtains similar performance as the proposed two-pass scheme.

The advent of High Definition television has increased the demand for displays capable of displaying the higher resolution pictures. However, the availability of high definition video broadcast is still very limited. Therefore, one might expect that only in these scarce cases one can fully appreciate the fine details associated with a High Definition picture. However, this is not completely correct. The high display resolution can be exploited by up-converting the Standard Definition video towards the higher resolution, creating a close to High Definition experience. In this paper we will present a methodology to convert standard definition video into video with up to a four times higher picture resolution.

In this paper, we propose a new deinterlacing algorithm utilizing directional interpolation and motion compensation. In particular, intrafield interpolation is first performed in the spatial direction that shows the highest correlation. Then, motion estimation is performed between two fields of the same parity. In order to interpolate missing lines prior to motion estimation between opposite parity fields, the proposed method does not require a prefilter such as line averaging, since block matching is performed between the same parity fields. Finally, we apply a test and use either directional interpolation or motion compensated interpolation depending on the test result. Experimental results are promising.

The introduction of HDTV asks for spatial up-conversion techniques enabling the display of standard resolution material. Recently, X. Li and M. Orchard proposed the 'New Edge-Directed Interpolation' (NEDI) algorithm for high quality up-scaling of natural images. We shall show that the method, although it generally behaves well, introduces annoying artifacts in fine-textured areas. Based on an analysis of these artifacts and taking advantage of the temporal correlation between video images, we propose an improved NEDI algorithm. In our evaluation, we compare the performance of the original and the improved NEDI method on a significant set of test images. We conclude from both subjective and objective measures that the proposed modifications improve the overall performance of NEDI.

This paper presents two novel methods to encode HD (High Definition) video at low bitrates (~5 Mbps) using the MPEG-2 Main Profile@Main Level standard (be compatible with current digital video devices; e.g., DVD players, digital video recorders, etc.) with the embedded HD-relevant information (E-data) in the bitstream. Due to the low bitrate constraint, traditional coding based approaches (e.g., MPEG-2 layered coding or scalable coding) cannot satisfy this requirement. Therefore, we developed our system from the video-enhancement point of view. At first, the HD video is down converted to SD (Standard Definition). During the down-conversion, extra data (E-data) is saved. This E-data is used to re-create HD effects when the encoded SD is upconverted prior to the display. For HD re-creation, we developed a novel multilevel resolution-enhancement method that makes an upconverted image emulate the quality of the original HD picture. Further, we designed a visual-based fine detail injection method to add more details into the picture to achieve near HD quality. Based on several test video sequences, we conclude that our approaches have the potential to create HD visual effect on the upconverted SD video.

Superresolution techniques attempt to reconstruct a higher resolution image (or video) from a low resolution input video sequence. The motivation here is that camera and scene motion lead temporal frames in the source video sequence contain similar but not identical information. In this paper, we present a novel warping synthesis combined with an error correction scheme to reconstruct an SR video from an input video sequence as an alternative to the conventional `sliding window' approach. Simulation results demonstrate that the proposed scheme reduces dramatically the computational load with little or no loss of reconstruction quality.

Since the standardization of JPEG-LS, several improvements and variations to the original algorithms have been proposed. In this paper we propose a Region of Interest (ROI) based coding strategy for JPEG-LS that has the additional ability of providing effective rate controlled image compression. Given a single ROI or multiple ROIs of a fixed or arbitrary shape, the scheme we propose is able to compress a given image by a required ratio, while maintaining the subjective image of the ROIs at either lossless or at a quality specified by a Target Compression Ratio (TRC) of the ROI. We provide experimental results to compare the performance of the proposed rate-control algorithm with the state of the art near lossless rate control schemes. We show that the proposed scheme is able to achieve much higher TCRs, at increased accuracy and better objective image quality, using a less computationally intensive rate control algorithm. Finally we demonstrate that the proposed ROI based coding scheme can be used to extend the applicability of JPEG-LS to medical and satellite imaging applications and provides a useful alternative to JPEG-2000 based ROI coding.

Mobile devices are more often capable for locating the user on the globe by using GPS or network-based systems. The location is given to the user in a meaningful context such as map or aerial image. We study compression methods for reducing the amount of data required by aerial images. We consider the two well known lossless graphics file formats GIF and PNG, the compression standards JPEG and JPEG2000, a fractal compressor known as FIASCO, and commercial wavelet-based method developed for aerial images. It is also expected that the devices support fewer than 256 gray levels. It is therefore possible to get further reduction by quantizing the images prior to compression, for example, to two-bit per pixel, and then apply lossless comrpession methods such as JBIG, GIF and PNG. For four color mobile displays error diffusion dithered images approximate the original 8-bit color images quite well. The trade-off in dithering is that the lossless compression ratios decrease. Solution to this might be to store the 8-bit images compressed by using a decent lossy compressor such as JPEG2000. Quantization and dithering should happen only at the moment when the image is displayed.

The removal of unwanted, parasitic vibrations in a video sequence induced by camera motion is an essential part of video acquisition in industrial, military and consumer applications. In this paper, we present a new image processing method to remove such vibrations and reconstruct a video sequence void of sudden camera movements. Our approach to separating unwanted vibrations from intentional camera motion is based on a probabilistic estimation framework. We treat estimated parameters of interframe camera motion as noisy observations of the intentional camera motion parameters. We construct a physics-based state-space model of these interframe motion parameters and use recursive Kalman filtering to perform stabilized camera position estimation. A six-parameter affine model is used to describe the interframe transformation, allowing quite accurate description of typical scene changes due to camera motion. The model parameters are estimated using a p-norm-based multi-resolution approach. This approach is robust to model mismatch and to object motion within the scene (which are treated as outliers). We use mosaicking in order to reconstruct undefined areas that result from motion compensation applied to each video frame. Registration between distant frames is performed efficiently by cascading interframe affine transformation parameters. We compare our method's performance with that of a commercial product on real-life video sequences, and show a significant improvement in stabilization quality for our method.

In this paper a novel stochastic image model in the transform domain is presented and its superior performance in image denoising applications is demonstrated. The proposed model exploits local subband image statistics and is based on geometrical priors. Contrarily to complex models based on local correlations, or to mixture models, the proposed model performs a partition of the image into non-overlapping regions with distinctive statistics. A close form analytical solution of the image denoising problem for AWGN is derived and its performance bounds are analyzed. Despite being very simple, the proposed stochastic image model provides a number of advantages in comparison to the existing approaches: (a) simplicity of stochastic image modeling; (b) completeness of the model, taking into account multiresolution, non-stationary image behavior, geometrical priors and providing an excellent fit to the global image statistics; (c) very low complexity of the algorithm; (d) tractabiity of the model and of the obtained results due to the closed-form solution and to the existence of analytical performance bounds; (e) extensibility to different transform domains, such as orthogonal, biorthogonal and overcomplete data representations. The results of benchmarking with the state-of-the-art image denoising methods demonstrate the superior performance of the proposed approach.

Temporal and spatial random variation of luminance in images, or 'flicker' is a typical degradation observed in archived film and video. The underlying premise in typical flicker reduction algorithms is that each image must be corrected for a spatially varying gain and offset. These parameters are estimated in the stationary region of the image. Hence the performance of that algorithm depends crucially on the identification of stationary image regions. Position fluctuations are also a common artefact resulting in a random 'shake' of each film frame. For removing both, the key is to reject regions showing local motion or other outlier activity. Parameters are then estimated mostly on that part of the image undergoing the dominant motion. A new algorithm that simultaneously deals with global motion estimation and flicker is presented. The final process is based on a robust application of weighted least-squares, in which the weights also classify portions of the image as local or global. The paper presents results on severely degraded sequences showing evidence of both Flicker and random shake.

By looking up pixels of image sequence as moving gas molecules, a novel concept, image temperature, has been proposed to describe the natural property of the image motion. The idea comes from the reveal of Maxwell-Boltaman Distribution Law in gas dynamic theories. Further, we develop another concept of energy flow corresponding to optical flow. Based on the concepts and the method of optical flow analysis, we developed the method of the Energy Flow Equation (EFE) to estimate image motion. Experiment indicates better performance of the proposed EFE scheme with significantly reduced false motion estimates than the traditional Optical Flow Equation does.

The current paper proposes a fast stereo matching algorithm based on a pixel-wise matching strategy that can produce a stable and accurate disparity map. Since the differences between a pair of stereo images are only small and such differences are just caused by horizontal shifts in a certain order, matching using a large window is not required within a given search range. Therefore, the current study adopts a disparity space image(DSI) for efficient pixel-wise matching. In a DSI, each disparity value is derived from the matching path. As such, when the matching path in a DSI innately satisfies the ordering constraint, continuous matching is possible. In addition, since this continuous matching also considers neighbor pixels, it assumes the characteristics of region-based matching, similar to window-based matching, yet with a lower computational load. To avoid the unstable characteristic of the pixel-based matching, we modified the directions in the cost array in the dynamic programming(DP). Experimental results demonstrate that the proposed method can remove almost all disparity noise and produce a good quality disparity map within a very short time.

Computing dense disparity fields from large-baseline stereo is a difficult problem because of long-range correspondences involved. A typical solution to this problem is to use optical flow or block matching methods implemented over a hierarchy of resolutions. However, these approaches cannot easily cope with disparity discontinuities. Recently, we have proposed a novel approach that combines feature matching and Delaunay triangulation. In this approach, first feature points are extracted using intensity corner detector, and then corresponding feature-point pairs are found using cross-correlation. These two steps result in a reliable but sparse map of disparity vectors. In order to compute a dense disparity field, the third step involves Delaunay triangulation followed by disparity interpolation based on an affine (planar) model. The resulting disparity fields are continuous everywhere, and thus are not realistic; typical stereo image pairs exhibit disparity discontinuities at object boundaries. To address this problem, in the past we subdivided some Delaunay triangles into smaller ones. Although this approach has significantly improved the rendition of disparity discontinuities, it did not always work reliably. In this paper, we propose an adaptive interpolation over Delaunay triangles. As before, the interpolation is distance-dependent, i.e., accounts for Euclidian distance between the position of disparity under interpolation and three vertices of a triangle. The distance-dependent weights, however, are now additionally adapted so that the interpolated, pixel-based disparities within each triangle afford discontinuities. The new method has been applied to natural stereoscopic images. The resulting dense disparity fields exhibit clear, although subtle, discontinuities at object boundaries, and are more realistic than disparity fields obtained by the prior approach.

This paper develops a bit-plane coding technique to achieve the subtitle enhancement in MPEG-4 for very low bit rate streaming video. The original video frame that contains subtitle information is encoded by the use of MPEG-4 codec and then the MPEG-4 decoded Y frame is subtracted from the original one to generate a residue. Because the subtitle information usually appears on the special region of a video frame, one can pre-set the position of the subtitle region and just process this residue subtitle region for subtitle enhancement. The pixel value of the residue subtitle region will be transferred from decimal number to binary number. Thus one can obtain 8 bit-planes of the residue subtitle region. For each bit-plane, (RUN, EOP) symbols are formed and then encoded via the variable length coding to produce the output bitstream. It has been shown that such a bit-plane coding technique is very efficient and its decoding procedure can be easily performed. From various experimental results, using MSB, MSB-1 and MSB-2 planes can obtain a satisfied subtitle visual enhancement with only increasing 5% bit rate.

Along with growing up of wireless network and mobile devices, more and more users expect to access multimedia content not only by PC terminal, but also through those pervasive devices. In this paper, we proposed a content-based video-streaming framework for universal multimedia access (UMA). The framework that mainly combines the video analysis with the video streaming technology can make the pervasive device access the multimedia information at anywhere, anytime and anyhow. Our contribution of this paper is to streaming video data according to the video analysis result, the video data during transmission is no more “meaningless information” bits. We have implemented a UMA server, which customizes the video stream according to the video analysis result and the capabilities of client, and a UMA client that receives the adapted video stream.

This paper proposes a new stereo sequence transmission technique using digital watermarking for compatibility with conventional 2D digital TV. We, generally, compress and transmit image sequence using temporal-spatial redundancy between stereo images. It is difficult for users with conventional digital TV to watch the transmitted 3D image sequence because many 3D image compression methods are different. To solve such a problem, in this paper, we perceive the concealment of new information of digital watermarking and conceal information of the other stereo image into three channels of the reference image. The main target of the technique presented is to let the people who have conventional DTV watch stereo movies at the same time. This goal is reached by considering the response of human eyes to color information and by using digital watermarking. To hide right images into left images effectively, bit-change in 3 color channels and disparity estimation according to the value of estimated disparity are performed. The proposed method assigns the displacement information of right image to each channel of YCbCr on DCT domain. Each LSB bit on YCbCr channels is changed according to the bits of disparity information. The performance of the presented methods is confirmed by several computer experiments.

An unequal loss protection framework for transmission of JPEG2000 codestreams over packet erasure channels is presented. A joint source-channel coding approach is adopted to form the JPEG2000 codestream and assign the appropriate amount of protection to different sections of the codestream. Experimental results indicate that the proposed scheme yields excellent performance across a wide range of packet loss rates.

Most video coding standards, including MPEG-4, use variable length codes to encode the discrete cosine transform (DCT) coefficients of intra frames and inter frames, and the motion vector information. Although variable length codes can achieve good compression, they are very sensitive to channel errors, especially in wireless channels, which are characterized by bursty errors. Joint source-channel decoding (JSCD) is emerging as a potential alternative to traditional error resilience methods for dealing with this sensitivity to channel errors. This paper describes one such JSCD for the reliable transmission of MPEG-4 video over wireless channels. We apply the maximum a posteriori probability based JSCD developed for first order Markov sources to both inter and intra coded macroblocks. Experiments indicate that the proposed decoder gives significant improvements (9 dB in some cases) for MPEG-4 video with error resilience at various channel error rates.

Error concealment is becoming increasingly important because of the growing interest in multimedia transmission over unreliable channels such as wireless channel. At present most concealment method has its own advantage as well as applicable limitation. In different case, it can achieve different concealment effect. In our paper, we present a novel feature-based image error detection and error concealment algorithm to improve the image quality which was degraded during its transmission over wireless channel. First a simulation channel based on Rayleigh mode is implemented to emulating the actual wireless fading channel characterized by fading, multipath and Doppler frequency shift. The damaged image blocks are detected by exploring the contextual information in images, such as their consistency and edge continuity. The statistical characteristics of missing blocks are then estimated based on the types of their surrounding blocks (e.g., smoothness, texture and edge). Finally different error concealment strategies are applied to different types of blocks in order to achieve better visual quality. Instead of assuming random errors in packets, we simulate the errors of wireless channel based on the Rayleigh model. The proposed algorithm is tested on a number of still images. Simulation results demonstrate that our proposed algorithm is effective in terms of visual quality assessment and PSNR.

This paper concentrates on exploiting fast human face detection techniques for home video surveillance applications. The proposed method uses successive face detectors with incremental complexity and detection capability. The detectors are cascaded in such a way that each detector progressively restricts the possible face candidates into fewer areas. The proposed detectors, listed in the order of usage and complexity, are: (1) skin-color detector, (2) face structure detector which uses probability-based facial feature verification, and (3) three parallel learning-based detectors which take several representations of face candidates as inputs. The adopted representations are the pixel representation, the partial profile representation and the eigenface representation. The initial pruning of large areas of non-face regions significantly decreases the number of input windows for the learning-based face detector. This largely reduces the high computation cost for most learning-based detection approaches, while retaining the high detection accuracy and learning capabilities. Experimental results show that our proposal achieves an average of 0.3 - 0.4 second per frame processing speed with an image resolution of 320 by 240 pixels. An average of 92% detection rate is achieved for a test set composed of downloaded photos, standard test sequences and self-made sequences.

The current paper proposes an efficient method for edge detection in original and noisy images using Waerden's statistic. Edges represent a significant amount of information on an image. For example, edges reveal the location of objects, their shape and size, and something about their texture. Since edges represent where the intensity of an image moves from a low value to a high value or vice versa, edge detection is often the first step in image segmentation. As a field of image analysis, image segmentation groups pixels into regions to determine the image composition. Therefore, the current paper describes the nonparametric Wilcoxon test and parametric T test based on statistical hypothesis testing for edge detection. Here, the threshold is determined by specifying a significance level, whereas Bovik, Huang, and Munson considered a range of possible test statistic values for the threshold. In the current study, the test statistic is calculated based on pixel gray levels obtained using an edge-height parameter and compared with the threshold determined by a significance level. Experiments were conducted to evaluate the performance of these methods in both original and noisy images. As a result, the Wilcoxon and T test was found to be sensitive to a noisy image, whereas the proposed Waerden test was robust in both noisy and noise-free images under α=0.0005. Furthermore, when compared with Sobel, LoG, and Canny operators, the proposed Waerden test was also more effective in both noisy and noise-free images.

We propose a multi-agents platform for an interactive broadcasting system based on the MPEG-7 and TV-anytime standards. In the system, an intelligent agent technique is adopted from FIPA that provides software standards for interacting heterogeneous agents. Using the MDS (Multimedia Description Scheme) of MPEG-7, TV-anytime standards and interactive functions such as a user preference and an audio/video contents summary, are applied. In this paper, we propose the technique to use multi-modal features as well as multiple MPEG-7 features. We evaluate a video summary and filtering based on MPEG-7 on top of intelligent agent.

This work presents a new video feature extraction technique based on the Generalized Hough Transform (GHT). This technique provides a way to define a similarity measure between two different frames, which establishes the basis for scene cut detection algorithms. Moreover, GHT allows to calculate the differences between two frames in terms of rotation, scale and displacement. This provides a framework for the development of global motion estimation algorithms. In addition, gradual transition detection algorithms (fades, dissolves, etc.) can also be developed. To illustrate the posibilities of this technique, a scene cut detection algorithm is also proposed. This algorithm works with MPEG video in compressed domain, achieving real time processing. An improved thresholding technique is also stated. This technique uses two different sets of similarity values making the scene cut detection algorithm perform quite well with different types of videos. The thresholding process reports two different kinds of cuts: real cuts and probable cuts. Also, it detects the location of dynamic scenes, which can be used to perform further semantic analysis. Finally, the use of the improved thresholding technique and a set of optimized parameters results in an algorithm where no human intervention is needed. Several tests have been carried out using long videos, including more than 1400 cuts. Comparison with another well-known cut detection algorithm has also been performed.

The English city of Nottingham is widely known for its rich history and compelling folklore. A key attraction is the extensive system of caves to be found beneath Nottingham Castle. Regular guided tours are made of the Nottingham caves, during which castle staff tell stories and explain historical events to small groups of visitors while pointing out relevant cave locations and features. The work reported here is part of a project aimed at enhancing the experience of cave visitors, and providing flexible storytelling tools to their guides, by developing machine vision systems capable of identifying specific actions of guides and/or visitors and triggering audio and/or video presentations as a result. Attention is currently focused on triggering audio material by directing the beam of a standard domestic flashlight towards features of interest on the cave wall. Cameras attached to the walls or roof provide image sequences within which torch light and cave features are detected and their relative positions estimated. When a target feature is illuminated the corresponding audio response is generated. We describe the architecture of the system, its implementation within the caves and the results of initial evaluations carried out with castle guides and members of the public.

An algorithm is presented for indexing and retrieving images using an inverted index that stores colour features. The features are extracted using a modification of the Multi-modal Neighbourhood Signature (MNS). Images are divided into regular patches and modes of the colour distribution are derived using the meanshift algorithm. The colour values of patches with one, two or three dominant modes are recorded, and quantised into bins that form the colour feature terms. The terms and their frequencies are stored in an inverted index implemented in a relational database. Retrieval is performed using four different techniques, including a variation of the Term Frequency, Inverse Document Frequency (TF/IDF) algorithm used in text retrieval, that weight the query image features against those in the index. This new approach is compared to our previous work with indexed features and more traditional colour retrieval algorithms. The comparison is performed against a database of photographic images containing a wide variety of scenes. Two types of retrieval are tested - full image and sub-image queries. The performance of the algorithms are presented both in terms of computational speed and retrieval accuracy.

We design and implement a research friendly software platform, which aims at the flexibility and the abstraction of MPEG-7 application prototyping. We studied and analyzed the MPEG-7 standard, including a typical scenario of using MPEG-7. In order to fulfill to needs of researches, in addition to the normative parts of MPEG-7, additional requirements are included. By examining these requirements, we propose a research friendly software platform. The architecture consists of a framework, utility units, and the descriptors. Because this system is implemented using Java, it also incorporates the features of the Java environment, and thus it is flexible for developing new components and prototyping applications. We demonstrate the flexibility of this testbed by constructing an example program which allows users to manipulate image related descriptors.

Abrupt video edition effects (mainly called scene cuts) are transitions in which the image changes globally from one frame to the following one. The automatic detection of this set of effects is a research area widely analyzed, which has produced a number of working technique families. Two of these families are the methods based on geometric features of the images (basically, edges and contours) and the ones based on intensity or chromaticity of the images. Feature-based techniques are highly reliable for cut detection, but they are sometimes unfeasible due to the time spent to obtain results. Intensity-based techniques are not so accurate to detect cuts, although they work very fast. Thus a new hybrid technique is proposed. This method has been stated by combining both techniques. The intensity-based method is used to locate 'probable cuts' with a considerable speed and then the edge-based method is applied to check whether each 'probable cut' actually exists or not. This way, a great time reduction is achieved with little or no loss of accuracy. Results of five well-known algorithms for automatic cut detection are shown. Besides, the results of the proposed method are given and all of them are compared, obtaining the new technique very good results.

In this paper, we describe a novel approach to vision based navigation. In this approach, an airplane's position at each sampling time is estimated through a two-step process. In the first step, the plane's 3D motion is estimated from the current and previous image frames to produce an initial estimate of the plane's position. In the second step, the error in the initial position estimate is corrected by using a test image generated from a digital elevation map of the flight area and the previous frame. Experimental results demonstrated the efficacy of this approach--the correction step reduces position estimation error and with it, the error does not increase with time.

In modern traffic surveillance, computer vision methods are often employed to detect vehicles of interest because of the rich information content contained in an image. In this paper, we propose an efficient method for extracting the boundary of vehicles free from their moving cast shadows and reflective regions. The extraction method is based on the hypothesis that regions of similar texture are less discriminative, disregarding intensity differences between the vehicle body and the cast shadow or reflection on the vehicle. In this novel algorithm, a united likelihood map that based on the relationship of texture, luminance and chrominance of each pixel is initially constructed. Subsequently, a foreground mask is constructed by applying morphological operations. Vehicles can be successfully extracted and different vehicle components can be efficiently distinguished by the related autocorrelation index within the vehicle mask.

For exploitation of temporal interdependencies between consecutive frames, in existing 3D wavelet video coding concepts a blockwise motion estimation (ME) and compensation (MC) is employed. Because of local object motion, rotation or scaling, the processing of occlusion areas is problematic. In these regions, the calculation of correct motion vectors (MV) is not always possible and blocking artifacts may appear at the motion boundaries to the connected areas, for which uniquely referenced MV could be estimated. In order to avoid this, smooth transitions can be included around the occlusion pixels, which means to blur out the block artifacts. The proposed algorithm is based on the MC-EZBC 3D wavelet video coder (Motion-Compensated Embedded video coding algorithm using ZeroBlocks of subband / wavelet coefficients and Context modeling), which employs a lifting approach for temporal filtering.

Today, 3D subband (3DS) video coding schemes are close to current standard solutions in terms of coding efficiency while they add the scalability functionality through embedded bitstreams. Spatial scalability may be regarded as a key-feature brought by such codecs, enabling adaptation to varying terminal capabilities and display sizes. However, this functionality still suffers from a lack of coding efficiency when motion compensation is used. Though enabling motion compensation at the temporal filtering stage dramatically improves energy compaction, it generates a strong motion vector overhead and introduces a reconstruction drift when decoding at lower spatial resolutions. This is because the Discrete Wavelet Transform and the motion compensation (MC) are not commutative. Some authors solve this problem by transmitting the drift signal as side information, but this increases the bit-rate. In this paper we present a low-resolution optimized MC-3DS scalable codec with no drift, that does not generate any overhead. Its structure is fully compliant with any subband-tree entropy encoder and preserves all the other scalability functions (temporal and SNR). Tests and simulations show that the new scheme remains quite efficient when decoding at full resolution, while it outperforms the previous solution as far as spatial scalability is concerned, especially with high-activity sequences.

The JPEG compression standard is a popular image format. However, at high compression ratios JPEG compression, which uses block-transform coding, can produce blocking artifacts, or artificially introduced edges within the image. Several post-processing algorithms have been developed to remove these artifacts. This paper describes an implementation of a post-processing algorithm developed by Ramchandran, Chou, and Crouse (RCC) which is fast enough for real-time software-only video applications. The original implementation of the RCC algorithm involved calculating thresholds to identify artificial edges. These calculations proved too expensive for use in real-time software-only applications. We replaced these calculations with a linear scale approximating ideal threshold values based on a combination of peak signal-to-noise ratio calculations and subjective visual quality. The resulting filter implementation is available in the widely-deployed Open Mash streaming media toolkit.

The paper presents a novel Orthogonal Logarithmic Search (OLS) method for block base motion compensation. The performance of the algorithm is evaluated by using standard QCIF benchmark video sequences and the results are compared to a traditional well-known full search algorithm (FSA) and a sub-optimal method called the Three Step Search (3SS). The evaluation considers the three important metrics, time, entropy and PSNR (Peak Signal to Noise Ratio). The paper also shows that the strength of the algorithm lies in its speed of operation as it is 95% faster than the FSA and over 60% faster than the 3SS. Investigation into motion estimation algorithms is one of the important issues in the video coding standards such as ISO MPEG-1/2 and ITU-T H.263. These international standards regularly use a conventional FSA to estimate the motion of pixels between pairs of image block matching algorithms. Since a full search requires intensive computations and the distortion function needs to be evaluated many times for each target block to be matched, the process is very time consuming. Therefore, the main aim of this investigation has been to alleviate this acute problem of search speed and accuracy.

In this paper we propose a scalable video coding scheme that utilizes the embedded block coding with optimal truncation (EBCOT) compression algorithm. Three dimensional spatio-temporal decomposition of the video sequence succeeded by compression using the EBCOT generates a SNR and resolution scalable bit stream. The proposed video coding algorithm not only performs closer to the MPEG-4 video coding standard in compression efficiency but also provides better SNR and resolution scalability. Experimental results show that the performance of the proposed algorithm does better than the 3-D SPIHT (Set Partitioning in Hierarchial Trees) algorithm by 1.5dB.

In this paper, a wavelet-based spatially scalable video coding scheme with in-band prediction is investigated. The discrete wavelet transform is employed for the spatial decomposition of each video frame. A blockwise motion compensated prediction is computed for the individual subbands. For motion compensation the shift-invariant overcomplete discrete wavelet transform is exploited to achieve half-pixel motion vector accuracy. A blockwise decision is made between the coding of the motion compensated prediction error and the corresponding motion vector data and the wavelet coefficients. JPEG 2000 quantization and arithmetic coding are applied to the resulting prediction error. The performance of the motion compensated predictive in-band coder is evaluated for different block sizes and sub-pixel accuracy.

It is very important to control the bit-rate for video communication under a limited capacity of communication channel like wireless communication system because of large amout of data. In this paper, we propose a new adaptive pre-filter to improve the bit-rate in video compression, which preserves visually significant portion of the original image and smooth only insignificant portion actively. We focus on the differential signals between the original image and its filtered one and control the degree of smoothing by considering the power of local differential signals to produce various kinds of intermediate images betwwen both images. We conducted some computer simulations to check the performance of our scheme for the standard MPEG-4 video compression algorithm. Consequently, we could achieve further improvement of compression ratio while improving visual image quality than a conventional scheme.

In this paper, we demonstrate the performance of a lossless video-coding algorithm based on the existing H.26L test model. The lossless coding algorithm is formed by simply skipping the transform and quantization operations in H.26L; therefore, the main elements in this lossless coding algorithm are motion compensation, Intra prediction, and entropy coding tools of H.26L. It has been demonstrated that H.26L has significant good performance at both low bit rates and high bit rates for lossy video coding. Here we further demonstrate the competitive performance of lossless video coding using H.26L tools. The H.26L lossless video coding results are compared to those of Motion JPEG2000. When Inter coding tool is applied, H.26L provides on average 32% bit reduction comparing to Motion JPEG2000. We evaluate the performances of H.26L Intra and Inter coding separately. The motion compensation techniques adopted in H.26L proves to be also effective for lossless coding. There is roughly 40% bit saving when Inter frame coding is turned on. The performance of Intra prediction of H.26L for lossless coding is not as impressive as that of the H.26L motion compensation; the bitrate increases on average by 19% comparing to that of Motion JPEG2000, which also uses only Intra coding tools. The results indicate that H.26L Intra coding could further be improved. Possible improvements for H.26L lossless video coding are suggested in the paper.

The Block-based Discrete Cosine Transform (BDCT) is one of the most widely used transforms in image and video coding. However, it introduces annoying blocking artifact at low data rates. A great deal of work has been done to reduce the artifact. In this paper, we propose a transform domain-based Markov Random Field (TD-MRF) model to address this problem. Based on this new model, a transform domain maximum a posteriori (MAP) algorithm is presented to remove the blocking artifacts in images and video. It is shown that our new approach can reduce the computational complexity dramatically while achieving significant visual improvements.

Additive noise in images is estimated by the concept of geostatistics, which then is implemented in the space-variant Wiener filter for adaptive image restoration. The algorithm is tested on several image versions degraded with Gaussian white noise. The present results are more favorable than those obtained by standard statistical methods.

Polarization in optical waveguides is always an aspect of optical loss and consequently impacts device performance. Wavelet image processing allows a means to detect optical signals buried under noise. Orthogonality is an essential element in wavelet bases. Wavelet transforms are orthogonal. They can also be biorthogonal. Wavelet image processing will be applied to laser beam propagation in lithium niobate and nonlinear polymer waveguides to achieve detection of signals below noise and a better understanding of polarization as an aspect of device performance.

A Hyperspectral image is a sequence of images generated by collecting contiguously spaced spectral bands of data. One can view such an image sequence as a three-dimensional array of intensity values (pixels) within a rectangular prism. We present a Three-Dimensional Set Partitioned Embedded bloCK (3DSPECK) algorithm based on the observation that hyperspectral images are contiguous in the spectrum axis (this implies large inter-band correlations) and there is no motion between bands. Therefore, the three-dimensional discrete wavelet transform can fully exploit the inter-band correlations. A SPECK partitioning algorithm extended to three-dimensions is used to sort significant pixels. Rate distortion (Peak Signal-to-Noise Ratio (PSNR) vs. bit rate) performances were plotted by comparing 3DSPECK against 3DSPIHT on several sets of hyperspectral images. Results show that 3DSPECK is comparable to 3DSPIHT in hyperspectral image compression. 3DSPECK can achieve compression ratios in the approximate range of 16 to 27 while providing very high quality reconstructed images. It guarantees over 3 dB PSNR improvement at all rates or rate saving at least a factor of 2 over 2D coding of separate spectral bands without axial transformation.

We evaluated some arbitrary-shape ROI (Region of Interest) coding techniques for three-dimensional volumetric images, and through the observation we propose a flexible ROI coding with efficient compression performance. In arbitrary-shape ROI coding, the object in an image is coded with higher fidelity than the rest of the image, together with the shape information, which indicates the region of the object. In our proposed method (named as SA-ROI), in which shape-adaptive wavelet transform and scaling-based ROI coding are incorporated, the samples within the object are transformed with three-dimensional shape-adaptive wavelet transform according to its shape-information. If necessary, the background is also transformed by shape-adaptive wavelet transform independently. Then the samples within the object are scaled up by a certain number of bit-shifts, and encoded from the MSB (Most Significant Bit) plane by plane so that coefficients within the object are encoded earlier and have higher fidelity than the background. Compared with the scaling-based ROI coding using ordinary wavelet transform, which has almost the same ROI coding functionalities, SA-ROI outperforms by 5% in lossless compression ratio. Also in lossy coding, except at very low coding rates, SA-ROI has better compression performance.

In this paper, we propose a new low bitrate system to encode monochromatic images that combines DWT and SVD techniques. At the encoder side, the input image is divided into blocks (tiles) of 64×64 pixels. A criterion based on the average standard deviation of 8×8 subblocks is used to choose DWT or SVD. If the tile exhibits a high average standard deviation, it is compressed by using SVD otherwise by DWT. Eigenvalues and eigenvectors are scalar and vector quantized respectively. The Daubechies 9/7 filter bank factored into lifting steps is used for the DWT. The maximum number of levels of subband decomposition for each tile is 3. Each subband is coded using the HC-RIOT. At the decoder side, the header of each compressed tile is read and the information is sent to the HC-RIOT decoder or to the inverse quantization, in order to recover the tile. The proposed system is applied to image coding and its performance is discussed.

We present a new proxy-based system to allow public use handheld devices to access instantly to the Low Resolution Picture Taking (LRPT) satellite weather image data and display regional weather image. First the location-based transcoding is done from non frame based satellite image to frame based CIF/QCIF image for handheld device. GPS information from expansion module is used in the location of the region of interest. A robust fixed-length joint source and channel coding scheme is then implemented to achieve robust wireless transmission. Experimental results show that the proposed system is suitable to the time varying, low bandwidth wireless channel and power constraint on wireless handheld devices.

The Bluetooth specifications currently include many application profiles that define the requirements for some usage cases. In this paper the attention is focused on the 'Basic Imaging Profile' BIP. The BIP could add important features to classic Digital Still Cameras, such as the possibility to send images to a Bluetooth Printer, or to send the image on Internet with a Bluetooth cellular phone.

The eXtensible Markup Language (XML) provides a good tool to represent the Web content due to its semi-structured characteristics. A new XML-based Web architecture is proposed in this work, upon which various applications are built. The proposed XML-based Web architecture contains four major components: layout, content, digital right management (DRM), and security. Each component is described by one or more specific XML languages. Then, different XML languages are harmonized under one framework for efficiency and ease of manipulation. The XML-based schema of webpages can be organized easily to meet both internal and external requirements. That is, the Web data should be managed more easily by content owners while customization should be made for different users with different interests and access means.

In image coding, the distribution of quantization error is uniform across the transformed domain (i.e., coding error). To minimize visually perceptible noise, the quantization step size should suit the most sensitive area in the color space used. Consequently, we can expect to improve the coding efficiency while achieving the same subjective quality by adopting a perceptually uniform color space. Currently used image coding schemes such as JPEG and MPEG, make extensive use of the YCbCr color space. However, its perceptual non-uniformity is significant. In this paper, we introduce the uniform color space 'LST', which is obtained by smoothly deforming the CIE xy chromaticity diagram with a look-up table with 333 morphing points so that every MacAdam JND (just noticeable difference) ellipse is mapped onto or close to a unit circle. Test color images that use this LST color space, as well as the CIELAB, CIELUV and YCbCr spaces, are JPEG coded. The coding algorithm itself does not need to be changed. Subjective comparisons by six subjects of the decoded images to the originals show the superiority of LST to the three other spaces; the most commonly-used YCbCr was ranked last despite its highest PSNR. No significant difference was observed between CIELUV and CIELAB.

This paper proposes a semiautomatic algorithm for the accurate extraction of an athlete from color diving sequences. Change detection techniques and edge detection techniques are combined to extract the moving object. Color information and interactive information are used to get rough region of the athlete interested. A robust edge map is derived from the difference between successive frames, then further refining of rough athlete region is applied by the information of the robust edge. The proposed method is useful in applications with a relatively still background, Experimental results show that the method provides accurate extraction with pixel-wise precision, thus providing a reliable input to further analysis or applications such as MPEG-4.

In this paper, we propose fast hybrid interpolation methods that employ more than one interpolation algorithm and choose the most appropriate interpolation algorithm that provides high quality images with a minimum processing time. In the proposed algorithm, we first apply a test to predict which interpolation is most appropriate for a given pixel in terms of complexity and performance. Then, a simple interpolation algorithm is used for pixels for which the simple interpolation algorithm provides acceptable performances and a complex interpolation algorithm is used for pixels for which the complex interpolation algorithm significantly outperforms the simple interpolation algorithm. Consequently, it is possible to obtain high quality images without significantly increasing the processing time.

In this paper, we extend our previous resolution enhancement results by proposing a technique for the estimation of the regularization parameter based on the assumption that it should satisfy the following properties: It should be a function of the regularized noise power of the data and its choice should yield a convex functional whose minimization would give the desired high-resolution image. Experimental results are presented and conclusions are drawn.

This paper describes the new DM642 Digital Media Processor designed to support a broad range of video over packet applications. The DM642 includes the C64x VLIW DSP core optimized for video processing and flexible peripheral interfaces including three 20-bit video ports, a multi-channel audio serial port (McASP), Ethernet MAC, and PCI. The DM642 supports the full range of mature and emerging video compression algorithms including MPEG-2, MPEG-4, H.261, H.263, H.264, Windows Media, Real Video, and On2 VP5.

Due to the increasing significance of development costs in the competitive domain of high-volume consumer electronics, generic solutions are required to enable reuse of the design effort and to increase the potential market volume. As a result from this, Systems-on-Chip (SoCs) contain a growing amount of fully programmable media processing devices as opposed to application-specific systems, which offered the most attractive solutions due to a high performance density. The following motivates this trend. First, SoCs are increasingly dominated by their communication infrastructure and embedded memory, thereby making the cost of the functional units less significant. Moreover, the continuously growing design costs require generic solutions that can be applied over a broad product range. Hence, powerful programmable SoCs are becoming increasingly attractive. However, to enable power-efficient designs, that are also scalable over the advancing VLSI technology, parallelism should be fully exploited. Both task-level and instruction-level parallelism can be provided by means of e.g. a VLIW multiprocessor architecture. To provide the above-mentioned scalability, we propose to partition the data over the processors, instead of traditional functional partitioning. An advantage of this approach is the inherent locality of data, which is extremely important for communication-efficient software implementations. Consequently, a software implementation is discussed, enabling e.g. SD resolution H.264 decoding with a two-processor architecture, whereas High-Definition (HD) decoding can be achieved with an eight-processor system, executing the same software. Experimental results show that the data communication considerably reduces up to 65% directly improving the overall performance. Apart from considerable improvement in memory bandwidth, this novel concept of partitioning offers a natural approach for optimally balancing the load of all processors, thereby further improving the overall speedup.

The TMS320C64x DSP is a generation of high-speed DSPs with a rich instruction set and an efficient memory system for multimedia processing. Digital video decoding is one of the key applications in multimedia processing. It is a computationally intensive application, which requires high bandwidth to external memory and an efficient DMA engine. Reference models for video decoders typically follow a simple data flow that operates sequentially on one macroblock (MB) at a time. This structure leads to inefficiencies in real-time implementations including less than optimal utilization of program caches and DMA bandwidth. These issues become more significant with high-performance devices like the C64x DSP because the CPU efficiency and high-clock rate allow the core processing to occur much faster than on other processors. At the same time, the bandwidth to external memory has not increased at the same rate as the processing performance. This can lead the performance bottleneck to be I/O bandwidth instead of processing unless the system data flow is carefully designed. This paper describes an optimized flow for MPEG-2 decoding, which processes multiple blocks at a time to obtain optimum cache performance and DMA bandwidth efficiency. With this approach, system overhead is reduced from as high as 100% for worst-case B frames with the conventional flow to less than 20%.

A new chip using a DSP with a novel vector architecture is described. It uses a Very Dense Instruction Word (rather than a VLIW) and exploits the parallelism and narrow data typical of image processing to gain high performance at low cost and power. It contains eight 32-bit datapaths all working off a single instruction, and can do sixteen 16-bit MACs per cycle or four 32-bit memory accesses per cycle to 128 KB of on-chip memory. It also contains a serial datapath for handling low-performance code and OS functions. The chip includes memory, video and IO interfaces on an industry-standard bus. It also includes camera-specific IO such as videos DACs for NTSC/PAL and analog LCDs, an I2S audio interface, and USB 1.1. It is built in 0.18 um CMOS, runs at 233 MHz, and draws 300 mW. It uses no fixed-function blocks, microcode, or coprocessors, but can capture and compress video at 30 fps at VGA resolution using JPEG, or at CIF resolution using MPEG-4.

This paper presents an efficient VLSI implementation of a lifting coprocessor for mobile multimedia applications. To reduce the hardware complexity, we designed and implemented rational lifting coefficients. This approach allows the floating-point arithmetic units to be replaced by the integer arithmetic units in the design. Consequently, footprint and power consumption of the coprocessor are reduced. To improve the throughput of system, a fully pipelined parallel architecutre is designed. With the rational coefficients and parallel approaches, the proposed lifting coprocessor provides efficient computing power but requires very low power consumtion. The lifitng scheme coprocessor was implemented in VHDL using HCMOS8D 0.18 μm technology. It can run at 25 MHz withthe power supply of 1.55 volt and requires only 1.191 mW.