1.

Introduction

Layered coding combined with unequal error protection (UEP) is a promising method for video transmission over error-prone channels. Data partitioning¹ in the H.264/AVC standard² is an effective layered coding technique and is utilized in many UEP schemes, owing to its lower overhead and better error resilience.^{3, 4} With data partitioning, each partition in one slice has unequal importance for the reconstructed video quality due to their different dependency relationships.

In one group of pictures (GOP), which consists of one intraframe [I-frame or instantaneous decoder refresh (IDR)-frame] and a set of interframes (P-frames and B-frames), the intraframe is the predictive reference frame of subsequent interframes, thus the intraframe is more important than the interframes. Furthermore, previous P-frames are more important than succeeding P-frames, and B-frames are the least important in the event that hierarchical B-frames are not applied. Based on this fact, some video transmission schemes provide unequal error protection to frames with different encoding types.⁵

From the prior analysis, the importance measurement at different coded levels, e.g., slice level, frame level, or GOP level, is a prerequisite for designing an effective UEP scheme. In this work, we develop an algorithm for deriving distortion as a metric to evaluate importance, jointly considering the unequal importance of different partitions in one slice and frames with different positions in one GOP. Moreover, a priority sorting method, which employs the characteristics of wireless channels, is proposed.

2.

Distortion Derivation

According to the definition of data partitioning,^{1, 2} the encoded video data of one slice is rearranged into three different partitions, namely, partitions A, B, and C. Without loss of generality, we adopt a simple slice mode, i.e., one slice per frame, thus the encoded data of one frame is divided into three partitions. The average distortion over one GOP is selected as the metric to assess the importance of different partitions in different frames. A GOP is composed of one I-frame followed by $N_G-1$ P-frames, where $N_G$ denotes the total number of frames in one GOP. Due to its importance, the I-frame is assumed to be perfectly protected and correctly received.

Let $f_n^i$ denote the $i$ th pixel in frame $n$ , and ${\widehat{f}}_n^i$ and ${\tilde{f}}_n^i$ denote its corresponding reconstructed pixel at the encoder and the decoder, respectively. Moreover, let $d_n^i$ denote the distortion in terms of the mean squared error (MSE) of $f_n^i$ , which can be calculated by:

To analyze the importance of different partitions in frame $n$ , we derive the distortion of each partition in frame $n$ under the assumption that only the partition under discussion is lost and other partitions in the current frame and all partitions in subsequent frames are error-free. As for the error concealment scheme, the modified temporal replacement is adopted, in which the intracoded macroblock (MB) or intercoded MB with lost motion information is replaced by the colocated MB in the previous frame.

To compute $d_n^i$ using Eq. 1, ${\tilde{f}}_n^i$ is required. For the determination of ${\tilde{f}}_n^i$ , three cases are considered, depending on which partition in frame $n$ is lost. We refer to an intracoded MB as I-MB, and an intercoded MB as P-MB.

3.

Priority Sorting

Based on the analysis of ${\overline{D}}_{n,1}$ , ${\overline{D}}_{n,2}$ , and ${\overline{D}}_{n,3}$ for frames from frame 2 to frame $N_G$ , we present a priority sorting method jointly taking into consideration the unequal importance of partitions in one frame and P-frames at different positions in one GOP. That is, all partitions of the first $m$ P-frames and partition A of the middle $n$ P-frames following the preceding $m$ P-frames are labeled as high priority (HP), and partition B and partition C of the $n$ P-frames and all partitions of the remaining $p$ P-frames in one GOP are labeled as lower priority (LP).

For efficient delivery of the coded video data over wireless channels, the coded video data with HP must be delivered to the decoder with small probability of being affected by bit errors, providing basic reconstruction video quality. For each subcarrier in wireless channels employing orthogonal frequency division multiplexing (OFDM) technology, the extent of attenuation subject to fading is different, which can be utilized to categorize the subcarriers into two groups. The subcarriers with the signal-to-noise ratio (SNR) above $S_0$ are assigned to the high quality (HQ) subchannel group, otherwise the subcarriers are assigned to the low quality (LQ) subchannel group. Thus, the threshold $S_0$ used for grouping subchannels is determined by the bit error rate (BER) requirement of the coded video data with HP. Let $C_H$ and $C_L$ denote the available transmission capacity provided by HQ and LQ subchannel groups during $N_G-1$ P-frame intervals, respectively. And $C_0$ refers to the summation of all partition A for total P-frames. Moreover, let $b_{i,1}$ , $b_{i,2}$ , and $b_{i,3}$ signify the total coded bits for partitions A, B, and C in frame $i$ , respectively, and $b_{i,0}$ be the summation of $b_{i,2}$ and $b_{i,3}$ for simplicity. Therefore, $m$ , $n$ and $p$ for determining the priority of the total coded video data in all P-frames should satisfy the following constraints:

Given $C_H$ , $C_L$ , and video encoding parameters, $m$ , $n$ , and $p$ can be determined by a heuristic algorithm summarized as follows:

4.

Experimental Results

We first examine the performance of the distortion derivation algorithm. The sequences “Grandma” and “Silent” in QCIF format with a frame rate $30\mathrm{Hz}$ were encoded using H.264/AVC reference software JM15.1. The GOP consisted of one I-frame and $N_G-1$ P-frames without B-frames, where $N_G$ was set to 15. In addition, the UseConstrainedIntraPred option was set up, and the quantization parameter (QP) was fixed at 28 for all experiments.

Figure 1 shows the distortion of all subsequent frames versus frame index when partition A of the first P-frame is lost for “Silent.” From Fig. 1, we can observe that the distortion derived accords with the distortion simulated to a large extent, so the distortion determined by Eq. 5 can be used as the metric to evaluate the importance of different partitions of P-frames in one GOP.

Fig. 1

Comparison of derived distortion and actual distortion for Silent.

Next, the performance of the priority sorting method is verified. Five video sequences of 300 frames, “Mother&Daughter,” “Grandma,” “Salesman,” “Silent,” and “Foreman,” were encoded with the same parameters used in the first set of experiments. Assume that the BER of HP coded video data is ${10}^{-6}$ , the BER of LP data is ${10}^{-3}$ and ${10}^{-2}$ , and $C_H$ and $C_L$ satisfied $C_H:C_L=1:2$ . For comparison, the rations of $m$ , $n$ , and $p$ to $N_P$ were fixed as $1∕6$ , $2∕3$ , and $1∕6$ , repectively.

Comparing the proposed priority sorting scheme with data partitioning in H.264/AVC at the frame level, the average peak signal-to-noise ratio (PSNR) for all sequences and the PSNR versus the frame number for sequence “Silent” are given in Table 1 and Fig. 2, respectively. From Table 1 and Fig. 2, we can see that the proposed method can improve the reconstructed quality due to its better error resilience and adaptability, especially for sequences with medium motion.

Fig. 2

PSNR comparison of the proposed scheme against original data partitioning for Silent.

Table 1

Average PSNR comparison of the proposed scheme against data partitioning.

5.

Conclusions

A distortion derivation model is proposed by jointly considering the unequal importance of different partitions in one slice and each frame in one GOP. Furthermore, a heuristic algorithm is designed to determine priority sorting. Simulation results show that the distortion derivation model matches the actual distortion well, and the introduced priority sorting method improves the reconstructed video quality under error-prone environments.