Note: Descriptions are shown in the official language in which they were submitted.
CA 02522835 2004-03-03
METHOD OF SELECTING A REFERENCE PICTURE
Technical Field
The present invention relates to coding and encoding a
moving picture.
Background Art
Moving picture coding systems compensate for motion
using motion vector information. In the case of moving
picture coding system that uses multiple reference pictures,
reference picture index information as well as motion vector
information is required for motion compensation. The
reference picture index is a value that is used to
distinguish multiple reference pictures from each other. A
coder transfers the reference picture index to a decoder.
The decoder selects the reference picture as indicated by the
reference picture index, and performs motion compensation
using the selected reference picture.
Generally, scanning methods for images are classified
into either progressive scanning or interlaced scanning. In
the case of the progressive scanning, an image of one frame
consists of data that are sampled at the same time. In case
of the interlaced scanning, an image of one frame consists of
data that are sampled at different times, and the samples are
alternated line by line.
The interlaced image of a frame is usually divided into
two fields called a top field and a bottom field. In the
interlaced scanning, one frame is divided into two field
images. In this specification, an image is treated based on a
unit of a picture. A frame or a field may be referred to as
a picture.
Three coding methods for an interlaced moving picture
are proposed. A first coding method is a field picture
coding method which performs a coding process, considering
each field as an independent picture at a picture or slice
level. A second coding method is the frame picture coding
method which performs a coding process after combining two
1
CA 02522835 2004-03-03
fields into one frame at a picture or slice level. A
third coding method is a frame picture coding method with
frame/field macroblock, which performs a coding process by
combining two fields into one frame and selecting a frame
mode or a field mode at a macroblock level.
In the third coding method, two vertically adjacent
macroblocks are bound into a pair and the coding is performed
in unit of a macroblock pair. In FIG. 1, numbers assigned in
the macroblock pairs indicate macroblock addresses that are
used to distinguish the macroblocks from each other.
Here, a frame macroblock is a macroblock which is coded
at upper and lower macroblocks of the macroblock pair in
units of a frame. In other words, each macroblock is coded
in units of a frame after two fields are combined into one
frame.
Meanwhile, a field macroblock is a macroblock that is
coded at upper and lower macroblocks of macroblock pairs in
units of a field. The macroblock pair is divided into top
and bottom field components and each field is coded
independently. At this time, the macroblock pairs are
rearranged so that the upper and lower macroblocks may
respectively contain only a top field component and only a
bottom field component. The upper macroblock is called a top
field macroblock and the lower macroblock is called a bottom
field macroblock.
A reference buffer for storing multiple reference
pictures is configured in units of a frame. In the frame
picture coding, all reference pictures are considered as
units of a frame in which two fields are combined into one
frame, and one picture among the reference frame pictures is
used for motion compensation. Accordingly, values are
allocated to the reference picture indexes in units of a
frame.
The reference picture index for a P frame is obtained
by sorting all the reference frames in an order reverse to a
coding order and then sequentially allocating an index, which
is increased by one, to the sorted reference frames. The
2
CA 02522835 2004-03-03
reference picture index for a B frame is classified into a
list 0 and a list 1, and is determined based on a display
order of the reference frame.
First, in case of the reference frame list 0, indexes
are allocated in a reverse order to the reference frames
whose display order lags behind the B frame, and then, the
remaining indexes are allocated in the display order to the
reference frames whose display order leads the B frame. In
case of the reference frame list 1, in contrast to the
reference frame list 0, indexes are allocated in the display
order to the reference frames whose display order leads the B
frame, and then, the remaining indexes are allocated in a
reverse order to the reference frame whose display order lags
behind the B frame. FIG. 2 shows a reference picture index
for a P frame when a size of the reference buffer is 5, and
FIG. 3 shows the reference picture index lists 0 and 1 for a
B frame .
In the field picture coding, the fields of all
reference pictures are considered as independent pictures and
one of the reference field pictures is used for motion
compensation. Accordingly, values are allocated to the
reference picture indexes in units of a field. At this time,
the reference fields are combined in the unit of a frame, and
then, the reference picture indexes in each field of the P
frame are sorted in an order reverse to a coding order of the
frame. Then, index values that are increased by one are
alternately allocated in an order that starts from a
reference field having parity equal to a current picture to a
reference field having parity different from the current
picture, while visiting the sorted reference frames in
sequence.
In addition, in the case of the reference picture index
lists 0 and 1 for each field of a B frame, all the reference
fields are combined in units of a frame and then a reference
frame is sorted in the same manner as the method of
determining reference frame index of a B frame. Thereafter,
the indexes that are increased by one are alternately
3
CA 02522835 2004-03-03
allocated in an order that starts from a reference
field having parity equal to a current picture to a reference
field having parity different from the current picture, while
visiting the sorted reference frames in sequence.
FIG. 4 shows reference picture indexes of top and
bottom fields of a P frame when a size of a reference buffer
is 5, and FIG. 5 shows reference picture index lists 0 and 1
of top and bottom fields of a B frame that can be used as a
reference.
In order to improve coding efficiency, the moving
picture coding system using multiple reference pictures
provides a function of reordering the reference picture
indexes before decoding a picture or a slice. The reference
picture indexes are arbitrarily reordered so as to enhance
the coding efficiency after determining an initial reference
picture index for the frame and field pictures. FIG. 6 shows
a result of determining an initial reference picture index
for a P frame and arbitrarily reordering indexes according to
the above-described method.
When the moving picture coding system using the
multiple reference pictures codes an interlaced moving
picture into a frame picture having frame/field macroblocks,
all the reference pictures for the frame macroblock are
considered as a frame unit and one of the reference frame
pictures must be used for motion compensation. In addition,
all the reference pictures for the field macroblock are
considered as independent pictures and one of the reference
field pictures must be used for motion compensation.
Accordingly, in the frame picture coding that selects
the frame/field coding modes at the macroblock level, it is
necessary to allocate values to the reference picture index
in frame unit or field unit according to the coding mode of
each macroblock.
Summary of the Invention
An embodiment of the present invention provides
4
CA 02522835 2004-03-03
reference picture information used for motion compensation
by
determining the reference picture index differently
according a coding mode of macroblock when interlaced moving
picture is coded with a frame picture having frame/field
macroblocks in a moving picture coding system using multiple
reference pictures.
In one embodiment, the method of selecting a reference
picture for processing a field macroblock includes selecting
at least one reference picture for processing a field
macroblock from at least one reference picture list using
reference picture index information. Here, each selected at
least one reference picture is a field. For example, the
reference field may a top field or a bottom field.
In another embodiment, the method may further include
processing the field macroblock using the selected at least
one reference picture.
In a further embodiment, the method may include forming
the at least one reference picture list. For example, the
forming step may include initializing the at least one
reference picture list, and reordering the at least one
reference picture list.
In another embodiment, if the reference picture index
information is an even index number, then the selected at
least one reference field has a same field parity as the
field macroblock.
In yet another embodiment, if the reference picture
index information is an odd index number, then the selected
at least one reference field has a field parity different
from the field macroblock.
In one embodiment, the method of selecting a reference
picture for processing a field macroblock includes selecting
at least one reference picture for processing a field
macroblock from at least one reference picture list using
reference picture index information. Here, each selected at
least one reference picture is a field, and a parity of the
selected at least one reference field may be based on the
5
CA 02522835 2004-03-03
parity of the field macroblock and the reference picture
index information.
Brief Description of the Drawings
The accompanying drawings, which are included to
provide a further understanding of the invention and are
incorporated in and constitute a part of this specification,
illustrate embodiments of the invention and together with the
description serve to explain the principles of the invention.
In the drawings:
FIG. 1 shows macroblock pair in frame picture having
general frame/field macroblock;
FIG. 2 shows reference picture indexes of P frame
coding in a moving picture coding system according to the
related art;
FIG. 3 shows the reference picture index lists 0 and 1
of B frame coding in a moving picture coding system according
to the related art;
FIG. 4 shows reference picture indexes in a coding of
top and bottom fields of a P frame in a moving picture coding
system according to the related art;
FIG. 5 shows reference picture index lists 0 and 1 in a
coding of top and bottom fields of a B frame in a moving
picture coding system according to the related art;
FIG. 6 shows reference picture indexes reordered in a
coding of a P frame in a moving picture coding system
according to the related art;
FIG. 7 shows reference picture indexes of a P frame and
a B frame having frame macroblocks in a moving picture coding
system according to the present invention;
FIG. 8 shows reference picture indexes of P frame and B
frame having field macroblock in a moving picture coding
system according to an embodiment of the present invention
(case 1);
FIG. 9 shows reference picture indexes of a P frame and
a B frame having field macroblocks in a moving picture coding
6
CA 02522835 2004-03-03
system according to another embodiment of the present
invention (case 2);
FIG. 10 shows reference picture indexes of a P frame
having field macroblocks in a moving picture coding system
according to another embodiment of the present invention
(case 3);
FIG. 11 shows reference picture indexes of a B frame
having field macroblocks in a moving picture coding system
according to a further embodiment of the present invention
(case 3); and
FIG. 12 shows reference picture indexes of a P frame
and a B frame having field macroblocks in a moving picture
coding system according to another embodiment of the present
invention (case 4).
Detail Description of Example Embodiments
Hereinafter, example embodiments of the present
invention will be described in detail with reference to
accompanying drawings.
As stated previously, moving picture coding systems
compensate for motion using motion vector information. In
the case of a moving picture coding system that uses multiple
reference pictures, reference picture index information as
well as motion vector information is used for motion
compensation. The reference picture index is a value that is
used to distinguish multiple reference pictures from each
other. A coder transfers the reference picture index to a
decoder. The decoder selects the reference picture as
indicated by the reference picture index, and performs motion
compensation using the selected reference picture.
When an interlaced moving picture is coded into a frame
picture having frame/field macroblocks, a moving picture
coding system using multiple reference pictures performs a
motion compensation from a reference frame in order for frame
coding in the frame macroblock. In other words, the
7
CA 02522835 2004-03-03
reference pictures may be considered as a frame unit. The
moving picture coding system, however, performs a motion
compensation from a reference field in order for a field
coding in the field macroblocks. In other words, the
reference pictures may be considered as a field unit.
Accordingly, the frame pictures having frame/field
macroblock use indexes having different structures at the
macroblock level. For example, the frame macroblock uses a
reference picture index of a frame unit and the field
macroblock uses a reference picture index of a field unit. It
should be considered that the reference picture indexes may
be reordered at a picture or slice level.
Therefore, the frame picture having frame/field
macroblocks should consider the reference picture in a frame
unit at a picture or slice level and have the reordered
reference picture indexes of a frame unit. The reference
picture index may be determined according to a coding mode of
macroblock based on the reference picture indexes of the
frame unit at the macroblock level.
The present invention provides an embodiment of a
method for determining reference picture indexes in a frame
picture having frame/field macroblocks. The method includes
a process of determining reference picture indexes at a
picture (or slice) level and a process of determining
reference picture indexes at a macroblock level.
[1] Determination (or Initialization) of Reference
Picture Tndexes at a Picture (or Slice) Level
The reference pictures may be considered in a frame
unit at a picture or slice level and reference picture
indexes of frame unit are calculated.
In the method of determining a reference picture index
for a P frame, reference frames are sorted in an order
reverse to a coding order and indexes are allocated while
visiting the sorted reference frames in sequence.
Reference picture indexes for a B frame are determined
8
CA 02522835 2004-03-03
based on a display order of the reference frames. First, in
case of a reference frame list 0, indexes are allocated in a
reverse order to reference frames whose display orders lag
behind the B frame and the remaining indexes are allocated in
the display order to reference frames whose display order
lead the B frame.
In case of a reference frame list 1, in contrast to the
reference frame list 0, indexes are allocated in the display
order to reference frames whose display orders are higher
than the B frame and the remaining indexes are allocated in a
reverse order to reference frames whose display orders are
lower than the B frame.
In this and the other embodiment described below, in
order to improve coding efficiency, the moving picture coding
system using multiple reference pictures may provide a
function of reordering the reference picture indexes before
decoding a picture or a slice. The reference picture indexes
are arbitrarily reordered so as to enhance the coding
efficiency after determining an initial reference picture
index for the frame and field pictures. When the reference
picture indexes are reordered so as to enhance a coding
efficiency, the reference picture indexes of a frame unit may
be reordered.
[2] Determination (or reordering) of Reference
Picture Index at Macroblock Level
The reference picture index is modified according to
the coding mode of the macroblock while performing a coding
in unit of a macroblock pair with respect to the reference
picture indexes of a frame unit, which are obtained at the
picture (or slice) level. This process will be described
below.
[2.1] Case of Frame Macroblock
Since motion compensation may be performed to frame
macroblocks from a reference frame, the reference picture
index may be a frame unit. Accordingly, the reference
picture indexes of a frame unit, which are obtained at the
9
CA 02522835 2004-03-03
picture (or slice) level, are used.
A reference frame stored in a reference buffer may be
configured with a field pair having parities (e.g., top and
bottom) opposite to each other. FIG. 7 shows reference
picture indexes for a frame macroblock in a P frame and a B
frame when a size of the reference buffer is 5.
[2.2] Case of Field Macroblock
Since motion compensation may be performed to field
macroblocks from a reference field, the reference picture
index may be a field unit. In this case, a field pair having
parities opposite to each other is also present in the
reference buffer. Accordingly, each reference frame is
divided into two fields while sequentially visiting the
reference frame according to the reference picture indexes of
a frame unit, which are obtained at the picture (or slice)
level. Then, the reference picture indexes are newly
allocated to each field. The reference picture indexes for
two fields of each reference frame are allocated in various
methods (Cases 1 to 4).
(Case 1)
Regardless of the top and bottom field macroblocks of a
current field macroblock, a lower index is allocated to the
top reference field and a higher index is allocated to the
bottom reference field while sequentially visiting the
reference frames in an order of the reference picture
indexes, which are obtained at the picture (or slice) level.
In other words, the top and bottom reference field
indexes may be given by the equation below.
Top reference field index = 2 x picture index of
reference frame
Bottom reference field index = 2 x picture index of
reference frame + 1
FIG. 8 shows reference picture indexes for field
macroblocks of a P frame and a B frame.
CA 02522835 2004-03-03
(Case 2)
Regardless of the top and bottom field macroblocks of a
current field macroblock, a lower index is allocated to the
bottom reference field and a higher index is allocated to the
top reference field while sequentially visiting the reference
frames in an order of the reference picture indexes, which
are obtained at the picture (or slice) level.
In other words, the top and bottom reference field
indexes may be given by an equation below.
Top reference field index = 2 x picture index of
reference frame + 1
Bottom reference field index = 2 x picture index of
reference frame
FIG. 9 shows reference picture indexes for field
macroblocks of a P frame and a B frame.
(Case 3)
Indexes that are increased by one are alternately
allocated to reference fields, starting from the reference
field having a parity equal to the current field to the
reference field having a parity different from the current
field, while sequentially visiting the reference frames
according to a reference picture index order of the picture
(or slice) level. FIGS. 10 and 11 show reference picture
indexes for the field macroblocks of a P frame and a B frame.
(Case 4)
Regardless of the top and bottom field macroblocks of a
current field macroblock, a lower index is allocated to
reference field close to the current field in view of time
and a higher index is allocated to reference field farther
from the current field in view of time while sequentially
visiting the reference frames in an order of the reference
picture indexes of the picture (or slice) level.
In other words, the reference field indexes may be
given by the equation below.
Reference field index close to the current field = 2 x
picture index of reference frame
11
CA 02522835 2004-03-03
Reference field index far from the current field = 2 x
picture index of reference frame + 1
In FIG. 12, there are shown reference picture indexes
for a field macroblock of a P frame and a B frame.
While the present invention has been described and
illustrated herein with reference to the example embodiments
thereof, it will be apparent to those skilled in the art that
various modifications and variations can be made
therein without departing from the spirit and scope of the
invention. Thus, it is intended that the present invention
covers the modifications and variations.
According to a coding a moving picture method of at
least one embodiment of the present invention, when a moving
picture coding system using multiple reference pictures codes
an interlaced moving picture into a frame picture having
frame/field macroblocks, the moving picture coding system
efficiently provides reference picture information used for
motion compensation by differently determining reference
picture indexes according to coding modes of macroblocks.
12
