Note: Descriptions are shown in the official language in which they were submitted.
CA 02440007 2003-09-11
1
This application is a division of Canadian Application No. 2,415,375 filed
June 18, 1998 for Signal Adaptive Filtering Method, Signal Adaptive Filter and
Computer Readable IVledium for Storing Program Therefor.
Technical Field
The present invention relates to data filtering, and more particularly, to a
signal adaptive filtering method for reducing blocking effect and ringing
noise, a
signal adaptive filter and a computer readable medium for storing a program
therefor.
Background Art
Generally, picture encoding standards such as MPEG of the International
Standardization Organization (ISO) and H.263 recommended by the
International Telecommunication Union (ITU) adopt block-based motion
estimation and discrete cosine transform (DCT) blocks. Vllhen an image is
highly compressed, the block-based coding may cause the well-known blocking
effect. A typical blocking effect is grid noise in a homogeneous area in which
adjacent pixels have relatively similar pixel values. Another blocking effect
is
staircase noise which has a staircase shape and is generated along the edges
of the image. Also, ringing noise is a typical Gibb's phenomenon occurring by
truncation when the coefficients of the DCT are quantized so as to highly
compress the image.
Grid noise shows traces of the block-based process at the edges
between blocks when the compressed data is displayed on a screen after being
restored. Thus, one can identify the edges between blocks. Also, staircase
noise has a staircase shape at the edges of the image, so that one carp notice
a
bumpy edge on the image. Also, one can notice overlapping of images with a
predetermined interval due to the ringing noise.
In order to reduce the blocking effect and the ringing noise
occurring when block-based coding is performed, several methods have
been suggested. According to Fi.261 encoding, a simple 3 X 3 low-pass
filter (LPF) is used as a Loop Biter to reduce the blocking effect ["Video
Codec for Audiovisual Services at P X 62 kbitls°', . . . . . . . .
CA 02440007 2003-09-11
2
CCIT~',Recommendation H.261, December 14, 1990]. Also, a simple edge loop
filter has been suggested so as to reduce the blocking effect and mosquito
noise
[G.Bjontegaard, "A Simple Edge Loop Filter to Reduce Blocking and llfosqacito
Noise", ISOIIEC JTCIlSc29/VJG11 I~fPEG96/0617, 3anuary, 1996, and ".4 Simpde
S Edge Loop Filter to Reduce Blocking and .Ivlosquito Noise" , ITU SC 1 S hBC
Expert
Group ITU-LBC-96-032, January, 1996). The edge loop filter makes linear values
of two pixels adjacent to the block boundary and replaces the two pixel values
by
the linearized values. Such edge loop filter can reduce the blocking effect
but not
the ringing noise. In order to reduce the ringing noise, a non-linear filter
using a
binary index has been suggested [Y.Itoh, "Detail Preserving Nonlinear Filter
using
Binary Index, "ISO/IEC JTCIfSC29/WG11 MPEG9Sl03S7, hlovember, 1995].
However, the non-linear filter cannot reduce the blocking effect.
Disclosure of the Invention
To solve the above problems, it is an object of the present invention is to
provide an signal adaptive filtering method for reducing blocking effect and
ringing
noise in high compression encoding, a signal adaptive filter, and a computer
readable medium.
According to an aspect of the present invention, there is provided a signal
adaptive filtering method capable of..reducing blocking effect and ringing
noise of
image data when a frame is coanposed of blocks of a predetermined siae, the
method comprising the steps of: (a~ generating blocking information for.
reducing
the blocking effect and ringing information for reducing, the ringing noise,
from
coefficients of predetermined pixels of the upper and left boundary regions of
the
data block when a frame obtained by deconstructing a bitstream iraage data for
inverse quantization is an intraframe; and (b) adaptively filtering the image
data
passed through inverse quantizadon and inverse discrete cosine transform
according
to the generated blocking information and ringing information:
Preferably; the step (a) further comprises the step of setting the blocking
information and ringing information of the previous frame corresponding to a
motion vector as blocking information and ringing information of the current
frame
if the frame is an interframe, and setting the ringing information to
°° 1'° which
CA 02440007 2003-09-11
3
represents the image data requiring filtering if a residual signal of the
inverse
quantized current block exists, and the blocking information and ringing
information are determined according to coefficients of a pixel A 1~cated at
the
upper left corner of the block, a pixel B located to the right of the pixel A
and a
pixel C located below the pixel A.
Preferably, the blocking information is constituted of horizontal blocking
information and vertical blocking information, and the horizontal blocking
information is set to "1 °' which means the image data requiring
filtering when only
the coefficient of the pixel A is not equal to "0" or any coefficient of the
pixels of
the left boundary region of the block is not equal to "0", and the vertical
blocking '
information is set to "1" which means the image data requiring filtering when
only
the coefficient of~the pixel .~ is not equal to "0" or any coefficient of the
pixels of
the upper boundary region of the block is not equal to "0", and the ringing
information is set to "1" which means the image data requiring filtering when
any
coefficient of the pixels other than the pixels A, B and C of the block is not
equal
to "0"
Preferably, the blocking information is constituted of horizontal blocking
information and vertical blocking information, and the horizontal blocking
information is set to " 1 " which means the image data requiring .filtering
when all
coefficients of the pixels A, B and C of the block are not equal to
"0°' or any
coefficient of the pixels of the left boundary region of the block is n~t
equal to "0",
and flee vertical blocking information is set to °' 1 " w~rhich means
the image data
. requiring f ltering when a1i coefficients of the pixels A, B and . C are not
equal to
"0" or any coefficient of the pixels of the upper boundary region of the block
is not
BS equal to "0", and the ringing information is set to "1" which means the
image data
requiring filtering when any coefficient of the pixels other than the pixels
A, B and
C of the block is not equal to "~".
Preferably, in order to reduce the blocking effect, in the step (b), the
horizontal (or vertical) filtering is performed using a weighted filter having
a
predetermined weighted value when the horizontal (or vertical) blocking
information of the block is equal to "1" and the ringing information is equal
to "0",
and when the horizontal (or vertical) blocking information of the block is not
equal
CA 02440007 2003-09-11
4
to "1" or the ringing information is not squat to "~", an absolute value of
the
difference between adjacent pixels and a Q value used as a dividend for
quantizing
the block. are compared, and then filtering is performed with a predetermined
value
according to the result of the comparison.
According to another aspect of the present irsvention, there is provided a
signal adaptive filter capable of reducing blocking effect and ringing noise
of image
data when a frame is composed of blocks of a predeterrrained size, comprising:
a
mode flag checking unit for checking a flag to determine whether or not a
frame
is an intraframe or an interframe when a bitstream image data is deconstructed
for
inverse quantization; an infra filtering inforrmation generator fir generating
blocking
information for reducing the blocking effect and ringing information for
reducing
the ringing noise, from coefficients of predetermined pixels of the upper and
left
boundary regions of the data block when the frame is , determined as an
intraframe
by the mode flag checking unit; an inter filtering information generator for
setting .
the blocking information and ringing information of the previous frame
corresponding to a motion vector as blocking information and ringing
information
of the current frame if the frame is an interframe, and setting the ringing
information to "1" if a residual signal of the inverse-quantized current block
exists;
and an adaptive filter for adaptively filtering the image data passed through
an
inverse quantizer and an inverse discrete cosine transformer according to the
blocking information and the ringing information generated by the infra
filtering
information generator and the inter filtering information generator.
The invention may be embodied in a general purpose digital computer that
as running a program from a computer usable medium, including but not limited
to
storage media such as magnetic storage media (e.g., 12~M's, floppy disks, hard
disks, etc.), optically readahle media (e.g., CI?-It~Ms, ~VI)s, etc.) and
carrier
waves (e.g., transmissions over the Internet). Hence, the pxesent invention
may
be embodied as a computer usable medium.
According to still another aspect of the present invention, there is provided
a computer readable medium having embodied thereon a computer program for a
signal adaptive filtering capable of reducing blocking effect and ringing
noise of
image data when a frame is composed of blocks of a predetermined size, wherein
CA 02440007 2003-09-11
. . .
the signal adaptive filtering comprises the steps of: (a) generating blocking
information for reducing the blacking effect and ringing information for
reducing
the ringing noise, from coefficients of predetermined pixels of the upper and
left
boundary regions of the data block when a frame obtained by deconstructing a
bitstrearn image data for inverse quantization is an intraframe; (b) setting
the
blocking .information and ringing information of the previous frame
corresponding
to a motion vector as blocking information and ringing information of the
current
frame if the frame is an interframe, and setting the ringing infornnation to
"1"
which represents the image data requiring filtering if a residual signal of
the
inverse-quantized current block exists; and (c) adaptively filtering the image
data
passed through the inverse quantization and inverse discrete cosine transform
based
on the block according to the generated bloekang information and ringing
information.
Also, there is provided a computer readable medium having embodied
thereon a computer program for a method of filtering ringing noise caused when
decoding a block-based compressed image data, wherein the ringing noise
frltering
method comprises the steps of: (a) performing a gradient operation on the
block
subjected to inverse quantization and inverse discrete cosine transform using
predetermined one-dimensional horizontal and vertical gradient operators; (b)
generating a binary edge reap representing whether or not each pixel. is an
edge
pixel, using an absolute value of the difference between the gradient-operated
value
of one pixel arid the value of the adjacent pixel, and a Q value used as a
dividend
for quantizing the block; and (c) performing a faltering by applying a
predetermined
filter window to the generated b° edge map.
brief Description of the Drawings
FIG. 1 is a block diagrarra of a signal adaptive filter for reducing blocking
effect and ringing noise according to the present invention;
FIG. 2 is a flowchart illustrating a signal adaptive filtering method
according
to a preferred embodiment of the present invention;
FIG. 3 shows an inverse-quantized block having 8 x8 pixels;
FIG. 4 is a flowchart illustrating the step of generating information used to
CA 02440007 2003-09-11
6
filter an intraframe;
FIG. 5 is a flowchart illustrating the step of generating information used to
filter an interframe;
FIG. b shows the location of pixels adjacent to the block boundary for
illustrating the filtering for reducing blocking effect; and
FIG. 7 shows the location of pixels to be processed in the current block-
Best mode for carrying out the Invention . ~~
In FIG. I, a signal adaptive filter for reducing blocking effect and ringing
noise according to the present invention includes a mode flag checking unit
120,
an infra filtering information generator 130,-an inter filtering information
generator
140 and an adaptive filter unit 150. When bitstream image data is
deconstructed
for inverse quantization, the anode flag checking unit 120 checks whether or
not the
frame is an infra frame or an inter frame. T'he infra filtering inf~rmation
generator
130 generates blocking iriforrnation for reducing blocking effect and ringing
information from coefficients of predetermined pixels of the upper and left
boundary regions of the data block when the frame is determined as an infra
frame
by the mode flag checking unit 120. 'then the mode , flag checking unit 120
determines the frame as an inter frame, the inter filtering information
generator 140
generates blocking information and ringing information of the previous frame
corresponding to a motion vector as blocking information and ringing
information
of the current frame. If a residual signal of the inverse-quantized. current
block
exists, the ringing information is set to " I °' . ~'he adaptive
filtering unit 150
adaptively filters the image data of the block which has passed an inverse
quantizer
(Q-') 100 and an inverse discrete cosine transformer (I)CT-' ) l 10 according-
to the
blocking information and ringing information generated by the infra filtering
information generator I30 and the inter filtering information generator 140.
Meanwhile, 'a signal adaptive filtering method according to a preferred
embodiment of the present invention will be described. FIG. 2 is a flowchart
illustrating the signal adaptive filtering method according to the present
invention:
Bitstream image data coded by an encoder is decoded by a decoder to be
reproduced. To end this, the bitstream data is deconstructed and then inverse-
CA 02440007 2003-09-11
7
quantized by the inverse quantizer 1a0 {step 200). I-Iere, the image data is
constituted of a plurality of frames, and each frame is constituted of a
plurality of
blocks. FIG. 3 shows an inverse-quantized block having 8x8 pixels which form
the frame.
Before filtering the frame data with the inverse discrete cosine transform
(IDCT), a flag is checked in order to make a determination as to whether or
not the
frame is an intraframe or an interframe (step 2.10). If the frame is an
intraframe
(step 220), information used to filter the intraframe is generated (step 230).
If the
frame is an interframe, information used to filter the interframe is generated
(step
240). Then, the frame data which has passed through the II~CT l I0 are
adaptively
filtered according to the genes ated filtering information, thereby
eliminating
blocking effect and ringing noise {step 250).
FIG. 4 is a flowchart illustrating in detail the step of generating
information
used to filter the intraframe. As shown in FIG. 4, if the frame is determined
as an
intraframe by the mode flag checking unit 120, the coefficient of pixel A of
FIG.
3 is checked (step 400). If only the coefficient of the pixel A is not equal
to "0",
horizontal blocking information {HIBI) and vertical blocking information
(~'BI) are
set to "1" (step 410). If any coefficient of pixels {8 pixels including the
pixels A
anti B) belonging to the upper boundary region 300 of the block shown in FIG.
3
is not equal to "0" (step 420), the VBI is set w "1'° (step 430).
Otherwise, the VBI
is set to "0" (seep 440). Also, if any coefficient of pixels (8 pixels
including the
pixels A and C) belonging to the left boundary region 310 of the block shown
in
FIG. 3 is not equal to "0" {step 450), the I-IBI is set Iro '° 1 "
(step 460). Otherwise,
the HBI is set to "0" (seep 470) .
After the IiBI and VBI are set, ringing information (RI) used to filter the
ringing noise is generated. That is, if any coefficient of pixels other than
the pixels
A, B and C of the block shown in FIG. 3 is not equal to "0°' (step
480), the RI is
set to "1" (step 490). Otherwise, the RI is set to "0" {step 495). I-iere, the
IIBI
and VBI are set to " 1 " when only the coefficient of the pixel A is not equal
to °' 0 °'
(step 400). I-Iowever, by setting the HBI and VBI to '° 1 " even if all
coefficients of
the pixels A, B and C are not equal to °°0°', favorable
effect to some extent can be
obtained when a signal adaptive filtering is performed later.
CA 02440007 2003-09-11
~'S
FIG. 5 is a flowchart illustrating the step of generating information used to
filter the interframe. If the frame is determined as an interframe by the mode
flag
checking unit 120, the HBI, VBI and RI of the intraframe are transferred to
HBI,
VBI and RI of the interframe according tc~ the motion vector (step 500).
l~Iso, if
a residual signal exists after motion compensation (step 510), the I2I is
updated
(step 520).
When the blocking information and ringing. information for filtering are
generated as described above, filtering is adaptively performed according to
the
information: ~ First, a filtering method for reducing blocking effect will be
described. The filtering for reducing the blocking effect is classified as one
of
horizontal filtering and vertical filtering. Here, the horizontal' filtering
will be
explained. FIG. 6 shows the location of pixels adjacent to the block boundary,
for
illustrating the filtering for reducing the blocking effect. A determination
is made
as to whether or not the I~hI and RI of blocks I and J~ of FIG. 6 are equal to
"0".
i5 If the IiBI and RI of blocks I and J of FIG. 6 are equal to "0", a weighted
filtering
is performed on pixels A, B, C, D, E and F of FIG. 6 using a 7-tap { 1,1,1, 2
,1,1,1
low-pass filter (I:.PF).
If the HBI or the RI of blocks I and J of FIG. 6 is not equal to
°'0", a
fcltering is performed on the pixels B; C, D and E using the following
algorithm.
d=D~--C;
If (ABS(d) < Q) f
D= D - (d!2); ~ C = C + (d!2);
d = E~_ D;
if (ABS(d) ~ Q ) E = E - (d!4);
d=C-B;
if (ABS(d) ~ Q) B = B -+- (d/4);
else {
if (ABS(d/2) <_ 2Q)
If (d > 0) {
9
CA 02440007 2003-09-11
9
C = C + (Q - ABS(dl2));
else ~
D = D + {Q - ABS{d/2));
C = C + (Q - ABS(d/2));
d=E_D;
if (ABS{d) <_ Q) E = E - (d!4).;
d=C_B
if (ABS(d) _< Q) B = I3 -- (d/4);
In the above algorithm, ABS represents an absolute value, and Q represents
a dividend used when the blocks forming the frame are quantized.
In detail, the absolute value (ABS{d)) of the difference (d) between the
pixels D and C is equal to or less than Q, the current pixel value of the
pixel D is
set by subtracting d!2 from the current pixel value, 'and the current pixel
value of .
the pixel C is set by adding d/2 to the current pixel value. Also, the
absolute value
(ABS(d)) of the difference (d) between the pixels 1E and D is equal fo or less
than
2U Q, the current pixel value of the pixels is set by subtracting d14. from
the current
pixel value. Also, if the absolute value (ABS(d)) of the difference (d)
between the
pixels C and B is equal to or less than Q, the current pixel values of the
pixel B is
set by subtracting d/4 from the current pixel valu~a. In the same manner as
the
above, the pixel values of the pixels B, C, I~ and E are set according to the
algorithm other than the above, which is obvious to those skilled in the art,
thus the
explanation thereof will be omitted. Also, vertical filtering is performed
according
to the same principle as the horizontal filtering.
i~Text, a faltering method for reducing ringing noise will be described.
First,
the generated ringing inforrnatior~ is checked. If the ringing information is
set to
°' 1 ", the filtering is performed. Dtherwise, the filtering is not
performed. To end
this, edge pixels of the block which have been subjected to inverse
quantization and
IDCT are determined. In order to determine the edge pixels, a gradient
operation
CA 02440007 2003-09-11
1~
is performed on the blocks which have been subjected to inverse quantization
and
IDCT using one-dimensional horizontal and vertical gradient operators.
'Then, an absolute value of the difference between one gradient-operated
pixel value and the adjacent pixel value, and Q value used as a dividend when
quantizing the block, are used to generate a binary edge snap representing the
edge
of each pixel. here, the block has 8 x 8 pixels, and the size of the binary
edge map
is represented as a two-diaiaensional array edge[10][ld] as shown in FICa. 7.
In order to generate the binary edge map, 'vertical edge detection and
horizontal edge detection are performed. Algoritl~ns for vertical edge and
horizontal edge defections are as follows.
I * Vertical edge detection ~'I
Al = ABS(PtrIW age[0] - Ptrlmage[I));
A2 = (ABS(PtrImage[0] - PtrImage[-1]);
is if (((A1 > Th)(A2 > T~)) I I (A1 > s~Th~z) I l (~> > ~*Th/~))
Edge[m][n] = Ii; / ~ edge */
For the vertical edge detection, an absolute value {Al) of the difference
25 between the gradient-operated results of the pixel {Ptrlanage[0]) on which
a
determination as to whether or not the pixel is an edge of flee block is made,
and
the right pixel (PtrImage[I]) of the pixel (PtrImage[4j) is calculated.
°I°hen, an
°" ~ absolute value {A2) ~f the difference between the gradient-
operated results of the w
PtrImage[0) and the left pixel (Ptrlmage(-lJ) of the pixel {PtrImage[~]) is
30 calculated. Then, a determination as to whether or not the pixel is an edge
is made
accot~ding to the logic values obtained after the absolute values A1 and A2
are.
compared with a predetermined threshold value Th, and then the above process
is
CA 02440007 2003-09-11
1I
performed on all pixels of the block. 'rhe vertical edge detection is
performed
according to a logical forrnuia of (AI > Th)&&(A2 > Th) ! ~ (A1 > 5*Th~2) ( ~
(A2) > 5*T'hl2). 1f the logical formula is true, the pixel is determined as a
vertical
edge. ~therwise, the pixel is determined to not be a vertical edge.
The horizontal edge detection is performed according to the same: principle
of the horizontal edge detection. First, an absolute value (A'1) of the
difference
between ttae gradient-operated results of the pixel (PtrImage[0]) on which a
determination as to whether or not the pixel is edge of the block is made, and
the
Lower pixel (I'trImage[width]) of the pixel (Ptrimage[0]) is calculated. Then,
an
absolute value (A'2) of the difference between the gradient-operated results
of the
pixel (PtrImage[0]) and the upper pixel (PfrImage[-width]) of the pixel
(Ptrlmage[0]) is calculated. 'Then, a determination as to whether or not the
pixel
is an edge is made according to the logic values obtained after the absolute
values
A' 1 and A'2 are compared with a predetermined threshold value Th, and then
the
above process is performed on all pixels of the block. The horizontal edge
detection is performed according to a logical formula of
(A' 1 > Th)&&(A'2 > Th) j ~ (A' 1 > 5*Thl2) ~ ~ (A'2) > 5*'I'hl2). If the
logical formula
is true, the pixel is determined as a horizontal edge. Gtherwise, the pixel is
determined to not be a horizontal edge. I-Iere, '~&' represents logical ANi~,
and
' J l ' represents logical ~R.
Next, f ltering is performed by applying a predetermined filter window to
the generated binary edge map_ The filtering may be performed by a general
filtering method by applying a filter window having a predetermined size:
However, in this embodiment, filtering is not performed if the central pixel
of the
filter window is an edge, while the filtering is performed if the central
pixel is not
an edge. The filter window may be a general filter window,. In this
embodiment,
a 4-connectivity f lter window having five pixels arranged in a cross shape
centering one central pixel, as shown in FIG. 7, is used. In FIG. 7, ~
represents
an edge pixel, and the regions other than the regions with "%" represent non-
edge
pixels.
Also, if the filter window has no edge pixel, ordinary filtering is performed,
while weighted filtering is performed if the edge pixel exists. An example of
CA 02440007 2003-09-11
z2
weighted filtering is shown in FIG. 7. In F'IG_ 7, " < < " represents a shift
to the
left, and " > > " represents a shift to the right.
The invention may be embodied in a general purpose digital computer that
is running a p~_ogram from a computer usable medium, including but not limited
to
storage media such as magnetic storage media (e.g., RGNI's, floppy disks, hard
disks, etc.), optically readable media (e.g., CD-ROMs, DVDs, etc.) and carrier
waves (e.g., transmissions over the Internet). Hence, the present invention
may
be embodied as a computer usable medium having a computer readable program
code unit embodied therein for signal adaptive filtering, the computer
readable
program code means in the computer usable medium comprising: computer
readable program code means for causing a computex to effect generating
blocking
information for reducing the blocking effect and ringing information for
reducing
the ringing noise, from coefficients of predetermined pixels of the upper and
left
boundary regions of the data block when a frame obtained by deconstructing a
bitstream image data for inverse quantization is an intraframe; computer
readable
program code means for causing a computer to effect setting the blocking
information and ringing information of the previous frame corresponding to a
motion vector as blocking information and ringing information of the current
frame
if the frame is an rote e, and setting the ringing information to "1" which
represents the image data requiring filtering if a residual signal of the
inverse-
quanti2ed current block exists; and computer readable program code means for
causing a computer to effect adaptively filtering the image data passed
through the
inverse quantization and inverse discrete cosine transform based on the biocle
according to the generated blocking information and ringing information, for
instance. A functional program, code and code segments, used to implement the
present invention can be derived by a skilled computer programmer from the
description of the invention contained herein.
Industrial A.pplicabilit~r
As described above, the present invention can remove the blocking noise
and the ringing noise from an image restored from an image compressed based on
blocks, thereby enhancing the image restored from compression.
