Note: Descriptions are shown in the official language in which they were submitted.
CA 02440013 2003-09-11
1
This application is a division of Canadian Application I~o. 2,415,375 filed
June 18, 1998 for Signal Adaptive Filtering nllethod, Signa! Adaptive Filter
and
Computer Readable ll~edium for Storing Program Therefor.
Technical Field
The present invention relates to data filtering, and more particularly, to a
signal adaptive filtering method for reducing blocking effect and ringing
noise, a
signal adaptive filter and a computer readable medium for storing a program
therefor.
Background Art
Generally, picture encoding standards such as MPEG of the International
Standardization Organization (ISO) and H.263 recommended by the
International Telecommunication Union (ITU) adopt block-based motion
estimation and discrete cosine transform (DCT) blocks. V1lhen an image is
highly compressed, the block-based coding may cause the well-known blocking
effect. A typical blocking effect is grid noise in a homogeneous area in which
adjacent pixels have relatively similar pixel values. Another blocking effect
is
staircase noise which has a staircase shape and is generated along the edges
of the image. Also, ringing noise is a typical Gibb's phenomenon occurring by
truncation when the coefficients of the DCT are quantized so as to highly
compress the image.
Grid noise shows traces of the block-based process at the edges
between blocks when the compressed data is displayed on a screen after being
restored. Thus, one can identify the edges between blocks. Also, staircase
noise has a staircase shape at the edges of the image, so that one can notice
a
bumpy edge on the image. Also, one can notice overlapping of images with a
predetermined interval due to the ringing noise.
In order to reduce the blocking effect and the ringing noise
occurring when block-based coding is performed, several methods have
been suggested. According to H.261 encoding, a simple 3 ~ 3 low-pass
filter (LPF) is used as a loop filter to reduce the blocking effect
[°'Video
Codes for Audiovisual Services at f' X 62 kbitls'°, . . . . . . ,
CA 02440013 2003-09-11
CCITT Recommendation H.~6I, December 14~, 3990]. Also, a simple edge loop
filter has been suggested . so as to reduce the blocking effect and mosquito
noise
(G.Bjontegaard, "A Simple Fdge Loop Filter to Reduce .~lockir~g cznd
Ivlosquito
Noise", ISOJIEC TTCIJSc29/WG11 I~iFEG96/0617, January, 1996, and '°A
Simple
S Edge Loop Filter to Reduce docking rznd .Mosquito Noise", ITU SC15 LBC
Expert
Group ITU-LBC-96-032, .Ianuary, 1996]. The edge loop filter makes linear
values
of two pixels adjacent to the block boundary and replaces the two pixel values
by
the linearized values. Such edge loop filter can reduce the blocking effect
but not
the ringing noise. In order to reduce the ringing noise, a non-linear filter
using a
binary index has been suggested [Y.Itoh, "Detail Preserving Nonlinear Filter
using
Binary Index, "ISUJIEC JTC1lSC291WCa11 Mfl'EG95/0357, November, 1995j.
However, the non-linear filter cannot reduce the blocking effect.
Disclosure of the Invention
To solve the above problems, it is an object of the present invention is to
provide an signal adaptive filtering method for reducing blocking effect and
ringing
noise in high compression encoding, a signal adaptive filter, and ~a computer
readable medium.
According to an aspect of the present invention, there is provided a signal
adaptive filtering method capable of.reducing blocking effect and ringing
noise of
image data when a frame is composed of blocks of a predetermined size, the
method comprising the steps of. (a) generating blocking information for
reducing
the blocking effect and ringing information for reducing the ringing noise,
from
coefficients of predetermined pixels of the upper and left boundary regions of
the
data block when a frame obtained by deconstructing a bitstream image data for
inverse quantizarion is an intraframes and (b) adaptively filtering the image
data
passed through inverse quantization and inverse discrete cosine transform
according
to the generated blocking information and ringing information:
Preferably the step (a) further comprises the step of setting the blocking
information and ringing information of the previous frame corresponding to a
morion vector as blocking information and ringing information of the-current
frame
if the frame is an interframe, and setting the ringing information to " 1 "
which
CA 02440013 2003-09-11
3
represents the image data requiring filtering if a residual signal of the
inverse-
quantized current block exists, and the blocking information and ringing
information are determined according to coefficients of a pixel A located at
the
upper left comer of the block, a pixel ft located to the right of the pixel A
and a
pixel C located below the pixel A.
Preferably, the blocking information is constituted of horizontal blocking
information and vertical blocking information, and the horizontal blocking
information is set to "1'° which means the image data requiring
filtering when only
the coefficient of the pixel A is not equal to "0" or any coefficient of the
pixels of
the left boundary region of the block is not equal to °'0", and the
vertical blocking
information is set to " 1 " which means the image data requiring filtering
when only
the coefficient of~the pixel A is not equal to "0'° or any coefficient
of the pixels of
the upper boundary region of the block is not equal to "0", and the ringing
information is set to "1" which means the image data requiring filtering when
any
coe~cient of the pixels other than the pixels A, P and C of the block is not
equal
to "0" .
Preferably, the blocking information is constituted of horizontal blocking
information and vertical blocking information, and the horizontal blocking
information is set to "1" which means the image data requiring .filtering when
all
coefficients of the pixels A, ~ and C of the block are not equal to "0" or any
coefficient of the pixels of the left boundary region of the block is nbt
equal _ to "0" ,
and tie vertical blocking information is set to °' 9.'" which means the
image data
requiring filtering when all coefficients of the pixels A, B and.C are not
equal to
"0" or any coefficient. of the pixels of the upper boundary region of the
block is not
equal too "0", and the ringing information is set to "1" which means the image
data
requiring filtering when any coefficient of the pixels other than the pixels
A; B and
C of the block is not equal to "~" .
Preferably, in order to reduce the blocking effect, in the step (b), the
horizontal (or vertical) filtering is performed using a weighted filter having
a
predetermined weighted value when the horizontal (or vertical) blocking
information of the block is equal to " 1 " and the ringing information is
equal to "0",
and when the horizontal (or vertical) blocking information of the block is not
equal
CA 02440013 2003-09-11
to °' 1 " or the ringing inforanation is not equal to °'0" , an
absolute value of the
difference between adjacent pixels and a Q value used as a dividend for
quantizing
the block are compared, and then filtering is performed with a predetermined
value
according to the result of the comparison.
According to another aspect of the present invention, there is provided a
signal adaptive filter capable of reducing blocking effect- and ringing noise
of image
data when a frame is composed of blocks of a predetermined size, comprising: a
mode flag checking unit for checking a flag to determine whether or not a
frame
is an intraframe or an interframe when a bitstream image data is deconstructed
for
inverse quantization; an infra filtering information generator for generating
blocking
information for reducing the Mocking effect and zinging information for
reducing
the ringing noise, from coefficients of predetermined pixels of the upper and
left
boundary regions of the data block when the frame is . determined as an
intraframe
by the mode flag checking unit; an inter filtering information generator for
setting
the blocking information and ringing . information of the previous frame
corresponding to a motion vector as blocking information and ringing
information
of the current frame if the frame is an interframe, and setting the ringing
information to "i" if a residual signal of the inverse-quantized current block
exists;
and an adaptive filter for adaptively filtering the image data passed through
an
inverse quantizer and an inverse discrete cosine transformer according to the
blocking information and the ringing information generated by the infra
filtering
information generator and the inter faltering information generator.
The invention may be embodied in a general purpose digital computer that
is running a program from a computer usable medium, including but ndt limited
to
storage media such as magnetic storage media (e.g., i2~M's, floppy disks, hard
disks, etc.), optically readable media (e.g., CIA-It~Ms, L)V'Ds, etc.) and
carrier
waves (e.g., transmissions over the Internet). kience, the present invention
may
be embodied as a computer usable medium.
According eo still another aspect of the present invention, there is provided
a computer readable medium having embodied thereon a computer program for a
signal adaptive filtering capable of reducing blocking effect and ringing
noise of
image data when a frame is composed of blocks of a predetermined size, wherein
CA 02440013 2003-09-11
the signal adaptive filtering comprises the steps of: (a) generating blocking
information for reducing the blocking effect and ringing information for
reducing
the ringing noise, from coefficients of predetermined pixels of the upper and
left
boundary regions of the data block when a frame obtained by deconstructing a
bitstream image data for inverse quantization is ~an intraframe; (b) setting
the
blocking information and ringing information of the previous frame
corresponding
to a motion vector as blocking information and ringing information of the cura-
ent
frame if the frame is an interframe, and setting the ringing informati~n 'to
"1"
which represents the image data requiring filtering if a residual signal of
the
inverse-quantized current block exists; and (c) adaptively filtering the image
data
passed through the inverse quantization and inverse discrete cosine transform
based
on the block according to the generated blocking information and ringing
information.
Also, there is provided a computer readable medium having embodied
thereon a computer program for a method of filtering ringing noise caused when
decoding a block-based compressed image data, wherein the ringing noisy
filtering
method comprises the steps of: Via) performing a gradient operation on the
block
subjected to inverse tpuantization and inverse discrete cosine transform using
predetermined one-dimensional horizontal and vertical gradient operators; (b)
generating a binary edge map representing whether or not each pixel is an edge
pixel, using an absolute value of the d'iff'erence betv~een the gradient-
operated value
of one pixel azid the value of the adjacent pixel, and a C2 value used as a
dividend
for quantizing the block; and (c) performing a filtering by applying a
predetermined
Biter window to the generated binary edge map.
Brief Description of the Drawings
FIG. 1 is a block diagram of a signal adaptive f°~lter for reducing
blocking
effect and ringing noise according to the present invention;
FIG. 2 is a flowchart illustrating a signal adaptive filtering method
according
to a preferred embodiment of the present invention;
FIG. 3 shows an inverse-quantized block having 8 x 8 pixels
FIG: 4 is a flowchart illustrating the step of generating information used to
CA 02440013 2003-09-11
6
filter an intraframe;
FIG. 5 is a flowchart illustrating the step of generating inforrr~ation used
to
filter an interframe;
FIG. 6 shows the location of pmxels adjacent to the block boundary for
illustrating the filtering for reducing blocking effect; and
FIG. 7 shows the location of pixels to be processed in the current block.
Best mode for carr~rin~ out the Invention _
In FIG. 1, a signal adaptive filter for reducing blocking effect and ringing
noise according to the present invention includes a mode flag checking unit
120,
an infra filtering information generator 130, an inter filtering information
generator
140 and an adaptive filter unit 1~0. '6~hen bitstx~ea~n image data is
deconstructed
for inverse quantization, the mode flag checking unit 120 checks whether or
not the
frame is an infra frame or an inter fine. The infra filtering information
generator
130 generates blocking information for reducing blocking effect and ringing
information from coefficients of predetermined pixels of the upper and left
boundary regions of the data block when the frame is determined as an infra
frame
by the mode flag checking unit 120. then the mode flag checking unit 120
determines the frame as an inter frame, the inter filtering information
generator 140
generates blocking anformatioar and ringing information of the previous frame
corresponding to a motion vector as blocking information and ringing
information
of the current -frame. If a residual signal of the: inverse-quantized current
block
exists, the ringing information is set to "1". a adaptive filtering unit 150
adaptively filters the image data of the block which has passed an inverse
quantizer
(Q-') 100 and an inverse discrete cosine transformer (I)CT'-' ) i 10 according
to the
blocking information and ringing information generated by the infra filtering
information generator 130 and the inter, filtering information generator 140.
Meanwhile, .a signal adaptive filtering method according to a preferred
embodiment of the present ynvention will be described. FIG. 2 is a flowchart
illustrating the signal adaptive filtering method according to the present
invention.
Bitstream image data coded by an encoder is decoded by a decoder to be
reproduced. To end this, the bitstream data is deconstructed and then inverse-
CA 02440013 2003-09-11
quantized by the inverse quantizer 100 (step 200). I-Iere, the image data is
constituted of a plurality of frames, and each frame is constituted of a
plurality of
blocks. FIG. 3 shows an inverse-quantized block having ~ x8 pixels which form
the frame.
S Before filtering the frame data with the inverse discrete cosine transform
(IDCT), a flag is checked in order to make a determination as to whether or
not the
frame is an intraframe or an interframe (step 2I0). If the frame is an
intraframe
(step 220), information used to filter the intraframe is generated (step 230).
If the
frame is an interfrar~ae, information used to filter the interframe is
generated {step
I0 240). Then, the frame data which has passed through the IDCT 110 are
adaptively
filtered according to the gene: ated filtering inforrrtation, thereby
eliminating
blocking effect and ringing noise (step 250).
FIG. 4 is a flowchart illustrating in detail the step ~f generating
information
used to filter the intraframe. As shown in FIG. 4, if the frame is determined
as an
15 intraframe by the anode flag checking unit 120, the coefficient of pixel A
of FIG.
3 is checked (step 400). If only the coefficient of the pixel A is not equal
to "0",
horizontal blocking information (HBI) and vertical blocking information (VBI)
are
set to "1" {step 41~). If any coefficient of pixels (~ pixels including the
pixels A
and B) belonging to the upper boundary region 3of the block shown in FICi. 3
20 is not equal to "0" (step 420), the VBI is set to "1 ~' (step 430).
Otherwise, the VBI
is set to "0" (seep 440). Also, if any coefficient of pixels {8 pixels
including the
pixels A and C) belonging to the left boundary region 310 of the block shown
in
FIG. 3 is not equal to '°0" {step 450), the HBI is.set to "1" (step
460). Otherwise,
the HBI is set to "~" (seep 470).
25 After the HBI and I are set, ringing information {ItI) used to filter the
ringing noise is generated. t is, if any coe~cient of pixels other than the
pixels
A, B and C of the block shown in FI(i. 3 is not equal to "0" {step 4~0); the
RI is
set to °'I" {step 490). Otherwise, the I2T is set to "0" {step 495).
Here, the HBI
and VBI are set to "1'° when only flee coefficient of the pixel A is
not equal to "0"
30 (step 400). However, by setting the I and VBI to °' I " even if all
coefficients of
the pixels A, B and C are not equal to '°0°', favorable effect
to some extent can be
obtained when a signal adaptive filtering is performed later.
CA 02440013 2003-09-11
g
FIG. S is a flowchart illustrating the step of generating inforrtr,ation used
to
falter the interframe. If the frame is determined as an interframe by the mode
flag
checking unit 120, the HBI, ~J'BI and It.I of the intraframe are transferred
to IiBI,
VBI and RI of the interframe according to the motion vector (step S00). Also,
if
S a residual signal exists after motion compensation (step 510), the laI is
updated
(step S20).
~Ihen the blocking information and ringing information for filtering are
generated as described above, filtering is adaptively performed according to
the
information. First, a filtering method for reducing blocking effect will be
described. °The filtering for reducing the blocking effect is
classified as one of
horizontal filtering and vertical filtering. Flere, the horizontal filtering
will be
explained. FIG. b shows the location of pixels adjacent to the block boundary,
for
illustrating the filtering for reducing the blocking effect. A determination
is made
as to whether or not the HBI and R.I of blocks I and J of F$G. 6 are equal to
''0".
iS If the HBI and RI of blocks I and J of FIG. 6 are e~.qual to "~", a
weighted filtering
is performed on pixels A, B, C, IJ, E and F of FIG. 6 using a 7-tap
(1,1,1,2,1,1,1)
low-pass filter (I,PF).
If the HBI or the ItI of blocks I and 3 cof FIG. 6 is not equal to
"0'°, a
filtering is performed on the pixels B, C, I3 and F using the following
algorithm.
d=D-C;
If (ABS(d) S Q) {
I~= D - (dl2); C = C + (dl2);
2S if (ABS(d) ~ ) E = E - (dl4);
d=C--°B;
if (ABS(d) <_ ~) B = B + (dl4);
else {
if (ABS(d/2) < 2Q)
If (d > 0) {
-. dl2)) 9
CA 02440013 2003-09-11
9
C = C + {Q - ABS(d/2));
else {
D = D -t- {Q - ABS(d/2});
S C = C + (Q - ABS(d/2));
d~E_D~
if (ABS(d) ~ Q) E = E - (d/4);
d=C-By
IO if (ABS(d) < Q) B = B - (d/4);
In the above algorithm, ABS represents an absolute value, and Q represents
a dividend used when the blocks forming the frame are quantized.
IS In detail, the absolute value (ABS{d)) of the difference (d) between the
pixels D and C is equal to or less than Q, the current pixel value of the
pixel D is
set by subtracting dl2 from the current pixel value, and the current pixel
value of
the pixel C is set by adding dt2 to the current pixel value. Also, the
absolute value
(ABS(d)) of the difference (d) between the pixels E and D is equal to or less
than
20 Q, the current pixel value of the pixels is set by subtracting d~4 from the
current
pixtl value. Also, if the absolute value (ABS(d)) of the difference (d)
between the
pixels C and B is equal to or less than Q, the current pixel values of the
pixel B is
set by subtracting d14 fr~an the current pixel value. In the same ynanner as
the
above, the pixel values of the pixels B, C, D and E are set according to the
25 algorithm other than the above, which is obvious to those skilled in the
art, thus the
explanation thereof will be omitted. Also, vertical filtering is performed
according
to the same principle as the horizontal faltering.
1'lext, a filtering method for reducing ringing noise will be described.
First,
the generated ringing inforrnadon is checked. If the ringing information is
'set to
30 "I", the filtering is performed. C,therwise, the filtering is not
performed. 'lo end
this, edge pixels of the block which have been sub3ected to inverse
quantization and
IDC'T are determined. hi order to determine flee edge pixels, a gradient
operation
CA 02440013 2003-09-11
is performed on the blocks which have been subjected to inverse quantization
and
IDCT using one-dimensional horizontal and vertical gradient operators.
Then, an absolute value of the difference between one gradient-operated
pixel value and the adjacent pixel value, and Q value used as a dividend when
5 quantizing the block, are used to generate a binary edge map representing
the edge
of each pixel. Here, the block has 8 x8 pixels, and the size of the binary
edge map
is represented as a two-dimensional array edge[10)~lU] as shown in FIG. 7.
In order to generate the binary edge map, vertical edge detection and
horizontal edge detection are performed. Algorithms for vertical edge and
10 horizontal edge detecLions are as follows.
l * Vertical edge detection *!
Al = ABS(Ptrimage[0] - PtrImage[1]);
A2 = (ABS(PtrImage[tI] - Ptrlmage[-1]);
if (({Al > Th)&&{A2 > Th)) ~ ~ (A1 > 5*Th/2) ~ ~ (A2) > 5*Th/2))
~ge(ml f nl = I ~ l * edge *!
else ~ /~ Horizontal edge detection */
A'1 = ABS(PtrImage[0] - Ptrlmage[width]);
A'2 = (ABS(Ptrlmage[0] - PtrImage[-width]);
if t((A'~'Th)&8~dA'2>'1'h))II(A'1'S"~T'hl2)II~I~'2)'S'rt-hl2)>
Edge[m][n] = 1; / '" edge '~/
For the vertical edge detection, an absolute value (A1) of the difference
between the gradient-operated results of the piacel (Ptrlmage[0]) on which a
determination as to whether or not the pixel is an edge of the block is made,
and
the right pixel (PtrImage[I]) of the pixel {Ptrlmage[OJ) is calculate. Then,
an
°'" ~ absolute value (A2) of the difference between the gradient-
operated results of the
PtrImage[0] and the left pixel (PtrImage[-1]) of the pixel (PtrImage[0]) is
calculated. Then, a determination as to whether or not the pixel is an edge is
made
acco~~ding to the logic values obtained after the absolute values A1 and A2
are-
compared with a predetermined threshold value Th, and then the above process
'is
CA 02440013 2003-09-11
Il
performed on all pixels of the block. The vertical edge detection is performed
according to a logical formula of (A1 > Th}&&{A2 > Th} [ j {A1 > S*Thl2) [ J
{A2) > S*Th/2). If the logical formula is true, the pixel is determined as a
vertical
edge. Otherwise, the pixel is determined to not be a vertical edge.
o.
S The horizontal edge detection is performed according to the same principle
of the horizontal edge detection. First, an absolute value (A°1) of the
difference
between the gradient-operated results of the pixel (PtrImagej0]) on which a
determination as to whether or not the pixel is edge of the block is nnade,
and the
Iower pixel (Ftrlmage[width]) of the pixel (Ptrlmage[0]) is calculated. Then,
an
absolute value (A'2) of the difference between the gradient-operated results
of the
pixel (PtrImage[0]) and the upper pixel (PtrImage[-width]) of the pixel
(PtrIrnagej0]) is calculated. Then, a determination as to whether or not the
pixel
is an edge is made according to the Iogic values obtained after the absolute
values
A' I and A'2 are compared with a predetermined threshold value Th, and then
the
1 S above process is performed on all pixels of the block. The horizontal edge
detection is performed according to a logical formula of
(A' 1 > Th)&&(A'2 > Th) ( I (A' 1 > 5 *Th/2) [ ~ (A'2) > S*Th/2). if the
logical formula
is true, the pixel is determined as a horizontal edge. Otherwise, the pixel is
determined to not be a horizontal edge. Here, '&&' represents logical AND, and
' [ ~' represents logical ~I~.
Next, filtering is perfoamed by applying a predetermined filter window to
the generated binary edge snap. The filtering may be perfornned by a general
filtering method by applying a flier window having a predetermined size.
However, in this embodiment, filtering is not performed if the central pixel
of the
filter window is an edge, while the filtering is performed if the central
pixel is not
an edge. The filter window array be a general filter window. In this
embodiment,
a 4-connectivity filter window having five pixels arranged in a cross shape
centering one central pixel, as shown in FIG. 7,, is used. In FIG. 7, ~C
represents
an edge pixel, and the regions other than the regions with "X" represent non-
edge
pixels.
Also, if the filter window has no edge pixel, ordinary filtering is performed,
while weighted filtering is performed if the edge pixel exists. An example of
CA 02440013 2003-09-11
12
weighted filtering is shown in FIG. 7. In FIG. 7, " < < " represents a shift
to the
left, and " > > " represents a shift to the right.
The invention rnay be embodied in a general purpose digital computer that
is running a p<-ogram from a computer usable medium, including but not limited
to
storage media such as magnetic storage media (e.g., R~M's, floppy disks, hard
disks, etc.), optically readable media (e.g., CD-IZOMs, DVDs, etc.) and
carrier
waves (e.g., transmissions over the Internet). I-Ience, the present invention
may
be embodied as a computer usable medium having a computer readable program
code unit embodied therein for signal adaptive filtering, the coynputer
readable
program code means in the computer usable medium comprising: computer
readable program code means for causing a computer to effect generating
blocking
information for reducing the blocking effect and ringing information for
reducing
the ringing noise, from coefficients of predetermined pixels of the upper and
left
boundary regions of the data block when a frame obtained by deconstructing a
i5 bitstream image data for inverse quantization is an intraframe; computer
readable
program code means for causing a computer to effect setting the blocking
information and ringing informatioru of the previous frame corresponding to a
motion vector as blocking information and ringing information of the current
frame
if the frame is an interframe, and setting the ringing information to "I"
which
represents the image data requiring filtering if a residual signal of the
inverse-
quantized current block exists; and computer readable program code means for
causing a computer to effect adaptively filtering the image data passed
through the
inverse quantization and inverse discrete cosine transform based on the block
according to the generated blocking information and ringing information, for
instance. A functional program, code and code segments, used to implement the
present invention can be derived by a skilled computer programmer from the
description of the invention contained hereiri_
Industrial An~alicability
3m As described above, the present invention can remove the blocking noise
and the ringing noise from an image restored from an image compressed based on
blocks, thereby enhancing the image restored from compression.
