Note: Descriptions are shown in the official language in which they were submitted.
~ WO94/03013 214 ~ 8 :9 3 ~ PCT/AU93/00368
TITLE
IMAGE PROCESSING SYSTEM
FIELD O~ THE INVENTION
The present invention relates to an image
pxocessing system particularly, although not exclusively,
en~isaged ~or use in reduclng the bandwidth requirements for
transmission of video siynals (i.e. for high definition
television(HDTV), TV, video and video-phones), increasing
the resolution of a video picture for a given bandwidth
0 (HDTV) and in digitlsing signals for storage onto video
cassette tapes.
~ The pres~ent invention is able to do this by taking
into consideration the psychophysical attributes of the
human visual perception system, namely BETA APPARENT
15 MOVEMENT and SUPER EDGIMG ( the Julesz stereopsis
experiment ) .
~: ` ; ACP~GROUN~D TO THE INVENTION
The design of the modern televislon has been
nfluenced by prevlous discoveries in the cinema industry
and are based on three basic desiyn ~arameters:
1. Frame Format - in cinema it was discovered that a
frame rate of 16 frames per second was needed to
sustain smooth apparent motion. This was
increased to 24~ frames per second in order to
r~produce sound. Then the rate was increased to
half of the mains frequency (25 Hz in Europe and
;~ ~ 30Hz in US~) so as to avoid mains interference.
2. Synchronisatlon - achieved by sequentially
~i scanning video pictures starting at the top left
of the frame and progressing to the bottom right
on a line by line basis, and repeating the process
for subsequent frames.
3. Fiic;~er - which has important effects on selêc~iGLl
of the frame frequency. If the frame frequency is
; 35 too low (say 16 Hz or less) flicker results; at
medium frequencies (say 25 Hz) stro~oscopic
effects are created (i.e. wheels appearing to
~ i
W094/03013 ~40~9~ PCT/AU93/D0368
rotate counter to their actual movement; whereas
at too high a frequency (50 Hz) the bandwidth
required is too large and costs become
prohibitive.
In relation to the latter, video compression
techniques have been developed to reduce the bandwidth
requirements. However, most of these so ~ ions are based on
algorlthms which are mainly designed t~-detect and predict
changes such as those described ln`~the proposed CC~TT
standard H261. .
Conventional display techniques including
teIevision and electronic displays work on the principle
hat for an image to be seen by a viewer, the complete image
has to be represented by the illumination of the appropriate
l5~ picture elements (plxels) in the form of the image. When
the~e pixels are lit in their appropriate positions, the
human~eye can reconstruct the complete image, preserving the
positional relationships of the pixeIs.
; The biggest drawback of this method of display is
20 that for images that require a highe~ level of resolution to
be~ properly displayed (i.e.~ graphlcal images, including
graphical company logos like Co a-Cola), a display
consisting~ of a higher number of pixels is required.
Slmlla~ly,~ ~or~ dlsplays that scroll across the display
25~screen~ to~appear to be movin~ smoothly, large numbers of
columns of~these pi~xels are required, so that the resolution
of the image t as it moves across the screen, can be
preserved.
However, it was discovered that for moving
~objects, the human vlsual system is able to function
; perfectly~on signlficantly~less information than previously
was assume~ required. An e~ample of this is an effect known
as the BETA APP~RENT MOVEMENT or picket Ience ~flect, where
an observer ~i ws a moving object through a pic~et fence
that re~eals no more than 10~ of the object at any instant,
yet the viewer is able to s~e the object perfectly. It is
the mo~ement of the object behind the picket fence which is
t
~94/03013 21~ 0 8 9 3 plcT/Au93/ao36s
-- 3
fundamental to the viewer being able to see the entire
object.
Most drivers may notice that when moving (even at
low speeds), the rails of bridges, low density vegetation,
and the common picket fence become effectively transparent.
Similarly, objects become quite visible through the spokes
of rotating wheels, or the blades of rotating propellers.
In these cases, up to 90% of an objec~ can be
obscured by a moving obstacle, and still the full details of
the object can be resolved The converse also applies; a
moving object can be shielded by a fixed a barrier which is
~ up to 90% solid and~the full details of he object can still
be seen.
This is not due to the "persistence of vision",
lS which in the past was ascribed to retinal retention. It is
in fact due to the "BETA APPARENT MOVEMENT" effect. This
e~fect is the correct reason for humans being able to
interpret and connect different images into a sensation of
motion. Converse1y, the integration of partially presented
images, scrolled over time, is al~p mxde possible by the
;same principle.
The BETA~effect was used in Australian Patent No.
493435 where, instead of relying on large numbers OL pixels
in order to display graphical images of high resolution, the
BET~ e~fect was simulated and the human visual system was
tricked into perceiving a resolution much higher than was
actually bei~g displayed.~
In Australian Patent No. 493435 instead of using a
display fu11y covered with plxels, a relatively small number
3D of columns of pixels, spaced relatively far apart, were u~ed
;~(like the gaps in the fence that are the sole source of
information). If the image is then moved across the
display, either left to right or right to left, at a
suitable rate, it was discovered that the human visual
35 system would "fill in the gaps", giving the viewer the ~
;perception that the viewed image was being displayed at full -
~resolution. The viewer thus would not notice that the image
W094/03013 2 ~ 40 ~ 3 PCT/AU93/00368
at any yiven instant contained only a small fraction of the
complete image. The process of deleting a significant
: portion of the image and regenerating it over time for
recombination by the human visuai system is herein referred
: 5 to as "IMAGE DEPLETION" - the image was~EPLETED.
There are two main adva~ ages to using image
depletion. First, a display that~`only requires a column
where every tenth column should b`e results in significant
:~cost savings ln the production and operation of such a
display. Secondly, if this display is fed with information
~through an existing information network, significantly less
;~ information has to be transmitted for each "field" of the
~isplay to represent an image. This in turn has a two-fold
advantage of either transmitting at a higher field rate than
would be possible for the full non-depleted image, or
transmitting on a network of narrower bandwidth than would
otherwise be required for the non-depleted image.
The main problem with pictures, when stored and
transmltted, is that ~hey have a lot of data. Typically, an
20 :a~erage television picture needs, ~hen digitised, ~uffers
running into millions Qf bytes. A PAL standard colour
television picture needs three monochrome components, each
o some 520,~33 bytes.: This image is assumed to have 625 x
625 = 390,625 pixels each with an aspect ratio of 4 x 3, add
up to :520,833 bytes. The total, for a colour picture,
e~uals~three times this number, or approximately 1,562,500
bytes. : :
::: To send such a picture on a TV channel, the
; ~:`` channel would need to be able to handle a rate of 1,562,500
,~ .
30~ x 8j or approximately 12~:.5 megabits, 25 times each se~ondl .
Thls is equivalent to 12.5;x 25 i.e. 31~.5 mega bit/sec. . :
The challenge: is to reduce this data rate to the .
capabi~ities of ISDN network communication speeds Ol the
: order of 64 kb/sec.
The first step in handling such a huge task is to ~
implement G system which can compress the pict~re, free it :
~;
of any redundant information, and further reduce
;
~ W~94/03013 21 4 0 8 9 3 ; P~T/AU93/00368
:, . .
transmission times by sending only the field and block
differences rather than the whole field information. This
is ~he intention of the proposed CCITT H261 JPEG/MPEG
document covering the Discrete Cosine Transform (DCT),
5 Motion Estimation Prediction (MEP) and ancillary algorithms. :
Some of the reduction in transmission bandwidth
requirement can be handled by these algorithms, but they
have finite limits. We have discovered, however, that
enhancement to picture quality, as well as reductions in
transmission bandwidth requirement, can be achieved by
improving the machine-to-human interface by the use of
Psychophysics.
Prior art video systems (with the exception of
Australian Patents 493435 and 573024) have been designed on
the basis of a machine-to-machine interface and have not
taken the special needs of the human perception system into
consideration.
THE ROLE OF PSYCHOPHYS ICS
In order to compress a video i~age for low
: 20 bandwidth, we use the potential: of~ the visual perception
: system of: the observer to fill-in missins details. This
requires a series of techniques which rely upon certain
unique featurés in the manner in which the human brain
functlons. Hence, we can produce a considerable difference
betwe~n actual and perceived resolution and so enhance the
: machine:-to-human interface.
` :` :
These techniques are chosen from:-
1. The BETA APPARENT MOVEMENT effect (the BETA
` effect);
,
~30 2~ Two dimensional interlacingi
::
3. ~ Statistical fill-in; and,
4. Super Edging.
:~: The latter two of which constitute a VISUAh EN~AN~Ei~lri~T
:~ I
: TECHNIQUE (VET).
~;:: 35~ The present invention relies on these techniques j ;
to enable considerable depletion of an image without causing
signi~icant loss of visual intelligibility OL the image.
~ .
W094/030l3 ~ ~089 ~ ~ ~ PC~/AU93/00368
The BETA effect is used in a scanning and raster
technique which works equally well whethex the video imaye
is stationary or moving and is referred to as a DEPLETIOM
OPTIMISATION TECHNIQUE raster (DOT raster)~. The DOT raster
adopts the BETA effect described in Australian patent
4~3435, except that, instead of requiring the video image to
move with respect to a plurality of spaced apart stationary
columns of video elements, a raster of columns are moved
backwards and forwards :with~xespect to a fixed or moving
video image. :Hence, the pixels are moved backwards and
forwards to create apparent movement - even in still video
:images. This is equivalent to taking the picket fence
analogy of Australian patent 4934~5 and moving the picket
fence with respect:to the video image instead of moving the
I5 video image with respect to the stationary picket fence.
The:~ef:fect can be simulated by closing one eye, spreading
; your~fingèrs slightly apart and waving them in your field of
view.
It~ is the mo~ement o~ the column raster which
2:0~ enables the~present~invention to appl~ the~3ET~ effect to a
stationar~ video: image. Also, a row raster can
simultaneously be used to provide a horizontal depletion -
to create :a two:~dimensional interlace raster, as shown in
Figure ~la~
25 ~In fact the raster movement of the pixels does not
have to~ be~:limlted~to llnear~movement in rows and columns,
but c~n~ be~:random ln~ Z:dimensions, as shown in Figure lb.
The DOT~ raster produces highly viewable ~ideo
images~even~ where; the fields: are reduced from 625 to `6d
30pic~ture columns, e:ach with only 12~8 out of 625 vertically
arranged~ picture elements (p1xels). This represents a
horizontal~;depletion of lO:~l and a vertical depletion of
4:~1,:: givin:g:a~ overall depletion of 40:1. That is, each ~ -
field is depleted by 40:1, but a time separated interlace of
35~ a::plurality of the fields which build u~ a full video image
having a resolution of 625 x 625 pixels. :~
" : Hence, the horizontal resolution which can be
WO94/030l3 2 ~ 4 0 8 9 3 PCT/AUg3/00368
achieved is not dependent upon the number of pixels in a
single field, but is dependent on the number o~ unique
positio~s that the DOT raster can acquire over time.
The columns can be broken up and rearranged into a
two dimensional interlaced array (a checker board pattern as
shown in Figures 2a, 2b and 2c). This has the effect ol
allowing the entire image to build up over a number of
fields and to redùce the tendency for the viewer to lock
onto the moving checker board.~
Alsot the DOT raster does not preclude the use of
other da~a compression techniques such as Discrete Cosine
- Transform ~DCT) and motion prediction algorithms. Hence,
even greater savings in bandwidth are possible.
Further, statistical fill-in methods can be used
to reduce any artefacts introduced into the video image by
the DOT raster. This is achieved with a VISUA~ E~HA~CEMENT
TECHNIQUE ;(VET) which relies on the injection of band
limited noise into the video signal, once decoded, and
manifests itself as pixels appearing between the other
;~ 20 pixels of the decoded checker board Battern. The luminance
va~lue o~ the noise approximates that of the intensity of
neishbouring pixels. This has the very surprising effect
that the addition of noise increases the sharpness of the
video image. AIso, the injected noise creates pseudo pixels
25~ and hence can double the apparent resolution. As the noise
do~s~not have a fixed position within the DOT raster but is
o~erlapped by true pixels, by the action of the Beta effect,
the viewer perceives the average of the ~noise and active
image. ` : ; `
; 30 ~ The increase in sharpness can be explained by a
consideration of the SUPER EDGING effect ~see Figures 3a and
3b) r where any random structure improves the sharpness of an
image, this is the property of the viewer to "seel' non-
existent details between certain points. The edges of the
shapes shown in Figures 3a and 3b appear extremely sharp
even though the edyes are formed fr~m a random arrangement
of pixels. The sharpness of the edges in these exam~les
t
': ~,. I
WOg4/03013 PCT/AU93/00368
2i 40893
would be unobtainable by simple connecting lines. It is the
psychophysics of the human perception system which
~; interpolates the random pixels and creates the perception of ; an extremely sharp edge - hence SUPER EDGING.
- 5The ability of the ~iewer'~;visual per~eption
system is such that, lac~ing full i ~ mation, the missing
lnformation is "created" in dramati~ detail It is this
ability which enables a v~I~ewer to appreciate
mpressionistici' pa1ntings. Further examples of the viewer
"creating"~ the~ missing information to achieve a more
complete image is~shown in Figures 4a and 4b. In each case
~ t~e~viewer can perceive a triangle, yet if the spots are
~; taken individually ~(by covering the other two) the viewer
"sees" only what is present.
~ SUMMARY OF THE INVENTION
There~ore, it is an object of the present
invent1on~ to prov1de an 1mage processing system relying upon
the~BETA~APPARENT MO~EMENT effect to enhance the perceived
resolu~ion of~a~ ~ideo image.
;Z O~In~ ac~cordance;~ with one ~spect~ of the invention
there is~prov1ded~an~image~processing~system having: ~
; inter1Ock1ng~ means for~interlacing the~ pixels of
subsequent ~ields~of pixels horizontallyi
whereby~,~the inter1a~c1ng causes~ thé pixels to move
25~ ba~kwards~and~forwards at a rate substantially imperceptible
to ~a~ human:~vièwer~ for creatlng a perceived reso1ut1on of a
; video~ im~àge~ formed~ by~a~plurality of the fields so
interleaved, wherein the perceived resolution is greater
than the actual~resolution.~ ~`
30~In~aGcordance~wit~h~another aspect of the invention
there~is~;pro~ided an image~processing system for compressing ~ -
a video image re~erred to as an oniginal ~ideo image, the
mage proc~essing system comprising: ~
;a~digitiser ~means for digitising the original video ~ - 35 ~;image;, ~the orig1nal video image being formed of a plurality
of original video fields each having M rows of pixels and N j `columns of pixels; and, ~3 ,
~ W094/03~l3 . 2140893 ;~ PCT/A~93/00368
g
a process control means for processing the digitised
original fields, the process control means selecting one
pixel out of euery d pixels and for deleting the remainder
of the d-l pix~ls for generating depleted ~ields in a
depleted video image, the depleted field ha~ing m rows of
pixels and n columns of pixels where m is less than M and n
is less than N;
whereby, a receive~ means can recei~e the depleted
video image and can generate d-1 pixels from each selected
pixel and can display:each selected pixel and its associated
d-l generated pixels in a manner to simulate movement of the
~ pixels on the display to substantially reconstruct the
original video image by relying on the BETA APPARENT
MOVEMENT effect.
15: Xn a~c~rdance with another aspect of the invention
there is pro~ided a method for compressing a video image
ref erred to as an oniginal video image, the method
comprising the steps of:
~; digitising a field of the original ~ideo image into a
pluraIity of data bytes referred to ~s pixels, each original
video field having M rows of pixels and N columns of pixelsi
selecting one pixel out of every d pixels;
deleting the remainder of the d-1 pixels; and,
generating a depleted: field of pixels in a depleted
~ideo image, the depleted field having m rows of pixels and
n columns:of pixeIs where m is less than M and n is less
than N;
whereby,~ a~; receiver means can receive the depleted
;; video image, generate d-1 pixels from each selected pixel
and: dlsplay each selected pixel and its associated d-l
: generated pixels on a display means in a manner to simulate
:
~ movement of the: pixels to :substantiaIly reconstruct the
:: ~ original video image by relying on the BETA APPARENT
;: MOYEM~NT effect.
In accordance with another aspect of the invention
there is provided an image processing system for
decompressing a video image referred to as a deplet~d video
2 ~ 4 0 8 9 3 PCT/AU93/00368
image, the image processing system comprising:
a digitiser means for digitising the depleted video
image, the depleted video image bei~g formed of a plurality
of depleted video fields each havin~ m rows of pixels and n
columns of pixels; and,
a process control means ~f'or processing the digitised
depleted fields, the process control means generating d-l
:~ pixels from each selected pixel and displaying each selected
pixel and its associated d-1~ generated pixels over a period
lo of time imperceptible to a viewer for simulating movement of
::
the pixels on ~a ~display~to regenerate an original video
- image having~M rows of pixels and N columns of pixels;
whereln,~ the process controller relies upon the BETA
APPARENT MOVEMENT effect in regenerating the original video
15 image:. ` ~
: In accordance :with another aspect of the invention
:
there lS provided:a method for decompressing a video image
referred to as a depleted video image, the method comprising
the~steps of~
~ di~itising the~depleted video i~ge, the depleted video
image being formed~ of a plurality of depleted video fields
each~havlng m rows~ of pixels and n columns of pixels;
sélecting~a pixel from the:~depleted video:lmage;
generating~:d-l pixels from each selected pixel;
;;25 : ; ~displaying~ each ~selected pixel and its associated d-1
generated:~pixels: o~er a period of~ time imperceptible to a
viewer~for simulating mo~ement of the pixels on a display
mqan!s~for reconstrjucting an original video image wi~h the
selected pixels and the generated pixels, the reconstructed
3:0:; :video lmage~havlng M rows of pixels and N columns of pixels,
where M is greater than m and N in greater than n;
wherein, the simulated movement relies upon the BETA
: APPAR~NT MOVEMENT:effect in reconstructing the original
video~image~.
35 ~ Preferably, the ` luminance companent of the
rljected noise is congruent :with the luminance of adjacent
pixel~s. Also, the injected noise is preferably band
21~ 0893
W~9~03~13
PIcrfAug3~00368
- 11
limited.
Prererably, luminance determining means is
provided to determine the luminance of the adjacent Dixels
and to set the luminance component of the pseudo pixe; to a
value proximate the actual value but different enough so as
: to induce an observer's~perception system to select visually
the most probable value~.
BRIEF INTRO UCTION OF THE DRAWINGS
One embodiment, being an example only, of the
present invention ;wlll now be described with reference to
the accompanyi~g drawi~gs,: in which:-
- Figure la shows a two dimensional interlace of pixels,
being in an QDD field "O" and an EVEN field "E";
Figure lb shows a two dimensional random raster
~15 movement of pixels for a modulo-3 raster scheme;
Figures 2a :to 2c show the break-up of a plurality of
columns of video (Flgure 2a~ into a single depleted field
(Figure ~b) and into two interlaced depleted fields (Figure
2c); ~ :
20`~ ~Figures 3a and 3~ are diagrams i~powing the effect known
: as SUPER EDGING in relation:to~a square and a triangle;
Figures 4a and 4b are diagrams showing the effect known
as FILLING-IN~in relation to a triangle;
Fi~ure~5 is~ a block diaqram~ of an image processing
25~: system ln accordance with the present invention; -.
Figure 6 is a block diagram of a receiver of the image
processing system shown in:Figure 5;
Figure 7~ lS a~ block diag~am showing~a transmitter of
:the image processing system shown in Figure 5;
Figure 8 iæ ~a graphical representation of a two
: interlaced regenerated~fields ~J and G of the receiver shown .
n Figure 6; and,
~ ~ .
~ :Figure 9 shows a modulo-4 raster scheme.
;
: DESÇRIPTION OF THE P~EFERRED_EMBODIMENT
The following relates to an embodiment of the
image processing system 5 capable of providlng a video
depletion of about 40:1. The image processing system 9 ~ ~ -
WO94/03013 21~ PCT/AU93/00368
- 12 -
comprises a transmitter 10 and a receiver 12, shown in
Figures 6 and 7 respecti~ely. A CODEC 14 is connected to an
output of the transmitter 10 and another CODEC 16 is
connected to an input of the receiver~. A video source 18
5~ is connected to an input of the transmitter 10 and a video
; monitor 20 is conne~ted to an ~put of the receiver 12.
~; Optionally, a monitor 21 is connected to the transmitter 10.
.
RECEIVER
The receiver 12 has a video signal conditioner 32a
and a sync separator 32b connected to ~ video input 33 which
.
:: : lS connected to the CODEC 16. The~ sync separator 32b is
; connected to a clock circuit 32c and a timlng controller 32d
whlch controls a~clock timer 32e. The clock circuit 32c
typically operates at a frequency of 12 megahert~ and is
15 ~hereinafter referred to as the "pixel clock'l 32c. The video
conditioner 32a is connected to an analogue dis?ital
converter;34a and thereby to a field store 34b. The out~ut
of;the field store 34b is typically an 8-bit databus which
lS shown ~as~ a thi~k line in Figure 6. The field store 34b
20~ is ~onnected to a delay circuit 34c~and thereby to an 8-bit
latch 34d. The~delay of the delay circuit 34c is typically
about l~microsecond so as to enable correct syncing. The 8-
bit latch~ 34d typically has a refresh rate of~about 5
million times per~se~ond.
25~ The~ ~sync and colour subcarrier components are
extracted from the~video~signal at the vi~eo input 33 by a
colour extractor 32~ connected to~the video input 33. The
colour extractor 32 is connected to a colour processor 34
which typically includes a control circuit substantially t;~e
~ ;
30 ~;same~ as the remainder of the receiver 12. The colour
extractor 34~ ope~rates on standard B~Y and R-Y signals sent . 7
in~the video~signal from the transmitter 10, Typically, the
B-Y and R-Y signals are at a reduced resolution, such as . 3
half the resolution of the video signal processed by the
35; plxel processor 26 (the compressed Y signal).
The analo~ue to digital converter 34a and the
field store 34b digitise the video luminance signal from the
$
~ 21~089~ 1
wo94/03013 :- ~ . PCr!AU93/~0368
!
-- 13 --
video conditioner 32a. The analogue to digital converter ~,
34a generates digital values corresponding to the luminance
of each pixel (hereinafter referred to as the ~Ipixels~) "on
the fly" as the luminancP signal is processed by the video
conditioner 32a. : The pixels are then stored in the
appropriate memory address of the ~ield store 34b.
: Typically, the luminance for each pixel is determined to be,
for example, one of 256 luminance levels per pixel depending
~: on the instantaneous voltage of the video signal (i.e. one
i~ 2~).
~; For ease of understanding the conception of the
` ~ - receiver consider that the fie_d store 34b collects one full
field of the luminance signal and stores it for further
processing. That is, consider that the receiver 12 operates
one field behind the incoming video signals at the video
input 33.
: The clo:ck circuit 32c controls the timing of the
analogue to digitaI con~erter 34a so as to digitise the
s:lgnal :received from the video co~ditioner 34a at the
`:
appropriate time:to correspond to eac~ pixel location of the
~ideo source 18. The sync separator 32b re-aligns the cloc~
circuit 32c at the beginning of each line of the video
signal by use~ of the horlzontal sync signal contained
therein.
~ The: outputs of the fieId store 34b and the ~-bit
Iatch~34d are connected to a pixel processor 36. The pixel
processor 36 has a process controller 36a connected to
: latches and buffers 36c, 36d and 36e. The latch and buffer
~'~ 36c is connected to an ALU unit 38c and thereby to a latch
and buffer 38d. The clock timer 32e is connected to the 8-
bit latch 3~d, the latches and buffers 36c, 36d, 36e and
38d. The clock timer :32e combines the clock and s-~nc
signals trom the sync separator 32b and the clock circuit
32c for controlling the tl~ing of the pixel processor 36. ~j `
The process controller 36a reads the pixels from
~; : the field st~re 34b and allows them to proceed to the latch
and buffer 36d. Also, the process controller 36a can allow
; ~ .
~ ' I'
.
W094/03013 2~4893 ~ !.. PCT/AU93/~0368
- 14
passage of the pixels to the latch and bu~fers 36c and 36e.
The pixels which pass to the latch and buffer 36d correspond
to those un-depleted pixels which were transmitted to the
receiver 12.
A pseudo noise generator 3~8a is connected to the
ALU unit 38c via a latch 38b. The pseudo noise generator
38a allQws injection of noise pixels into the video image
~; displayed on the monitor 20. The pseudo noise generator 38a
typ~cally generates pixels with luminance values which have
a random value between the values of the adjacent pixels.
The outputs of the latches and buffers 36d, 36e
~ and 38d are connected to a digital to analogue converter 40
which is connected to a video output 42 and hence to the
monitor 20.
~ ~ ~ The pixel from the latches and buffers 36d, 36e
and 313d are recombined with the colour information at the
digltal to analogue converter 40 via the colour processor 34
and with the sync information from the sync separator 32b
hence forming a vldeo signa1 corresponding to a regenerated
20 ~ideo~ field for`display on the monito~ 20.
The process~controller 36a reads pixels from the
fiel;d store 34b one at a time in sequential order,
corresponding to ~rows of the video signal from the video
source~18~i.e. U11, U13,...Ul,n (where n~is the number of
25~ col~umns~ as shown in Figure 8. However, once the pixels
U11,~ U13...U1,n~ have been~read~the process controller 3~a
genera~:es pixels ~12, G14,...Gln-1 and sends them to the
latch~and buffer 3~d before their each of associated pixels
U11, U13,...~Uln are;sent to the latch and buffer 36d via the
8-blt latch 34d. That is, the generated pixel G12 is sent
` to~he latch and buffer 36d before the depleted pixel U13
and so on. Hen~e, the process controller 36a generates the
pixels between the un-depleted pixels from the video slgr.al
as it reads the un-depleted pixels and sends each generated
~; 35 pixel~ for display before it sends the last pixel read from
the field store 34b to the latch and buffer 36d. The pixels
are referred to as '~un-depleted pixels'~ because they are the
~: : . . . . . .
f. 21~0893
WO 94/03013 . P.~/AU93/00368
i,
- 15 -
j pixels which remain after the depletion process of the
¦ transmitter 10, described hereinafter.
The control processor 36a generates the ge~eratedpixels G by considering the luminance values (between and 0
and 255) of the last two pixels read from the field store
34b and makes the generated pixel lu~inance value
statistically dependent thereon. For example, the value of
the generated plxel could be a polynomial interpolation of
: the values of the two un-depleted pixels.
: Alternatively, th value could be a random value
with the values of the two depleted pixels as its upper and
lower limitsO In this case the process controller passes
control to the latch and buffer 36c and the pixel is
generated by the pseudo noise generator 38a and sent to the
~: 15 DAC 40 via the latch and buffer 38d.
The process controller 36a then reads the pixels
U~1, U33,...U3,n of row three of the video image. These are
the next pixels in the field store 34b. The control
processor 36a also generates the pixels G32, G34,...G3n+1.
Then the cvntrol processor 36a gen~rates the pixels G21,
: G22,...G2n for the line of pixels between the first and
third lines of pixels read from the field store 34b.
The effect of the above is that each deple~ed
:~ fleld commenc~es with only: 312 out of 625 pixels across and
2~5 312 out of 625 pix~ls down in each column but results in a
fu11 62:5 line video signal. Hence, each depleted field had
only one quarter of the pixels of the original field and the
recelver 12 regenerates the other three ~uarters of the
pixels, as shown graphically in Figure 8.
:30 The above~ description relates to a mo~ulo-2
: depletion of the pixels of the original video signals. It
amounts to a horizontal and vertical interlace of two
fields, namely Un,m/Gn,m and un,m/gn,m.
The horizon~al interlace has the effect of moving
~ 35 the image back and forth behind "a picket fence" and hence
; induces the BETA effec~ to give the illusion of hi~her than
~:~ actual resolution to the viewer. The vertical interlace
W 0 94/03013 2 ~ 4 0 a 9 3 PC~r/AU93/00368
- 16 -
gives higher vertical resolution.
In Figure 9 there is shown a graphical
representation of a modulo-4 raster scheme. In modulo-4
four fields are effectively interlaced h~rizontally and step
through positions marked "1" throug~ '7". In this scheme
the receiver 12 generates pixels G~ nd g for three out of
every four columns. Vertical in~èrlacing could be included
in this raster scheme. Further, as shown in Figure lb the
raster could be random (a random 2 dimensional modulo-3) ln
which over subsequent: fields each pixel described the path
shown by lines enumerated 1 to 9.
,
T~ANSMITTER
The transmitter 10 has a transmission signal
conditioner 22a and a since separator 22b connected to a
vldeo input 23 which~is driven by the video source 18 (such
as a video camera~or HDTV program or VCR or the like). The
sync separator 22b is connected to a clock circuit 22c and a
timing controller 22d which controls a clock timer 22e. The
: clock ~circuitry 22c typically operates at a frequency of 12
megahertz and is hereinafter referre~ to as the pixel clock
~;;`: : 22c. The:video conditioner 22a is connected to an analogue
digital converter 24a and thereby to a field store 24b. Th~
output of the field stoxe 24b is typically an 8-bit databus
which is shown as a thick line in Figure 7. The field store
24b is connected to a delay circuit 24c and thereby to an 8-
bit latch 24d. ~The delay of the delay circuit 24c is
typical~y about 1 microsecond so as to enable correct
syn¢ing. The 8-bit latch 24d typically has a refresh rate
of about 5 million times per second.
~ The signal conditioner 22a extracts the luminance
component from the video signal. A colour extractor 22
extracts the colour component from the video signal received
from the video source 18. The colour extractor 22 is
: : connected to a colour processor 27 which typically includes
: 35 a control circuit substantially the same as the remainder of
:~ the transmitter 10. The colour extractor 27 operates on
~ standard B-Y and R-Y signals sent in the video signal from
WO94/030l3 21 ~ o ~ g 3 PCr/AU93/00368
- 17 -
the transmitter 10. Typically, the B-Y and R-Y signals are
at a reduced resolution, such as half the resolution of the
~ video signal processed by the pixel processor 26 (the
¦ compressed Y signal).
!: 5 The analogue to~ digital converter 24 and the
I field store 24b digitise each field received from the video
I source 18 and store it for digital processing. The analogue
I to digital converter 2~a generates digital values of the
luminance of each pixel (hereinafter referred to as
I 10 "pixels") "on the fly" as the luminance signal is processed
j by the video conditioner 22a. Typlcally, the luminance is
determined to be, for example, one of 256 luminance levels
~er pixel (i.e. one in 28).
The clock circuit 22c controls the timing of the
analogue to digital sonverter 24a so as to digitise the
signal received from the ~ideo conditioner 24a at the
appropriate time to correspond to each pixel location of the
video sou~ce 18. The sync separator 22b re-aligns the clock
circuit ~2c at the beginning of each line of the video
~: 20 signal by use of the horizontal ~ync signal contained
: ::
therein. :
The output of the field store 24b and the 8-bit
latch 24d is zonnected to a pixel processor 26~ The pixel
processor 26 has a process controller 26a connected to
latches and buffers 26c and 26d. The latch and buffer 26c
is connécted to an:~ALU unlt 28a and thereby to a latch and
buffer 2~b. The cloc~ timer 22e is connected to the 8-bit
latch 24d and the latches and buffers 26c, 26d and 28b. The
: clock timer 22e combines the clock and sync signals from the
30 sync separator 22~b and the clock circuit ~2c ~or controlling
the timing of the pixel processor 26.
; The process controller 26a reads luminance values
from the field store 24b one at a time in sequential order,
corresponding to rows of the video signal from the video
:35 source 18. s
For ease of understanding the conception of the
transmitter consider that the field store 24b collects one
W094~0311l3 93 PCr/AU93/1)0368
- -- 18 --
full field of the luminance signal and stores it for further
processing. That is/ consider that the transmitter 10
operates one field behind the incoming video signals at the
video input 23
5Ref erring to Fig~ ~ 8 the process controller 26a
selects, for example, th~.pixels corresponding to the ODD
rows and the ODD columns', i.e. Un,m for all values of n and
: m. These pixels are referred to as the "un-depleted pixels~'
and are allowed~to~proceed to the latch and bu~fer 26d for
: 10 transmission to the` receiver 12. Also, the process
controller 26a allows passage of~other ones- of the pixels
^ th~ latch and buffer: 26c. These pixels are referred to as
"depleted pixels". The pixels which pass to the latch and
buffer 2~c are used to be displayed on a monitor 15 by the
15: ALU unlt 28a, the latch and buffer 28b ana a digital to
:analogue: converter:31. The digital to anaIogue converter 31
; is al~so:~connected to an output oL the latch and buffer 26d
and therefore, the monitor 15: can show a video signal
similar~to that wh1ch will be shown on the monitor 20 at the
:20 rece:iver i2. :~
: A pseud~ noise~ generator 28c îs connected to the
ALU unit: 28a via a latch 28d. ~ The~ pseudo noise generator
28c allows~ injection of noise pixels into the video image
displayed on the monitor~15~: :
: 25 ~ The process:controller 26a has the effect of, for
example, selecting ODD pixels- from OD~ video fields and ~VEN
pixels from EVEN vldeo ~ields. The combined effect is a
;~ field having ;~two fields which are interlaced both
horizontally and:vertically as shown in Figure 8. That is,
every second column and every second row from the original
ideo signal has been deleted. Hence, the resultant video
signal for transmission is re~erred to as "depleted".
,,
:; The output of the latch and buIIer 26d is
: connected to a digital to analogue converter 3C which is
connected to the CODEC 14 for transmission viar for example,
~: an antenna. The sync separator ~2b is connected to the DAC .,
30 to control its timing.
~,
~ ;
WO94/03013 21 ~ 0 8 9 ~ PCT/AU93/U036X
-- 19 --
The effect of the a~ove is that each field is
depleted from 625 down to 312 pixels across and from 625
dow~ to 321 pixels down in each column. Hencej each
depleted field has one quarter of the information of the
original field. This correspo~ds to a modulo-2 depletion of
the pixels of the original video signals.
The Iuminance of the depleted pixels is recombined
~: with the sync;:and colour information at the digital to
analogue converter 30 via the colour processor 27 and the
sync separator 22b respectively.
~~ As with the receiver 12 other forms of depletion
`: ~ of the video signal could be used, such as, for examplP,
~ modulo 3, 4, 5 etc. or ~even a relatively random form of
:~ depletion as shown in Flgure lb in which over subsequent
:~: 15 fields each pixel describes:the path shown by lines 1 to 9.
This is a two ~imenslonal version of modulo-3.
: In modulo-4, as sho:wn in ~igure 9, each field
image lS digitised and saved in the field store 2~b in the
following manner:
Z0 -~ during field l~r startinq from the first pixel, ,`
on~ly 1 pixe~l in 8 is saved on each line. The
; : dis:played picture loo~s simllar to Figure 2a where
part of a 64 column pattern is depicted; ~,
during:field 2, starting from pixel 3, only the
:25 ~ : three-modulo-8 elements are saved;
during fields 3:and 4 the same process is used
for pixel 5 and 7 respectively~
:~ The combined effect is a frame of 4 fields that looks
::~; similar to ~igure 2bf in which every second line has been ,-
deleted - both horizontally and vertically.
The imag~ processing system of the present
invention allows ~or considerable compression of a video
signal by relying on ~the psychophysical attributes of the
; : human perception system to undertake appropriate
interpolation of the video image to provide a perceived
resolution which is substantiall-~ the same as that of the
uncompressed signal - even though the actual resolution has
~ .: ` f~:
W~94/03013 PCT/AU93/00368
~40~9 - 20 -
been severely depleted. The DOT system relies on the BETA
APP~RENT MOTION effect to achieve the high compression by
deletion of all information from the video signal which is
not necessary for the human perception system. The BETA
effect is ~chieved in still video images by moving the
pixels, such as, backwards and forwards. Also, the
` viewability of the interlaced fields is enhanced due to the
very high statistical correlation - between the pixels of
subsequent fields (by stepping the pixels bac~wards and
forwards). The VET then adds random noise to the signal,
once received, in order to make the resultant video image
sharper - by relying on the phenomenon of SUPER EDGING.
Also, the system of the present invention pref;erably
operates in a digital format and so avoids analogue
artefacts (i.e. overshoot and smear) which are very
; difficult to remove. Statistical manipulation of the video
signal can allow for even greater compression. For example,
the ~ideo signal, as compressed by the DOT raster, could be
processed by a DCT to give a further compression of 40:1,
thus giving a total compression of ~600:1. Hence, a video
signal can be compressed and transmitted via a telephone
:
line having a bandwidth of 64kHz - thus being applicable to
video telephone without requiring special ox multiple
telephone lines.
~lso, the DOT can be used to enhançe the
resolution of a standard video signal to achieve a high
definition video result. And, such high definition can be
achieved simply~ by processes at the receiver. Still
further, the DOT could be used to digitise video signals for
video recorders and ~ideo cameras.
Modificatlon and variations such as would be
apparent to a skilled addressee are considered within the
scope of the present invention. For example, other systems
of movement of the pixels could be used to take advantage of
the BETA effect e.g. circular or random movement of pixels.
'
.
