METHODE ET APPAREIL DE TRAITEMENT DE SIGNAUX VIDEO

VIDEO PROCESSING METHOD AND APPARATUS

Abrégé anglais

A method and an apparatus are set forth for encoding and
decoding video to achieve bandwidth compression. In one form of
the disclosure, two video signals (810, 850), representative of
different images, can be transmitted using only the bandwidth (880)
generally allocated to a single video signal, with little or no
perceived degradation of image quality. In another form of the
disclosure, motion indicative signals (970) are used in a technique
that dynamically modifies the frequency band information to be
stored and/or transmitted.

Revendications

CLAIMS
1. A method for encoding a video signal, comprising the steps
of:
dividing frames of the video signal into a multiplicity of
tiles;
separating the picture content of each tile into a
plurality of frequency bands having a predetermined priority order;
determining motion at each tile from changes in picture
content of the tile from frame to frame; and
generating signals for each frame that include an
indication of the motion status for each tile and a representation of
a frequency band for each tile; the frequency band representation for
each particular tile being selected as a function of the motion status
for said particular tile.
2. The method as defined by claim 1, wherein said plurality
of frequency bands comprises at least three frequency bands.
3. The method as defined by claim 1 or 2, wherein said step
of generating a signal that includes an indication of the motion
status for each tile includes generating status signals that indicate,
for each particular tile not exhibiting motion, the period for which
said particular tile has not exhibited motion.
4. The method as defined by claim 3, wherein different
frequency band representations are selected for each motion status.
5. The method as defined by claim 3, wherein one of said
motion statuses results in selection of no frequency band
representation for the tile having said one motion status.
6. The method as defined by claim 1, 3 or 5 wherein said step
of separating the picture content of each tile on a frequency basis
includes applying an orthogonal frequency transform to the picture
content of each tile, and dividing the resultant coefficients into
band groups.

7. The method as defined by claim 6, further comprising the
step of discarding coefficients representative of relatively high
frequency diagonal frequency components of the picture information of
each tile.
8. Apparatus for encoding a video signal, comprising:
means for dividing frames of the video signal into a
multiplicity of tiles;
means for separating the picture content of each tile into
a plurality of frequency bands having a predetermined priority order;
means for determining motion at each tile from changes in
picture content of the tile from frame to frame; and
means for generating signals for each frame that include
an indication of the motion status for each tile and a representation
of a frequency band for each tile, the frequency band representation
for each particular tile being selected as a function of the motion
status for said particular tile.
9. Apparatus as defined by claim 8, wherein said means for
generating a signal that includes an indication of the motion status
for each tile includes means for generating status signals that
indicate, for each particular tile not exhibiting motion, the period
for which said particular tile has not exhibited motion.
10. A method for encoding a video signal, comprising the steps
of:
separating said video signal into a relatively low
resolution component and a relatively high resolution detail
component;
dividing frames of the detail component into a multiplicity
of tiles;
separating the picture content of each tile into a
plurality of frequency bands having a predetermined priority order;
determining motion at each tile from changes in picture
content of the tile from frame to frame; and
generating signals for each frame that include an
indication of the motion status for each tile and a representation of
a frequency band for each tile, the frequency band representation for

each particular tile being selected as a function of the motion status
for said particular tile.
11. For use in conjunction with a method for encoding a video
signal, comprising the steps of: dividing frames of the video signal
into a multiplicity of tiles; separating the picture content of each
tile into a plurality of at least three frequency bands having a
predetermined priority order; determining motion at each tile from
changes in picture content of the tile from frame to frame; and
generating signals for each frame that include an indication of the
motion status for each tile and a representation of a frequency band
for each tile, the frequency band representation for each particular
tile being selected as a function of the motion status for said
particular tile; a decoding method comprising the steps of;
storing frequency band representations for each tile;
selecting stored frequency bands as a function of said
motion status signals; and
generating a decoded video signal from said selected stored
frequency bands.

Description

CA 02379330 2002-04-16
WO 91/15929 - . PCT/US91/02228
1
. . . . . -_- DESCRIPTTON ~_ -- _. _ :.
VIDEO-PROCESSING:METHOD AND APPARATUS "
FTELD OF THE INVENTION
This invention relates to video signals and, more
particularly,: to,.apparatus_and method for encoding and
decoding video signals..for-use in.televisionvand in high
definition television~systems as well asrin other '
applications including storage and/or transmissionvover any
suitable,medium, of:moving images,~or combinations of moving ..
and still images,_in a form: that requires reduced storage
capacity and/or reduced bandwidth channels.v Some of the
techniques hereof can be employed; for example, for
transmitting through the air or through conducting or optical
cable, a plurality of video signals~using only the bandwidth
generally allocated to a single video signal, and with little
or no perceived-degradation of image quality. Some of the
techniques hereof can be employed, for example, in so-called
"compatible" high definition television approaches, as in
so-called "simulcast" approaches wherein independent high
definition television signal is sent simultaneously with a
conventional transmission of the same program information.
Some of the techniques hereof can also be employed in
so-called "enhanced definition" approaches that send picture
enhancement information (but less than the information needed
for full high definition. performance) on the same channel
with a conventional television program.
BACKGROUND OF. THE-INVENTION'
Available spectrum is becoming increasingly burdened by
ever greater demand-for_.video information channels.
Traditional airwave.spectral. space has beenvcrowded for many
years, and burgeoning .video program~irig for 'such applications
as home cable, teleconferencing,-picture''phonesand~computerw
video transmission:has now crowded conductive and optical' '~
c~les, phone ,lines,:: and sattelite ~ conmunication ~channels.~

CA 02379330 2002-04-16
wo 9ins~z9 Pc-rms9l~~nxfz~
a
i
The desirability of techniguss for increasing the amount of
video i»formation that aan bs ser~t~ °,. over these transmiss.ian
media is evident. ~. A~s~c~~~.. as mere. .,via~o.'..~.nfi~'s~RiatiQn~ is~
stored,
It is des~.rable to develop techniques -chat #increase the
amount of video that cast be,- stcxed.in a given storage size.
As hf.gh definition tal~vis,ion (HhTV) becomes mere w ''
prevalent, improved systems- ar~~. needed for. trartsmissiaw ahd v
r~cepti.an of. the.; addi~i.o~al, infcrmatior~_ reqN.~red far r
presenting HDTV images,. Any- new- services which provides
higher de~initian te~ev,.is3.an_:than is convent3.onally broadcast
(i.e., mate elements per ~.ine and lines3 per frame, and thus a
widor bandwidth naaos$axy far.txar~em3ssioy ahoWld serve
existing home telev~.sipn receivers with e~3sent~al~.y all the
picture attribut~s and quality of which the receivers are
c$pable. Also, receivers dasf.gned far new (high definition)
service, should be capable af,_operating using the
pre°existing transmissions and dex~.ve from them a ,result not
inferior to that provided bV pre-existing . receivers:
A variety of HDTV schema hive heen proposed. in U.S.
Patent No.s 4,$17,597, 4,b2B,34y 4y52,909, 4,7~1,78~, and
4~800,42~, assigned, to the same asa~.gnee as the prBaent
application, as well. as 3n the,. publ.ioation nHDT~t Compatible
Txar~smisaion syabmm~~, W.r. Glenn, Hatl.onax Association of
Braadcastexs, Apr~.).. 198b, there is disclosed an HDTV system
that utilizes an auc~sntation approach which permits
compatible tra~tsmisafon of.. HDTV. A s~parate auxiliary or
~~augmentation~~ chsnne.l 3e used to send picture c3etaix
informat3.on that augnsenta conventionally'received t~rl,evis3on
~.n~ormation to nhtain high d~firtit3on performance. The
disclosed techniqe~es slap have .,dpp1#.ca~Cion to viddo bandwidth
compression and to redac3,ng,vid~o_starage capacity.
As r3eseribed in the rexerer~ead patents axed publication;
an ele~ctronia video signal , (e. g.~; a ~ telev3,siori signal ) can bev
encoded at reduced bandwidth by Iowering:the f=ate refresh
rata ~x~ tho..high apafital, gr~guenay Gongsone»ta, whf.ler . . r ,_
maintaining tl~e . iramet refresh ; rake , of : at 1~ast a ~po~t.ion ' o#'~ ~
'
the ,low spatial ~.~f~equen~yr , cop~ponsnts.~ , atv th~ standard rate o~~

CA 02379330 2002-04-16
wV y1i I~YLy : v ... PCT/U591/02228
3
If done i~ a. specified. mannerwthis: will= not'- cause=- substantial-
degradation in. the: ultimately displayed. image,=-~-sincew .humane ~
°~
vision cannot' perceive changes, ~in:~ high spatial'- resolution'w ~ ~ '
information at as. fas a. rate as: it caw perceive changes in -
low spatial resolution-information. Accordingly; as~has~been
previously , set - forth, - an. , electronic video r encoding and - ~ - ~ -
decoding system can"be devised_which-takes ~advan~tage'of=this,
and other,. characteristics_of,human.w vision by encodingvhigher
spatial. resolution..video, components; to be.at~'a temporal
information rate.which< approximately corresponds~to the w
highest rate actually,: perceived by human.vision~ifor such w -
components; thereby.. eliminating the.need to encode these . -
components at a higher rate,-which inherently-wastes
bandwidth.. Also, as shown ~in referenced patent and v
publication,. the low spatial resolution information can be
generated in a form which is compatible with standard
television video, for examgle.NTSC video used in the U.S. It
has also been recognized that a number.of-frequency
components can.. be transmitted at specified rates [see e.g.
W.F. Schreiber et al., Reliable EDTV/HDTV Transmission In Low
Quality Analog Channels, SMPTE: Journal, July 1989, and the
abovereferenced patents of-the present assignee], with
components selected according to degree of motion in order to
have higher spatial resolution in-scenes with little~motion
and higher temporal resolution in scenes with a great deal of
motion.
Fig. I illustrates.a compatible. high definition
television transmission.,and receiving system of the general
type described in the above-referenced patents and
publication. A transmitter 200 includes NTSC processing
circuitry 21,0 which processes television signals from a
source such as a television camera system (not shown) or a
video recording system.(not,shown)....The'circuitry 210 is
coupled to transmitting circuitry,215,.which typically --
includes modulation circuitry::and other=suitable=circuitry .
for groducfng a signal:to-be,:transmitted over:a=standazd NTSC ~ -
channel . The television. signals. fromr the televisidnm r camera

CA 02379330 2002-04-16
WO 91I15~tZ9 v. .' .. PCFlE1S91I0?.Z2~ .
4
system, or,wideQ.;, r~entder. (wh~.ch is~~assumed~~to have a high
definition__vidaa;~caga~i;lit~r)' are also prab~~sed by high .
def~.nitian.:telev~si,on., (Hp~V) processing circuitry Z6o which
produces detail.; sigma#s. that can be~ ut3~li:zed to enhance
corive»tional televiaibn sis~»als to'obtain HDTV signaxs, as .
described in_ the, abovsre#era»ced paten'ts~~ and publication.
[ As furthar: deac~ihdd ~.n the. referenced t1. S : pate»t I~o. ~ .
~, ss2, 909, . the,:,detail ~ aign~si can. be obtained ~ from a'. separaxe
camera. The datail=~t~.c~nals are coup#ed'to ~uxther airGUitry'
275, which trans~tits ~ the detail signa~.~~o~rer a' second
.(auxiliasy).:chann~3.~that ~.s typ#caZly not adjacent to the
(maf.n) NTSC ahan~a~.v.uscsci fox .tranami.a~s3c~n of the staridax'd ' .
pprtion of the ~keieVision it~~oxmatfon. The NTSC signal is
received by receivers auch~as.rece~,ver 31Q which has only a
capabil.#~ky.. of pracl~ec#ng a te7.ev,ision picture at substantially
coaventioral r~rsol;ution e.g. canvant~ivnal disp7Lay 315.
Receivers such as. receivBr 360, which have a capabixity for
receiving, prACessirig, and. displaying high defir~itio~r
talevis~.on sig~tals, xe~oive both the mai» chann~1 carrying
the NTSC sig»a~. and the aWx#lia~ry channel carrying the detail
signa3s to be used for augmentation of the NTSC vid~o signal
so as to produce a hi.c~h de~i~n3tion television. signal #'or
display on an HDTV dtspiay 3b5:
In the refexenceci patents and pglalicat,io», the spatia3
detail. is trans~ntitted at a relatively clew frame rate, such
a,s 15 or 7.5 frames per secpnd. "Juttexr' (jerky edge motion)
was obsarved when thh.detai# frat~e rate was reduced too far.
Th~.s artifact constrains the augmentation channox bar~dwidth _
to be ~.argex than wou~.d athexwise be fnd#ce~ted by
psychophysicai,stud#es. Camera lag, c~iua~ed by the
i»tegration o~ image energy o» the face v~ the camera tube,
which attenuates, detail i.n moving araas~af tHo picture; can '
be exploited to reduce ~uttsr, but some'reductian in image
dataii can be obsesved~ in mov3.riq~ ab jectw When ' their are' ~ - ..
v#~aaa~.ly tracked.., . .ttv. ia~.amv»q they obyecta~ of the present ~~ '."
.invention ;to' pro;vir~~,; improva~ritay in' perfb~ince and° is ~- . .
,
bandwidth - ca~press#.ori with respect. to: ths~ techn,id~ues ~ ' ~ '

CA 02379330 2002-04-16
wv yii i~yiy PCT/US91/OZ228
..
described. above, - and.. with respect to:- other-priorvartv = - -
techniques . It ~ is- also; - among=. the- objects hereof' to" provide. " ' '
'
such improvements.- in a- :- system; that. can. be' made compatiblev
with existing:,.or:,.future~_.; television:. standards ( for example,
NTSC, or other standards.such as PAL-or SECAM). ~ ~. - ~ -
As further background;to-the.invention; reference can be -
made to the following U.S.:Patents.which relate;to~~~--=- ~~-~ -
compression,;transmission.and/or other processing of videow-
signals and/or still picture:-information:~-=U. S. Patent
4,196,,-448,_ 4,210,931;- 4,:224,678°;:4;302;775;:4,394,774;- y
4,541,012, 4,605,952, ,4,630,.099-;= 4,661,862;w4,672,425~.-'
4,675,733, 4,675,750,:,4,729,012,_ 4,774,562, 4,780,761,-
4,791,598, 4,807,029,.4,821,119,-_.4,845,562, 4,85T,906;~ -
4,870,489 and 4,873,573..
The FCC recently announced that it prefers planned
terrestrial HDTV transmission in the U.S. to be broadcast
using a simulcast format: i.e., with the same program content
sent simultaneously both a conventional,.television channel and
a separate HDTV channel. -It has been anticipated that, in
time, television viewers,_will.replace standard NTSC receivers
with high definition sets,: thereby allowing the present NTSC
channels to eventually be reassigned for other application.
In order for this concept to.work, however,wiewers~must be
motivated,to purchase receivers:designed.to accept this new
format. Even when wide-screen HDTV becomes available,'a
significant demand-will always exist for smaller=screen
receivers. The image quality of small screen-size television
receivers is generally not limited. by transmission
considerations, but by,human visual~acuity. The optimum
viewing distance for popular 19-20'.'.:conventional receiver, for
example, is between six and seven feet. A similar screen-size
HDTV receiver has an optimum viewing distance of about three
feet;~clearly impractical in most viewing situations:v. The
goal of abandoning.-the,conventional NTSC channehs iri the
foreseeable future may be::impractical because therevwill
always be a..consumer,demand;,for-inexpensive smaller screen
television sets . . . =. - .. _ ..

CA 02379330 2002-04-16
WQ 9111~9Z9 . . . , pCf/U891/QZ22$ '
s
It fs~ alsQ. amongv the;. ol?jea~s: af~ the pxesen~t- invention- to ~'.. _.. .
provide. ,improvement~.~ .1» enrodingw aid decoding ~ of ~r3deo
i~formatian: which addresses: the~° d~scr~.bed problems and '
~.imitatio»s of.~the; prior_.$rt, achieves substantsal~ bandwidth ' '
savings, increases the efficiency of v,~deo transm~.ssion and
storage,,aRd..proYides a capability.for~.higher definition
tea,ev3s,ion xansmission=in.~he bandwidth pf~a single . ,
conventional te.leviaiow channel. it is a~.so among the
objects of the, present.invsntion to provide a technigue
whereby two video signals, represantative~~'of differe»t images
can be transp~itted uainc~ only the bandwidth r~enerally
olloee~'ked tc~ a es~.ng~.e v3:dcP signal, ~i~.tlt .little of »o
pezceived degradation of image quality. It is also among the
objects of the present invention ~o provide a method fox
broadcasting video signals w~.th improved interference
immunity.
~ . : : SUMMARY f~F THE INY~NTI~1N
A method and an appax~tus are sit ~oxth~fox~ encoding and
decoding video to-achieve bandwidth corppt~ession. I» one form
of the ~.r~vention, two video sig»als, representative of
different images, can be transmitted using only the bandwidth
generally allocated to a single video s~.gnal, with little ox
rno pQrceivad degradation of f~nagd qe~ality. In another farm pi
the i=tvention, motion indicative signals-aye uaed in a
technique that dynamically ~oclifies.the'frequency band
intarmation to be stared andfor transmitted.
Further foatWro$ aac~ advantaqQa 'of' the invention will
become mcre,readily apparent frog the following. detaiied
description when taken it con~unGtion with the accompa»yirtq
drawings.
BRIEF p~SCRIP~'IU~THE hRAWINGS
Fig. 1 ~.s a siatp~.ifasd brlock diagram of' a prior
campatibi.a high, dett~i~ioa teleVti3.io=i' 9ysteilt. ' .. ' ..
Fig. ~ i~_,a pc~Iar: plot .~llustrati~ig data that sie~aareg
the obligtse eftect. ; _ .

CA 02379330 2002-04-16
WO 91/15929 . :w - PCT/US91/02228
F.ig:.. s.; 3A,.; 3B and._;3C:-. respectively- 'il~liistrate= cardinal
sampling, quincunx.sampling,and=quincunx sampling-with-w-' ~ -
reduced sampling rate. . .- . ... . . . . . ..
Fig s 4A, 4B and:4C respectively illustrate spectra for
the Fig. 3A, 3B and 3C situations.- -. ~ -.~
Figs 5 and 6 illustrate spectra referred=to in the
description.. -. . : ,... :'.= r. ~ _ : -. -:;. v v ;-: :. : . ;::
Fig. ? illustrates the four quadrant pass band of~a ~~ -
two-dimensional diagonal.:filter.having its vertical and
horizontal cutoff frequencies:.at half the video sampling
rate . .: ::.: ~. : :.. : . ::. ~. .. ': .
Fig. 8 is a block..diagram of an embodiment of an encoder
and encoding.method in accordance with a form of the
invention.
Fig. 9 illustrates an example of the coefficients of a
9x7 filter kernel array....
Fig. 10 is a block diagram of circuit which can be
utilized to implement two-dimensional convolution with a
filter kernel.
Fig. 11 illustrates an:.example of two-dimensional
spectrum folding around a diagonal which occurs as a result'
of a two-dimensional image modulating a two-dimensional
subcarrier.
Fig s 12A and 12B respectively show a pixel array before
and after two-dimensional:modulation. . w
Fig. l3 illustrates a circuit for implementing two-
dimensional modulation:
Fig. 14 shows a block diagram of an embodiment of a
decoder and decoding method which can be utilized to recover
signals encoded in accordance with a form of the invention.
Fig s 15A, 15B,: 15C 15D and 15E shown illustrative
spectra. , ,.-
Fig. 16 is a block diagram.of an embodiment of an
encoder and encoding_method in accordance with another form
. . -. . . _ . _; _ : , ._ , o _- , . :. . ..
of the invention. . .. _ _ _. . . : ... .. . _ .
Fig. 17-is~a diagram=of-:.a.decimator Which can be
" ,.
utilized in an embodiment of the invention.

CA 02379330 2002-04-16
WO.91/15929 ~ ~~. , . PCFlUS9llOZZZ~3
F~,g. :1~.. is: ,a, b~oc~:.,d~.agram.~.of_ anv embodiment of a decoder
and decoding, msthcad. which can.. be utii.iaed to decode the
encoded signals o~ a l'orm of the inrrenkion. .. _. ..._.. _._ ....... . _ .
Fig.. 19. is a diagram of a zero padder which can be
utilized in embod.lments of the iltvention. -
FI:q. 2p is a block,diagram of an embodiment of an
Encoder and~encoding method in accordance with another .fo=m
of the invant~.on. ... . . . . . . , . , .. . .
Fig. 2l is a block diagram of an.embod.iment of a decoder
and decoding method whfch can be,u~iliaed ~a decode signals
encoded in accordance with a further form of the invention.
Fig. .2a is a .black diagram of a» smb~ad~.ment of a high-
definition t~levl.sion syt~tem and. method- in accordance with a
further form of, the invention.
Fig. x3 illustrates a furthax form of the invsation that
,is used to minimize int$rferenca botween traasmi.tted v~.deo
signals.
F~.g.s 24 anc3 25 illustrate exemplary. band divisions of
tha Fig. 6 apactrum.
Fig. 2C is a block dl.agrata of an encoder in accordance
with an embodiment. of the irwention, ~ alld. whf.ch can be used to
praGtfce an embodiment of method o~:th~ fnventl.on.
F3g. 2? ~;a a black diagram of a system for encoder scan
conversion.
Fig. 28 3s~a block diagram of a portion of the encoder of
the Fig. 26 embodiment. _ . .
Fig. 29 is a flcW diagram a routine for coxltrol.ling the
tile control processor of Fig. 2,8. .
Fig. 30: ~i~3 a blpck d#:agram of an embodiment of the mpt3on
detection circuit of Fig.. a8. ~ -
Fig. 31 is ~a bloick dl.agram of- a decoder fn $ccordance
with as embodiment of the invention and which can be util~.z~d
to practice an embodiment,, of. the, dl,~c~;ps~ decoding method.
Fig. 32,.;which includes Fig:s.32A and 328..~laced one
below at~nthQx, ire a #low .diagram of ther ~cotstinc for ~ w
cantro1.1.1.ng vthe attc~rtentation in~rwt ~ processor.: of~,the Fig: 3 i
embodfonent. . .. , . --_. ~ .

CA 02379330 2002-04-16
WO 91/15929 -_ _~ PCT/US91/02228
g ,.
Fig.s.. 33 .and 34~, are- flora diagramsvof the- routine'~for" -
implementing -the- spectral-to-detail converter' control ~°"
processor of , the. Fig .. 31° embodiment'.' ' " -
Fig. 35 is, a block diagram of a fifo circuit utilized in
the Fig. 31 embodiment..
.., DETAILED.. DESCRIPTION
Subjective:.vision_studies have indicated~that perceived
resolution is anisotropic°(not equallyrprecise in all'-
directions).;. The eye is more sensitive~to detail along the'
horizontal and. vertical r. axes- than' to~ that along diagonals:
[See, for example, W.E. Glenn et.al:, "Imaging System Design
Based On Psychophysical Data," Proc.~of the SID, Vol 26/1, pp.
71-78, Jan. (1985); NYIT STRC "Visual Psychophysical Factors
as Applicable to the Design and Development of Video Systems
for Use in Space, Final Report," NASA Report, May (1989); 6.C.
Higgins et al., "Variatiow of. Visual Acuity with Various
Test-Object Orientations and Viewing Conditions,''' J. Opt.
Soc. Am. 40, pp. 135-137 (1950); F.W. Campbell et al.,
"Orientational Selectivity of the Human Visual System," J.
Physiol., 1.87, pp. 437-445, (I966); and S: Appelle,
"Perception and Discrimination as a Function of Stimulus
Orientation: The "Oblique Effect"'in Man and Animais,"
Psychological Bulletin, VoI. 78, No. 4, pp. 26f-278, (1972).]
Fig. 2 illustrates this oblique effect, plotted in polar form,
as characterized by various researchers. While results differ
somewhat due to the different types of testing employed; the
curves of subjective resolution have a similar shape and
diverge from the isotropic resolution shown by the outer
circle. It is known that bandwidth and. display element
density can be, reduced by taking advantage of the anisotropic
spatial response.characteristics'of thewisual system. Fig s
3A, 38 and 3C respectively illustrate cardinal sampliwg,
quincunx (or diagonal) sampling, and quincunx sampling with
reduced sampling. rate., Figs 4A'~'48 and 4C show'the~
respective discrete spectra . for-- the ~ sa~epling- of Fig. s 3A~, 3B
and 3c, where fs _-. 1/D. _ The,, quincunx samplingshown in Fig:'s

CA 02379330 2002-04-16
WA 9!/59'l9 P4~f/~fS9tlp~
1~
3B and 3C re"suits. ;irk the. rotation. of. the spectral coaxdinatr
axes by 45 degrees...(see,.; for.. example, . R.C.-. Gonzales et al. , .
Digital image ProceSS~.nc~, Readi»q~ Mass. , . Add~.son-Wesley ~ . .
(1957); 5. Dubois et al.,'!Three-173mensional spectrWm and
pracpssion ~of Digital NTSC color S3.gnels, ~~ SMPTE ,3ouraal, pp.
3?2-3?8, April ( 1982 ) ; and 8. Wes~dland et al _ frOn Picture
duality of Same Telev3.sion Signal Frocessihg~~Teahnic~ues, ~~
5M'FTE Jaurnalr:PP~,935-9x2, Oct., I19~4j], thereby more
closea.y matching_.'the characteristics of vision. This method
cart be used to-. xeduce the. 3.rifo=matiort content by a factor of
two withoctt degx$dation_ia perceived 3;mage quality. Half. tone
pr3.nt~s anr~, more recently, CCD cameras and LcD displays axe
sttecessfully uti3izf,ng ;this tech»igue: soma of the systems
described in the. patents seferencsd fn the Background portion
hereof utilized gti.inct~r~x sampling tp reduce the sampling rate,
arid therefore thg augmentaxien ba~widtn, by a factor of two.
In an embod~.ment described below; information cont~ant is
reduced bar eLi.minating high diagonal freguency components
appxoxiatate.iy to the upper right of the dfagnnal line 5 in the
discrete spectral domai.ri ~llustratad tn Fig. 5. The NTSC
luminance spectrum is il3.ustxated approximately in xh~ lo~wor
lefthartd box'of Fig. 5. rn a subseguently described
embodiment, far an aWgmentatioh System wherein the NT5C
apaatral portia» is avai~.ah~.es from a GonV8nt10»al. channel:,
the approximate rs~ainiag spectrum used for transmission an
the augmentation ~ahattnel .~s.shown 3.n the shaded region of Fig:
A Sow-pass filter can be used to restrict the treguency
cnatpopents of a cardinally sampled fmape to the region within
the diamoad~shaped perceptivity curve of Fiq..2. A telev~.sion
viewer, postt3:oned at the mutate favorable vfewing distance
roughly six screQt~ hoights fot conwent~.ons~, NTgC~ 525 line
video); is not obis to resolve the (vertical) video raster,
yet 3.e sill. able tQ,app=eciate image detail.. Fi~u. re ? shows
the tour quadrant pass--bar~a ~f a.: two-dimensional ~ diagonal v .
filter hav~nq both its . verti,csl attd- horizontal cutaf f
frequencies, set to~~o~e--half the video sample rate ( fox
r

CA 02379330 2002-04-16
WO 91/I5929':' PCT/US91102228
1 I -.....
example, one-half the vertical sampling rate):- One-half ttiewy
total spectrah- area~~is :passedw by this-. filter: The effective
two-dimensional bandwidth-for-.-imageswmatched to the
characteristics of.the human..visual system is only one-half of
that created by cardinal sampling. In- accordance with a
feature of~the present~invention, useful video information is
two-dimensionally modulated so as to be positioned in the ~w
shaded portion of Fig..:7; i.e.,~ into:a spectral region that
has beew effectively:unused and,genex°ally-wasted in prior art
systems.. In an embodiment hereof, two different television
images, for example, can. be encoded on a single transmission
channel by effectively placing. one in each of the two distinct
spectral regions. Applicant has discovered that each
television picture. can maintain substantially the full
subjective resolutiow found in the original, and is completely
separable from the other television picture.
Referring to Fig. 8; there is shown a block diagram of an
apparatus in accordance~with an embodiment of first form of
the invention, and which can be used to practice a form of
the method of the invention. Two electronic video signals
are produced, as represented by the blocks 810 and 850,
respectively. The electronic video signals may be generated
by any suitable means, for example by video- cameras; video
storage, graphics or animation generators, or medical or
other imagers, etc. It wilf be understood that the blocks
810 and 850 may represent respectively different types of
sources of electronic video signals. As an illustrative
example, it-can be assumed that the blocks 810 and 850
represent electronic video camera systems directed at
different scenes. As described hereinbelow, the signals can
also be representative of different components'of~the same
image.. Also, it will become understood that~the techniques
are applicable to: various formats of electronic video signals
and to conventional as well~as low o= high' definition video:
An example is initially:set forth-iri' terms of avmonochrome ~~
video.signal.having.conventional' television resolufion,-v
although...the techniques~hereof=are- also generally acceptable '

CA 02379330 2002-04-16
wo 9~i~s~z9 - . ". ~cr~uss~iiQZxzu
12..
c ,
to color video_signals:~:~. . _ .. . " " .. ,... ._ ._ .__. .... . _ .. . .._.
. _ _.....,.. .._.
The "outputs:, of video signal sources . 81D.. and 85Q are - . - ---~- ---
reapeative7.lr coupled ta.~I,ow;pa~s-:fijaers BI5 a»d- a55~ (see -.:~,.
also two-dim~ansional spectra si5A and 855A), a»d then to
analog-to-digital cpnvettexs 824 and 860. The analog--to-
di.gital converters can be operated atany suitable clock
rate, in knowin. fashi.~», to obta~.n frames of digital pixels
which are .stored-,~,n digital buffers 821. and 861,
xeapea~~.vely:; The k~uf,fers may be frame buffers. or portions
thereof . each pixel- of. each Exams- c$» have a luau»ance value
convent~.4»ally rispresent~ed by an n-hit digital word.'
Thu outputs of fray buffe~rt~. 8~1 artc~ 861 are ,
respectively caupled,to two-d~:mensional diagonal low-pass
f3.~.tsrs 8~5 and 865. Faah of these Tilt~xs is operative to
remove high fregusncy two-dimdusiona7, diagonal fregueacy
componentr~ from the fr~a~tcea of digitised video r~ic~naZ. Fos
example, for the approx3.mately squaxe spectrum of the first
guadxa»t Fig. 7,. the filtering of the px~serit embodiment wil.I
p=efesably result i» a.spectrum having an approximately
triangular shape as in. the unshaded .regfat~ in Fig. 7 ( s~:e
a3so the sketches 825A and 8fi5A). It ~ri~l be understood,
however, that the, line joining the highest paaseci vertical
arid horizortta~. freg~~nc~.as [ ( f~« ~ constant ] ca». genera l 1y be
aosslgcnsg as a-boundary. As noted in co»~;u~ct.ion with the
description of.,Fig.,2,:_there are inveatigatars who have
determined that event:som~ freguenciea within the i»dicated
triangular regsoa will De Substantially attenua~ad by the
human vfauai $y~tem. The precise shape of the fi.lto~c can b~
determined frog present and/or future studies on ths~ humavrt
visua?.:system, arid/_or aan be adjusted empirically.
The twa~dime»~sic~al diago»al low-pass filter (825 and
a85) aan,be.impleme~ted,by any su~aable technique. For
examgle, a commercial programmable fflter kernel can be
utilised,to obtain"the desired;fi3teriag function. . F~:g. g
..il.laatrat~aa ,ans.. exs~m~ale of._~GhB coatf3ci.ente~ of 3 9x7 'fi.lter. '
.
kernel array; that, cart, be., ~utllized to implement two--diate»sional
diagQnar. lob-paaa_ l~il~ex3.ng.., ; The. filter. kernel. cart be applied

CA 02379330 2002-04-16
WO 91/15929 . PCT/US91/02228
' 13 __
by convolving the . array.-with the frame of ~ pixels tow be '
filtered. , Techniques -far--:implementation- of the filtering " _
process, are.known in.the art. Fig.~lOvshows~-a block diagram ...
of a circuit which can-be,utilized to.implement w
two-dimensional convolution,.-.and which. can be employed, with
appropriate weighting coefficients, in the present embodiment
to implement a two-dimensional.diagonal low-pass filter: In -
the circuit of Fig..: 10,: an: array of : coefficients kij, 'are ' w
applied to an (mjx(nj moving group of pixels by using m line
delays 1020 and n pixel~.delays which are indicated in Fig.. 10'
by representative register rows 1025, each of which has
individual stages with respective one. pixel delays. Shift
registers or,FIFOs may be used for~this purpose. Each pixel
and delayed pixel is multiplied by a coefficient, kij, with the
coefficient values being implemented by applying corresponding
signal levels to the multipliers 1050. The coefficients can
be in accordance with the selected array for a particular
filter kernel, for example the array illustrated in Fig. 9.
The outputs of multipliers 1050 are summed by a summing
circuit 1080 which produces each convolved output signal as
the array "moves" over the frame. It will be understood that
other filter implementations, including commercially available
chips, can be utilized,, if desired, and that end conditions
can be handled by techniques known in the art.
Referring again to Fig. 8, the output of two-dimensional
diagonal low-pass filter 825 is coupled to one input of
summing circuit 880.' The output of two-dimensional diagonal
low-pass filter 870 is coupled-to a two-dimensional modulator
870, which serves to fold the.spectrum of the filtered signal ,
into the spectral space normally. occupied by the high
frequency diagonal; components. Fig. 1l illustrates an example
of the two-dimensional spectrum folding which occurs when a
two-dimensional image modulates a two-dimensional subcarrier
having (in this casej,.,a"horizontal frequency of half the
sampling rate and a vertical frequency of'half the sampling
rate. In.,general, the.image spectreim will be folded:~around~~
the diagonal ,demarcation<linewand.reversedw'so-that'~hi.gh

CA 02379330 2002-04-16
wo 9ms~ra . rc-~iu~~imzz~a
i~
l r
freguency hvrizonta3w co~pp~ahents:-. fl f the original: image becamQ
high freguency. vertical components and vicewersa~ as sshown ~n
Fig. l1.. A carist3n~ gx'ay~level (dc~ on'thawencoded~~~ma~ge~wfll .
then appear as the highest possible.frequency which~cari be w
represented with the sampling:Paremeters,v f5/2, f8/2. ~in ether
wards, , the spectra. location ( t1, A ) ~ after suoh ts~o-dimensional .
modulation. wi,l~.,_be a~ (f~12, f~12) and~coristant ihtensity
~.cve3 will generally e~ppeh~c as a high freQiiericy checkerboard
pattern. [For this modula~kion in the 3:mage bs3.ghtness domain.
being modulated in. this casewas brightly above the average
gray 3evel as below itr: the entire image.content would be~last
and appear as n blonk gxay ac~een to :x viewer more than six
screen heights from the display, since the spectral content
is outside the range at hcuuan percaptivity. i.e. the he-man
visual system would act as a diagonal low-pass filter. The
spectral location (0, tx), after much two-d.~menaiortetl
modulation, will be at (~~/~, f~12-f,~), and so on. The
two-dimensional t~advlation to achieve folding around the
diagonal. can be implemented by reversing the po3,aritlr of every
other pixel on evesy line, with the polarity order reversed
every other ~.ine, as illutstrated in Figs l~A and Ira. Fi9.
12A shows a pixel arrap before two-dimensional modulation on
the two-dimensional. subcarrier, anc~ Fig. lab shows the pixel
arzay after such modt~lati.on. F'ig. 13 illustrates a aiFCUit
far implementing this ipadulatfon. A mult~.plexer 1350
receives, at one inpux, the pixel output of two-di8ensional
filter ~s5 and, at another inpux, tMe pixel output inverted by
invexter 1310. The modulator receives pixel and lino
ind#.cations and al~etnates~~its input line select~.on sequence
in accordance with the~polarity a~quence shown in Ffg. 128.
The encoded output of summing circuit 8~0 contains video
s.~gna~.s representativg of both 3.mag~as (see ewo-d3.mQnsicnal
spectral sketch 8~~A.,which denotes the respective vidso~sfgnal
spectra as x and Z), can be stored andlar transmitted, as
rwpresented by;tha b~.aak 89p.. . . .
Referring ,to. i~~g.: :,.I4, there: is ahpwn a blocl~ diagram of a .
decoder which can be utiZized.to recovex and record and/o~c

CA 02379330 2002-04-16
WO 91115929 .... PCf/US91/02228
15 -
display the encoded video-signals: The block 1410-represents ~~
the receiving of the encoded signal-or the reading thereof .-
from storage. The two-dimensional frequency spectrum is shown
in the sketch,at,1410A.,,._,An,optional pixel.storage-buffer 1415 --
can be used to store frames of-information or. portions
thereof. The output of buffer:1415 is coupled to a
two-dimens'ionah modulator.which can operate in the same manner
as modulator 865 (F.ig...8), as illustrated in conjunction with
Fig s 11-13. The_previously.de5cribed spectrum folding
reverses the spectral positions of the video signals. The
output/of modulator 1450 is coupled to,a two-dimensional
diagonal low-pass filter 1470, which may again be of-the type
illustrated in ~conjunction_with Figs 9-10. The output of
buffer 1415 is also coupled to such a filter.(1420): The
respective outputs of two-dimensional low-pass filters 1420
and 1470 are coupled to digital-to-analog converters 1425 and
1475, and then to analog (horizontal) low-pass filters 1430
and 1480. The output analog video ignals can be recorded
and/or displayed, as represented by the blocks 1435 and 1490,
respectively. In operation, it is seen that the
two-dimensional, modulator operates to "reverse" the spectral
locations of the signals identified as "1" and "2" (see sketch
1450A). The filters 1420 and 1470 can then be utilized to
filter undesired spectral components and obtain the
respective separated video signals. (see also sketches 1420A
and 1470A). After conversion to analog form and suitable ....
low-pass filtering, the recovered signals can be recorded
and/or displayed, as desired.
In accordance with a further form of the invention, the
size of an image may be reduced,.. or squeezed, by :;ub-sampling
after first filtering to remove frequencies which would
otherwise cause aliasing. ,As will be described, an image can
be reduced to half its original.size by throwing away, or
decimating, alternate pixels, horizontally and._yertically.-
Spatial decimation can.also.be;used to reduce~.the.data content
and associatedytransmi sion_bandwidth of_non-squeezed~iniages -
without substantially.: degrading-, the subjective resolution'.- - = w-- .

CA 02379330 2002-04-16
W4 91!159Z9 PGT/US9I/D22'tii
3.6
$e~ore-constder.ing' two-~dtmensi,onai~ ddctmat~,on; i~°'is ~helpfu~.
'.. _
tea examine theweffeGts of- one-dimensional decimation: ~tf it'
is assumed that avcontinuc~us' ' analog s~:grial is~ satppled using ~a
d#.gitizer which has an ~.nfin3.te?:y small, sampl~,ng aperture, '
the digitized i.xsput card bo considered as a conttilllat~ts dndla~
sislnal mult3.plied by a series of impulses spaced Ti secar:ds
apart. The samp3ed spectrum'cnn$ists of th'e oxigin3l ana~.og
spectrum convolved with spectraa. impulses located at n~~ where
f~~i/Ti. Repetitions of the analog spectrum w1~.1 thus be
centered about integral, rnultiple~, of the samplfitg frequency as
shown in Fig. 15A. No a3;~.asing will occur if the analog ~~
base~aand~spectrum fs resxricted to frequencies less than fg/2
prior to sampling. Assume, now, that :even and odd saatples are
separated using the fc~llowtn~ even and odd dec,imat~.ng
functions:
Dk.''Cove~ ' ti/Z:)r1 t aos2pi*(f~/Zj(X/t~)~
(1/~)~l. + coa(pi;x)j
1~EC,~d -- (l/2) ~l - cos(pi*x?
where x s integral sample number'
Apply~.ng DEC~,.A t° the data set wLll force the odd date
elements to zero whii.$ pEC~4 wLll i~orca~ the even elements to
0. The spectrum of an evenly decimated image is obtained by
convclving the sa~rip3ed spectrum with the spectrum of DEC~n.
rig. Z~a shows that dscimat.ion caused another repetition of
the analog spectrum tc~ he c=sated about f~/2. Alids3ng wf.Il
pacux for fregusncies greater than f~/4: DEC ~4 viii create a
similar spectrum, but the a3.fased components wsll bd f,nverted
with the =aspect to the alta~shd spectra dram DECD. When the
add and even compapanta are. added, the wnslf.aeed ori,gina.l
components add, but aliased components canc$l. leaw~.ng the
origf.hal spectrum. While this result ~.s f:ntereeting, it is
of little pract3.ca~, value in one d,~mens3on= the same result
is obtained by s~amplinA the analog .i.nput at fw/~.
Spatial deGimat~on, however, does not lead te'the
trivial. rasu~.t encountered in'~ one dimension;' provided a ~ '
diagonal decimating-pat~ernvis used: ~A d~:aqonal decf.matox ~ ~ .
ata~aks an image.with a checkerboard pdttern of ~a.g': black)

CA 02379330 2002-04-16
W4 91./15929 ' PCTlUS91l02228
17 .- .
dots. The even decimator can be represented as:
', DECdia9 =. (1/2) ['1: +: cos(pi*x}cos(pi*y) l .-. _....-..
The spectrum_.of.the~decimator is-composed of two
frequencies,- --one _ at DC : ( 0, 0 ) , the.: other- at half -the w
horizontal and vertical.sampling rates (fh/2,f~12).~ It is
analogous to the one-dimensional'decimation spectrum except
that frequencies are two-dimensional. An image to be
decimated can: be first. passed:: through a two-dimensional
diagonal pre-filter of the type previously described. The
decimated spectrum is obtained by convolving the filtered. ~ _.
image spectrum with the spectrum of the decimator. The
original-filtered.spectrum is repeated about~multiples of the
decimating frequency, i~.e... ..
_-(2m+1)f$/2,(2n+1)fs/2 for.all m and n
a portion of which_is shown in Fig. ISC. Passing the
decimated image through another diagonal filter regenerates
the undecimated image without degrading the image quality.
To summarize the. foregoing discussion, when a diagonally
pre-filtered image is.spatially decimated diagonally the
spectral components introduced by the decimation process do
not extend into the original spectral region, and hence may
be removed by filtering.
Diagonally decimated images contain half the number of
pixels per line of cardinally sampled images. The time
necessary to send a line of.video can be reduced by half if
the bandwidth is maintained constant, so that the image will
be squeezed horizontally- The second half of each video'line
may then be used for other purposes, such as carrying a
second image or carrying high definition components of the
hasic image. Squeezedwimages'can be reconstructed at the
receiver by reinserting~.the zeros created by the decimator
and passing it through a two-dimensional low-pass filter.
Fig. 15D is-a representation of a two-dimensional
frequency spectrum.of a:decimated image which is composed of '~
repetitions of:the-cardinally-sampled spectrum about -
multiples of~ the decimating frequency, 3.a. at w ' '' w'
. . ( 2mi'l ) f ~~~~n2 ~ ( 2n'f'1 ) f ~u~u~ ' . ., _ ..

CA 02379330 2002-04-16
wo ~ii~~9z~ . pcrms~~razzz~
1e
fax all intagra~. values of m and~ »:: -__. . . . . - _ .. . .
Freguency- alias,~ng, will occur .if' the image is' not
prapexly pre-filtered and likewise, ~.f the'zsrc--padded'
xeconstructl:on is;nvt adeguately post-filtered; F~.g. 15~
'shows that a tuo-dime»sianal fiit~r w~.th 4 diamond-shaped
pass-band that can be used fos both. the pre- a»d post=
filter. It removes anly,thase specxral~compo»e»ts outside ..
the se»si~kiv#~ty range of.,the .human ,visual system. pictures
with the same subjective s~ual.ity of the or3.ginal image can,
therefore, be, reconstructed from a diagonally-decimated
image.
Referring to Fig., 1b,. there is,, spawn a blank diagram of
an embodiment af'a form of thg invention which uti~.izea
pr~.nc~.p~,es dust set forth. Twc~ sources of video s.~gnal, .1610
a»d iGSh are provided, as prerr~ously described in aan~unction
w~.th the desc=iptinn of f'~:g. ~ . ~e si.gnals are harizonta~.ly
lAw-pass f3Ztered and a»alorl-to-cii93x$1 oouverter as
~epres~ented by the blocks 1615, 1620 and 1655, 165D,
respectively (see axso sketches ~.615A and ib55A). The
outputs of ths.analog-to~-digital canvexters'are respecti.vel.y
two-dimensiorial~.y'low-pass filtered, such as in the manner
previously described. as represented by the blocks 1625 and
16b5. The resultant ~ssgecttve two-dfmeresional spectra are
rep~raaented .in. sketcttss i62SA anQ 1665A~ Tile fl-lured 3.magQs
are then d~aci.mated, i» the Manner previously dgs~cribed, apt
represented by the blocks,l634 and 160, respectively. The
pattern of pixels selected by the decimators are illustrated
in the diagrams 1631 and 16'i1, respectively. in xhe present
embodiment, only the selected alternate pixels (as
.illustrated) are passed by the deeira,~tors, sod this ears be
implemented, far example, by using.vo~aage--controlled gates
as illustrated irt Fish. 17. zn pe~xticular, li»e and pixel
information provid~d to the controlled gate deteria~.nes
whether the ;pixel. 3;s passed by: _ the decia~ator.y ~~,tex»at.~vely,
a flip-flop, which.. is: set to. .. a different inf.tial~ status aaGh
Line, could be utilized. The outputs of decf.matars ~:f30~ and
157p are respectively coup~.ed to t#:me bhse'eompressors~xb35

CA 02379330 2002-04-16
pcrrvs9iro2zzs
19
and 1675 which. operate:.. to. compress each line of pii~els into
respective. halves. of the- original .tine time: Time base
correction is very; well .known.in the art, and any suitable
time base compressor can -be.utilized for this purpose. - The
compressed two-, dimensional spectra of the resultant signals
are represented.- in_sketches 1635A:and I675A, respectively.
It is seen,that the horizontal component~.is doubled. A ' w
multiplexes 1690;is then utilized to combine the signals
during successive time. slots, as illustrated in the diagram
1690A. The resultant ;signal can be stored-andrbr
transmitted, as represented by the. block 1695.
Referring to Fig. 18, there is shown a block diagram of
an embodiment of a decoder: which can be=utilized to decode
the encoded-video signals of-the Fig. 16 circuit. The block
1810 represents the receiving or the reading;from storage of
the encoded signal. A demultiplexer 1815 is utilized to
separate the signals in the two time slots of each line, and '
the demultiplexer outputs are respectively coupled to time
base correction circuits 1820 and 1850, respectively, which
perform the converse of the operations that were performed by
corresponding time base correctors 1635 and 1675 in the
encoder. The outputs of the°time base correctors are
respectively coupled to pixel storage buffers 1825 and 1855,
and these may comprise, for example; half-frame stores. The
outputs of the buffers-are respectivelg coupled to zero
paddsrs.,1830 and 1860 which operate to insert zeros at
diagonally alternating pixel positions, as represented in the
diagrams 1830A and 1860A, res3pectivel.y. A circuit which can
be utilized to implement-the padders is illustrated in~ Fig.
19. In particular, a~multiplexer can be utilized which
receives,. as inputs, the pixels output.froia the buffers (1825
or 1855) and a-signal representative pf~a zero level.
Selection of the value to be-pa~sed..by-the'padder will depend
on the line and pixel information input'to-the- multiplexes, '
in accordance with the illustrated altetiiating=- pattern. The
outputs of padders 1830-and:.1860~are :resgeetively coupled to ~ ~ .~ '
two-dimensional diagonal low-pass -filters-w1835 and 1865,'

CA 02379330 2002-04-16
~vo stiis~z~ - ' ~cr~us9~iun2~ . _... . _.,
xe~pectively.- which, ~s previausly.,....ae~cribed; ap~srato~ tov _ .
remove the alias,inq~ componerita.v They. outputs of~~ the' filters"
are coup~:ec~ to cifgital.-ta-analog corivsrtsrs 1840 and 1~7E1,
respectsveiy,, and then carp be analog ~;ow-pass filtered
lharizontaily), if desired. and diSpleyed andr'Qr recorded, as
represented by the biacks'1843 and 1875,. respectively. ' 'v
Referring. to Fig. gIl.:, l:hexe #a shown an embodiment of~ a .
form of thB. invention wherein two. video. si.grtals can be
combin~d.; on a s~n~~.e Gha»ne~~ w~.th reduced bandwidth '
reriuirsment, aaalag pxacess~.ng hein~ utilized in this '' .
embodia~e»t. As berfore; two video in8ut signals' are received
from vidcp.signa3 aos~rces (2pi0 r~tsd 2050) . The present
inver~t~.or~ ( in both dig~.tai .arid anaToq prflcessine~
ia~plemert at~.ons ) many sometimes be practiced without initial
two-dimensional diagonal low-pass filtering, since most
rsatura3, images have ~.itt~.s spectral energy to began with ~,n
the high diagonal freguency region of the spectrum. The
embodiment of Fig. 20 is an example of praaessittg ~r~.thout
such prefi3.tering: In the embodiment of Fig. xg 3t is
assumed, also as an examp~.e, that the video comprises
~.nteriaced video. Also in this embodiment, he two-
dimensional modulation, is implemeht~d by separate vertfcal
asu~ riorizonta~. e~odulati.o». In part#,aular, the b~.cck 2D~a
represents the vertical madulat~.on. For an intorlsced
display, al.texnate lines occur in successive fields, so
v~rtical modulat3:on can be obtained by invertl.nc~ during every
ether field. A multiplexar ZOa~r which is contralied oa a
lane 20218 by a signal des~.vwd fro~a fie~.d ~aynch~aniaation,
selects either the video receiaed via an amplifier ~p2~ or
the video reoaivec( vi,a. #.nverting ampli:fiar 2023. The
re~uxtant "a3te~nafie line inverted" s~,gnal .fs used to drive a
double balance$ modulator (txhich implements the horizontal.
modulatiort,comporent), the mixing freguency (f~) of which,
received on line., a~I30B, : i.s selected as th~ maxi~au~n pass-band
fragWency, for ax~upls' 4.~;M#~z.. tn oxder,:'to maintain the
unipolar. nature o~ the, video, a bias equal: . ~o half the' . ~ w
maxf.muta unipola,r video level r i~ subtracted prior to encoding

CA 02379330 2002-04-16
.. ~ .., .~»~ PCTlUS91l02228
21
by difference_'circuit 2005;: and is ~ then ire~iiis~erted after the ' ~ ~~
modulation., process, ~ by.. summing circuit '- 2050, ~ v The modulated '
and unmodulated.video-signals are added by summing circuit
2060, low-pass,_filtered (block 2070),and'-recorded and/or -'
transmitted (block 2080). ' ~ -
Referring to-Fig: 21;. after storage and/or transmission,
the combined signal-::iswreceived andlor read~from storage
(block,21I0),-andv can be decoded.to~recover-thew original
signals. The combined signal is~coupledwto=a~-two-dimensional
diagonal- filter _2120 to obtain the~~lo~a two=dimensional '.
diagonal fzequencies on a linev2020C, and the high
two-dimensional~frequencies on a line 20208. The diagonal ..
filtering may be implemented in analog~fashion or by
converting to digital farm, separating as above, and
reconverting. The. signal 21200 can be recorded .and/or
displayed, as represented by block 2180.v The signal 21208 is
two-dimensionally modulated-,. in one-dimensional stages
analogous to the encoding process of Fig:- 20. In particular,
a multiplexes 2140, which is controlled on a~line 2140B by a
signal derived from field synchronization-; selects either the
output of an amplifier 2130 or of inverting amplifier 2135.
The result is,then passed-through a double-balanced mixer 215 0
which receives, as its other input, a signal at the same
frequency as that used at the transmitter;w i.e, a 4.2 MHz
signal in.this~example. Bias can be suitably added by
summing circuit 2160 and the resultant signal stored and/or .
displayed, as represented by block 2170.
The high frequency diagonal portion of~the
two-dimensional spectrum can also be utilized to carry
components of the primary image (for example, high definition '
components, color components, additionalv image area to modify
aspect ratio or for other purposes, ete.)~, or to carry other
information such. as data, still pictures, audio, etc: Fig. 22
is a block diagram of a high definition televisioru~system
which two-dimensionally modulates the high''definition
telev.ision;system which two-dimensionallyv~modulates the high -
definition components:~into. the high frequency two=dimensional

CA 02379330 2002-04-16
Wt.~ 91/15429 , ... pC'ftr1S91tQ2?.28
ZZ
dictganal regf.on.~ ' A_soWroe c~f. hir~h dsfinit3an (which- eon' be ' ..
considered as: en~ompass~.ng- a signalw w.ith~..substantiaily higher ~ . ..
dafin.it.idn. than current cpnventional television video) 2205 is
provided. .The.outp~~,thexeo~;.is,ccupled.to the positive input '
of a difference a rcuit 220, and to a block 2210 which
represents twA~,dimens~.o~tal low--passwdiagonal filtering of the ,
high-definition v.ideo,.siqnal. ~ This fr~itering may bar
i~tp~.emented, fo,r ~examp7.e,. ,as descFibed above by converting to
digital form and iptpiementing the filtering a$ described
above. The two-d~:mensional ,spsctrua~ o~ thewese~ltant signal
is represented at 2210A, and .~t~ is seen thax~ the original.
horixonta~.,;and vertical. freguencies, at half the high
dafinitiatt sampling raga, axe reduced to within f.he diagonal
hand at the .lower frec,~uency standard definition
two-dimensional frequency cutoffs. The output of the fi~:ter
~21p is received by decimator 2215, which operates, irt this
embodiment to decimate every other vertical lire and retain
every fourth horizontal pixel. The pattern of decimation is
illustrated~in anothsx park of the diagram at 250, and it
wi~.l be understopd that thla decimation c$n be achieved by the
decimator preuiot~sly 3l:l.u~strated, with the line and field
inputs causing refection of the pixels to be retained irt
accordance with the desired pattern: The resultant relative
imago density is illudtraterd irt the diaqram'at zZOp.''Atter
time base correction .in the manner previously described, the
output a~ the dee,imatox~ i$ coupled to one input of multiplexes
212p. The output o~ the t~u~.tiplexer i~ coupled to a zero
padder 2210 and to a storage buffer for storing detai3
coe~f~;cient~3. The output o~ zero phdder 2260 reconstructs the
pattern shown at_226OA, using a padder of the type previ.ous~.y
i~rlustrated. This signs?: is coupled to a standard definition
two-dimensional .d3ac~s~na~. , low-gee Titter 2260, ~rh~3ch operates
in the same . ~aruser , as ~~.lter 22111, and produces a P$atrum as
iLiustrated at 2265A, without;aiiasing. The output of f.i~,ter
22b5 is one input td a summing circu~;t a2yD.. Tha other .
muitip7.exsr ~outpu~ , ih coupl,ed xo detail coefficient storage v
buffer a2ao whoBe output ts;cq~pi~d.to. an inv~rse co$ine
r. l.. ~ '~ , awI ..:.

CA 02379330 2002-04-16
WU 91/15929 '-. ' PCT/US91lOZ228
23
transform circuit . 2265,-: which: may be. a; chip of the' type--
described in the above referenced Canadian Patent Application
Serial No. 2,079,318. The output of circuit 2285 is the other ---w
input to summing. circuit 2270..~The output of summing circuit
2270 is, in turn, the negative input to difference circuit
2207 whose output~is coupled.:. to cosine transform-circuit 2135;
and the output. of .'this circuit: is coupled'to'a detail'
component. selection circuit 2237.. The circuits 2135 and 2137
may, again, be.of the.type described in the abovE referenced
Canadian Patent Application Serial No: 2,079,318. The output of
circuit 2237 is time base,corrected (block 2240), and is then
the other input to multiplexes 2120. In operation, this
circuit uses (as an:example), selected transform components in
the time slot made available by decimation. The decoder,
after demultiplexing,. can utilize circuitry in the dashed
enclosure 2250 to recover the components of the signal and add
them to obtain an HDTV output .. It will.be understood that the
same principles could be applied for non- transformed detail
components.
It will also be understood that the two-dimensional
modulation of video into the high frequency~diagonal portion
of the spectrum (for subsequent recovery) can be utilized to
reduce interference between signals, for example in a
co-channel situation. Fig. 23 illustrates transmitters A and
B in separate broadcasting regions, and a receiver which can
receive interfering signals from the transmitters: If one of
the transmitters is utilizing a two-dimensionally modulated
video signal (for subsequent: conversion, as described above),
perceived interference will be substantially reduced.
It is.known that pictures can alternatively k~e
represented in terms of their spectrah content. The discrete
Fourier transform.of~ an entire image having N spatialv pixels
will contain N unique. frequency components and therefore
generally requires the..same transmissiorrwbandwidth needed to
send spatial. image information.-:: In°: the' embodiment to de
described.next; spectrally transforming.thevaugmentation
detail can provide advantages,;for example'because the lower

CA 02379330 2002-04-16
W4 ~lr~s~z~ . . . pcTrus~~io3zZ~
2~
spectral components, usua~,ly, sent. in- the - conventfonal channel
need not be- dupi~.ca~ed,,,and~ the txansformeid augmentation
comgane~ts. can be dynamical~.y. selected in a ma»ner which can
better satisfy the needs of this human visual system, g~.g.s, ~9~
and 25 illt~sx.tate exemplary band divfsians of the Ffq. s
spectrum into_twa arid fouW binds, respectively. In Fig. 25,
tha bands have;app~oxima~ely equal:spectra~, order, and ire
numbered~in descondirg: ordex cf v.iaua3~prioritx: A tour band ,
augmentation tech».ic~ue ~is set forth ~ i.» the embadilnent to be
d8saribed, but otheac band selections (both as to the number of
bands ahd the spectral shapes and apportionments thereof)~cett
lie employed. , .
Refexxinq ,to F~,g. 2b,. there is shown a b~.r~ak diagram of
a» encadex ~.3~ ae~ordance with a further embodi~sat of the
invention, and which cart be used to practicevan embod~.ment, of
thQ mQthod,of the invQntian, The enaodesr may be utiiiaed, for
example, inthe type of system ille~s~rated in ~'ig. 1. Tha
use of codes or synch"rorii:zing s#,~nals, or of othex means for
iaentifyzng car synehxonixing channels, mey be ix~ accordance
with the teachinc~~ of 11.S. Patent No. 4,8flD,~2fi~ or other
abovereferenced patents ar publications:
The high definition videp signal ~a coupled to a
complementary a-aitnen~s.iana~. sgatfal filter 910. The f3.lter,
described fu5tther in conjanatipn.~"~ith b'ig. ~~, ~apetatea, inter
alia, to separate the received ~tzTV signal into a conve»tional
resvlutlo» video sigt~t~xl a»d $~dekail signal. The encoder
spatial filter 91o can be 1» xre form of two fine d~imerisional
low pass finite impulse respnn~se (FZR) f~.J.tars ~ one operating
in the vertical spatial direction cascaded with orte operating
in the hor3xo»tal spatial direction. The cascading operation
acts to Convo~.ve the vertical: anci horizontal spatial images.
The two dimensiana7. spectral xespo»se is ebta3ned by ,
multiplyi»g the responseh p~ the tWO filtars'the~eby providictg
a rectangular spectral wi»dow in ttie two di~enstonal freguency
da~aair. Raferanee can, oleo be made-: to the: ah~pverefexenced
U. S . Patent No . , ~ , 6~ ~, 34 ~ ., ( Of : caurae, if sepaiate hi gti ~
and~~''
low rssolntiori cameras ox.othex.sdurces. uf,video campon~nt

CA 02379330 2002-04-16
wo 9ms9z9 Pcrius~~iozng
2 5 : -.
signals are. employed;:..the_ filtering=.may not be necessary ar
may be of a different form:.j-::The conventional--resolution
video signal is coupled- to~ awscanwconverter 920 which is
utilized to produce.video having conventional scanlirie and
element format, for_example 483 visible lines as in NTSC. The
scan-converted video;is,..coupled to°encoder 930, f.or example an
NTSC encoder. The encoded.. conventional resolution video
signal can be stored, transmitted,'or otherwise processed~in -
known fashion. A motion-indicative:signal-can also be
included in the conventional. video.channei in order to obtain
enhanced operation at.receivers~having certain processing
capabilities. ~ , ::
In the present_.embodiment, the high definition detail
- component is processed using predetermined regions (or
"tiles"} of the picture.- In this. embodiment, the tiles are
square and there are 8x8 pixels per tile, although other tile
sizes and configurations could be utilized.
The detail augmentation channel includes transformation
of the detail signal~.using an orthogonal transform into the
frequency domain, such as by cosine transform, as represented
by the block 950. The particular transform bands to be
utilized (transmitted; in this case} are dynamically selected
as a function of motion for each tile,~as represented by the
blocks 960 and 970, and described further in conjunction with
Fig. 28. in accordance with a feature of this embodiment., the -
particular bands selected are determined, for those tiles at
which there is currently no. substantial motion, as a function
of the motion history of such tile. In particular, the longer
the picture information at the tile remains substantially the _
same, the more"detail information'is provided (transmitted, in
this case} for the. tile, until all available detail has been
provided. In the present embodiment, the motion indicative
signals are generated by detecting motion of the transformed
tiles (e. g. Figs 28 and 30); although-it will be understood
that motion could,be detected~using the detail component or~
ether suitable video representation. The selected band
components (if any)~,for each_tile, and motion status signals

CA 02379330 2002-04-16
W~ ~~~~~~~ ~~~~;~
. i
are coupled, ..,in ;the, pre~~,n~: embod;ment, via mu~ltiplexer 9Qt~, . _._..
._ ..
td, the transmitter ,andlo~: .storage me~lit~m. .
F~.q. 27. illustrates a preferred- arrangement for the w -
encoder scan convex~s,ion.: The low ~ or conventicana~l ) resolution
output of the.2p spdtial filter.~9lQ (e.g. at 1125 tinac) aan
be coupled tQ a first dawh, scan converter 2'l20 that convexts
to conventional 525 line (NTSC) format, for coupl~.itg to the
conventional. transmtt~er (e.g. alp in Fig: l). An up scan
convertor 270, that is identica)..to one .hat will be employed
.in the recei~rex's decoder te.g. block 31I~ of Fig. 31 b~low),
can than bemused to up-ccpvert.back to.~.125 lines. The result
can be subbrocted tram the high resolution comp~onerlt, using
di~ference.circuit 2?aA; to obtain the desired high-pass
detail components. A domper~ating delay 2710 can be employed,
as shown, and throughout the description hereof it w~11 be
understood that ,ashy neceas~sry competisaxing 4eldys can t~
emg3.ayed, as Itnown in the art .
Referring to Fig. a8, thexe is shown, in further c~e~ail,
a block diagram of the portio~t of the encoder that procasa~s:
the detail s~.qnal for ccuplinq to the auc~mentati,on channs~, and
which was repre~cented in Fig..~6 by the blocks 950, g60, 9~0
and 980. A demult p3.exar IIpS, line gifos x.110 and
multlplexsr ~lli5. serve, in known fash3.on, to put the cletaii
signal in appropxiata format to be recafved by the ~,ransform
chip II~O which, is the present embodime~lt may be he INMOS I~dB ..
A7.21 Discrete Coa3.ne Transform chip. the tt~nsform chip 1120
receives a pixel cloak, which ~,8 also received by a
coefficient cnuntar ilZa.. Th~ txan$~arm chip alas receives a
..next tile" irtd.~eatinn, which can be derived as the pixel
count modulo b4r and this indication ~s a7a3o coup~.ed to the
coefficient counter.l~Z5 and_to a tile po$ition counter 1130.
Ths til~ position counter ~.~.3o a7,sa reoefves a "next frame~~
.indic~tt3.on. .The aonf~icient counter 1125y keeps track of
which of theft=anaform coefficients ~6~ of them; fcsr this
exarn~ale) is .being output from the tr$nt~fo~n 'ohip I120, ahd the . '
tile position aountex,113a keeps track of:tho position (x,y)~
of the tile being pxocesssd, the t~.les typically being

CA 02379330 2002-04-16
WU 91!15929 - -' PCTlUS91l02228
27
processed sequentially, a~ row at. a~ time'.-=~-R=look-up table
1135, which; may.__be a. read-only. memory- (.'-'ROM!'-j.,=-translates the'
coefficient identification:information'from counter 1125 to
part of an address at which-coefficients. from transform chip
II20 are .to be stored,in two band storage RAMs 1150~and 1155,
respectively._.;The_band storage RAM.1150 is. dual ported; that
is, it;.can..be.accessed at either an input port, using and -
input address;.or, independently; at an output port using an
output address (which,".in this case, is--.obtained-from a tile
control processor 1175). The. band storage:~RAM 1150 is used to
store the band. components; that is, the groups-~o~ transform
coefficients that. comprise individual bands'of the frequency
spectrum, such, as, the four bands labeled 1, 2, 3 and 4 in Fig.
25. In.the present.example it.is assumed that-each of the
four bands, 1-4, has five coefficients, which that means that
twenty of the possible sixty-four coefficients are actually
utilized to represent the shaded spectrum-shown in Fig. 6, the
other forty-four coefficients approximately representing the
remaining regions shown:in~Fig. 5, and.not being-necessary, as
first explained above. As will be explained, the appropriate
band component will be read out of the band storage RAM 1150
to a multipl,exer 1165 which also receives indications of
motion flag storage status from the flag storage RAM ii70.
These signals are applied to a fifo 1185 and eventually to a
transmitter (or storage; as the case may be); such as
transmitter 275 of Fig. i.
The other band storage RAM 1155 may be-single ported, and
is used in this embodiment for the purpose of motion
detection. The RAM 1155 receives the same coefficient
information as the RAM 1150, but each..time it receives a new
coefficient, it reads out the corresponding coeff~.cient from
the previous.frame (thatis, the~coefficient stored at the
address to be occupied by,_the corresponding coefficient of the
current frame),.so:,that motion;detection can be implemented by
the circuit'1160,.which:is.,shown.:in:further detail in Fig. 30:v
Referringnmoiaentarily~to"Fig:-..-30;.:-the: current=band component -
( from transform.chip l I20.. -_ dig: ,: 28 ) and ~ the y~corresponding

CA 02379330 2002-04-16
wo 9tns9~. . . Pcx~us9~~ozzzs
previous trams.; bandL cPt~ppnent.~ ( from. harid~ storac~a~ ~~ 1155 -
Fig. 28) are..,receiv~d, hlr;,a:di~terence, circwit 131Q whose
output ins coupled to . an abso~.utev vaZu~e.. c.~rcuit 1320 . An
accut~u~.ator 1330 accumulates the' total of the absolute values
of the differences for each t~.le ,the xesett3~nq and road-oWt
of the accumulator being enabled by a next tile indicattan),
and its output. is compared again$ a predetera~fned thiceshald
level. If the. t1'~esho~.d is exceeded, a fiction tntl~.Cation
signal is output..f,rom camparatar 134Q. As seen, i.rt Fig. 2g,
this signal, '.is received by flag storage RAb! 11?t~, which ~is~
alao a dual ,ported RA,~.-.~; Yn the pre:e»t embodiment there are
five ~neti~rn; ~~ag, atatuses,. yea. fvl~ows: motion flag statusvl
indicates atotiun, and that the first of the~four bands should
be sent; motion fxag.,status 2 ind3.cdtes thht there has been nn
motion for. orte frame, and that band 2 should be sent; motion
~lag status 3 t»di;cataa theft thrx~r hens beep 110 mot~.on for two
frames, and that~~band 3 should beW ent: motion flag 4
.indicates that there has been no motion at the tile for three
fra~ses, and t~tat band . $ . should be sent; and motion f~.ag status
U lndicates,that there has been no-motion at the tile for four
or more frames, at~d that nn spectrhl infotmatio» should be
sent (all four bands havipg alreadx been sent). The tile
avntrol proaesacrr 13.95, which may ba $ny suitablsr
microprocessa=_or part thdrgof, dedioated logic, or integ=atod
circuit. acntrols the flag storage RAM 1170, as will be
described. yig. 29 is a flow diagram of a routine for
control~,xng trie tile coritrAl proceasar 11?5.. Thd block ~~p5
represents the init$aiizi.nc~ of indices ~, and j ~rhich represent
the position of tha tile in the two df~tenaf.onal tale array. A
sync coda is then cent (block X210). as indicated in Fig. 2~
by art output of, ti~:e. control processor xav a .sync code
generator xiBO which outputs a s~raa aiqnal to ona input of an
analog~muxtiplexe~.l=9~. The motia~e flag status for tile
1 is then., aeAt..to the fifo. 11a5. from the fl.eq~ stcrage~ RAM
t i~0 via the multip~,exex . l ib3, as rsprese~tted ~ by the block '~ ' ' . '
1215 . This . is . ~mpletpe,~t~d, by ~nd3.ng the'~add=ess ~ ~.; ~ j~ ~to the
outpWt port of the.fl~g s~urage x.1170; and'enabl3:ng the ~'
s '

CA 02379330 2002-04-16
WV 9/15929- ~ PCT/US91/02228
29
multiplexer. 1165 to. pass._ the: flag status-information~-to the =
fifo 1185. .. Inquiry,. is...then made (diamond 1220) as to whether'- '- w
the motion flag status is zero. If so, there has been no
motion at this tile, for-at least the last four frames, and a1,1
of the band information has already been sent, so na spectrally
information will-be-sent for this tile. In such case, the
diamond 1250 is entered directly. If the motion~flag status
is not zero, band information will be sent from the band v
storage RAM 1150 to.;the,:fifo 118:5, and this is implemented by
the loop 1243. An index k is initialized at zero, as _
represented by the block 1225. The index k is used to
identify the five components of each band. Each time k is
incremented (block 1240) the band component at the address
[i,j,flag(i,j),k] is loaded from the band storage RAM 1150
into the fifo 11.85 via the multiplexes 1165, the multiplexes
being controlled to load information during this phase from
the band storage RAM,1150 by the line in Fig. 28 labeled
"select flag or band component". The address is seen to
include the tile position [(i,j)l, the motion flag status
[flag {i,j) - which determines the band to be used, in
accordance with the above-listed rules), and the component of
the band [k]. When the loop 1243 is complete, the inquiry of
diamond 1235 will be in the affirmative, and block 1245 will
be entered, this block representing the updating of the motion
flag status in accordance with the above-listed rules. Stated
another way; the motion flag status at the address (i,j) is
accessed in the flag storage RAM i170 and is incremented
modulo 5. Inquiry is then made (diamond 1250) as to whether
the last tile of the row has been reached. If not, l is
incremented {block 1260), block 1215 is re-entered, and the
Ioop 1263 is continued~until the row is complete" Index l is
then initialized for the next row, and inquiry is then made
(diamond 1255): as to whether the last row of tiles has been
considered. If not, j is.;~incremented (I2?0), block 1215 is
re-entered, and the loop l2?3 is.continued until all tiles v
have been. cons dared, whereupon: the: block 1205 is re-entered
for processing of.the tiles of:.the next frame: "'

CA 02379330 2002-04-16 _ " _,__,
WO 9y115929 . FGT/LtS9ll
3Q
The., ~,nfor'm3tion.: i.n.:fi.f.Q .1195 is -clocked aot preferab~.y at'
a fixed: dock . rh~.e for a given. Pox~ion of the transmitter
frames, ta. di9ithl-to-a»alog convertsr~ 1190, aRd-'then- to v _.
t~cansmi.tter via analog multiplexer 1195. The t~u3tiplexer 1195
~.s cantrolle~,d, to pass the output of di:gixal-to-~aaaLog
cp»vertex~ 1190, except when a sync code is being a~aplied, as
previously descr3;bed.
R$f$xring to Fig. 31, them ~.s, shown ~! Dlc~Ck diagram of
an embodiment. of the decQde~. In general. the decoder
receives a convehtio»a7. channel (e.g: 525 l.itte input) and
converts .it f.a a =s~,atively low resolatian image at the same
number of titres as tha HpTV to b~ llt.tme3tely displayed or
recprded. Lnfermatfott .fr~am th$ augmentation ahannol is
received aver a separate path. aRd is p~oces ed and stored n
a spectral memory (block 14~a in Fig. 31). The output of the
epeatraJ. memory ,~~ processad by an #nvers~e translaXm chip
(block 3160 in Fig. ill, for example a» inverse cosies
transform for the present embodf,~ptetet, to crbtair~ image detail.
which i.s added td the re,~atfvely Sow resolution scan--co»verted
imago obtained from the aonventiona~, resolution ehd»ne~..
This is impleme»ted by the sinner 3x70 ~.n th~ Fiq. 33
embodiment. ~'h~s signal is converted to analog forip and can
c3~en be coupled to a sWitable HD~!~V display and/or record3~ng
means.
It can be i.»idally ;toterd that. the timing rslat3o»ship
between the conventional channel, and the augmentation channel
is non-cr~,tical, and aan bra txeated independently. ~Th.is-is
berceuse small delays k~attresn the low xesolul~i.an components and
ttie detail components will generailly not be noticeable to the
viewer.]
I» the F,~g. 31 decoder emlQOdi~nent, an augmentation input
processor 3140 aad a spectral-to--detai~i canvartar contxel
proces'sox are employed. Theae ~unctiuns can be i~tplemented,
for example, by sharing a slngae rnic~oproces~r~~ os with
se~aaratv microproce~eo~sr o= py ded,icated logic or int~grated .
c.ircu.it means. . The roatfne ~gor controll~.ng the a~g~enta~ian
input processor is deacribed.~n:con~unction with the flaw '

CA 02379330 2002-04-16
WO 91/1S929 , - PCT/US91/02228
3I
diagram. of .Fig::.. 32,.: and. the-. routine...-.for~~implementing the '
spectral-to-detail.converter..control~processar is described in
conjunction with the- floww diagrams of Fig. s 33' and 34. ~~ The' - - ~~
pixel stream.output;from the inverse transform chip 3160 is '
coupled to a fifo circuit 3165 which is described further in
conjunction with Fig. 35. -
Turni-ng..now to the.detailed-operation of the Fig. 31 '
embodiment,-the. video-from the conventional~receiver portion
is coupled to an--analog-to-digital converter 3105 and then to
a scan converter.3115, the scan converter also receiving the
necessary_synchronizing information which is extracted by sync
detector 3110, and is to be used by the scan'~converter 3115 and
the spectral-to-detail converter control processor 3155. . w
- The augmentation signal from the augmentation channel
receiver portion is digitized using analog-to--digital
converter 3120, and the augmentation~sync~is extracted (block
3125) and coupled to the augmentation input processor 3140
which, if in. the. middle of an operation, will asynchronously
terminate the process in progress and-return t:o the start of
t:he frame sequence in order to re-synchronize. The digital
data output from analog-to-digital 3120 is coupled, via a
demultiplexer. 3130, to the processor 3140 or to one input of a
~nultiplexer 3135, the,other input of which is a logical "0".
As will be recalled, the data has a flag status indication
followed, where ,appropriate, by spectral components, and the
augmentation.ingut processor 3140 operates to control the
demultiplexer 3130 to couple motion flag status indications to
the processor 3140 and spectral component, data; via
multiplexer 3135, to the dual ported spectral data RAM 3145.
- Reference can be made at this pofnt to the flow diagram
of Fig. 32 which illustrates a routine for controlling the
augmentation-input.processorv3140. The diamond~1502, and the
associated loop, represent the. waiting for detection of sync,
whereupon;~the;-,tile position indices, i,j, are initialized
(block 1504).. Inquiry is.theca madev(diamond~1506) as to
whether the motion flag status is-. 0. ~'=- If . so, no- spectral data
will follow.the,motion status flag, the next portion.of the

CA 02379330 2002-04-16
wo 9m5~x9 ~ rcrms9noxzxs
32
a
rAUtine is bypassed..: ~~dJ:~~~.. diamond 15?5 ~ is en~ared~ diroctly°:
. . . .
xf tt~e. flag status, is. r~o~..'~._.irtguiry_ is made. (diamond x.508)
as to whexher the ~lag status is 1. ~If ~ not, the block 1560
is entered directly. If;sp, however, motion at'the cur=ent
tile is i~diCated;, dnd the h:I9hw resolution bands f.n~spectral
data RAID! 3145 ~thex~#ore contain obsolete values. In such
case, the next portion o~ .the ~ roW~fne is utfl~.zed 'to. remflve
~khese obsolete va~.ues , dram. hAb~ v : 315 e~nc1 xo ~.°i:nserx a"0"
via
multiplexer 3135. In;, particular,. the bandvindex is
initialized at 2 (b~.ock.lSID),~, it being understood that it is
not necessary to zentave ex.istinq data from band l of stcraga,
rinse the subseguer~~ .. r~pere~k~an, will cause inse=tiolt of new
spectral data into bard ~. (t3ie motion flag status being 1 for
this branch o,t the routinel. The bandcomponent index k ie
then ~.n~.tialJ.zed block 15~;~), and the b7.ack 151, is entered',
khis black repxecontir~g the aett'sng of the cr~mpvnent in
spectral data RAM 314, at hddre~s-[i,~,b,k], try 0, xnguiry
is then made (diamvo~d 1517) as, to whether the last k has beers
reached. If nest, k is incremented (block l5xQ~. block 1514
is rs-entered, and the loop 1519 continues until ali band
components have been considered. Then, .ingu~.xy i.s made
(diamond 1525) as to whether the last band hay been reached.
1 f not, the band index ~.~ increp~entec~ . (block 1529 ) , block
1512 is re-entered and the ~.oop 1530 ~,s continued until all.
bands have been considered. The block 3560,i.s then entered,
and k 3s initial.~.xed. Tht received spectra. component will
then be stored in the spectral.rlatd RAM at addtest~
I 3 , ~ . tla9 ( i. ~ ) . k ] . dt~ r~~tx'esen.ted by the bloctk 15s5.
Ir~qt~3.xy
is rhea made (diamond 1.560 ifs to whether the last k has been
reached. It not, k is..inare~tented (lock 1569), and the loop
1570 is cp~tinued until ail component of the received
spectral band have been read into their appropr~.ate addre~aes
in spectral data RAl~~3i15. The diamond 1595~is than entered
(and is also entered directly from the nyes~~ output branch of .
a3amond l5osj: arid ingpiry.is made as to whether xa:t i (that' .
is, the last tiles of, the rc~w~ has been ' re~ahed. If not, .~ is
fnare~ae~tted (block 1578), diamohd 106 is re-entered, and the

CA 02379330 2002-04-16
WO 91,/15929., PCT/US91 /02228
33
Loop 1580_.is...continued until ~he last~~i is~-reached. When this
occurs, i. is initialized..to ; begin -a new row - (block 1582 )-; 'and - ~ "
inquiry is made as to whether thewlast~row has been reached
(diamond 1585). If not, j is incremented (block 1587),
diamond 1506. is. re-entered, and the loop 1590 is continued
until all tiles have been processed, whereupon the diamond
1502 is re-entered.
Referring again to Fig. 31, the spectral-to-detail
converter control processor is synchronized to the output~of
scan converter 3115. When an indication of a start of frame
is received by processor 3155, it begins the routine of
controlling inputting of spectral data information from RAM
3145 to inverse transform chip 3160 via the multiplexer 3150.
Referring, in this regard, to the flow diagram of Fig. 33,
the sync is awaited (diamond 3302 and associated loop), and
the tile indices are then initialized (block 3305). A
coefficient index, c, is then initialized (block 3308), to
consider all coefficients [for example,w 64 coefficients for
an 8x8 pixel.tileJ to be coupled, for each tile, to the
inverse transform chip 3160. Inquiry is made (diamond 3310)
as to whether c is used (it being recalled- that only some of
the coefficients are utilized). If not, a "0" is sent to the
inverse transform chip 3160 by sending a command to the
control line of multiplexer 3150. [Alternatively, if it is
viable to permanently disable the not-used coefficients of
inverse transform chip 3160, this operation would not be
necessary.] If the coefficient is used, the block 3320 is
entered, this block representing the sending to the inverse
transform chip of the component in the spectral data RAM at
address '[i;j,cj. Inquiry is then made (diamond 3330) as to
whether the last coefficient has been reached. If not, c is
incremented (block 3332), diamond 3310 is re-entered, and the
loop 3335 is continued until all coefficients have been read
into the inverse transform chip 3160. When this has been
done for the-;current tfle, the inverse transform aperation isw
initialized (block 3340, and the "start" line in Fig: 3I).
Inquiry is then made (diamond 3360) as to whether the last

CA 02379330 2002-04-16
WU 91/lar9Z9 . FG'~'/US91/tlZZ2~
34
tile of the ,. , row , hays, begirt, reaahed . ~- I f ncrt, s is
irtcreme3lt~ed
(b~.oak, 33~~), black 330$ as: re-entered, and the ioop: 335p is
continued until .the row is completed: Tt~e index i is then ' '
initializec!'for the next row block 33fi5), and inguiry i~
trade (diamond 33~9j as tp whether the last ~:ow hay been
processed. If not, ~ ~.s :incremented (block 3380), bibck 3308
is ra-entered, and the loop 3385 is continued until all rows
of tiles have been process~d. The diamond 3302 is then
re--entered : to again away t the sync . " . .
The rout.,fne illustxa~ed by the flow dfagram of F~.c3. 3~
fs used to control the loading of tiles of output pixel data
into, and then out af, the f~fa ci~auit ' 3165 of F3.g. 31, t?~e
fife circu3.~ boiriq shown ire F#g. 35. ~n the present
embodiment there are eight.fifos, x521-35~~, and they each
receive the inputs from the 3:»verse ~r$nsfAxmch~.p 3160.
ttowsvsr, only one t~.fo is enabled to load at a time, under
cc~atrol of demnitiplexer 35tp. The demult~,plexer 351Q
receives the inverse ~rattsform clock and a fifo select
control from processor 3155: 1» particular, refe7crin~ tv this
routine of Fig. 34, the d3,am4nd 3~1Q, and the assoc~.ated
loop, represei~t$ the wa3tinc3 foxy sine of ~n~ outprat video ~:u
be genex~ated~ The complsrtion of the inverse tsansfarm
aompu~ation for the Current tile is th~n awaited (diamond x.715
and the c~asociated loflp) , and a pixel ~,ncie;~ is initialf zec~
(block Z7~0): The demuhtiplexer 3510 is then controlled to
select the fi:fo for the current pixel Gaunt (block x,725).
Ingu3.ry is then made (diamond 1730) as to ~rhether the last
pixel has bs~n rer~dhed. If not, the pixel. index is
incxernented (b~.ork 1735), the block i'725 is re-entered, and
the loop ~7~0 continues until a:L1 pixels for the current tile
have been read into the ~f3.foe. Inquiry is then made (diamond
x750) a$ to whether all t3.les have been processed. It not,
the tile index is irtcreatented (block 17551, diamond 115 is
re-enterBd, and the loop ly5i continues until all. tiles have
peen proeessed,.wheraupon~~:he diamond i?i0 is re-entered.
The pixel in~armat~,on iri the f3.fos 1e clocked out under
control of demultiplexer 350 whfch rece~.veh the video nut
a

CA 02379330 2002-04-16
WO 91/I5929 PCT/US91/82228
clock and the l;ne out enable, as seen in Fig. 35. The
demultiplexer is-.controlled by the output~of line counter
3550 which receives'thewline-out' enable, and, in the present
embodiment, isya modulo_8 counter. The counter 3550 output
also controls the multiplexer 3530 to select~which fifo
output is coupled to:summer 3170 (Fig. 31):, so that the
information is read out a line at a time, after the 8x8 pixel
data is read into the fifos 3521-3528. w
It will-be understood that the techniques hereof are
applicable regardless of the original resolution, and could be
used to advantage for bandwidth compressing moving picture
video information at any original bandwidth. It will also be
inderstood that when a substantial portion of the scene is
still (not in motion) for'a substantial number of frame
periods (e.g. more than five frame periods~or l/6 of a
second), very little picture information will be transmitted
[since, as noted above, most tiles wi3i be status "0"]. In
such case, the additional bandwidth could be used to
periodically send update. information. Also, statistical
multiplexing among a number of channels of the type described
could take particular advantage of the dynamic bandwidth
characteristics of. each channel.

Dessin représentatif