Note: Descriptions are shown in the official language in which they were submitted.
CA 02220483 1997-11-07
WO 96/37075 PCT/US96107236
COMPRESSED DIGITAL-DATA INTERACTIVE PROGRAM SYSTEM
' S
BACKGROUND OF THE INVENTION
1. Field of the Invention
The present invention relates generally to interactive response systems, and
more particularly to an interactive television system which provides
interactive
programming using compressed, digital data having more than one video signal
on a broadcast channel, or a multiplexed signal within a digital format, or
both.
The invention also relates to seamlessly switching befween video signals while
viewing a first video signal, even though the video signal switched to may be
on
a different broadcast channel, or on the same channel multiplexed with, the
currently viewed video signal.
2. Description of the Prior Art
Interactive systems are well known in the art. By synchronizing parallel
tracks of
an information storage media, and relating the content of the various tracks,
it
was found that interactive activity could be simulated. For example, commonly
owned Freeman, U.S. Patent No. 3,947,972 discloses the use of a time
synchronized multi-track audio tape to store educational conversations. One
track is employed to relay educational interrogatories to a user, and the
remainder of the tracks, selectable by a switching mechanism, are used to
convey
responsive messages.
These systems progressed to interactive television, wherein multiple broadcast
or
cable channels were switched in response to user selections to provide
interactive
~ operation. Commonly owned Freeman, U.S. Patent No. 4,847,700 discloses an
interactive television system wherein a common video signal is synched to a
plurality of audio channels to provide content related to user selectable
responses.
SUBSTITUTE SHEET (RULE 26)
CA 02220483 1997-11-07
WD 96/37075 PCTIUS96/07236
Commonly owned Freeman, U.S. Patent No. 4,264,925 discloses the use of a
conventional cable television system to develop an interactive system.
Standard
television channels with time synchronized content are broadcast to a
plurality of
users. Each user switches between channels responsive to interrogatories to
provide interactivity.
These systems have been tailored to include memory functions so that the
system can be more interactive, individually responsive, and so that
customized
messages may be given to the various categories of users responsive to
informational queries. Freeman, U.S. Patent No. 4,602,279 discloses the use of
a
memory to store demographic profiles of television viewers. This information
is
stored to be recalled later for providing target specific advertising, for
example.
Prior art interactive television systems were generally concerned with
providing
one signal (i.e. one video signal) per channel, whether the channel is on
cable
television, broadcast television, or a VCR. Because cable and broadcast
television
channel capacity is becoming limited as more and more cable channels are being
utilized for conventional programming, and interactive systems of the type
described require multiple channels, it is desirable to reduce the channel
capacity
required for such systems while still providing at least the same level of
interactivity. Disadvantages of the prior art are overcome by the present
invention which provides an interactive television system which employs
multiple, time-synchronized, content-related video signals on one or more
broadcast channels.
SUMMARY OF THE INVENTION
The present invention is an interactive cable televisian system which utilizes
digital video signals to provide customized viewing responsive to user
selections.
A standard cable or direct broadcast satellite television distribution network
is
utilized for transmitting the interactive and other programming to users. The
present invention allows a plurality of viewers to be simultaneously provided
with a plurality of different program information me ssage signals related in
time
and content to each other. The interactive program comprises a plurality of
video signals related in time and content to one another.
2
CA 02220483 1997-11-07
WO 96/37075 PCT/US96/07236
The video signals are converted into digital format for transmission. In a
digital
format, it is possible to transmit more than-one video signal per cable
television
channel. Further, it is possible to transmit video signals via conventional
telephone lines. If desired, the various digital video signals may be
compressed
before transmission. Compression allows an even larger number of video signals
to be transmitted over a channel of the transmission media. A multiplexer
combines the various digital signals into a reduced number of transmission
data
streams for transmission. The various NTSC television channels may be
allocated in a predetermined fashion to maximize the number of simultaneously
transmittable signals. The multiplexer in conjunction with the television
transmission system multiplexes the desired data streams onto the desired
channels, and transmits these signals over the NTSC channels. The number of
video signals which may be multiplexed onto a data stream on a single
transmission channel will vary depending on the video signals to be
transmitted.
The television channels containing a data stream of multiplexed video signals
may be transmitted over a standard cable television distribution network, or
direct broadcast satellite transmission system. A receiver receives one or
more
television channels, some or all containing a multiplexed data stream of video
signals or non-multiplexed digital video signals, and in conjunction with a
signal
selector, selects a particular data channel/ data stream for playback, then
selects a
particular video signal from the data stream's multiplexed signal, and finally
expands the video signal, if necessary, for playback to a television monitor.
A multiple choice controller operates to control the receiver and signal
selector to
select a particular video signal for playback. If more than one channel is
received,
the multiple choice controller may be programmed to map the different
channels, and the multiple signals thereon, to a serial numerical channel
representation to simplify use by the user. The signal selector includes the
necessary expansion apparatus corresponding with the compression scheme in
use.
In practice, a user selects a desired interactive program to be viewed. Using
the
multiple choice controller, the user selectably responds to information
displays or
interrogatory messages and the signal selector selects a particular
multiplexed
3
CA 02220483 1997-11-07
WO 96/37075 PCT/US96/07236
video signal and de-multiplexes, expands and displays the selected video
signal.
Alternatively, the signal selector may select a video signal based on personal
profile information stored in memory.
If more signals are needed for an interactive program than were mappable to a
data stream on a single channel, the signal selector in conjunction with the '
receiver is programmed to switch between the various video signals within a
multiplexed data stream as well as between data streams among the various
broadcast channels to provide the necessary level of interactivity.
The various information segments in the various video signals preferably
relate
in real-time and content so that an interactive conversation can occur as the
video signal is played back and the user responds to the various
interrogatories
on the video signals. The use of multiple signals per channel may be used for
many types of interactive programs, including those disclosed in the
previously
mentioned U.S. Patents, for example, field synchronized multiple camera angles
from a sporting event, or an interactive game show. I-3:owever, the present
invention also covers the use of various video signals :not related in real-
time
and content.
In a two-way embodiment, the various signals which comprise the interactive
program may be switched at the head end rather than a.t the receiver. This
embodiment may be used in a cable television system, a direct broadcast
satellite
system, a conventional telephone system modified to receive digital video
signals, or any other appropriate transmission system capable of sending
digital
video signals. The multiple choice control unit, rather than the hand-held
multiple choice controller, selects a desired video signal by relaying the
multiple
choice selections of the user through a relay box back to a remotely located
switching station, preferably the cable television source. The multiple choice
selections may be relayed to the switching station in an.y conventional means,
such as two-way cable television, telephone, or FM transmission. If the
interactive programming is being transmitted over a telephone line, the
multiple
choice selections may be relayed back over the same telephone line. The
switching station receives the multiple choice selection of the user and
routes the
correct signal down the appropriate cable channel, telephone line, or other
4
CA 02220483 1998-02-27
transmission media for the particular user. In such an arrangement, only a
single
link is required between the subscriber or receiver and the head end so that
the
one channel link can be used to receive a plurality of different channel
selections
dependent on the interactive choice relayed from the receiver to the video
switch
at the head end.
If desired, the two-way link may be used for other purposes, such as to
transmit
user demographic data back to the programming source for commercial reasons,
or to allow an interactive game show player to win prizes, for example. The
system of the present invention allows improved performance during switching,
making the channel switches transparent. When a channel change is required by
a
user response to an interactive interlude, a slight imperceptible delay is
programmed to allow the expansion algorithm an opportunity to adjust to the
rapid change from one video signal to another.
During the delay, previously obtained video information is displayed while the
interactive system locates, receives, demultiplexes, decompresses, decodes,
and
processes the new video signal. This allows the interactive system to switch
to
the new video signal without flicker or distortion appearing on the TV screen,
i.e., a seamless switch.
Disclosed are different methods to achieve this seamless switching. One
involves
an analog video frame buffer. Another uses two tuners. Other alternatives
include: (a) using two digital video buffers; (b) using a large memory; (c)
using a
large buffer in an embodiment similar to that of (b); and (d) switching at the
cable
head end.
Accordingly, in one aspect, the present invention relates to an interactive
television system comprising: a receiver for receiving a program which is
filmed
using multiple cameras situated at various camera viewpoints, the broadcast
program having a plurality of digital video signals, wherein the plurality of
digital
video signals corresponds to the various camera viewpoints; a display unit to
display the selected channel, the switch from one channel to the selected
channel a
seamless digital switch; and a remote controller remotely coupled to the
display
5
CA 02220483 1998-02-27
unit, the remote controller for selecting corresponding digital video signals
to
thereby choose a desired camera viewpoint.
In a further aspect, the present invention relates to a method for providing
live
interactive digital television, comprising the steps of: obtaining video
signals
from a plurality of video cameras, one or more of the cameras relaying a
different
view of an event; digitally encoding one or more of the video signals;
transmitting
the digital video signals; receiving the combined digital program stream;
gathering
a viewer selection from a remote controller; seamlessly switching to the
selected
digital video signal; and displaying the selected video signal on a screen.
In a still further aspect, the present invention relates to an apparatus for
seamlessly switching between digital video signals, comprising: a receiver
interface, the receiver interface receiving a plurality of digital video
signals; a
processor, the processor selecting a video signal for display; and a seamless
video
switch, connected to the processor, the switch seamlessly switching from a
previously displayed digital video signal to the selected video signal.
In a further aspect, the present invention relates to an apparatus for
seamlessly
switching between digital video signals, comprising: a receiver interface, the
receiver interface receiving a plurality of digital video signals and program
data
codes; a user interface for receiving subscriber selections; a processor, in
communications with the user interface, for selecting a video signal for
display,
wherein the selection of the video signal is based on the program data codes
and
on one or more subscriber selections; and a seamless video switch, connected
to
the processor and the receiver interface, the switch receiving an instruction
from
the processor identifying the selected video signal, and the switch seamlessly
switches from a previously displayed digital video signal to the selected
video
signal.
In a still further aspect, the present invention relates to a method for
seamlessly
switching between digital video signals, comprising the steps of: receiving a
plurality of digital video signals and program data codes; gathering
subscriber
selections; selecting a video signal for display with a processor, wherein the
Sa
CA 02220483 1998-02-27
selection of the video signal is based on the program data codes and on one or
more subscriber selections; receiving an instruction at a digital switch from
the
processor identifying the selected video signal; and seamlessly switching from
a
previously displayed digital video signal to the selected video signal.
BRIEF DESCRIPTION OF THE DRAWINGS
FIGURE 1 is a block diagram of the Interactive Television System of the
present
invention.
FIGURE 2 is a block diagram of the system of the present invention in a two-
way
transmission configuration.
FIGURE 3 is a block diagram of one embodiment to achieve seamless switching
between video signals.
FIGURE 4 is a block diagram showing an alternative embodiment to achieve
seamless switching between video signals.
Sb
~.a> ~i.' ~ , a; I il
CA 02220483 2002-07-15
FIGURE S is a block diagram showing another alternative to achieve seamless
switching between video signals.
FIGURE 6 is an alternative method of achieving seamless switching by using the
S memory shown in Figure S as a buffer.
FIGURE 7 is an alternative embodiment of achieving seamless switching by
switching channels at the cable headend.
DESCRIPTION OF THE PREFERRED EMBODIMENT
The present invention is an interactive television system in which a plurality
of
viewers are simultaneously provided with a plurality of different program
information message signals. A plurality of video signals 1 are provided.
Video
signals 1 may be, for example, various field and/or audio synchronized camera
angles of a sporting event, or a game show having a content and host acting
1 S responsively to user selections. Alternatively, video signals 1 may be any
video
signals suitable for interactive conversation, such as a video signal
containing
multiple audio signals multiplexed therein via pulse amplitude modulation,
pulse
duration modulation, pluse code modulation or various forms of digitizing;
multiple
video signals multiplexed from various video sources (such as conventional
video
tape recorders) into a single multiplexed video signal; video signals which
may be
individualized and/or tailored to specific users or groups of users based upon
profile
information; any conventional video signals carrying synchronized information
messages related in real time and content to one another, such as those
provided by
synchronized video discs, tape recorders, and so forth; or other forms of
interactive
signals containing multiple information channels from which a user may select,
as
may be more fully described in U.S. Patent Nos. 4,847,700, 3,947,972,
4,602,279,
4,264,925 and 4,264,924.
6
- ~_=~i~i, ~=~ :) -
CA 02220483 2002-07-15
Various types of time and content related video signals exist which are
suitable for
interactive operation. In previous systems, these various signals would be
transmitted to a receiver on separate broadcast or cable channels, each
requiring a
separate 6 Mhz NTSC channel. According to the present invention, video signals
1
are directed to analog-to-digital ("A/D") convertors 2 which convert the
various
video signals into digital format for transmission. A/D convertors 2 may be of
any
conventional type for converting analog signals to digital format. An AID
convertor may not be needed for each video signal 1, but rather fewer
convertors, or
even a single convertor are capable of digitizing various video signals 1.
Interactive
video programs may also be delivered to a cable or other distribution network
in
pre-digitized and/or precompressed format.
Digital conversion results in very large amounts of data. It may therefore be
desirable to reduce the amount of data to be sent, allowing more signals to be
sent
over a single transmission channel. For example, a single frame of digitized
NTSC video represents over 350 Kbytes of data. Therefore, two hours of
standard
6a
CA 02220483 1997-11-07
WO 96/37075 PCT/US96/07236
video is about 80 Gbytes. Since there are 30 frames/sec in such video, the
data
transfer rate is 22 Mbytes/sec. This large amount of data is preferably
reduced by
digital compression.
In order to reduce the data transfer requirements, the various digital video
signals are preferably compressed before transmission. The video may be
compressed by any conventional compression algorithm, the two most common
types being "processor intensive" and "memory intensive."
The processor intensive approach performs compression by eliminating non-
changing aspects of a picture from the processing in the frame-to-frame
transfer
of information, and through other manipulations of picture information
involving mathematical computations that determine the degree to which a
given motion in a picture is perceptible to the human eye. This approach
depends on high-speed processing power at the transmission point.
The memory approach involves division of a picture frame into hundreds of
minuscule blocks of pixels, where each block is given a code representing its
set of
colors and variations in luminance. The code, which is a much smaller
increment of information than all the information that would describe a given
block of the picture, is transmitted to the receiver. There, it calls up the
identically coded block from a library of blocks stored in the memory of the
receiver.
Thus, the bit stream represents a much smaller portion of the picture
information in this approach. This system is generally limited by the variety
of
picture blocks which may be stored in the receiver, which relates directly to
memory size and microprocessor power.
Examples of commonly known compression techniques which may be used with
the invention are JPEG, MPEG1 and MPEG2.
Data Compressors 3 are provided to reduce the data for each video signal which
must be transmitted. Data compressors 3 may be of any conventional type
commonly known in the art for compressing video images, such as those
previously described. Compression of the various video signals might be done
with fewer data compressors 3 than one compressor per video signal. In a
conventional analog NTSC system, by way of example, it is customary to
transmit
7
CA 02220483 1997-11-07
WO 96/37075 PCT/US96/07236
one video signal per 6 MHZ channel. By digitizing the video signal, it is
possible
to send a data stream containing more than one video signal in one channel.
Compressing the digitized signals, allows even more video signals to be
transmitted over a single transmission channel. The number of signals which
may be sent over a single channel is generally related to, for example, a) the
type
of video being sent; b) the video compression scheme in use; c) the processor
used
and memory power; and d) the bandwidth of the transmission channel.
Compression techniques exploit the fact that in moving images there is very
little
change from frame-to-frame. Editing out the redundancies between frames and
coding just the changes allows much higher compression rates. The type of
video
which normally contains a great deal of high-speed movement, such as occurs at
live sporting events, will, therefore, have the lowest compression rates.
Movies,
on the other hand, which normally have a lower frame rate and less frame-to-
frame change than a live sporting event will achieve higher compression rates.
Currently, commonly known compression schemes have compression rates that
vary from 2:1 to 10:1 for satellites, and 2:1 to 5:1 for cable television
systems,
depending on the degree of motion.
Once the various video signals 1 have been digitized and compressed,
multiplexer 4 combines the various digital signals into a reduced number of
transmission data streams for transmission. For example, if 68 NTSC channels
are available, and each channel is capable of transmitting either 4 digitized,
compressed slow moving video signals (e.g. movies) or 2 digitized, compressed,
high-speed video signals (e.g. sports), then the various NTSC channels should
be
allocated in a predetermined fashion to maximize the number of simultaneously
transmittable signals.
As an example, the broadcast frequency corresponding to a first NTSC channel
may contain a data stream of separate digitally compressed non-interactive
movies. On this frequency, the data stream would con~Eain video signals
representing a number of movies. However, the video signals, unlike those of p
an interactive program, are not related in time and content. The frequency
corresponding to a second channel might contain a digital data stream of an
interactive sports program, consisting of two multiplexed compressed high-
speed
8
CA 02220483 1997-11-07
WO 96/37075 PCT/US96/07236
video signals that are preferably related in time and content. The frequency
corresponding to a third channel might contain a digital data stream of an
interactive movie consisting of four multiplexed compressed video signals
which
are related in time and content. The frequency corresponding to a fourth
channel
might contain an analog NTSC signal relating to local programming. Therefore,
using the invention, four NTSC channels could contain a channel of multiplexed
movies, an interactive sports program, an interactive movie, and local
programming.
Multiplexer 4 receives the incoming compressed, digitized video signals and in
a predetermined conventional fashion, in conjunction with transmitter 5,
multiplexes the desired video signal onto the desired channels, and transmits
these signals over the NTSC channels. Certain NTSC channels may contain only
one video or other signal, in analog or digital form.
As indicated earlier, the number of video signals which may be multiplexed
onto
a data stream on a single transmission channel will vary. Also, the number of
channels which use data streams may vary. The transmission data streams are
transmitted by transmitter 4 via transmission media 6 to a receiving station
7.
The transmitter 4, media 6, and receiver 7 may be any conventional means for
transmitting digital video signals including broadcast television, cable
television,
direct broadcast satellite, fiber optic, or any other transmission means.
Alternatively, the invention may be self-contained in a stand-alone system, as
explained below.
The transmission means may also be a telephone system transmitting a digital
video data stream. Thus, a multiplexed data stream containing several
broadcast
channels or an interactive program with related video signals may be sent
directly to a user over a single telephone line. The aforementioned digital
transmission devices may include means for transmitting analog signals as
well.
In one of the preferred embodiments, the digital transmission signal is
transmitted using a cable television system. Receiver 7 receives various NTSC
channels, some or all containing multiplexed or non-multiplexed digital video
signals. Ordinarily, more than one channel will be transmitted by transmitter
5
and received by receiver 7 as in an ordinary cable television system. However,
9
CA 02220483 1997-11-07
WO 96/37075 PCT/US96/07236
each of the different channels may have a data stream containing several
digitized video signals thereon. Therefore, receiver 7 preferably operates in
conjunction with signal selector 8 to select a particular NTSC channel for
playback, then to select a particular video signal from the data stream's
multiplexed signal, and finally to uncompress or expand the compressed video
signal, if necessary for playback to monitor 10.
Multiple choice controller 9 operates to control receiver 7 and signal
selector 8 to
select a particular video signal for playback. In practice, a user need not
know that
multiple signals per channel are in use. If, for example, 68 channels with 4
signals-per-channel were in use, controller 9, in conjunction with receiver 7
and
signal selector 8 might be programmed to represent these channels to the user
as
channels 12-72. Monitor 10 may be, for example, a conventional television.
Signal selector 8 preferably includes a conventional de-multiplexer for
selecting a
particular video signal from the data stream on the channel currently being
received by receiver 7. Signal selector 8 further includes the necessary un-
compression or expansion apparatus corresponding with the compression
scheme in use by compressors 3.
In practice, an interactive sporting event program might be transmitted on a 6
MHZ cable television signal using a compression-multiplexing scheme which
allows two sports video signals (A and B, for example) to be transmitted over
a
single NTSC channel (channel 34, for example). It might be desired to have
four
video signals (A-D, for example) for the particular interactive sporting
event. A
first video signal (signal A) may contain the standard broadcast signal of the
game; the second video signal (signal B) may contain a. close-up view of the
game
action; a third video signal (signal C) may contain a continuously updated
replay
of game highlights; the fourth video signal (signal D) may contain statistical
information. These four video signals (A-D) may, for example, be multiplexed
as
follows: video signals A and B multiplexed onto a data stream transmitted on
cable channel 34; video signals C and D multiplexed onto data stream
transmitted
on cable channel 35. Alternatively, all four video signals (A-D) could be
multiplexed into one data stream carried on one frequency channel. These four
signals may, however, be mapped by controller 9, or signal selector 8, to play
as
CA 02220483 2003-04-30
WO 96/37075 PCTIUS96/07236
separate channel displays for the user which, when the viewer makes choices on
the multiple choice controller, a seamless switch occurs therebetween. Each
video signal of this interactive program may include a label which reads, for
example, "Full-Screen Action -- Press A: Close-up Action -- Press B: Replay --
Press C: Statistics -- Press D."
As shown, if more signals were needed for an interactive program than were
mappable to a data stream on a single channel, signal selector 8 in
conjunction
with receiver 7 may be programmed to switch between the various video signals
1 as well as the various broadcast channels to provide the necessary level of
interactivity. However, preferably all the various video signals associated
with a
particular interactive program are multiplexed onto a single channel.
Additionally, the signal selector 8 may store information relating to current
and
previous user responses. For example, the personal profile of the viewer or
previous response patterns of the viewer could be stored in memory. This
information may be used in conjunction with commands transmitted within the
video signals, as discussed in patent No. 4,602,279. The stored
personal profile information and received commands may
be used to switch interactively between data streams and video signals without
any additional response from the user.
The multiplexed interactive program may be transmitted over a single telephone
line, if desired. In this embodiment, multiple choice controller 9 is
programmed
to switch between the various video signals on the single telephone line. If
additional channels were desired, a two-way configuration is used as described
below.
The system of the present invention may be utilized in an educational
embodiment. In this embodiment, information is stored on each data stream in a
plurality of reproducible information segments, each of which comprises a
complete message reproducible by the receiver directly in response to the
selection of the video signal by signal selector 8 responsive to a user
selection on
multiple choice controller 9. Each of the information segments in the various
data streams contain interrogatory messages with associated multiple choice
11
CA 02220483 1997-11-07
WO 96/37075 PCT/US96/07236
responses, responsive messages, informational messages, or combinations
thereof.
The various information segments in the various data streams preferably relate
in real-time and content so that an interactive conversation may occur as the
video signals are displayed and the user responds to the various
interrogatories
contained in the video signals. As a user answers a particular interrogatory
with
a multiple choice response, the information in the video signal associated
with
the particular selection is displayed by the signal seleci:or 7. The various
interrogatories, responsive messages, and informational messages may generally
be contained in any one, more than one or all of the various video signals.
The use of a data stream containing multiple video signals per broadcast
channel
may be used for many types of interactive programs, such as those disclosed in
the previously mentioned U.S. patents. Other interactive programs may be
developed which are within the scope of the present :invention.
The present invention may also be utilized as a stand--alone system with no
transmission means necessary. In this embodiment, tile digitized video signals
that make up an interactive program are stored in local storage means such as
video tape, video disk, memory (e.g., RAM, ROM, EPI;OM, etc.) or in a
computer.
Preferably, the digital video signals are multiplexed onto a standard NTSC
signal.
The particular storage means may be connected to any of the interactive boxes
disclosed in Figures 3-5, and described below. The interactive boxes would
then
be connected to a television set. Alternatively, the circuitry in Figures 3-5
below
could be implemented on a board and inserted into a standard personal computer
(PC). A separate microprocessor on the interactive board is not necessary for
this
configuration since the standard PC processor performs the functions of the
processor 108 shown in Figures 3-5.
As shown in FIG. 2, the system of the present invention may be operated in a
two-way configuration. In this mode, the various video signals 1 are processed
as
previously described, being digitized by A/D convertor 2 and compressed by
video
compressors 3. The signals are then routed to a central switching station 14.
In
this embodiment, the switching between the various video signals is
accomplished at the head end rather than at the receiver. Multiple choice
control
12
CA 02220483 1997-11-07
WO 96/37075 PCTlUS96107236
unit 9 relays the multiple choice selections of theuser through a relay box 17
back
to the remotely located switching station 14: The multiple choice selections
may
be relayed by relay box 17 to the switching station by any conventional means,
such as two-way cable television, telephone, or FM transmission. Switching
station 14 receives the multiple choice selection of the user and routes the
desired
signal to transmitter 5 which conventionally transmits the desired video
signal
down the appropriate cable channel for the particular user. If desired,
transmitter
5 may also transfer conventional programming on the cable television channels
not being used for interactive programming. Alternatively, switching station 4
may include multiplexing equipment as previously described, and thus operate
multiple interactive or noninteractive programs over a single television
channel.
For example, if it were desired to implement the interactive football game
program as previously described, a single NTSC cable channel may be allocated
for the program. However, in this instance, the video signals would be present
at
the transmitting end. In response to a signal from wireless controller 9, a
signal is
sent by relay box 7 to the cable TV switching station which routes the desired
video signal to the requesting viewer. Such a system requires very fast
switching
equipment, but can be implemented using digital imagery.
Alternatively, it may be desirable to transmit the interactive sporting event
over
a single telephone line. When the user enters a selection on controller 9, a
signal
is sent via the telephone line to the central switching station which routes
the
desired signal of the interactive program over the user's telephone line so
that a
single link handles both the interactive choice being made at the receiver and
the
transmission of that choice, out of a plurality of choices, from the head end
where
the actual switching takes place in response to the interactive selection made
at
the receiver.
The two-way link between the user and the switching station may be used for
other purposes. For example, demographic data may be transferred from the user
,. to the broadcast network for commercial purposes, such as targeted
advertising,
billing, sending a game show winner a winning number for pickup of a prize, or
other commercial or non-commercial purposes.
13
CA 02220483 1997-11-07
WO 96/37075 PCT/US96/07236
As previously described, compression systems generally perform less
efficiently
when frame-to-frame content includes many changes in pixel content (e.g.,
during fast motion or scenery changes). The system of the present invention
may
be advantageously programmed to ease the processing burden on the
uncompression program. When a key on the controller is depressed to select a
desired signal, a slight imperceptible delay may be effectuated if desired.
This
delay allows the uncompression or expansion algorithm a short period of time
to
adjust to the rapid change from one video signal to another which ordinarily
causes a degradation in the efficiency of the algorithm causing video glitches
to
appear on the screen display.
As shown in Figure 7, a two way link (similar to Figure 2) may also be used,
employing virtual channels back to the user. In this embodiment, multiple
video signals, preferably related in time and synchronous to each other, are
present at a cable headend 300 on multiple channels A, B, C, . . . N of a
video
signal bus 250. The signals may be locally generated or received from a remote
location (such as a sporfing arena) by receivers 200, 202, 204, and 206.
Alternatively, if the remotely received signals are digitally multiplexed onto
one
channel, a digital demultiplexer would replace receivers 200-206 and would
demultiplex the signals and place each signal on a separate bus channel. The
local or remote signals are synchronized by sync circuit 208. A number of
remote
control interactive switches 210, 212, 214, 216, and 218 are connected to
video
signal bus 250. The multiple channels on bus 250 are :provided synchronously
and simultaneously to the series of remote control interactive switches 210,
212,
214, 216, 218. These remote control interactive switches are dynamically
allocated
to users who request access to an interactive program. Each switch is
connected to
a frequency agile modulator 220, 222, 224, 226, 228 to assign the switch a
virtual
channel in order to connect a signal from bus 250 to a specific user at a
remote
site. Each switch is assigned to a single user so the nwnber of switches
present at
the headend is the limiting factor to the number of users who can interact
simultaneously. If it is assumed that only a portion of the users will
interact
simultaneously, an algorithm is used to determine the optimum number of
remote switches necessary to assure an acceptable percentage of access.
14
CA 02220483 1997-11-07
WO 96/37075 PCTIUS96/07236
After passing through the frequency agile modulators 220, 222, 224, 226, 228,
the
signals from video signal bus 250 progress through the cable (or broadcast TV)
system 260. The signals may pass through ItF feed 262 and amplifier 230. The
user's set top box 232, 234, 236, containing a frequency agile demodulator, is
tuned
to the frequency of the associated frequency agile modulator 220, 222, 224,
226, 228.
The decoded signal from the set top box 232, 234, 236 is displayed on
television
monitor 10.
When a user desires to interact, the user issues a command on the controller
9.
The command is received by the set top box 232, 234, 236. A user request is
sent
back down the cable or other transmission system 260 to one of the remote
switches 210, 212, 214, 216, 218. At the appropriate time, based on the user
request
and the algorithm for interactivity which accompanies the program, the remote
switch makes a cut during a vertical blanking interval from one signal on bus
250
to another signal on bus 250. The result of this switch is modulated by one of
the
frequency agile modulators 220, 222, 224, 226, 228 and sent down the virtual
channel to the user, who sees a seamless cut from one image to the other as a
result of the interaction. The signal delivered to the user may be full
bandwidth
or compressed video. Likewise the video signal on the bus 250 delivering the
simultaneous signal to the multiple remote switches 210, 212, 214, 216, 218
may be
compressed video. This embodiment allows for a relatively low cost remote user
box because the most costly switching equipment is located at the headend and
each remote switch may be allocated to any user. Therefore, the cost is spread
over the larger population of users.
As an example, it is assumed that the signal received by receiver 206 is
placed on
bus line 270 of the video signal bus 250 and is forwarded to set top box 236
and
displayed on monitor 10. At some point the set top box 236 causes a user
request
to be generated. The user request is based on a current or past entry on
controller
9 and/or information stored in set top box 236 (e.g., information stored could
be
previous user response information or personal profile information). The cable
TV system 260 may amplify the user request at amplifier 230 while carrying the
user request back to frequency agile modulator 226, which communicates the
request to remote switch 216. During the vertical blanking interval, the
remote
CA 02220483 1997-11-07
WD 96/37075 PCTlUS96/07236
switch 216 disconnects from old bus line 270 and switches to the appropriate
line
on the video signal bus 250, in this example line 280, based on the user
request.
This is shown by the dotted-line connection at 290. The signal from the new
connection (received by receiver 204) is sent through the frequency agile
modulator 226 on channel 47 and the cable TV system 260 to the user's set top
box
236. The new signal is seamlessly displayed on television monitor 10, without
any switching occurring at set top box 236.
As alternatives to the cable headend 300 and cable TV 260 of Figure 7, a
telephone
central office and/or telephone lines may be used. This alternative would
allow
the set tops 232, 234, 236 to receive interactive programming from a telephone
company or cable headend via telephonic communication.
Figures 3-6 show preferred embodiments of the receiver 7 and signal selector 8
of
the present invention to enable seamless flicker-free transparent switching
between the digital video signals on the same channel or different channels.
These embodiments may be connected to any transmission media or simply
connected to the output of any stand-alone storage means for the digitized
multiplexed interactive program. Preferably, the receiv er 7 and signal
selector 8
are both components of an interactive program box 11, which connects to a
television or other display monitor. Alternatively, the required functionality
of
the RF receiver 7, signal selector 8 and monitor could all be combined in a
standard personal computer by the addition of a few components to the personal
computer. To provide this capability, only an RF demodulator board, digital
demultiplexer, decompressor(s), frame buffer(s), and sync components need to
be
added to the personal computer. These items, and any other components, may be
connected to the PC processor and storage elements as disclosed in Figures 3-
5. In
this embodiment, the user makes selections via the computer keyboard.
Figure 3 shows an embodiment with a single analog frame buffer. Figure 4
includes pairs of RF demodulators, error correctors, an<i demultiplexers
and/or a
pair of digital video buffers, as described below. Figure 5 utilizes a single
digital
multiframe memory. Figure 6 is directed to an alternative method of using the
memory 190 described in Figure 5, and depicts buffering the memory contents
16
CA 02220483 2003-04-30
WO 96137075 PCTlUS96/07236
over time. Similar circuit elements appearing in Figures 3-5 are referred to
by
using the same reference numbers.
Figure 3 shows an embodiment which allows for a seamless video switch
between two or more separate digital video signals. As shown in Figure 3, a
microprocessor 108 is connected to RF demodulator 102 and digital
demultiplexer
106. The microprocessor 108 directs demodulation and demultiplexing of the
proper channel and data stream to obtain the correct video signal. The proper
channel is determined either by examination of the user's input from user
interface 130 and/or any other information or criteria (such as personal
profile
information) stored in RAM/ROM 120. For example, the RAM/ROM 120 could
store commands provided within the video signals as discussed in patent No.
4,602,279. The user interface 130 may be an
infrared, wireless, or wired receiver that receives information from multiple
choice control unit 9.
The RF demodulator 102 is part of the receiver 7, and demodulates data from
the
broadcast channel directed by the microprocessor 108. After the data stream is
demodulated, it passes through a forward error correction circuit 104 into a
digital
demultiplexer I06. The demultiplexer is controlled by microprocessor 108 to
provide a specific video signal out of a number of video signals which may be
located within the data stream on the demodulated broadcast channel. The
demultiplexed video signal is then decompressed and decoded by decompressor/
decoder 110. The decompressor/decoder 110 includes a digital to analog
converter to convert the decompressed signal into an analog signal. The analog
video signal is synchronized by a sync add circuit 150 and a sync generator
140.
The analog video signal is then buffered by an analog video frame buffer 160.
The buffered video signal is modulated by a modulator 170 into a NTSC
compatible signal.
By using an analog video frame buffer 160 and delaying the viewing of a given
signal, enough time is allowed for the decompressor/decoder 110 to lock onto,
decompress, convert to analog, and wait for the resultant vertical interval of
a
second video signal. For example, assume video signal A is currently being
processed and transferred through the circuit shown in Figure 3 and displayed
on
17
CA 02220483 1997-11-07
WO 96!37075 PCT/US96/07236
the television set or monitor 10. Based upon a user selection, the
microprocessor
108 directs the digital demultiplexer 106 and RF demodulator 102 to switch to
another video signal, video signal B. To accomplish this, the analog video
from
the first digital video signal, video signal A, complete 'with video sync, is
fed into
analog video frame buffer 160. This buffer can hold the full video picture for
"ri'
number of frames after which the signal is output to th.e display. In effect,
a
delayed video signal A is viewed "n" number of frames after the signal has
been
received by the television receiver. When the user selects a different video
path
by means of pressing a button on a keypad, the microprocessor instructs the
digital demultiplexer 106 to stop decoding signal A and lock onto signal B to
begin
decoding signal B instead of signal A.
While this is happening, even though the decompressor/decoder 110 is no
longer decompressing video signal A, the display is still showing video signal
A
because it is being read from the buffer 160. As soon a;> decompressing and
decoding occurs, the microprocessor 108 looks for the next vertical blanking
interval (VBI) and instructs the analog video frame buffer 160 to switch to
its
input, rather than its buffered output at the occurrence of the VBI.
Since the IZF demodulator 102, forward error corrector 104, digital
demultiplexer
106, and decompressor/decoder 110 require a certain time period to decompress
and decode the video signal B frame from its data stream, the size of the
buffer
160 has to be large enough so that this processing can take place without
interruption during the switching of the video signals. If desired, the system
may
continue to use the buffer 160 in anticipation of a future switch. By using
the
microprocessor 108 to manipulate the fill and empty rate of the buffer 160,
the
buffer 160 may be rapidly filled with video signal B frames and then after a
period
of time will be reset and ready to make another switch to another video in the
same manner. The buffer 160 may also be reset by skipping frames or providing
a
delay between sequential frame outputs for a short time in order to fill the
buffer
160. If a delay is used to maintain video signal or frame output while the
buffer ,
160 is being filled a slight distortion may occur for a brief amount of time.
Because a first video signal is always displayed as the output of the buffer
after the
delay, the buffered video masks the acquisition and decoding of a second video
18
CA 02220483 1997-11-07
WO 96/37075 PCT/US96/07236
signal. As long as the buffer is large enough to keep the first video running
while
the second video is being decompressed and decoded, a seamless switch will
occur.
Figure 4 shows an alternate, dual tuner embodiment for seamless switching
between separate video signals. In this embodiment, the microprocessor 108
controls the selection of the RF channel that is demodulated by RF
demodulators
102A, 102B. The demodulated data streams enter the forward error correctors
104A, 104B. At the output of the forward error correctors, the data streams
are
transmitted to the input of the digital demultiplexers 106A, 106B.
As with the RFdemodulators 102A, 102B, the digital demultiplexers 106A, 106B
are controlled by the microprocessor 108. This configuration allows the
microprocessor 108 to independently select two different individual time-
multiplexed video signals on different channels and data streams. If all the
video
signals of an interactive program were contained on a single channel or data
stream, it would only be necessary to have a single RF demodulator, forward
error corrector, and digital demultiplexer serially connected and feeding into
the
two digital video buffers.
Two data streams are provided from the digital demultiplexers 106A and 106B.
One data stream carries video information pertaining to the video signal the
user
is currently viewing. The second data stream carries the video signal selected
based on the user's previous and/or current interactive selections from the
user
interface 130, as determined by the microprocessor 108.
The digital information on each of the two streams is buffered in digital
video
buffers 164, 165. The buffered signals are then decompressed and converted
into
analog signals by decompressors/ decoders 110A, 110B which include digital to
analog converters. The decompressors 110A, 110B are preferably MPEG decoders.
A local sync generator 140 is connected to sync add 151, 152 and frame sync
153,
154 circuits. Because both streams are synchronized based on signals from the
same local sync generator 140, each stream becomes synchronized to the other.
In
particular, the signals on each stream are frame synchronized.
19
CA 02220483 1997-11-07
WO 96/37075 PCT/US96/07236
A vertical blanking interval (VBI) switch 180 is connected to the
microprocessor
108 so that the input may be switched during the vertical blanking interval of
the
current stream, resulting in a seamless switch to the viewer.
The embodiment of Figure 4 operates as follows. Based on user responses and
control codes, it is assumed that the microprocessor lOt3 determines that a
switch
from video signal A to video signal C should be performed. RF demodulator
102A and digital demultiplexer 106A are processing the currently viewed video
signal, video signal A, which is progressing through the upper branch
components 164, 110A, 151, 153, 180. A command is issued from the
microprocessor 108 to the RF demodulator 102B commanding a switch to the
channel and data stream on which video signal C is located. The microprocessor
108 also instructs the digital demultiplexer 106B to provide video signal C
from
the received data stream to digital video buffer 165.
At this point, RF demodulator 102A and digital demul~~iplexer 106A are still
independently receiving and processing video signal A, which continues through
the upper branch of the circuit.
At a certain point, the digital decompressor/decoder 1.10B in the lower branch
will begin filling up with video C frames. After the video signal C is
decompressed and decoded, it is converted into analog. A local sync generator
140
inserts both local sync and frame sync to video signal C via sync add circuit
152
and frame sync circuit 154 in order to synchronize it with the currently
displayed
video signal A, which is still being provided from digital video buffer 164.
At the
appropriate switch point, triggered by programming codes supplied with each
video signal A and C, the microprocessor directs the VBI switch 180 to switch
in
the vertical blanking interval from video A to video C, at which time video C
will then seamlessly appear on the television set monii:or 10.
Digital video buffers 164 and 165 may be used in the circuit of Figure 4, but
are
optional. However, in an alternative embodiment the buffers 164 and 165 would
be required to provide a seamless switch if the Figure 4 circuit was modified
to
incorporate a single RF demodulator 102, single forward error corrector 104,
and
single digital demultiplexer 106 (as in Figure 3), each with a single input
and
single output. In this alternative embodiment, the circuit cannot
independently
CA 02220483 1997-11-07
WO 96/37075 PCTIUS96/07236
receive and demultiplex two data streams on different frequency channels. One
buffer is used to store previously received video signals, while the other
buffer
quickly passes through the selected video signals.
Based on the same assumptions above, video signal A is progressing through the
upper branch of the circuit and it is desired to switch to video signal C.
However,
in this alternative embodiment, digital video buffer 164 is providing maximum
buffering to video signal A.
Because it is desired to switch to video signal C, microprocessor 108 directs
the
alternative circuit (containing a single RF receiver 102, single forward error
corrector 104 and single digital demultiplexer 106 connected in serial), to
receive
and demultiplex the data stream on which video signal C is located, which may
be different than that of video signal A. When video signal C is
demultiplexed,
the microprocessor 108 directs digital video buffer 165 to provide minimum
buffering of video signal C so that decompressor/decoder 110B may quickly
decompress and decode the digital signals. After decompression and decoding,
video signal C is synchronized with video signal A. At this time video signal
A
is read for display from digital video buffer 164. The digital video buffer
164 must
be large enough to provide video frames for output during the time it takes
the
RF demodulator and digital demultiplexer to switch to video signal C and the
time required for decompression, decoding, and synchronization of video signal
C.
When video signal C is synchronized with video signal A, the microprocessor
directs VBI switch 180 to switch from video signal A to video signal C in the
vertical blanking interval of video signal A, thereby providing a seamless and
flicker-free switch.
At this time, digital video buffer 165 will begin to utilize maximum buffering
by
altering its fill/empty rate as described above with respect to the Figure 4
embodiment. When adequate buffering is achieved, a switch to another video
signal may be performed in the same manner as described above.
Another preferred embodiment is shown in Figure 5. This embodiment also
includes an RF demodulator 102, a forward error corrector 104, and a digital
demultiplexer 106. However, the circuitry differs along the rest of the chain
to
21
CA 02220483 1997-11-07
WO 96!37075 PCT/US96/07236
the television set or monitor 10. In this embodiment, a large memory 190 is
incorporated and connected to the output of the demultiplexer 106 for storing
the
compressed composite digital video signal. The decornpressor/decoder 110 is
inserted at the output of the compressed memory 190. The
decompressor/decoder 110 decompresses the digital signal, converts the signal
to
analog and forwards the analog signal to the RF encoder 170 for transmission
to
the monitor 10. Once the composite compressed digital video signal is fed into
the compressed memory 190, the microprocessor 108 directs a pointer to be
placed
somewhere along the compressed digital video signal. Based on the placement of
the pointer, different frames and different segments of the composite digital
video signal will be read from memory 190 for decompression and decoding.
The different video signals are distinguished from one another because they
are
labeled, preferably by headers. Assuming that video signal A has been selected
for
play on the monitor 10, the compressed digital memory 190 fills up with A
frames. Assuming a switch to video signal C is desired, the microprocessor 108
directs the RF demodulator 102 and digital demultiplexer 106 to begin filling
the
compressed memory 190 with video C frames. The decoder pointer begins to
move down. As soon as a sufficient number of C frames have entered the
compressed memory 190, the pointer will then jump i:o the beginning of the C
frames. The C frames are then output into the decompressor/decoder 110 where
the digital frames are converted into an analog signal.
The digital video is multiplexed in a series of easily identifiable packets.
These
packets may contain full compressed frames of video (I frames) or may include
only the differences between full frames (B frames or 1' frames). To be able
to
reconstruct the full video images, the decompressor/decoder 110 needs to have
a
minimum number of I, P and B frames. In this embodiment the buffer memory
190 is filled with compressed packets from a given stream of data including I,
B
and P frames. When the microprocessor 108 instructs the digital demux 106 to
start sending packets from a different data stream there is no way to be
certain
that the next packet will be an I packet needed for decoding the second video
stream. To avoid a breakup of the video images, which would occur if the
decompressor, decoder 110 suddenly started receiving packets unrelated to the
22
CA 02220483 1997-11-07
WO 96/37075 PCT/LTS96/07236
stream it was decoding, the microprocessor 108 starts to fill up the memory
190
with video signal C packets until it is determined that a full sequence of I,
B and P
frames are available. As soon as the valid sequence is in memory the
microprocessor 108 moves the memory read pointer to the start of a valid
sequence of C video signal packets so that the decompressor decoder can
successfully decode the C signals. This results in a seamless switch from
video
signal A to video signal C.
This embodiment requires a data channel for enabling a synchronous switch
between a first video stream and a second video stream. This data channel
comprises the ACTV codes which link together the different program elements
and information segments on the different video signals. In addition, the data
channel also comprises synchronization pulses and a time code to signify to
the
pointer the proper time to skip from a memory location representing one video
signal to a memory location representing another video signal in order to
enable
a seamless switch.
The microprocessor 108 reads the data signal from the digital demultiplexer
106
and communicates pertinent data to the sync add circuit 150, which is
connected
to sync generator 140. The microprocessor 108 is then able to synchronously
communicate with the memory 190.
The time code sent will identify the timing for one picture, as well as for
multiple
pictures, and will lock the different pictures together. This is done through
the
use of similar clocks at both the transmission end and the receiver. A time
code
is used in order to keep the two clocks at both the transmission and receive
end
synchronously connected to one another. Once the clocks at both ends are
working synchronously, each of the multiplexed video streams must be
synchronized to the clocks. In order to synchronize the multiplexed video
stream
to the clocks, each of the individual channels must be referenced to a common
reference point and must be identified.
In the preferred embodiment, a packet header would be incorporated into the
transport layer of the MPEG signal to identify the various channels. The
packet
header will also include information as to where to insert the vertical
blanking
interval. In MPEG, the vertical blanking interval is not transmitted from the
23
CA 02220483 1997-11-07
WO 96/37075 PCT/US96/07236
headend. Therefore, the vertical blanking interval must be generated locally.
The packet header eye will identify at what time the vertical blanking
interval is
in existence in order to effectuate a seamless switch between analog pictures.
In summary, the combination of clock and the information imbedded in either
the transport layer of NIPEG or in a separate packet on a separate data
channel
effectuates the linking between each video signal and a corresponding time
point.
The data channel also includes information designating when all the various
video signals will be in synchronism with one another. It is at these points
that
the microprocessor 108 may direct the pointer to skip from one location to
another location, at a time (such as during the VBI) when a seamless switch
will
result.
Figure 6 shows a method to use the memory 190 described with respect to Figure
5. Figure 6 shows a timed sequence of video signals that are stored in the
memory 190. In Figure 6, the decompressor/decoder 110 requires, for example,
five specific video frames before decompressing and decoding can begin.
Specifically, the decompressor/decoder 110 requires frames A,a,a,a,A,
B,b,b,b,B, or
C,c,c,c,C, and so forth. Because of the MPEG compression algorithm used,
certain
frames of video signals (represented by the lowercase letters) cannot be
decoded
and decompressed without utilizing information within certain other
compressed frames (represented by uppercase letters).
At time TO the viewer is watching channel A, the memory is filled with frames
relating to channel A, and the pointer is pointing to frame Al.. At time T1,
A1 has
been output to the decompressor/decoder 110 and the :microprocessor 108 has
switched to channel C. As with a buffer, the memory contents are shifted so
that
new data cl is stored and data a2 is ready for output to the
decompressor/decoder
110, because the pointer is located at a2. At this point in time, the
microprocessor
108 is monitoring the data as it arrives to determine wizen data relating to
channel C is present. At time T2, c2 is stored, and the memory contents are
shifted so that a3 is ready for output. At time T3, C3 is read into the memory
and
the contents are again shifted so that a4 is ready for output. The pointer
position
is unchanged.
24
CA 02220483 1997-11-07
WO 96!37075 PCT/US96/07236
At time T4, compressed video signal c4 is stored. The A5 frame is presented
for
output at the pointer location. At time T5, c5 is stored, while the data is
again
shifted and a6 is ready for output.
In this example, the microprocessor 108 immediately recognizes channel C data
c6
and C~ at times T6 and T7, and continues to shift the data in memory without
moving the pointer. Therefore, channel A data frames a~ and A8 are output to
the decompressor/decoder 110. At time T8, there is enough channel C data to
begin a decoding sequence, assuming that the decompressor/decoder 110 requires
three frames (c,c,c) which reference two frames (C,C) for decompression and
decoding purposes. Therefore, at time T8, the pointer is moved to point to
frame
C3, which will be the next frame output to the decompressor/decoder 110. Frame
C3 is used in the decompression and decoding of one or more of frames c4, c5,
and
C6.
Rather than outputting the compressed video signals individually, the group of
C3, c4, cs, c6, and C~ may be output to the decompressor/decoder 110 at once.
This
is the preferable technique where C~ is required to decode preceding frames
(i.e.
c4, c5 or c6).
As with the examples described previously and shown in Figures 3 and 4, the
buffer may alter its fill and empty rates to provide maximum buffering. If
this is
performed, the pointer will be reset to its original position, as shown in
Figure 6
at time Tn.
Using Figures 3-6, and the previous description of memories and buffers, the
artisan will be able to provide a seamless switch for flicker-free switching
between
interactive channels of the interactive television system of this invention.
Other
variations using the above schemes are also possible, as realized by the
artisan of
ordinary skill.
Although the present invention has been described in detail with respect to
certain embodiments and examples, variations and modifications exist which are
within the scope of the present invention as defined in the following claims.
