Note: Descriptions are shown in the official language in which they were submitted.
CA 02425739 2003-04-29
A DIGITAL INTERACTIVE SYSTEIyt FOR PROVIDING FULL
INTERACTIVITY WITH LIVE PROGRAMMING EVENTS
This is a divisional of Canadian patent application serial number
?,?83,957 which is the National Phase application of PCT International
application no. PCT/US97/09045 published 17 September 1998 under
International publication number WO 98/410?0.
BACKGROUND OF TI-~E INVENTION
Interactive video and audio presentation systems are currently being
introduced into the entertainment and educational industries. A prominent
interactive technology that has been applied successfully in these industries
is based on providing interactivity in a one-way system through the
provision of multiple time-synched parallel channels of information. For
example, commonly owned Freeman et al. patents, U.S. patent nos.
4,264,925 and 4,?64,9?4, which provide both audio and video interactivity,
l 5 disclose interactive television systems where switching among multiple
broadcast or cable channels based on viewer selections provides an
interactive capability.
These systems have been enhanced to include memory functions
using computer logic and memory, where selection of system responses
played to the viewer are based on the processing and storage of subscriber
responses, as disclosed in Freeman patent, U.S. patent no. 4,507,680.
CA 02425739 2003-04-29
The benefits of providing interactivity through the use of different audio
responses is disclosed in Freeman, U.S. patent nos. 4,$4?,698, 4,847,699 and
4,84: ,700. These television systems provide a common video signal accompanied
by several synchronized audio charnels to provide content related user
selectable
responses. The audio signals produce different audio responses, and in some
cases, these are syllable synched to a first audio script and to the video
signal (such
as to a person or character on a display), providing the perception that the
person's or character's mouth movements match the spoken words.
Interactivity is brought to the classroom in the Freeman U_S. patent no.
x,537,1-~1. The distance learning system ciaimed in this application enhances
the
classroom educatienai experience through an innovative use of interactive
teciZnoio~% over transmission independent media. When an instructor, either
broadcast live on video or displayed rrom videotape, asks a question, each and
every student responds, preferably by entering a response or. a remote
handset,
land each student immediately receives a distinct and substantive audio
response
to his or her unique se!ecuon. The individualization o: audio response from
the
interactive program is a major aspect of the invention.
Individualization of audio is brought to the home based on the technology
disclosed in Freeman U.S. patent no. J,J85,858. i'his svs~em provides a
program
?0 that can be watched on any conventional television set or multimedia
computer
as a normal program. But if the v fewer has a special interactive program box
connected to the television, he or she can experience a fully functional
interactive
program. Each interactive viewer eniovs personalized audio responses and video
graphics overiaved on the screen. The interactive program can be provided to
CA 02425739 2003-04-29
WO 98141020 PCTIUS97109045
television sets or to computers by cable, direct broadcast satellite,
television
broadcast or other transmission means, and car, be analog or digital.
However, what is needed is an interactive presentation system for
providing tzue video, audio and graphics interactivity with digital programs
broadcast live. Such a system must efficiently package all the digital
elements of
the Iive interactive program at a centralized control studio and allow viewers
at
home to receive personalized interactive programming.
SUMMARY OF THE INVENTION
In accordance with the invention, there is provided an interactive digital
system allowing the viewer active participation in selecting digital video
streams,
associated with different camera angles, for example, and integrated audio
and/or
graphics segments. Further, Web pages from Internet Web sites can be
integrated
into the program. The invention is particularly suited for the environment of
1~ live events, such as the broadcast of live ~portin~ events. The viewer can
appear
to direct tre camera shots by instantly changing among various camera angles,
choose player interviews, or display associated statistical data on the team
or
players via graphics. In this manner, the system allows the individual
subscriber
to act as if he or she has control over how the program is directed and
presented
~0 on their personal television set. In addition to selecting different camera
angles,
various audio options, closeups, slow motion, replays, graphics overlays,
graphics
or audio from Web sites, etc., are ail possible. Further, games can be
integrated
with the live shorts programming to increase viewer interest.
Thus, viewers can customize the content of programs. The interactive
digital programming of the present invention is particularly advantageous for
CA 02425739 2003-04-29
WO 98Id10.',0 PCT/US97/09045
viewing live sporting events. Viewers are not limited to selecting from
multiple
camera angles, but may also call up player statistics on demand, listen to
selected
player interviews, etc. Cameras can be focused on different segments of an
event.
Further, video options could include video replay, slow motion effects,
isolation
on a particular player or group, etc. Changes are seamless, thereby adding to
the
effect that the viewer is directing the television show just as a director now
does
from a control room.
This "director" role by the viewer is possible due to the interactive
technology of the present invention and also due to the digital compression
and
IO transmission scheme which allows for much greater information throughput
over a given bandwidth, allowing viewers to choose from angles that are
already
available but presently cut by the director.
The digital interactive System is based uDOn branches which occur in the
course of the lull-motion video. Branches are real-time parallel paths that
may
I~ be other hill-motion video segments, graphics which are integrated into the
video, audio segments, and/ or retrieved Web pages which are integrated into
the
live event.
Sometimes, the interactive digital system will act upon the viewer's
response immediately; other times, it will utilize ACTV's unique "profiling"
20 concept to act upon the response later. This technology enables the system
to
"remember" the viewer's responses and desires, and integrate them into the
video, audio, graphics and i or Web site information at a later point. For
example,
the newer could specify at the beginning of a football game to isolate the
offensive auarterback of a particular team. Thus, whenever the team of choice
is
'_5 on offense, the video isolation or the quarterback is displayed to the
viewer
CA 02425739 2003-04-29
WO 98/41020 PCTIUS97/09045
J
automakically. Or, based on how a v fewer has selected camera angles, replays,
etc.,
over the past five minutes, the system acts to mimic khese selections at later
times
during the program. The system of the present invention "learns" from the
viewer how they want to view the game, and thus, continues viewer selection
sequences made earlier.
At the source, the present invention comprises a plurality of video
cameras, each of the video cameras relaying a different predetermined view of
an
event. The video signals corresponding to the different cameras are forwarded
to
a central control studio. Further, one or more audio signals or graphic
statistical
IO overlays can be collected and sent to a central control studio. After
receiving the
video, audio, and graphics signals at the central control studio, these
signals are
digitized and compressed in digital video anti audio compressors. These
signals
are then combined with special data codes into a "digital package," and
subsequently, transmitted over a cable distribution system. The special data
codes
5 are the Kevs to unlocking the interactive potential of the program.
The digital program signals are transmitted to a receive site by any suitable
transmission means. Once received by a receive antenna, the cii~;ital program
signals are passed along on a digital cable television distribution system to
the
viewer homes. Further, some other signals or commercials can be inserted at
the
20 local head end. The signals are received and processed in a digital cable
box.
Selections of the video, audio, graphics display and!or Web pages can be made
as
a function of immediate viewer entrees, or to interrogatory responses
presented at
the beginning or during the program, or based on a prestored viewer profile.
once a decision is made to switch from one video option to another video
option, the di~itai switch is performed seamlessly.
CA 02425739 2003-04-29
WO 98/.11020 PCT/US9~/09045
6
As mentioned above, the program at predetermined times or immediately
upon user entry can retrieve and branch to il~formative segments from Web
sites.
For example, a viewer watching a sporting event, through the system of the
present invention, can receive a stream of Web pages which provide additional,
specific information relating to a favorite player, team or perhaps the
remaining
schedule for the sports team, as examples. In addition, users can take
advantage
of the two-way capabilities of the Internet to respond to polls or to link to
additional sites.
Another Internet-based application allows advertisers to speak more
directly to consumers by directly sending Web pa,es to the consumer instead of
merely disalaving Web addresses in their commercials. The particular
advertising information from Web sites can be targeted to viewers based on the
viewer prorile, stored either in the digital set top box or at the cable
headend.
Alternatzvelv, yveb site access can be initiated by the viewer by simply
clicking on
l~ the remote during the commercial. Thus, viewers have the ca~abilitv to
individualiv select Web sites if then want more informatzon from advertisers,
for
example.
The video programming and corresponding Internet pages can be viewed
either on personal computers eQUipped with a television card on special
digital
cable boxes with stored interactive Internet application software providing
Internet access, or on digital television sets, all of which would utilize the
specialized TV /Internet software of the present invention.
The present invention also has applications for other types of
programming. For example, viewers can direct the scenes of a murder mystery.
'_'~ Switching from one scene to another can. be done seamlessly without
noticeable
CA 02425739 2003-04-29
7
etfect on the viewer. Further, the present invention can be used for any kind
of live
or pre-recorded event. Fur instance, a music concert or a political convention
can be
enhanced in the manner of the invention.
.~ccordinaly, in one aspect, the present invention provides a live interactive
digital presentation system, comprising: a means for receiving live
programming,
wherein the programming contains a plurality of digital video and audio; a
viewer
interface for receiving viewer entries; a microprocessor, connected to the
viewer
interface. for selecting and switching to one of the videu and audio signals
based
on at least one of the viewer entries; a means for displaying the selection
video
i t) signal; and a means for playing the selected audio signal.
In a further aspect, the present invention provides a system for providing
livo interactive c i';ital programming, comprising: a means for receiving
video signals
some oT which are tram a plurality of~ video ctimeras, one or more of the
cameras
relaying a di tferent precieterminec~ vi~:w of a live went; ;~ means for
producing one or
I ~ more audio si~~nals corresponding to the live event; a means for
generating one or
snore graphics signals; at least one digital compression device, connected to
the
receiving and produciny~ means, for ~iigitallv oompressin'~ the video,
graphics and
audio aiv~nals; a diwita! rnultipie~er, connected to the digital compression
device. for
multipl~;cin'~ the video, graphics and audio siunals, into a combined digital
program
't) stream: and a means for transrnittin~~.r the combined digital program
stream.
CA 02425739 2003-04-29
l :l
In a still further aspect, the present invention provides a method for
providin';
live interactive di~~ital programming, comprising the steps of: obtaining
video signals
from a plurality of video cameras, one or more of the cameras relaying a
different
view of a live event; producing one or more audio signals corresponding to the
live
event; receiving the video and audio signals in a control studio; digitally
compressing the video and audio signals; digitally multiplexing the video and
audio
signals into a combined digital program stream; transmitting the combined
digital
program stream; receiving the combined digital program stream at a receive
site;
di~itallv demuttiplexing the video and audio signals; and displaying the video
signal
1 ~? on a screen.
In a further aspect. the present invention provides a method for providing
live
interactive ui'~ital pro~'ramming, comprisin<g: receiving live interactive
programmin<g,
the live interactive programming comprising ; plurality of digitally
compressed
video and audio, sad one or more information segment addresses specifying one
or
more addresses of related information segments, the reception system
comprisin':
obtainin~,~ viower entries; selecting one of the video and audio signals and
directing a
switch to the selected video and audio signals; decoding the information
segment
addresses; retrieving the one or more information segments residing at the
determined addresses; demultiplexing the selected video and audio signals; and
?0 presentin<g the video, audio signals and information se;~ments.
tn another aspect, the preaent invention provides a method for providing
~.ii~~ital v»ieo prot~rammin~', ~ompri>in~~ the steps of: obtaining a
plurality of video
CA 02425739 2003-04-29
?b
signals: d~Iavin'~ at least one of the video signals for a predetermined
amount of
time to create a replay video 5i~~nal: producing one or more audio signals;
digitally
compressing the video and audio signals into a combined digital program
stream;
transmitting the combined digital program stream.
In a further aspect, the present invention provides a system of providing to a
user digital programming at a receiver station, comprising the steps of: means
for receiving a plurality of digitally compressed video signals, each signal
corresponding to a different video option of a proy~ram; a processor, operable
connected to the receiving means. for selectin'1 one of the video options.
wherein
the video option is selected based upon a user profile resulting in a program
tailored
to the user: a means for digitally decompressing the selected video signal
corresponding to the selected video option: and a means for displaying the
selected
video signal correspondin;~ to the aelected video option. wherein visual
transition to
the selected video signal is seami~ss.
In yet a turther aspect, the present invention provides a live interactive
digital pro~'zrammin~; system, comprisin~~: a viewer television reception
system for
receiving live interactive progrrarnmin<~, the live interactive pro~~ramminv
comprising= a plurality of digitailv compressed video, audio, branching codes
and
~~~raphics signals, the reception system comprising: a viewer interface for
receiving
?0 vie~.ver entries: a microprocessor. connected to the viewer interface, for
selecting
one of the video and audio sig=nals based upon the branching codes and the
received
viwver entries; a demultiple~cer, for demultipiexing the selected video and
audio
~i~nals: a decompressor/decoder, connected to the demultiplexer for
decompressing
CA 02425739 2003-04-29
y
the demultiplexed selected video and audio si_Jnals: a means for displaying
the
selected video signal; and a moans feyr plavin;l the selected audio signal.
In yet a further aspect, the present invention provides a viewer reception
system for receiving interactive pro~~rammin~. the interactive programming
comprising a plurality of synchronized and compressed, live and prerecorded.
video
signals and graphics signals, a plurality of compressed. live and prerecorded
audio
signals, and branching codes which have been multiple~ced with said video.
graphics and audio signals. and sari viewer reception system vomprisina: a
viewer
interface for receiving a view entries; proceasin'~ means. responsive to the
viewer
interlace. for determining which ot'the received video. ~~raphics and audio
signals
to play: a demultiple~er for 5electin~g and demultipiering a apecitic video
simal
under the control of said processir.~~ means: a decompressor'decoder connected
to
the demultiple~.er for decompressm~.; the demultiple~ed selected video
si~?nals: and
a video frame buffer for receivin:; and delayin~~ play of a received video
signal
wher;:by a seamless switch to a further video si~~nal may be directed b~~ said
processing' means at a predetermined, video switchin~~ time. the further video
si'_nal
selecred and the predetermined video switchinvg time of the selection being= a
function of the branching codes; and display means for displaying the selected
video signal: and wherein the display means is additionally arran~,ged to
display
?0 ~~raphics si~~nals ~~ctracted by said demultiplexer a5 directed by said
processing
means: and the viewer reception system tirrthc:r comprisin~l: an audio switch
connected to the demultiplexer for selecting and receiving, a specific audio
signal as
Sirect~d by said processing means. whereby a switch to a lurther audio signal
may
CA 02425739 2003-04-29
7d
be directed by said processing means at a predetermined, audio switching time.
the
further audio si~~nal selected and the predetermined audio switchin~~ time
being a
function of the branching codes; and means for playing the audio signal
selected.
In yet a further aspect, the present invention provides a system for
providing interactive digital programming, comprising: means for receiving
video
signals from a plurality of video cameras, one or more of the cameras relaying
a
different predetermined view of a live event, the video receiving means also
receiving prerecorded video signals; means for receiving audio signals
corresponding to the live event, the audio receiving means also receiving
t 0 prerecorded audio signals: at least one digital compression device
connected to
the video receiving means. the audio receivin~~ means, and said generating
means.
for digitally compressing the video, graphics and audio signals; processing
means, connected to the compression device, for creating a set of data
commands
corresponding to the audio, graphics and video signals. the data commands
1 ~ includin~a branching codes; a digital multiplerer, connected to the
digital
compression device, for multiplering the video, graphics and audio signals,
and
the data commands into a combined digital program scream: and means for
transmitting the combined digital program stream.
In yet a further aspect, the present invention provides a method for
'?0 providin~,~ interactive digital pro~nammin'.:, comprisinti the steps of:
obtaining video
si~~nals from a plurality of video cameras, on or more of the cameras relaying
a
ditfc:rent view of a live went: obtaining prerecorded video signals; producing
one
CA 02425739 2003-04-29
;L
~.~r more audio si~~nala correspon ding to thc: live event: obtaining
prerecorded audio
si~~na(: creating one ur more graphics si'.:nals: di~;itallv compressin~,~ the
video.
graphic and audio si~~nals; producing a set of data commands which
correspondin~~
to the audio. graphics and video si~~nals. the data commands including
branching
codes; di~itallv multiplexing the video. ~~raphics and audio signals, and the
data
commands into a combined digital program stream: and transmitting the combined
di'7ital proy~ram stream.
In vet a further aspect, the present invention provides a method for
receiving interactive pro~.:rammina, the live interactive programming
comprising a
plurality of digitally compressed video, audio, and y~raphics signals and
branching
;:odes, the method c;omprisin~~ the steps of: selectin'z one of the video
signals and
directin~~ a seamless switch to the selected video si~~nal at a first
predetermined
time, the selection of the video si<~nal and the first predetermined time of
the
selection being a function of the hranchin~' codes: selecting one of the audio
si~~nals and directing a switch to the Selected audio si~~nal at a second
predetermined time. the selection of the audio si~4nal and the second
predetermined time of the selection being a function of the branching codes;
demultiplexin~ the selected video and audio signals; decompressing the
demultiplexed selected vide~~ and audio sr~~nals; displaying the selected
video
?t) si~~nal: and playing the selected audio signal.
CA 02425739 2004-02-05
7f
In a further aspect, the present invention provides a system for
providing live interactive digital programming, comprising: a means for
receiving video signals from a plurality of video cameras, one or more of
the cameras relaying a different predetermined view of a live event; a
means for producing one or more audio signals corresponding to the live
event; a means for generating one or more graphics signals; at least one
digital compression device, connected to the receiving and producing
means, for digitally compressing the video, graphics and audio signals; a
means for processing, connected to the compression device, wherein the
1 o processing means creates a set of data commands which link together the
various audio, graphics and video signals, the data commands including
branching commands; a digital multiplexer, connected to the digital
compression device, for multiplexing the video, graphics and audio
signals, and the data codes into a combined digital program stream; and a
means for transmitting the combined digital program stream.
In a still further aspect, the present invention provides a method for
providing interactive digital programming, comprising: receiving video
signals from a plurality of video cameras, one or more of the cameras
relaying a different view of an event; producing one or more audio signals
corresponding to the event; generating one or more graphics signals;
digitally compressing the video, graphics and audio signals; multiplexing
CA 02425739 2004-02-05
7g
the video, graphics and audio signals into a combined digital program
stream, wherein the combined digital program stream further comprises a
set of data commands which link together the various audio, graphics and
video signals, the data commands including branching commands; and
transmitting the combine digital program stream.
In a further aspect, the present invention provides a method for
providing interactive digital progranvning, comprising: receiving video
signals from a plurality of video cameras, one or more of the cameras
relaying a different view of an event; producing one or more audio signals
to corresponding to the event; generating one or more graphics signals;
digitally compressing the video, graphics and audio signals; multiplexing
the video, graphics and audio signals into a combined digital program
stream; and transmitting the combined digital program stream, wherein the
transmission further comprises at least one trigger point indicating when to
select and play at least one audio, graphics or video signal.
In a still further aspect, the present invention provides an interactive
digital presentation system, comprising: a receiver for receiving
programming, wherein the programming comprises a plurality of digital
video signals, a plurality of audio signals and data commands which link
2o together the digital video and audio signals, the data commands including
branching commands; a microprocessor for selecting and switching the
CA 02425739 2004-02-05
7h
video and audio signals; a display for displaying the selected video signal;
and a player for playing the selected audio signal.
In a further aspect, the present invention provides an interactive
digital presentation system, comprising: a receiver for receiving
s programming, wherein the programming contains a plurality of digital
video and audio signals and at least one trigger point indicating when to
select and play at least one audio or video signal; a microprocessor for
selecting and switching video and audio signals based on at least one
trigger point; and a player for playing the selected audio or video.
to Accordingly, a primary objective of this invention is providing an
enhanced digital live program allowing the display to be tailored to the
user's desires, choices or interests.
It is an object of this invention to personalize and enhance live
sporting events for the viewer.
is
DESCRIPTION OF THE DRAWINGS
Figure 1 is a diagram of the network and equipment for providing
live digital programming.
Figure 2 is a block diagram of an interactive digital cable box
2o allowing seamless switching between video signals.
CA 02425739 2004-02-05
71
Figure 3 is a block diagram of an alternative dual-tuner interactive
digital cable box allowing seamless switching between video signals.
Figure 4 is a block diagram of another alternative interactive digital
cable box allowing seamless switching between video signals.
Figure 5 is a time diagram showing a representation of trigger points
and corresponding video, audio and/or graphics segments, one or a
combination of which are selected for presentation to the subscriber
immediately after the execution of the trigger point function.
Figure 6 is a block diagram of an alternative embodiment of the
1o interactive system including Internet access.
Figure 7 is a block diagram of the two-way configuration of the
system.
CA 02425739 2003-04-29
S
DE ~RIPTION OF THE PREFERRED EIVIBOD1NIENTS
The present invention is an interactive digital system 1 for producing a
powerful personalized program allowing the home viewer an expanded set of
programming options. Digital TV streams are put into digital packages made up
of
video, audio, data codes and graphics, and are used to provide personalized
responses to viewer selections. Such responses can be further enhanced by
allowing access to Internet Web sites 170. In this manner, sports such as
golf,
football, baseball, basketball, etc. can now be watched with greater interest
and
involvement. However. even further enchanted interactivity is possible with
the
s; present invention due to the provision of various prc7tiling and memory
features.
.~s shown in Figure 1. the present invention begins will the gathering of
several pcassible video streams by way of cameras 10(), strategically located
at a
sporting event 10, for example. Currently, many cameras 100 are employed at a
>portin'~ event 10. Super Bowl covera4~e, for example, typic:allv encompasses
?~ to
~() cameras. These live video streams can be integrated with recorded video
streams which, for example, could include highlights from the current same or
past
~_ames, player profiles, etc. To describe the components and operation of the
present invention. the production and transrnission of a live sporting event
10, i.e.,
football, is chcosen to present the invention features. However. other
applications
n> can be appreciated by the reader, including several disclosed below.
CA 02425739 2003-04-29
WO 98!41020 PCTNS97/09045
9
A. The Control Studio
The interactive broadcast prol;ram is prepared at the control studio 5 into
digital packages. The control studio ~, as shown in Figure 1, allows a
producer to
create and introduce interactive elements during a Live broadcast. In one
S preferred embodiment, the producer pre-records a set of interrogatories or
instructions for the user. These interrogatories may include such questions as
the
following:
SELECT THE C:~I~iER.~ ANGLES/C>PTIONS THAT YOL' PREFER:
IO OPTIONS: END-ZONE
FIFTY YARD LINE
FOCUS ON THE QUrIRTERBACK
FOCUS ON 'rHE DEFENSE
FOCUS ON THE CHEERLEADERS
15 PROVIDE HIGHLIGHTS
WHO IS YOUR FAVORITE TEAVi IN THE G_al~iE?
OPTIONS: I3LL: E TLA~~I
'?p REL:~ TEAM
WHO IS YOUR FAVORITE PLAYER IN THE CWIE?
OPTIONS: RED RUNNER
?5 BLL: E QU a.RTERBACI:;
BLLIE DEFENSIVE END
REI7 QUARTERBr~CK
ETC .
3~ ARE YOU INTERESTED IN RECEIVING STATISTICAL SUMMARIES?
WOULD YOU ENJOY LISTENING TO PLAYER INTERVIEWS DURING
THE G A~IE?
3S Such interrogatories can bc~ presented to the viewer at the beginning of
the
broadcast or scattered throughout the program. Interactive responses to such
interrogatories would include video, audio and s;raphics personalized to the
CA 02425739 2003-04-29
WO 981410:0 PCTlUS97/09045
particular viewer. The preparation or the graphics for presenting such
interrogatories occurs off-line at the control studio ~ using chyron or any
graphics
language. These interrogatories or instructions will ultimately be displayed
to the
home viewer, preferably in the form of graphics, to facilitate the interactive
5 responses. The producer creates these graphic video slides of questions on a
computer using the text editor and chyron. .Associated with each question, the
producer enters a number of possible viewer options. Then, the producer
relates
each possible user entry to one or more corresponding interactive responses.
If
the response is information from an Internet Web page, the producer will
10 indicate the particular Uniform Resource Lecator (URL) of the Web page. The
producer sets a time stamp for when each particular quesrion will appear
during
the program. as explained oeiow in more detail, the viewer response to a query
will be used to direct ~~hicrt video (and 1 or audio, graphics, or W'eb page)
option
will be provided to the viewer. In the present invention, the interactive
response
1~ to the query can occur immediately following the entry of the viewer entry
or at
some preaetermined later time in the program using "trigger points," X00 as
explained in detail below.
With respect to the video segments, cameras 100 are preferably trained on
different segments or the sporting event 10. As is common with broadcasts of a
football game, for example, cameras 100 could be located in the endzone, press
box, the yield and at various other locations throughout the stadium. Further,
various video options can be created including video replay, slow motion,
isolation on cheerleaders, particular player or group of players. Instant
replays are
created by delaying the Live teed for a certain number of seconds. These video
?5 streams are sent to a control studio 5. The control studio ~ contains the
necessary
CA 02425739 2003-04-29
WO 98141020 PCT/US97/09045
11
equipment for packaging the program for deliven~ to the viewers. The studio 5
contains a video switcher 105 which receives the live signals from the cameras
100 by way of various input lines. Further, lines carrying recorded video
streams
from one or more VCRs I10 , computers or CD players feed into the video
switcher 105. The video switcher 105 also receives video inputs from the
control
computer I35. Further, various graphics screens, depicting, for example,
sports
team or player statistics can be designed ~r.~ith the control personal
computer 135
and forwarded to the digital video switch 105. The producer, via the control
PC
I35, directs which video options to pass through the video switcher 105. At
the
output of the video s~~itcher 105, each of the different output video streams
access
a separate encoder 1'_'S and are all GEVLOCIVED, so that each video stream is
synchronized with the other video streams.
after encoding, the video streams are input into a video compressor 125.
Preferably, the digital compression scheme is '~tPEG-~. Preferably, 6~
Quadrature
Amplitude Modulation (QAM) is used as the modulation scheme. In this digital
embodiment, four channels of digitally-compressed video content would carry
about 2' :~tbps using b NtHz of bandwidth. .~lternatmelv, if 256 QOM is
employed, seven-to-one video compression can be achieved with the MPEG-2
scheme. While MPEG-~ is the preferred compression scheme, the signals can be
compressed according to any known standard including MPEG-1, JPEG, or other
DCT coding scheme, wavelets, fractals or other transform or waveform based
technique.
The control studio 5 also contains an audio switcher 115 which receives
live audio signals from microphones or recorded audio from tape players 220,
''S CDs, VCRs 110, etc.. The control computer 135 sends commands to the audio
CA 02425739 2003-04-29
WO 98/41020 PCT/US97/09045
12
switcher 11~ directing which audio options should pass through the switch 115.
Further, in the audio switcher 115 the various audio signals can be aligned to
match the various video signals in time. In addition, VCR audio output is
received by the audio switcher 115. The present invention can accommodate any
number of audio signals as output from the audio switcher 115, as directed by
the
producer. The audio outputs are received by an digital audio
encoder/compressor 130. The audio signals are then preferably sampled, encoded
and compressed in the digital audio encoder/compressor 130. The encoding
technique can be a waveform coding technictue such as PC1V, ADPCM or DM.
Alternatively, the signals can be encoded using synthesizer or vocoder
techniques
such as '4LUSIC AM, Linear Predictive Coding fLPC), Adaptive Predictive Coding
(APC;, and Sub-band coding. Generally, the transmission rate is about 256 kbps
per audio for the stereo pair.
The timing and control for integrating the various multimedia elements is
provided by the ACTV authoring language, a unique set of interactive data
codes
to facilitate the interactive process. The data codes are stored in memory in
the
control computer 135 as part of the ACTV programming language. The codes
comprise commands, or branch codes, for branching between interactive options,
timing signals for controlling the interactive program, data or text, commands
for
~0 termination and initiation or interactive program viewing, or triggers for
executing macros. Preferably, these commands are output from the control
computer 135 and multiplexed mth the ~~ideo streams in the MPEG-2 compressor
1?S, as shown in Figure 1. Interactive options that can be branched to based
on
the branch codes include video segment~c, audio segments, graphics segments
and; or identi>=ied Web pages.
CA 02425739 2003-04-29
WO 98141020 PCTNS971090~t5
1~
There are several commands in the ACTy authoring language that
function to provide the interactive sports programming applications. These
commands are explained below in Section D, which details programming
applications.
r
B. Tl~, 'transmission Sv.~, stem
As shown in figure 1, the digital interactive system 1 uses an interactive
program delivery system with any transmission means including satellite 15,
cable 1~0, wire or television broadcast 1:5 to deliver the interactive program
(hereinafter "composite interactive program"~ from the control studio 5 for
distribution to subscribers in their homes. ~~t the control studio ~, the
signals
from the digital multiplexer 1~0 are converted to RF and distributed to a
microwave 1; 5, cable 1S0 or satellite i~ netSNOrk. Preferably, the digital
interactive
signal is forwarded from the control studio ~ to a cable headend 1~0, and
1~ subseauentlv, sent to the homes via the cable network.
The program is preferably the broadcast of a live event. For example, live
sporting events with added interactive elements can be broadcast from the
control studio 5. Such live interactive elements could be different camera
angles
100, slow motion video, etc., as discussed above, while also incorporating
?0 prerecorded interactive segments such as highlights. Alternatively, the
program
can be produced off-line and stored in a program storage means at the control
studl0 J.
In a satellite broadcast transmission, the digital interactive signals are
transmitted to unlink eauipment where they may be multiplexed, upconverted.
CA 02425739 2003-04-29
WO 98141020 PCTIUS97I09045
14
modulated, amplified and transmitted by satellite 15 to the receiver site 155
for
distribution to the homes.
At the reception end, the composite digital interactive signals enter a
receiver 155 where the signals are demultiplexed, downconverted, demodulated
and then passed to a cable distribution system that directs the signals to the
homes. Although a cable distribution system 150 is the preferred transmission
media to the homes, the digital signals may also be distributed by any
conventionally known technique including satellite I5 to digital satellite
receivers I~~ at the home, fiberoptics, low or high power broadcast television
1i5,
I0 telephone Lines, cellular networks, and similar technology can be used
interchangeably with this program delivery system.
C. The Intera~t~ve Di~~.'tal Box
The interactive digital box 25 is shown schematically in Figure 2.
I5 Preferably, the interactive digital box is a spYCiailv adapted digital
cable box ?S.
The controller ?60 determines what video, audio, hraphics and/or Web pages to
display based upon the interactive commands which it receives. Based upon the
commands, it plays the appropriate video, audio, graphics or Web page options.
The graphics can either be created and sent from the control studio 5 or the
20 graphical images can be created at the interactive digital box 25 based on
instructions preierablv in the interactive commands. The interactive digital
box
~5 connects to a television 165 or other di~plav monitor. Further, the
interactive
digital box ~5 can be connected to a digital television 195, in which case an
RF
modulator ?45 is not necessary. Each downstream transmission reaches the
?5 subscriber's house, shown in Figure '_, preterablv through a tap and drop
cable.
CA 02425739 2003-04-29
WO 98/4t020 PCT/U597l09045
I~
The user interacts with the program through the input device 20.
Preferably, the input device 20 is a tv~pical television remote. The user
interface
2?0 mat be an inrrared, wireless, or wired receiver that receives information
from
the input device 20.
Regardless of the type of input device 20, user inputs can be utilized by the
present invention immediately, or at a later time, to result in personalized
graphics, video and/or audio presentation. For example, the present invention
utilizes "trigger points," 5~0 as described below, to enable subsequent
branches
among multimedia segments during the show.
Certain commands are sent from the control studio 5 as part of the digital
interactive programming to facilitate the collection of user entries. These
commands are extracted at the digital demultiplexer ?10 and sent to the
controller
260 which performs the appropriate action based on the commands. Some of
these commands are explained below.
i5 a. BEGI'~l INPUT E~'TE\1DED
The Begin Inaut command starts an input period during which the user
may press one or more 5uttons ao select his or her choicets). The entry
format of this command is set forth as follows:
BEGIN INPUT {(N} }VALID KEYS} }DIFFERENT} (FEEDBACK}
where:
~I The maximum number of keys that can be pressed.
KEYS The valid keys that can be pressed.
DIFFERE:~1T Requires each key pressed to be different.
FEEDBACK The feedback type provided to the viewer.
b. BEGII~I VIDEO CHOICE E.YTENDED
T'ne Begin Video Choice begins an input period for disco mode. During
~0 disco mode, the vid~.~o snitches d~_~namicallv each time the viewer makes a
selection.
CA 02425739 2003-04-29
WO 98141020 PCTIUS97/09045
I6
BEGIiV_VIDEO_ CHOICE MODE (KEYS] (FEEDBACK (AUDIO]}
The disco mode allows the viewer to change channels at will, while the
OneShot mode allows only one change of channel.
r
MODE DISCO/ONESHOT
KEYS The valid keys that can be pressed.
FEEDBACK The feedback supplied to the user for the keys) which are
pressed.
The mode Disco allows the viewer to change channels at will, while the
OneShot mode allows only one change of channel.
c. BEGINAlIDIO CHOICE E.'~TENDED
The Begin Audio Choice begins an input period for disco mode.
BEGIN_AUDIO_ CHOICE MODE [KEYS] )FEEDBACK [AUDIO]}
MODE DISCO i ONESHOT
KEYS The valid keys that can be pressed.
FEEDBACK T'ne reedback supplied to the user for the kev(sj which are
pressed.
d. NI AP
T'.~e '~laD command is used to map video or audio tracks to keys, for use in
?5 connection with the Begin Audio Choice and Begin Video Choice
commands. If this command is omitted, ~;ev i cviil mao to Track or
Channel l, Kev ? to Track or Channel ? etc.. This command allows
mapping anv kev to any channel.
MAP KEYS (TRACKS (T1,...TN/Tl-TN) CHANNELS LC1,...CN/C1-CN)]
The map statement maps audio tracks or video channels to kevs, to enable
audio or video choice commands to effect changes to tracks other than the
default t.-acks, which are that kev 1 maps to track 1, kev '' to track ?,
etc.. All
the choice statements after a map statement will cause the tracks, video
tracks or channels to be changed to those specified in the map statement.
The acceptable numbers for the map command are from 1 to 8, for keys,
video channels and audio channels.
~0 Other commands include those which allow for the following applications:
( 1) viewer protiiins" to enable the set top box ?5 to "remember" viewer
CA 02425739 2003-04-29
WO 98/.11020 PCT/US97l09045
17
preferences; (?) uploading viewer responses to a central location; (3)
downloading
of text and graphics, for display using the graphics chip of the set top box
25; (4)
the ability of the viewer to prepare his own video, based upon his selections
of
camera shots 100 and audio, which can be stored and replayed for the viewer.
The interactive digital box 25 of the present invention enables seamless
flicker-free transparent switching between the digital video signals.
"Seamless"
means that the switch from one video signal to another is user imperceptible.
Because the video signals are running off the same clock, the interactive
digital
box 2~ is capable of providing a seamless digital switch from one video signal
to
I0 another signal. T'ne program clock reference necessary for the box to make
this
seamless switch is preferably embedded in the signal header.
.~s shown in i~igure 2, a CPL' 260 is connected to an RF demodulator 200
and digital demultiplexer 210. 'The CPU 260 directs demodulation and
demultiulexi~-tg of the proper channel arid data stream to obtain the correct
video
signal. reamiess switching can occur with ~.IPEG-. compressed signals since
there are points within the frame wherein seamless switching can occur.
Preferably, switches occur at an "I" frame, assuming the use of :vIPEG-2
compression. The selected video signal is determined either by examination of
the user's input from user interface 2i0 and,%or any other information or
criteria
(such as personal profile information) stored in R.~~I/ROIvI 265. For example,
the R.~~1 i ROW 26~ could store commands provided within the video signals as
discussed in more detail in U.S. I'at~nt I~o. ~,60?,?79.
T:~e RF demodulator 200 demodulates data from the broadcast channel
directed by the controller 260. :after the data stream is demodulated, it
passes
through a :onward error correction circuit 205 into a digital dernultiplexer
210.
CA 02425739 2003-04-29
WO 98141010 PC'TIUS97I09045
13
The demultiplexer 210 is controlled by the controller 260 to provide a
specific
video signal out of a number of video signals which may be located within the
data stream on the demodulated broadcast channel. The demultiplexed video
signal is then decompressed and decoded by decompressor/decoder 2I5. The
video signal is synchronized by a sync add circuit 220 and a sync generator
''.'S.
The video signal is then buffered by a video frame buffer 230. The buffered
video
signal is modulated by a modulator 245 into a NTSC compatible signal. Such a
modulator is not necessary if the selected Signal is sent to a digital
television 195.
Bv using a video frame buffer 230 and delaying the viewing of a given
signal, enough time is allowed for the decompressor% decoder 215 to lock onto,
decompress, cony ert to analog, and wait for the resultant vertical interv al
of a
second video signal. For example, assume video signal :~ is currently being
processed and transferred through the circuit shown in Figure ? and displayed.
Based upon a user selection, the controller 260 directs the digital
demultiplexer
I~ 210 and RF demodulator 200 to switch to another video signal, video signal
B. To
accomplish this, the analog video from the first digital video signal, video
signal
A, complete with video sync, is fed into video frame buffer 2~0. This buffer
230
can hold the Lull video picture for "n" number of frames after which the
signal is
output to the disnlav. In effect, a delayed video signal A is viewed "n"
number of
frames after the signal has been received. When the user selects a different
video
path by means of pressing a button on a keypad or entry by other means, the
controller 260 instructs the digital demultiplexer ?10 to stop decoding signal
A
and lock onto signal B to begin decoding signal B instead of signal A.
While this is happening, even though the decompressor!decoder 215 is no
'_'S longer decompressing video signal A, the display i5 still showing video
signal A
CA 02425739 2003-04-29
WO 98/41020 PCTIU597/09045
19
because it is being read from the buffer 230. As soon as decompressing and
decoding occurs, the controller 260 looks for the next vertical blanking
interval
(VBI) and instructs the video frame buffer 230 to switch to its input, rather
than
its buffered output at the occurrence of the VBI.
Since the RF demodulator 200, forward error corrector 205, digital
demultiplexer 210, and decompressor/decoder 215 require a certain time period
to
decompress and decode the video signal B frame from its data stream, the size
of
the buffer 230 has to be large enough so that this processing can take place
without interruption during the switching of the video signals. If desired,
the
5vstem may continue to use the buffer in amicipation of a future switch. Bv
using
the controller 250 to manipulate the till and empt,~ rate of the buffer 230,
the
buffer 230 may be ra~idlv filled with video signal $ frames and then after a
period
of time will be reset and ready to make another switch to another video in the
same manner. The buffer ?30 may also be reset by skipping frames or providing
a
1delay between sequential frame outtmts for a short time in order to till the
buffer
230. if a delay is used to maintain video si~Tnai or frame output while the
buffer
230 is being filled, a slight distortion may occur for a brier amount of time.
Because a first video signal is alwav s displayed as the output of the buffer
~30 after the delay, the buffered video masks the acquisition and decoding of
a
?0 second video signal. As long as the buffer 230 is large enough to keep the
first
video running while the second W deo is being decompressed and decoded, a
seamless switch will occur.
vVhile the digital interactive box Z3 of Fissure ? provides video
interactivin,-, audio andior graphics interactivity is also provided. For
example,
if, based on ;he viewer profile or W ewer response to c~uerv, it is determined
that
CA 02425739 2003-04-29
WO 98~at02o PCT/LTS97~09045
the vie~n~er's pnmarv language is Spanish, then that viewer could obtain
Spanish
commentary to the football, soccer, etc. game. Alternatively, if a viewer has
a
favorite athlete, the audio can switch to an interview with the athlete during
a
segment of the broadcast. Multiple digital audio options forming a set of
suitable
5 responses to an interrogatory message can be sent as part of the composite
digital
signal. :~s set forth in more detail in U.S. Patent No. 5.58,858,
there are a number of different ways to effectively forward the necessary
audio
options for a given live event to the digital interactive box 25. With the
present
invention, it makes no difference how the audio options reach the digital
10 interactive box 25, as long as they are available for selection and play at
the
appropriate times.
In Figure 2, the digital demultiplexer 210 extracts the digital audio signals)
and forwards them to the audio switch 250. Additional audio options are
available from the digital audio memory ~~5. A,t certain times during the
:5 program, the data codes will identify the selection or a particular audio
option
corresponding to previous user inputs. The controller X60 calls the
appropriate
audio options from internal memory 255 or directs the audio switch 250 to
select
a predetermined audio segment received as part of the received digital signal
for
passage to the RF modulator 245 for play to the subscriber. At the end of the
20 audio segment time period as indicated by the data codes, the controller
260
instructs the audio switch 250 to again pick up standard audio.
The digital demultiplexer ''IO sends the extracted graphics data or ACTV
data codes to the controller 260. The controller 260 interprets the extracted
data as
either control data, including instructions for switching between video
signals,
audio signals, or graphics data for on-screen display. It the data is on-
screen
CA 02425739 2003-04-29
WO 981410:0 PCTlUS97109045
?2
display data, the data is preferably prefixed by a command designating the
data as
on-screen display data, as opposed to control data. Further, the controller
260 also
examines the control data for the occurrence of a header code designating the
onset of a trigger point 500 in the program, explained below.
Figure 3 shows an alternate, dual tuner embodiment for seamless
switching between separate video signals. This embodiment presumes that two 6
MHz channels are used, each of which comprises compressed digital video and
audio streams. In this embodiment, the microprocessor 260 controls the
selection
of the RF channel that is demodulated by RE demodulators 200A, 2008. The
demodulated data streams enter the forward error correctors 205A, 2058. At the
output of the forward error correctors ''05A, ?C5B the data streams are
transmitted
to the input or the digital demultipiexers 20A, ''108.
As with the RF demodulators 200A, ''OOB, the digital demultiplexers 210A,
?108 are controlled by the microprocessor '60. Z nis configuration allows the
1~ microprocessor 260 to independently select tcvo different individual time-
multiplexed video synals on different chanr,eis and data streams. I: all the
video
signals of an interactive program were contained on a single channel or data
stream, it would only be necessary to have a single RF demodulator 200,
forward
error corrector ?05, and digital dernultiplexer ?10 serially connected and
feeding
into the two digital video buffers Z~OA, 2pOB.
Two data streams are provided from the digital demultiplexers ~10A, 2108.
One data stream carries video information pertaining to the video signal the
user
is currently viewing. The second data stream carries the video signal selected
based on the users previous and; or current interacnve selections from the
user
'_'S interface '_'; 0, as determined by the microprocessor ?60.
CA 02425739 2003-04-29
WO 98/41020 PCTNS97/09045
The digital information on each of the two streams is buffered in digital
video buffers 230A, 2308. The buffered signals are then decompressed and
converted into analog signals by decompressors/decoders 215A, 2158 which
include digital to analog converters. The decompressors 215A, 2158 are
preferably
IvIT'EG-Z decoders.
A local sync generator ~?5 is connected to sync add 220A, 2208 and frame
sync 380A, 3808 circuits. Because both streams are synchronized based on
signals
from the same local sync generator 25, each stream becomes synchronized to the
other. In particular, the signals on each stream are frame synchronized.
A vertical blanking interval ', VBI) switch 33~ is connected to the
microprocessor 260 so that the input may be switched during the vertical
blanking interval of the current stream, resulting in a seamless switch to the
newer.
The embodiment of Figure 3 operates as follows. Based on user responses
1~ and control codes, it is assumed that the microprocessor 260 determines
that a
switch from video signal A to video signal C should be performed. The RF
demodulator ?OO:A and digital demultiple:cer 210A are processing the currently
viewed video signal, video signal A, which is progressing through the upper
branch components. A command is issued from the microprocessor 260 to the
RF demodulator 200 A. 2008 commanding a switch to the channel and data
stream on which video signal C is located. The microprocessor 260 also
instructs
the digital demultiplexer ?108 to provide video signal C from the received
data
stream to digital video buffer ?308.
CA 02425739 2003-04-29
WO 981410:0 PC'TNS97109045
''3
At this point, the upper RF demodulator 200 A and digital demultiplexer
210A are still independently receiving and processing video signal A, which
continues through the upper branch of the circuit.
At a certain point, the digital decompressor!decoder 21SB in the lower
branch will begin filling up with video signal C frames. After video signal C
is
decompressed and decoded, it is converted into analog. A local sync generator
~?5
inserts both local sync and frame sync to video signal C via sync add circuit
SOB
and frame sync circuit 380B in order to svnr_hronize it with the currently
displayed video signal A, which is still being provided from the upper digital
video buffer 230A. At the appropriate switch point, triggered by programming
codes supplied with each video signal A and C, the microprocessor 260 directs
the
VBI switch 33~ to switch in the vertical blanking interval from video A to
video
C, at which time video C will then seamles~slv appear on the computer screen.
Digital video buffers 230A, 3.30B may be used in the circuit of Figure 3, but
1S are optional. However, in an alternative embodiment the buffers would be
required to arovide a seamless switch ithe I'i:;ure 3 circuit was modified to
incorporate a single RF demodulator 200, single forward error corrector 205,
and
single digital demuitiplexer 210, each witr, a single input and single output.
In
this alternative embodiment, the circuit cannot independently receive and
demultipiex two data streams on different frequency channels. One buffer 230A
is used to store previously received video signals, while the other buffer
230B
auicklv passes through the selected video si~,lnals.
Based on the same assumptions above, video signal A is progressing
through the upper branch of the circuit and a is desired to switch to video
signal
CA 02425739 2003-04-29
WO 981.11020 PCT/US97/09045
24
C. However, in this alternative embodiment,, the digital video buffer 230A is
providing maximum buffering to video signal A.
Because it is desired to switch to video signal C, the microprocessor 260
directs the alternative circuit (containing a single RF receiver 200, single
forward
error corrector 205 and single digital demultiplexer 210 connected in serial),
to
receive and demultiplex the data stream on which video signal C is located,
which may be different than that of video signal A. When video signal C is
demultiplexed, the microprocessor 260 directs the digital video buffer 230 to
provide minimum buffering or video signal C so that decompressor/decoder 215
may quickly decompress and decode the digital signals. After decompression and
decoding, video signal C is synchronized with video signal A. At this time,
video
signal A is read for display from di:;itai video buffer 230:x. The upper
digital
video buffer ?30A must be large enough to provide video frames for output
during the time it takes the RF demodulator 200 and digital demultiplexer 210
to
switch to video signal C and the time required Eor decompression, decoding,
and
synchronization or video signal C.
When video signal C is synchronized with video signal A, the
microprocessor ?60 directs VBl switch 335 to switch from video signal A to
video
signal C in the vertical blanking interval of video signal A, thereby
providing a
seamless and clicker-free switch.
At this time, digital video buffer ? 30 will begin to utilize maximum
buffering by altering its till/empty rate as described above with respect to
the
Figure 3 embodiment. When adequate buffering is achieved, a switch to another
video sitrnal may be performed in the same manner as described above.
CA 02425739 2003-04-29
WO 98/41020 PCTlUS97/09045
Another preferred embodiment is shown in Figure ~. This embodiment
also includes an RF demodulator 200, a forward error corrector 205, and a
digital
demultiplexer 210. However, the circuitry differs along the rest of the chain
to
the television set or monitor. In this embodiment, a memory 475 is
incorporated
and connected to the output of the demultiplexer for storing the compressed
composite digital video signal. The decompressor/decoder 215 is inserted at
the
output of the compressed memory. T'he decompressor/decoder 215 decompresses
the digital signal, converts the signal to analog and forwards the analog
signal to
the RF encode 2=15 for transmission to the monitor. Once the composite
compressed digitai video signal is fed into the compressed memory 475, the
microprocessor ''60 directs a pointer to be placed somewhere along the
compressed digital video signal. Based on the placement of the pointer,
different
frames and different segments of the composite digital video signal will be
read
from memory -175 for decompression and decoding.
The different video signals are distinguished from one another because
they are labeled, preierablv by headers. :~ssumin; that video signal :~ has
been
selected for play on the monitor, the compressed digital memory s75 fills up
with
A frames. Assuming a switch to video signal C is desired, the microprocessor
360
directs the P F demodulator 200 and digital demultiplexer 210 to begin filling
the
compressed memory -175 wuth video C frames. The decoder 215 pointer begins to
move down. As soon as a sufficient number of C frames have entered the
compressed memory 175, the pointer will then jump to the beginning of the C
frames. The C frames are then output into the c~ecompressori decoder 215 where
the digital frames are converted into an anaio~ signal.
CA 02425739 2003-04-29
WO 98/41020 PCT/US97/09045
Z6
The digital video is multiplexed in a series of easily identifiable packets.
These packets may contain full compressed frames of video (I frames) or may
include only the differences between full frames (B frames or P frames).
To be able to reconstruct the full video images, the decompressor/decoder
215 needs to have a minimum number of I, P and B frames. The decoder 215
needs only one I frame to decode an image. Conversely, two prior Anchor frames
("I's" and "P's") are necessary to decode B frames. In order to decode P
frames, the
decoder 225 only needs one Prior tlnchor frame. When the microprocessor
instructs the digital demultiplexer 210 to start sending packets from a
different
data stream there is no way to be certain that the next packet will be an I
packet
needed for decoding the second video stream. To avoid a breakup of the video
images, which would occur if the decompressor% decoder 215 suddenly started
receiving packets unrelated to the stream it vas decoding, the microprocessor
260
starts to fill up the memory 4i5 with video signal C packets until it is
determined
1~ that a full sequence of I, B and P frames are available. The decoder ~? ~
should
receive the last bit of the last B frame in a given, VOP (Group of Pictures)
before
the switch, in order to prevent glitches when decoding. Furthermore, the last
B
frame of the GOP must only be backward predicted, not forward predicted or
bidirectional predicted. As soon as the valid sequence is in memory ~:5 the
microprocessor X60 moves the memory read pointer to the start of a valid
sequence of C video signal packets so that the decompressor/decoder '?I~ can
successfully decode the C signals. This results in a seamless switch from
video
sig.-tal A to video signal C.
This embodiment reqmres a data channel for enabling a synchronous
~5 switch bettveen a first video stream and a second video stream. This data
channel
CA 02425739 2003-04-29
WO 98141020 PCT/US97/09045
comprises the AC'T~' codes which link together the different program elements
and information segments on the different vudeo signals. In addition, the data
channel also comprises synchronization pulses and a time code to signify to
the
pointer the proper time to skip from a memory location representing one video
signal to a memory location representing another video signal in order to
enable
a seamless switch.
The microprocessor 260 reads the data signal from the digital
demultiplexer 210 and communicates pertinent data to the sync add circuit 220,
which is connected to sync generator 2'_'S. The microprocessor 260 is then
able to
1Q synchronously communicate with the memory 4; 5.
The time code sent will identify the timing for one picture, as well as for
multiple pictures, and viii lock the different uictures together. T'nis is
done
through the use of similar clocks at both the transmission end and the
receiver.
:~ time code is used in order to keen the two clocks at both the transmission
and
receive end synchronously connected to one another. Once the clocks at both
ends are workin7 synchronously, each of t':~e multiplexed video streams must
be
synchronized to the clocks. In order to synchronize the multiplexed video
stream
to the clocks, each of the individual channels must be referenced to a common
reference point and must be identiried.
?0 In the preferred embodiment, a packet header would be incorporated into
the transvort layer of the MPEG signal to identify the various channels. The
packet header will also include information as to where to insert the vertical
blanking interval. In MPEG, the vertical blanking interval is not transmitted
from the headend. Z'heretore, tht vertical blanking interval must be generated
locally. The packet header eve w:ll identity at what time the vertical
blanking
CA 02425739 2003-04-29
WO 98/41020 PC'T/US97/090A5
78
interval is in existence in order to effectuate a seamless switch between
analog
pictures.
In summary, the combination of clock and the information embedded in
either the transport layer of MPEG or in a separate packet on a separate data
channel effectuates the linking between each video signal and a corresponding
time point. The data channel also includes inforrnation designating when all
the
various video signals will be in synchronism with one another. It is at these
points that the microprocessor 260 may direct the pointer to skip from one
location to another location, at a time (such as during the VBI) when a
seamless
switch will result.
D. Truer Points
Interactivity is further enhanced in the digital interactive embodiments
through the application of trigger points 500 scattered at various
predetermined
1~ times throughout the program, a ;imeline representation of which is shown
in
Figure ~. The trigger points X00 correspond to times when interactive events
are
scheduled to take place during the live sporting event 10. These interactive
events could be the selection and playing of video, audio segments, the
display of
graphics or display of VVeb pages accessed from Internet Web sites 170. For
example, when a viewer's favorite baseball player is at bat, graphics showing
past
or current performance statistics of the player can be overlaid on the screen
while
excerpts from an interview with the player can be played for the viewer. While
the choice of particular video, audio or graphics is still dependent on viewer
selections, the viewer selections in response to displayed graphical
interrogatory
messages are preferably made during a period at the onset of the program or
CA 02425739 2003-04-29
WO 98141020 PCTNS97109045
~q
when a viewer first tunes into the program. :alternatively, interrogatories
are
not necessary if the switches are based on the viewer profile stored in memory
265. These viewer selections are then utilized as inputs to macros called up
at
later times during the program by the controller 260 upon the occurrence of
the
trigger points 500, identified to the interactiv<~ computer by unique codes
embedded in the v ideo signal.
T"ne trigger points 500 correspond to the times when the conventional
program content can be altered and personalized for the viewers. The
programmer can place the trigger points 500 at any time throughout the
program.
Since the trigger points 500 are unknown to the subscriber, the subscriber
does not
kno~cv when they will receive a personalized message. In other words, an
interactive response can either immediately follow a corresponding user
selection :node to an interrogatory message or occur at a later time
corresponding
to a trigger point 500, or any combination of the two. Of course, timing of
the
interactive events should correspond to suitable times in the program where
branchin:; to interactive elements is sensible and does not clash with the
program
content or the conventional video still displayed on the television 165 or
other
display monitor.
At the onset of a trigger point 500, the controller ?60 will select one of
?0 several possible audio (or video or graphic display) responses for
presentation to
the subscriber. :~s mentioned above and shown in figure 5, some of the
responses may comprise a branch to either a video segment, graphics and / or
audio segments.
In combination with the use of tzigger points 500, the present invention
~'S allows for t:~Ze viewer to select certain options at the onset of the
program to suit
CA 02425739 2003-04-29
WO 98/41020 PCT/US97/09045
JO
the viewers' preferences. For example, if the program broadcast is a live
sports
event 10, at an early trigger point 500, the viewer could be queried as to
whether
the viewer would prefer to receive audio in English, Spanish, French, or
perhaps
hear the local announcer instead of the network announcer. Upon the viewer
S selection, the CPL 260 directs a branch to the appropriate interactive
segment.
Each trigger point 500 is identified preferably through the broadcast of
ACTV codes sent as part of the composite interactive program signal. The codes
preferably include, at a minimum, the following information: (1) header
identifying the occurrence of a trigger point .~00; ('_') function ID (e.g.,
selection of
audio or graphics responses, etc. j; and (3) corresponding interrogatory
messages)
or particular viewer characteristic or habit based on viewer profile. The
first bit
sequence simply identifies to the controller that a trigger point X00 is about
to
occur. The function ID designates the macro or other set of executable
instructions for the controller ~'60 to read arcd interpret to obtain the
desired
1~ result, e.g., a selected video andi'or audio response.
Upon extraction of the codes by the data decoder, the controller ~60 reads
and interprets the codes and calls trom memory ''6~ a particular user
selections)
designated by the trigger point X00 codes. The user selections correspond to
subscriber answers to a series of interrogatory messages preferably presented
at the
beginning of the program. After obtaining the appropriate user selection(s),
the
controller 260 reads and performs the executable instructions using the user
selections) as inputts) in the macro algorithm. The result of the algorithm is
either a selected video stream, audio and/or selected graphics response. The
video/audio response can be called from memory 265 if it is prestored, called
'_'S from external data storage, or the controller ?60 can command the switch
to
CA 02425739 2003-04-29
WO 98/41020 PCTIUS9?109045
J1
branch to the particular video audio stream if the response is broadcast
concurrently with the trigger point 5G0. After the selected video/audio
response
is played to the subscriber, the switch branches back to the standard program,
shown at time is in Figure 5.
As mentioned above, a series of interrogatory messages are preferably
presented when the subscriber begins watching the interactive program. These
interrogatory messages can be presented in any one of three ways. First, the
interrogatory messages can be presented as graphics displays overlaid by the
interactive computer workstation onto a video signal, wherein the graphics
data
is sent in the vertical blanking interval of the composite interactive signal,
or
alternatively stored on the hard disk or external storage. Second, the
interrogatory messages are presented as graphics displays as discussed above,
except the graphics data comes from local storage, external data storage
(e.g., CD
ROMt, cartridge, etc.), or a combination of data in the VBI and data called
from
either local or external data storage. Third, graphics data can be presented
in the
form of user templates stored at the interactive comDUter workstation.
User selections corresponding to answers to the n successive interrogatory
messages are received by the remote interface 270 at the beginning of the
show,
stored in memory 265 and used throughout the show at the appropriate trigger
points S00 to subtlety change program content as the show progresses.
Preferably,
each interrogator', has a set of possible answers. ivlext to each possible
answer will
be some identifier corresponding to a label on a key on the user interface.
The
subscriber cevresses the key corresponding to their answer selection. This
selection is decoded by the remote interface ?~0 and controller ?60, stored in
CA 02425739 2003-04-29
WO 9814 t OZO PCTNS97/09045
32
memory 265, preferably RAM, and used Iater as required by an algorithm
designated at a trigger point 500.
E. Internet
In addition to the central studio serving as a source of interactive option
responses, the Internet can be used as a source of personalized information
for
interactive responses.
.as discussed above, the video programming is preferably created at a
centralized location, i.e., the control studio ~ as shown in Figure I, for
distribution to subscribers in their homes. Referring to Figures 1 and 6, in a
preferred method, the operator at the control studio ~ must designate certain
Web
pages to correspond to one or the program options, such as audio and /or
graphics
options, using control PC I3~ and URL encoder 600. Instead or encoding the
actual content of the options at the control studio ~, as with the audio, Web
address identifiers, i.e., Uniform Resource Locators (CRLsI are encoded and
sent
as pare of the data codes from the control PC 1~S to the digital muitiplexer 1-
10.
After multiplexing, the URLs are sent as part of the program signal 615, as
described above. Preferably, the URLs, like the various audio and graphics
options, have associated time stamps which indicate to the remote digital set
top
~0 boxes 25 when, during the video program, to display the particular Web
pages
addressed by the URLs, the selection and display or which is preferably made
as a
function of viewer responses or viewer profile.
Preferably, each digital set top box ~5 has an Internet connection lb0 created
concurrently with the cable connection. The Internet connection 160 can be via
~5 nigh-speed linN, f;F, conventional modem. The digital set top box ?S has
Internet
CA 02425739 2003-04-29
WO 98/4100 PC'1'/US97/09045
., .,
J .~
access 160 via any of the current ASCII sofr'~~are mechanisms. In a preferred
embodiment, in the interactive digital set top box '_'S, the digital
demultiplexer 210
extracts the URLs along with the other data codes. In an alternative
embodiment,
a local URL decoder 605 at the user site extracts the URLs.
In a preferred embodiment, a JAVA enabled browser as well as specialized
software for performing part of the method of the present invention are
installed
on the interactive digital set top box ?5. The JAV A enabled browser allows
the
interactive digital set top box ~ to retrieve the Web pages and is preferred
software, since it is platform independent, and thus, enables efficient and
flexible
transfer of arograms, images, etc., over the Internet. The specialized
software acts
as an interface between the video p:ogrammmg and the Internet functions of the
present invention. The processor and software interprets these URLs and
directs
the jAV.-~ enabled browser to retrieve the particular relevant Web pages, and
synchronizes the retrieved Web pages to the video content :or display on the
telex ision monitor 165 at the appropriate times.
In the present invention, she viewer also h,as the capability to iink to a
channel website at will. For example, if a newer is interested in purchasing a
product described in an advertisement, by merely clicking on a button on their
remote ?0, the producer's GVebsite could be accessed by Internet connection
160
and displayed to the viewer. The viewer could then either obtain more
information about the product or order the product, if desired. As described
above, this application is possible by sending the URL associated with the
producer's Website to the digital cable boxes ~~ as part of the interactive
program.
Upon selection by the viewer, the web bro~NSer, located either in t'tae
digital set-top
CA 02425739 2003-04-29
WO 98/41020 PCT/US97/09045
J~
box 25 or externally in a connected PC 610, can retrieve the Web pages. The
specialized sofrivare then synchronizes the Web pages for video display.
F. Memorw_
The interactive digital set top box 25 of the present invention also has the
advantage of remembering subscriber responses and using these responses in
choosing a video / audio response, and / or graphics interrogatory message, to
present to the student. Memory branching is a technique of the present
invention where the algorithm assembles vi.deo%audio responses and graphics
IO interrogatory messages according to the current and previous user inputs.
Memory branching is accomplished by linking videoiaudio streams and/or
successive graphics interrogatory messages together in a logical relationship.
In
this scheme, the interactive digital set top box ?5 contains logic
(preferably, in the
software algorithm) and memory '?h5 to store previous subscriber selections
and
to process these previous responses in the algorithm to control future
videoi audio stream selection, as well as fui:ure graphics message selection.
G. Digital Viewer Pr files
In a preferred embodiment, the interactive digital cable box 25 can have a
?0 "vie'n~er profile" stored in its memory 265. :alternatively, the
accumulated profile
of viewer characteristics and; or habits can be stored at the control studio 5
or
cable headend. If the profile statistics are accumulated at some central
location,
they can also be sent to each viewer's home. On the other hand, if accumulated
in memorv,~ ''65 at each of the set top terminals ?5, the data could be sent
to the
~_5 central location for storage and dissemination by way of the digital back
channel
CA 02425739 2003-04-29
WO 98/41020 PCT/US97/09045
J5
Decisions regarding personalized advertising and viewing can then be made for
a
viewer or class of viewers based on the accumulated parameters.
The "viewer profile" preferably contains characteristics of the particular
viewer at that subscriber location, such as sex, hobbies, interests, etc. This
viewer
profile is created by having the viewer respond to a series of questions.
Alternatively, the viewer profiles could be created at a control studio 5 and
sent to
the interactive digital cable box ?5. This information is then used by the
cable box
software to create a compendium of the viewer's interests and preferences --
i.e., a
user profile. T:ne stored user profile would be used in place of the
IO question answer format, and thus, dictate t:he branches to interactive
segments of
interest to the viewer.
Alternatively, the interactive cable bc>x ?5 can be programmed to create a
user profile of each viewer based on the sewections made during one of the
interactive t,rograms. If the profile shows that a particular viewer does not
enjoy
violence, the system can automaticaiiv branch to another video signal at the
commencement or a fight in a sports program. Furthermore, such a user profile
could be modified or enriched over time based on selections made during future
interactive programs. For example, the 'memory' technique described above can
be used to modify the user profile based on user response over time.
CA 02425739 2003-04-29
WO 98141020 PCT/US97/09045
36
Event data is collected from the viewer when the following command is
received and processed by the controller ?60 in the interactive digital cable
box 25:
VIEWER PROFILE
This command is used to enable events which can be utilized for profiling
during a show. The events indicated in this command are those profiling
events which are possible during the show. The enabled events may be
either selected by the viewer during the show, or may be automatically
enabled based upon viewer selections.
VIEWER-PROFILE EVENTI,...EVENT24
Once the profile is created, the programming choices or interactive
responses can be triggered based on the content of the viewer profile itself.
For
1~ example, if the viewer profile suggests that the viewer is particularly
interested in
sports cars, a sports car commercial could be planed for the viewer at a
predetermined point in the program. .=~s another application, if a viewer's
profile
indicates that the viewer is interested in cooking, whenever the viewer
watches
such a pro~:ram, the user profile would trigger the interactive program to
download recipes and either daplav such recipes on the screen or send the
recipes
to an attached printer.
Viewer profile information can then be collected at the control studio ~
through polling of the viewer cable boxes 25 for viewer selection data.
Special
polling software is loaded into the digital set top box 2~ for performing the
polling
?5 runctions. _~lternativelv, the invention allows for the digital set top
boxes 25 to
send back data on command from the control studio .5 or periodically. The
command to initiate an upload of viewer profile data is as follows:
CA 02425739 2003-04-29
WO 98/41020 PCT/US97/09045
J/
LIPLUAD EXTENDED
This command is used to initiate the 'uploading of data to the central site.
UPLOAD-EXTENDED UPLOAD-IDENTIFIER VARIABLE-NAME
[PHONE_NUMBER]
UPLOAD_IDENTIFIER numeric constant identifying
uplaad.
VARIABLE NAME name of variable to be uploaded
PHONE!~iUMBER string, that consists of numbers
only. it can be omitted, if not
needed.
Regardless of whether a polling or periodic scheme is used, the statistics and
other user profile information is preferably sent back to the control studio 5
by
use of the back-channel.
H. .-~pniications
?0 The embodiments, described above, allow for several possible applications.
For example, in a live sports event 10, one channel could carry the standard
video
channel, with other channels carrying different camera angles 100 and/'or
ciose-
ups of particular ciavers. Other potential video options include instant
replay,
highlights, p'taver statistics via graphic overlays, etc. Graphics presenting
~5 statistical information on the players can be constantly updated using the
Chyron
system.
The provision or trigger points 500, exp'tained above, can provide for
seamless integration of such video option.. during the sporting event, based
on
CA 02425739 2003-04-29
WO 98141020 PCT/US97/09045
38
either the viewer responses to interrogatorie at the beginning or the program
and/or on the digital viewer profile.
Further, the viewer can become the director with the present invention.
For example, the viewer can choose which camera angles 100 to emphasize. In a
broadcast of golf, the viewer can direct whether they desire to follow a
particular
player from hole-to-hole, focus on one particularly difficult hole. In this
manner, the viewer can customize the sporting broadcast to meet his own
interests.
Further, the viewer can act as a director to create their own video. During
a live concert 10, for example, the viewer selects various camera angles 100
at
different times. These selections along with a time stamp, indicating the
program
time when each selection was made, are stored in memory 26~. ~'Vhen the
program is piavea baci' a second time, the processor ?60 will automatically
direct
branching between the video channels according to the stored selections at the
l~ time stoma. In this manner, a music video can be created by the viewer.
The W ewmg experience can be further enlightening for the viewer by
implementing games and contests during the live sporting event 10. For
example, graphics overlays can be developed that query the viewer during the
game. During a football broadcast, for example, viewers can be queried with
such
''0 interrogatories as the following:
What ~~i11 be the next plan? (RUNaPASS/KICKl;
Will the offense get the first down?;
Will then score on this possession?;
Pick the halftime score;
Who will win?
CA 02425739 2003-04-29
WO 98/41020 PCTlUS97/09045
39
Each viewer's responses can be sent back to the control studio 5 for
tabulation of scores. Preferably, the responses are packaged at the digital
cable box
2~ and transmitted to the control studio 5 via the digital backchannel upon
the
UPLOAD EXTENDED command. .alternatively, tabulation of scores can take
place at the digital cable box 25 through the utilization of certain software
in
memory 265. Each correct answer can correspond to a certain number of points.
.4t the end of the game, the interactive program preferably presents a graphic
showing the viewer point total. If desired, advertisers could present special
gift
certificates for excellent performance in such games. The provision of such
certificates would occur by displaying a certain code that a viewer can take
to a
store to receive the gilt. In this manner, viewer interests in sports events
can be
enhanced.
Further, the viewer has the option with the present invention to block out
viewing ut certain events. For example, if the viewer is adverse to violence
1J during a sporting event or other type program, the system can block out
such
options from the viewer mth the following data commands:
E VENT
This command is used to indicate occurrence of a certain event (e_g., a tight
breaking out during a football game).
?0 EVENT EVENT_NUMBER
EVENT :~tUNiBER is a numeric constant.
ON EVE~~IT EYECIITE I~L~CRO
This command is used to implement certain actions (;such as an automatic
branch to another video channel in order to block out violent event, for
?5 exam~le~, as soon as the event arrives.
CA 02425739 2003-04-29
WO 98!41020 PCT/US9'7/09045
~0
ON~EVENT EVENT_NUMBER MACRO__NU1~IBER
EVE:~TT_~1L'IVIBER is a numeric constant.
MACRO-NL~1VIBER is a numeric constant.
I. Two-Way Configuration
The live programming system of the present invention may be operated in
a two way configuration, as illustrated in Figure 7. In this mode, the various
video signals are processed as previously described, being digitized and
compressed at the control studio ~. The signals are then sent to a central
switching station, or headend 30.
In this embodiment, the switching between the various live digital signals
is accomplished wt the headend 30 rather than at the receiver. On the receive
end,
each digital set-top box i 60 relays viewer selections back to the remotely
located
switching >tation 30. Preferably, the viewer selections are relayed by way of
the
digital back channel i 0. However, the viewer selections may be relayed to the
switching station 30 by any conventional means, such as two-way cable
television, telephone or microwave transmission. The switching station 30
receives the viewer selection and routes the desired signal to a transmitter
7~0
which conventionally transmits the desired video down the appropriate digital
cable channel for the particular viewer.
At the cenrrai switching station 30, a demultiplexer ;10 demultiplexes the
compressed signals and places each on a separate bus channel ~'_'S. A number
of
remote control interactive switches 730, ; :~2, % 3~, I36 are connected to the
video
signal bus i''_S. Based on the viewer selections, an algorithm stored in
memory
~5 ~'b5 and under processor '_'b0 control at the central switching station 30,
a digital
CA 02425739 2003-04-29
WO 98/41020 PCTJUS97/09045
-~ 1
seamless switch is made and the selected video, audio and/or graphics are
forwarded to the viewer home for display.
Such a two-way embodiment could be implemented in a video dial tone or
video sen~er system. In such a system, only a single video channel 755 is
necessary for each home. Once the viewer selection is received at the server
site
at the cable headend 30, a switch is made to the appropriate video stream and
this
stream is sent on the single channel 755 to the harne.
Alternatively, it may be desirable to transmit an interactive sporting event
over a single telephone line. When the viewer enters a selection on their
remote
~0, a signal is sent by way of the telephone line to the central switching
station 30
which routes the desired sijnai of the interactive program over the user's
telephone Line so that a single 'ink handles both the interactive choice being
made at the receiver and the transmission of that choice from the headend 30
where the act~:al switching takes place in response to the interactive
selection
made at the receiver.
The two-way link between the vieu~er and syvitching station 30 may be
used for other purposes. For example, demographic data may be transferred from
the viewer to the broadcast network for commercial purposes, such as targeted
advertising, billing, or other commercial or non-commercial purposes.
V~ihile the present invention has been described primarily with respect to
live events, and in particular sporting events, it has equal potential for
enhancinb
content in other program cateories. .~, viewer can become their own director
of a
murder mystery ur other drama. By entering responses to displayed questions at
the initiation or car during the show, the program will branch to alternative
''S ~~ideo i audio segments as a result of the user selections. In this
manner, different
CA 02425739 2003-04-29
WO 98!41020 PCT/U597/890A5
viewers with different selections may end up with a different murderer at the
conclusion of the broadcast.
Using the foregoing embodiments, methods and processes, the interactive
multimedia computer maximizes personalized attention and interactivity to
subscribers in their homes in real time. Although the present invention has
been described in detail with respect to certain embodiments and examples,
variations and modifications exist which are within the scope of the present
invention as defined in the following claims.
