Note: Descriptions are shown in the official language in which they were submitted.
214~089
WO94/11858 PCT/US93/10930
. ,; . .:
8Y~TEM AND APPARATU~ FOR
INTERA~TIVE M~LTI~EDIA ENTERTAINMENT
FIELD OF THE INVENTION
The present invention relates to an interactive
multimedia system and more particularly to a system for
the interactive multimedia entertainment.
BACRGRr)~ND OF T~E INVENq~ION
The entertainment industry has become more diverse ~-
over time. With the advent of music video entertainment,
the ability to dub in different information and provide a
facility for the viewer or the listener to actively
engage with the entertainment is now more available. ;~
There have been new technologies that have been developed
as a result of the music entertainment. For example, -~
Karaoke is very popular in Asia. In Karaoke individuals ;
listen to music and read the words on a screen, watch the
lyrics and sing along. It is of equal interest that many
people are doing these same things within the confines of
their home~
Karaoke entertainment involves participants which
sing to a popular song in which the main vocal trac~ has
been removed. Many forms of Karaoke can exist including
audio tape only Karaoke in which participant uses a
script or memory for the lyric words, CD-ROM Karaoke
which uses still or non-unique video off the CD-ROM to
provide entertainment for the non-singing participants
and color changing lyric words to aid the singing
participant, and laser disk Karaoke containing unique
- video along with color changing lyrics. Typically such
~ W094/11858 21~808~ PcTlus93/l0930
systems can change pitch but only by changing the speed ~
of the song's play back, since the audio is inherently ~-
analog or streaming digital based information which is
not buffered in any way. Revenue is generated eit~er
through admission fees collected from Karaoke bar or
booth patrons or from purchases of disk or tape products
at record or Karaoke shops.
Many draw backs exist which make multimedia music
entertainment more difficult to enjoy or more costly to
obtain. In music listening entertainment, for example,
it is impossible to judge a song until it is aired and
heard, with many songs on a given record never given air
time. In music ~ideo entertainment air time is evPn more
restricted si~ce there are only a small number of cable
channels devoted to music video entertainment. Both of
these areas would gain immensely from an interactive
system of access; they would gain even more entertainment
value if creative participant interactivity could be
added. The present invention addresses both of these
points.
Several specific draw backs exist with existing
techniques of Karaoke entertainment. First, often the
medium of tape, CD-ROM disks, or laser disks require a
purchaser to buy one tape or disk that contains a dozen
songs although only one or two are of interest. Second,
though some songs are used by purchasers many times, in
the case of Karaoke bar, booth, or personal entertainment
W094/1185X ' 2 1 ~ 8 0 8 9 ~CT/US93/10930
purposes it is common to use a given song only once or
twice over a year, thus increasing the effective cost of
the entertainment. Third, though there are thousands of
Karaoke song titles, normal record stores or even~Karaoke
stores only carry a small proportion of the total
produced titles. A last disadvan~age of existing Karaoke ~-
systems is that they have only been used for one track of
the full piece, the vocals; the present invention
addresses all of the above-mentioned points.
S~MMARY OF T~E INVENTION
The present invention is directed to a method and
apparatus for interactive multimedia entertainment.
In a first facet, an interactive music listening ~
entertainment system comprises means for storing a -;
plurality o~ songs; means for selecting one of the
plurality of songs from the storing means; and means
responsive to the selecting means for receiving the one
song from the storing means by a recipient; the receiving
means also receiving information concerning the cost of
the one song from the storing means and including means
for limiting the number of plays for the one song.
In a second facet, an interactive multimedia video
entertainment system comprises means for storing a
plurality of songs and video information related the
plurality of songs; means for selecting a song and its
associated video information from the storing means; and
means re~ponsive to the selecting means for receiving the
'! '`'.. WO 94t1185~ 2 1 ~ ~ 0 8 ~ PCT/US93/10930
one song and its associated video information, the
receiving means also for receiving information concerning
the cost of the one song and its associated video
information from the storing means and for counti~g the
number of times the one song is played and charging the
recipient a particular price per play.
In yet a third facet, an interactive multimedia
karaoke system comprises means for storing multimedia
information to allow for the production of a video and
audio presentation; means for selecting a particular set
of multimedia information from the storing means; and
means responsive to the selecting means for receiving the
particular set of multimedia information and for
modifying the particular set of multimedia information so
as ~nhance the quality and interactivity of the
particular set of multimedia information.
BRIEF DESCRIPTION OF ~E DRAWINGS
Figure l is an interactive multimedia system
utilized for music listening entertainment.
Figure 2A is an interactive multimedia system for
video entertainment.
Figure 2B is an interactive multimedia system for
Karaoke entertainment.
Figure ~C shows an interactive multimedia Karaoke
presentation.
Figure 3 is a flow chart of a first embodiment of ~`
the optimization method which is utilized in a network in -~`
4 ` ~
~ WO94/11X58 214 ~ 0 8 ~ PCT/US93/10930
accordance with the present invention.
Figure 4 is a flow chart showing the operation of a -~
second embodiment of an optimization method which is
utilized in a network in accordance with the pres~nt
invention in which a matrix of compression and
transmisSion algorithms are utilized.
Figure 5 is a flow chart showing the separation of ~-
multimedia information in accordance with the present
invention.
Figure 6 is a chart showing the cooperation of a
compression algorithm with a transmission algorithm in
accordance with the present invention.
Figure 7 is a block representation of digital
information of an image file and a MIDI file.
Figure 8 is a block representation of the multimedia
call processing system (MCPS) in accordance with the
present invention.
Figure 9 is a block representation of an interactive
multimedia device in accordance with the present
invention. ;
DETAILED DE8CRIPT~ON OF THE INVENTION
The present invention relates to an improvement in
interactive multimedia entertainment systems. The ~`
following description is presented to enable one of
ordinary skill in the art to ma~e and use the invention
and is provided in the context of a patent application
and its requirements. Various modifications to the -~
~, WO~4/118~8 2 1 4 8 0 8 9 PC~/US93/10930
preferred embodiment will be readily apparent to those
skilled in the art and the generic principles and
features described herein.
Referring now to Figure 1, what is shown is an
interactive music listening entertainment system. The
music listening entertainment system lO comprises an
interactive multimedia mastering system (IMM) 42 which
receives the program source material as in the songs or
records of CD quality provided through a multimedia call
processing system (MCPS) 44 which in turn is activated by
a telephone and is also connected to an interactive
multimedia device (IMD) 52. The (IMD) 52 in this
situation would be capable of providing for storage of
certain songs which could be played on a stereo or the ;~
like. This type of system the IMD 52 could provide a CD
quality song which can be transmitted over the network.
The MCPS 44 can take credit or order information and can
ship an IMD S2 song over the network or can indicate that
a CD should be shipped in the mail. In the present
invention the IMD 52 will limit the number of times the
song can ~e played by encoding the song in a particular
way when it is sent and also can provide for a charge for ~
each time a particular song is played. In addition, the ~`
system would have the capability to make a personalized
album. For example, if the recipient of a particular
song wanted to record it directly from the IMM, he/she
could send a signal down to the IMM 42 via the IMD 52.
WO94/11858 2 1 ~ ~ 0 8 9 PCT/US93/10930
,.. ^, ~:
The IMM 42 would then provide a high quality audio song
to be recorded.
Figure 2A shows an embodiment of a music video
entertainment system 20 much like the previously
mentioned music listening entertainment system an IMM 42
provides enhanced music to a MCPS 4~. In addition, ~
program source material for video is provided to the IMD ~-
52 from another source (not shown). The IMD 52 would be -
utilized to send information to a TV 32 and/or stereo 34
related to audio, MIDI, instruments and the like. In
this embodiment, multimedia information can be sent down
a network such as stills, motion, lyrics, score and
synchroniz~d with the playback. Additionally one or all
of the music video tracks can be substituted by an actual
audio or MIDI instrument input. This would allow a
person to play as a member of the band for entertainment
or learning depending upon the graphics display which can
include a section with the current music rotation score
with color changes to highlight the current position of -~
the guidance of the accompanying musician. In addition,
the tempo and pitch can be adjusted to the control of
data generated from the IMD 52. Multimedia information
could also be Xeyed by type or generated to provide a
video overlay. ;
Figure 2B is a simple block diagram of a the Karaoke
entertainment system 30. Once again this system includes
an IMM 42 which with the IMD 52, a CD quality song can be
~ WO94/11858 214 8 0 8 9 PCT/US93/10930
transmitted to the TV 32 and the stereo 34. Psycho-
graphic information can be displayed such as equalization
and dynamic levels. Multimedia information can be sent
down a network to the TV 32 such as with stills, ~otion,
lyrics, scores, etc. and synchronized with the playback
aspects of the s~stem.
In this system 30 each of a plurality of instruments
along with voi~e (sing-along) can be enhanced via the
present invention. To more fully explain this feature
refer now to Figure 2C which depicts a multimedia Karaoke
presentation 400 or in accordance with the present
invention. As is seen in the Figure, one person 402 is
holding a microphone 404, has a monitor 406 in front of
her, and a video camera 408 is in front of her on top of
the monitor 406. A second person 410 has a similar
arrangement 410 except he is playing a MIDI drum 412.
Yet a third person 414 has a similar video camera/monitor
arrangement except he is playing a MIDI guitar 416. All `
of the individuals 404, 410 and 416 can view the Karaoke ~
entertainment screen via their respective monitors. At ;
the same time the Karaoke presentation can be provided to
an audience 450 on a screen 452.
In an arrangement such as this a multimedia
presentation of a popular song is introduced, each of the
players can have their instruments or voice (as the case
may be~ substituted for the same tracks in the original
song/video. In such an arrangement all of the original
~ WO94/11858 214 ~ 0 8 9 PCT/US93/10930
~ ,............................................................................ ..
tracks (voice, lyrics, drum, guitar and score) can be
substituted with the tracks of the entertainment
instruments (instruments or voices) of the individuals
404, 410 and 416. Guidance to performers for tem~o is
provided using the monitors to display portions of the
corresponding score for the instrument being played or
vocals being sung (drum track, guitar score, voice
lyrics) with the position in time being indicated by the
track changing color.
-~ In a preferred embodiment each of the tracks can be `;`
further enhanced by incorporating interactive video
sources from the different performers using a key video
color keying technique for each performers video source.
; ~ The scene viewed by the audience may be in one
. .
embodiment, the heads of the individual performers on the
bodies of the people in the actual video having used the
performers video sources as identified by the color key ;`
mastered in the video as a substitute for the original
video material. In so doing many of She enhancements and ,
the like which will hereinafter be described can be
; ~ performed to make the performance as realistic as ~
possible. ````
~- ~ In all of the above described embodiments an IMM 42
is critical to providing some interactive activity of the ~`
program. To more fully describe this optimization
technique refer now to Figure 3 which is a block diagram
of an optimization method in accordance with the present
WO94/11858 2 1 ~ 8 0 ~ 9 PCT/US93/10930
invention. The systems lO and 20 as above-mentioned
includes an IMM 42 that provides much of the optimization
of the program source material for presentation to MCPS
44. ^~
To more specifically describe this optimization
technique, refer now to Figure 4 which is a block diagram
of an optimization method in accordance with the present
invention. This optimization technique has been -`
described in detail in U.S. Patent Application Number
, entitled, _'Method for the Production and
Transmission of Enhanced Multimedia Information '~, having
a filing date of
assigned to the assignee in the ~-
. .
present invention, and that patent application is ;~
incorporated by reference in this application. The `
following paragraphs along with the ac~ompanying figures
will provide the details regarding the optimization
method and how it will be used advantageously to provide
an enhanced interactive multimedia system.
The purpose of the IMM 42 is to provide maximum
interactivity while at the same time providing maximum
retention of the program model. It is also important
that there be minimum transit time for the interactivity
while the information has maximum replication.
Therefore, it is very important that the program model
psychographic parameters be well described. For example,
t~e spatial, color, temporal, audio response, material
.
~{ WO94/1l858 2 1 4 8 0 8 9 PCT/US93/10930
concept, contention perception all should be very well
describ~d and defined in the program model.
Referring again to Figure 4 what is shown is a first
embodiment of a flow chart for providing an enhanced
interactive image that utilizes the principles of the ;~
present invention. The flow chart 100 comprises the
steps of providing a program model to a separator. The
separator 102 will divide the information into primary ~-
and secondary layers of interactive multimedia
information. The separation is automatic and can be
accomplished in a variety of ways. For example, thle
layers can be separated by production sources. In `
another example, separation can be accomplished through
key coding the layers. In yet a third example, the ;~
layers can be spatially separated or separated by the -
various colors. Finally, layers of information could be -`
separated by a filtering process. ~-~
The primary layers are provided to the compression
generation block 104~ There are a variety of ways that
the multimedia data can be changed or generated to use ~
less bandwidth. For example, compression algorithms or ~`
their equivalents could be utilized to reduce the
bandwidth used. In addition generators, such as in a
tone generator system, could be utilized to reduce the
bandwidth required. Finally, key coding systems could be
utilized to reduce bandwidth use. Such systems will be
discusæed in more detail later in the specification.
11
~ WO94/118~8 2148089 rCT/US93/10930
In this embodiment, the primary layer is provided to
an encoder where the primary layer is prepared for
transmission (block 108). The primary layer is then
decompressed (block llO). Thereafter the primary^~ayer
of information is decompressed (block llO). The primary ;
layer is then decoded and mixed with the secondary layer
of data information (block 112) to provide an enhanced -
interactive multimedia image (motion, video or other) or
data to a display. `~`
Similarly, the secondary layer is compressed through ~
block 114, encoded (block 116) and then transmitted to -
block 118 to decompress and mix block llO. The two
signals (primary and secondary) are then sent to display
112.
In this embodiment, for example, in the optimization -
.....
of video images, the primary layer can be the foreground -
image, the secondary layer can be background information.
Through the use of this type of optimization technique
multimedia information can be enhanced while at the same ;~
time utilizing significantly less bandwidth.
To more fully understand this feature refer now to
the following discussion. In a typical interactive
multimedia system the information is all sent along one
layer. The information that can then be transmitted is ~`
limited by the bandwidth of that layer.
In the prior art, the interactive multimedia ~;
information that could be transmittsd along typical
12
~. WO9~/11858 2 1 4 8 0 8 9 PCT/US93/10930 ~
,...
networks or transmission paths that are very limited
because, for example, in the case of video images the
bandwidth is not adequate to provide a high quality image
to a display.
Hence, in the present invention, by separating the
- multimedia information into primary and secondary layers
and thereafter compressing the more important information
- utilizing well known compression algorithms, a system is -~
described that can produce enhanced interactive
multimedia information that easily be transmitted over
existing networks.
To more fully describe the psychographic enhancement `
feature of the present invention refer now to Figure 4
which shown the various possibilities from a particular
program model. The program model is provided to the
separator 42 of the multimedia system.
Psychographic enhancements are critical to the
improvement in interactive multimedia transmission and
reception enhancements in the context of the present
application is information that is not transmitted but
operates on, operates in conjunction with, or operates as
a supplement to transmitted multimedia information.
There are three separate categories that will be
described that relate to psychographic enhancements.
The first category will be described as a cross
correlation between the information that is being
transmitted and being enhanced due to the presence of
13
~ W094/11858 2 1 ~ 8 0 8 ~ PcT/us93/1o9~o
.,.;; :
information that is not transmitted. Dithering of image
is an example of this in that dithering masks artifacts
of images that are present and that improves the image
quality. This type of system does not remove the~
artifacts but actually just masks imperfections.
secon~ example in the audio ~rena where secondary audio
materials such as a sound of an ocean or the like which
might mask problems in the audio quality of the primary ;~
sound (voice, music or the like).
The second category is where the sign~l is actually
changed without the use of any control signal; for
example, through interpolation or some other technique.
The typical example of that is a graphic equalizer in -
which certain frequencies are enhanced depending on the
range of the particular device. Another example of the
second category is to frequency or amplitude compress a
certain signal so as to further enhance the frequencies
that are being transmitted. It is also known to use
various filters to sharpen or provide certain information
that will actually modify the signal without controlling
it per se.
Finally, the third category is using the primary and
secondary information to drive the other generators that
might be present within the multlmedia system. This can
be utilized to either enhance the multimedia information
or enhance the program model. An example of this is the
use of real-time graphics frequency spectrum displays to
14
,;s~ WO94~1185~ ' ~ 2 1 ~ 8 0 8 9 PCT/US93/10930
enhance a music juke box type of program model. ~,
As is seen in Figure 5, the primary multimedia
information layer can be compressed to reduce the
bandwidth utilizing well known algorithms. It is~also -~
seen that the signal can be replaced by a generator that
is responsive to the p~imary/secondary layers signals.
Finally, a key code could be used to cause information to
be provided from a look-up table or the like. ~ '
Although all of the, above methods provide advantages
in accordance with the present invention, key coding has ''
some additional non-obvious advantages when utilized in
the optimization system of the present invention. In the
following paragraphs the use of various key coding ;
systems will be described generally along with their
attendant advantages.
Typicallyj when looking at an interactive multimedia ~'
information signal there are several components of that
information. The first component is the data or the ,
multimedia information itself that is being conveyed.
The second component is referred to as program model
dynamics. That is the changes that occur in the
interactive multimedia information due to for example, a
fade that allows for a transition from one scene in the
graphics or video image to another. Conversely, if you '~
want to wipe away an image there is information
associated with the multimedia data that would call out
for that transition to change efficiently.
~1 WO94/118~8 2 1 4 8 ~ 8 ~ PCT~US93/10930
..... :
Finally, the third category of interactive
multimedia information is what will be referred to in
this specification which will allow a particular device
or system to go from one category to another. In~a
typical interactive multimedia information system all
this information is required to adequately transmit such -~-
information.
In its simplest form, a key has an identifiable code
which dictates the commands on ~he other side of the
device. The clearest example of such a keying system
would be the very simple dual tone multi-frequency (DTMF)
signal. This type of signal can be used in the
telecommunications area ~o provide keying for low
bandwidth protocol. These keys would then command a code
table on the side of the network to provide certain
information about the multimedia information to be
displayed without requiring actual transmission of the
multimedia information.
A more specific version of this type of key coding
is what will be referred to in this specification as
control information keying. What is meant by controlled
information keying is where a key code is utilized to
access particular types of commands which can then be
used to control other items on the other side of the
network.
Such a table would then be utilized to access a
certain set of multimedia information in the network. A
~ WO94/11858 2 1 4 8 0 8 9 PCT/U~93/10930 -:
.;;,~.-. :
final version of key coding will be called program
branching ~eyinq is described by each of the keys
representing a certain branch identification. Thus in
this type of key coding the key is cross referenc~d to a
particular branch of the interactive multimedIa program
where each of the branches allows plurality of funf~tions
or commands to be accessed in order to replicate the
program model.
The important feature that is provided by all of
these types of keying coding arrangements is that
information already present on the network can be
utilized. Therefore, the processing power inherent in
the network or the system being accessed can be utilized
rather than having to have to provide that processing
power within the optimization system itself.
It is also important to develop means to improve the
transmission quality of the multimedia information, for
example, the information may be transmitted utilizing a
typical transmission algorithm such a CCITT v.32, or Bell
202 with standard communication file data transfer
protocols. The interactive multimedia information could
also utilize specialized protocols that are optimized for
the particular interactive multimedia information that is
to be transmitted. In so doing the algorithm for the
compression algorithm can be interactively matrixed with
the transmission al~orithm to provide the highest quality
information with the maximum interactivity with the
~ WO94/11858 2 1 4 ~ 0 8 9 PCT/US93/10930
minimum transmission line.
Referring now to Figure 6, what is shown is a flow
chart that shows the cooperation of the transmission
algorithm with the compression algorithm to produce a
high quality multimedia image. The flow chart comprises
pro~iding a program model in which the primary and
secondary lay2rs are separated (block 202). The primary
layer is compressed and encoded (blocks 204 and 206).
A control element (block 210) is utilized to control
a compression matrix and a transmission matrix. These
two matrices comprise a plurality of compression
algorithm and transmission algorithm respectively that
are interactively controlled such that as the various
algorithms are detected the quality of the multimedia
information and the speed of the transmission are
interactively determined.
The quality of the information could be determined
manually or through the use of some control circuitry.
It should be understood that these same matrices could
a~so be used on the secondary layer. It was not shown or
described for purposes of clarity and simplicity.
Referring now to Figure 7 what is shown is a block
representation of a matrix of compression algorithm with
transmission algorithm that could be utilized in
accordance with the present invention. The circles 302
aligned in the vertical direction are the compression
algorithms. The rectangles 304 aligned in the horizontal
WO94/1~858 i 2 1 4 ~ 0 8 9 PCT/US93/10930
direction are the transmission algorithms.
For example the compression algorithms could be JPEG
and a generator with MIDI. Similarly, the transmission
algorithms could be optimized for JPEG, data compression
for MID~, or DTMF for key transmission type algorithms.
To provide the highest quality multimedia information
while at the same time utilizing minimum bandwidth the
different algorithms can be selected in an interactiYe
manner.
Hence, a first compression algorithm could be
selected along with the first transmission algorit:hm.
The multimedia information is reviewed either for image
or audio quality than a second compression algorithm is
selected. The multimedia information is reviewed and if
the quality is not acceptable then a second transmission
algorithm is selected. The quality of the information is
reviewed. This process is repeated until the highest or
desired quality multimedia information and interactivity
speed are provided.
The multimedia information derived from the
compression/transmission algorithms can be analog or
digital in nature. However, in a digital signal there
are certain other features that can be taken to advantage
that can be utilized in accordance with the present
invention.
It is known that digital data information is
typically sent in a file which specifies certain
WO94/11858 2 1 4 ~ O 8 3 PCT/US93/10930
parameters of that data and the data information itself
and within the data information itself is information
which may not change for a certain set of files. In the
~ase of an image file, the header information may~specify
the dimensions, pixel depth, and certain other features
- - of the particular image. This file may take up a much as
twenty percent of the data file.
Conversely, in a file such as MIDI music file which
comprise a plurality or a series of music notes, the
header may include instrument information, related
information and characteristics of that particular file.
In both of the above mentioned examples, the header ~ `
information may not change, through the use of the
optimization method the amount of information may be
significantly reduc~d over time.
Hence, in the case of the image file, the header ;
could be sent first with no compression or with lossless
data compression as the secondary file because it will
always remain the same. The date file itself can then be
compressed down to its smaller size.
Another method for enhancing the psychographic
parameters is to provide some form of error detection and
adjustment. As has been mentioned before the detection
and adjustment can be accomplished via interpolation of
the error. An alternative method of error corrector is
through an error correction/transmission algorithm. What `~
is meant by this, is relating the transmission to the
"''.~
., . . , ..... ..... ~ .. ... ,. . ,.. :
WO 94tl 1~58 ' 2 1 4 ~ 0 8 9 PCr/US93/10930
,.......
... .
compression to enhance interactivity.
In this type of system before the file is sent the
base file is compressed and then decompressed. This
decompressed file is called an expanded compresse~ base
file. The expanded compressed base file is then compared
to the original base file and an error file is then
developed (the error file being the difference between
the base file and the expanded compressed base file).
The error f ilP is compressed and sent along with the
compressed version of the base file down the line. These
files are then combined and decompressed to provide an
enhanced image. ;~
Referring now to Figure 8 it is seen that a data
file utilizing this technique could initially be
separated into primary and secondary layers. The primary
layer could be compressed using a first compression
algorithm, the header could be sent first along a first
transmission path and the compression signal could be
sent along a second transmission path.
Therefore, the amount of storage necessary for the
file is significantly reduced through secondary
compression techniques. This information can then be
transmitted or stored across the network rather than
having to have all that information stored within a
particular device within the optimizàtion system.
It should be understood by one of ordinary skill in
the art will recognize that the number of algorithms is
21~08!~
WO94/11858 PCT/US93/10930
... . .
not limited to the number shown in the figures. In
addition it should be recognized that the order or the
selection of the algorithms could be changed and that
would be within the spirit and scope of the present
invention.
The present invention has been discussed in terms of
compressing the primary layer or layer and by compressing
and transmitting that primary layer in a particular way
the interactivity of the system is enhanced. It should
be understood that it may be equally important to enhance
secondary layers to produce the same effect.
Therefore, it may be important to enhance the
secondary layer, it may be important to enhance the
primary layer or it may be important to enhance both.
Therefore, the present invention through the use of ;~
compression and transmission algorithms and through the
psychographic enhancement of the program model can
enhance interactivity of a multimedia system.
It should also be understood that the function of
the compression and transmission algorithms can also be
done through other means; for example, a signal generator
could be used to provide the same information. That is,
a signal generator responsive to a particular layer or
layer of information could be utilized to provide that `~
information or some level of information that is
representative of that layer. For example, a tone
generator responsive to a signal from the secondary layer
WO 94/11858 2 1 4 ~ 0 8 9 PCT/US93/10930
~;
to provide the tone that would be representative of that
secondary layer.
Conversely, some type of graphics generator could be
utilized to respond to that same type of signal t~
provide a certain type of graphic image in a video
system. Finally, it should also be understood th~t the
psychographic parameters can be adjusted by human
operator or in the alternative can be adjusted or
modified by an automatic means. ;
The optimized multimedia information from the --
program source material is then processed by the MCPS 44 ~
(Figure 2) to be utilized interactively with IMDs 52. ~--
Referring now to Figure 8, what is shown is the preferred
embodiment of a MCPS 44. The MCPS comprises a
distributed computing architecture 500. The distributed
computing architecture 500 includes a master node 502 ~-
that has several server nodes 504 which in turn are
connected to several ports 506.
When connected via the existing networks such as to
a switched telephone network and then on to connection to
the IMD which has its own processing storage and
computing structure the entire network can be operated as
a massive distributed computing environment. This
environment shares all dimensions of computing, storage,
transmission and peripheral resources (printing, product `
ordering, mailing functions, etc.). This type of
computing architecture would include dynamic port
23
~ WO94/11858 2 1 ~ 8 0 8 ~ PCT/US9~10930
allocation and would include incremental failure
characteristics to allow for robustness of the MCPS 44.
It also is very important in any of the above-
mentioned entertainment systems to have an interactive
multimedia device 5~ which will allow for the receipt of
high quality multimedia information from IMM 42.
Figure 9 is a preferred embodiment of an interactive
mul$imedia decoder (IMD) 52A. The IMD 52A comprises
several components. The telephone line is coupled to a
multimedia modem S22. The multimedia modem is coupled to
a multimedia memory 524 which can be an expandable ~
dynamic random access memory (~RAM) 524. The multimedia ~;
modem chip 522 provides data to a multimedia decompressor -
device 526. The multimedia memory provides data to
graphics/character generator 536, speech generator S38
and music synthesizer 540.
In addition, the output of the generators 536, 538 ~;
and 540 are provided to a video control chip 532. Video
control ~32 provides signals to a standard television
display and receives signals from a standard television
source. The multimedia modem 532, the multimedia memory
524, the multimedia decompressor 526, the multimedia
digital!audio control 528, the video control 532 and
music synthesizer 544 are all ultimately controlled by an
interactive control interface 530 which manages the
operation of all of the above elements. The video
control 528 is coupled to a standard telephone keypad
24
W094/11~58 2 1 4 8 0 8 9 PCT/US93/10930
input or for a television remote-type device or a special
IMD remote can be utilized in a variety of ways which
will be discussed in detail hereinafter.
Personalized and demographic information (~ch as
the age, race, sex and other personal characteristics) of
the user along with the technical information associated
with the IMD 52 ~serial no., number of generators, type
etc.). Upon connecting the IMD 52 to the MCPS 44, the
IMD 52 can both forward this information either at the
beginning of the session or any time afterward. The
information can then be updated through the MCPS 44 or
directly IMD 52 through selection using a keypad or
remote control or by receiving data from the MCPS 44.
Another mode of operation is the delivery of
multimedia during the period when a communication network
is nQt in use. In this way, large amounts of multimedia
information can be efficiently transmitted and stored in
the IMD 52 for later review and enhanced interactivity.
The function of each of the different components in
a preferred embodiment is described in a summary fashion
below.
Multimedia Modem 522
A. Responsible for all communications between standard
phone line, optional serial port, lnterface to multimedia
memory, multimedia decode, audio control, and processor
control modules.
B. Supports standards protocol for half-duplex, full
~,, WO~4tl1858 2 1 ~ 8 0 ~ ~ PCT/US93/10930
duplex, and half-duplex high speed operation.
C. On-chip encode/decode capability, D~A, A~D for
voice, facsimile, and data functions.
D. ~ual tone multi-frequency (DTME) detect and
generation.
E. Auto-detect voice/~acsimile/data switch for
transparent mode transition.
F. Incorporates controller unit with binary file
transfer, facsimile, data, and voice modes, and o~tional
proprietary multimedia processor control optimized
protocol firmware.
G. Firmware allows IMD to use multimedia modem to
perform call processing function including telephone call
dialing and connection, unattended receipt of data and
fax among other functions.
Multimedia Memory 524
A. ~ominal DRAM or VRAM for image mixing/processing,
and auxiliary multimedia data store.
B. Nominal ROM for resident IMD control program.
C. Optional co-resident DRAM for multimedia data store
and program/data store.
D. Optional non-volatile storage (extendible).
E. Memory control unit for VRAM/ROM/DRAM and non-
volatile storage.
Multimedia Decode 526
A. Responsible for real-time decompression of images
transferred to or stored in the IMD 52.
094/1185g ! 2 1 ~ ~ ~ 8 9 PCT~US93/10930
B. On chip inverse discrete cosine transform processor
with multiple algorithm coding.
C. Reverse quantizer decoder/tables.
D. Built-in zoom, pan, chroma key, mix from compressed
data incorporates interfaces to video data bus,
multimedia memory, multimedia modem, video control, and
microprocessor control sections.
Video Control 532
A. Responsi~le for all IMD 52 video mixing,
enhancements, and display functions.
B. Pixel processor for mix, zoom, pan, chroma key,
transform on pixel data, transitions.
C. Graphics processor for figures (e.g., rectangles
with color fill) generation, sprites, text with foreign
characters, and scrolling.
D. Digital to analog conversion, analog to ~TSC, NTSC
video plus stereo audio to RF.
~ra~hics/Character Speech Generator Music Synthesizer
536 538 and 540
A. Responsible for enhancing received analog/digital
audio, music synthesis generation, and overall analog
mixing and audio effects.
B. Incorporates decoding burden.
C. Sampled instrument synthesis from compressed MIDI
input.
WO94/11858 214 ~ O 8 9 PCT/US93/10930
D. Built-in micro-controller for multi-task generation.
E. Dual analog source mix, digital audio and
synthesizer mix, analog audio control (volume, bass,
treble, ~alance) for output to analog left/right~audio.
Interactive Multi-Task Processor 530
A. Responsible for multi~task exe~ution of resident and
downloaded IMD code for operation in conjunction or
independently of the MCPS.
B. Master/slave microcontroller architecture for multi-
task control of communications, multimedia memory,
multimedia decode, digital video control, digital
audio/synthesis, and interface management.
Through the cooperation of the IMM 42, IMD 52 and
MCPS 44, an enhanced multimedia entertainment system is
provided. These devices when connected either through a
network or with a television or stereo equipment provide
for the presentation of multimedia information that has
enhanced quality as well as improved interactivity. In
addition, this type of system will provide the facility
to generate revenues and track the characteristics of
individuals who are the users of the program source
material (songs and videos~.
Although the present invention has been described in
accordance with the embodiments shown in the figures, one
of ordinary skill in the art recognizes there could be
variations to the embodiments and those variations would
be within the spirit and scope of the present invention.
WO94/11858 2 1 4 8 0 8 ~ PCT/US93/10930
.~.
Accordingly, many modifications may be made by one of
ordinary skills in the art without departing from the
spirit and scope of present invention, the scope of which
is defined solely by the appended claims. ~
