Note: Descriptions are shown in the official language in which they were submitted.
CA 02382650 2010-07-12
VIDEO CONFERENCING USING AN ELECTRONIC BOOK VIEWER
Technical Field
The invention relates to electronic book viewers. More particularly, the
invention
relates to electronic book viewers with capabilities for conference calling
and associated
communication systems and processes.
Background OF The Invention
Sparked by the concept of an information superhighway, a revolution will take
place in
the distribution of books. Not since the introduction of Gutenberg's movable
typeset printing
has the world stood on the brink of such a revolution in the distribution of
text material. The
definition of the word "book" will change drastically in the near future. Due
to reasons such as
security, convenience, cost, and other technical problems, book and magazine
publishers are
currently only able to distribute their products in paper form. The electronic
book system of the
present invention solves the problems encountered by publishers.
CA 02382650 2002-02-22
WO 01/18665 PCT/US00/24819
2
1 Summary Of The Invention
2 The present invention is an improvement to electronic book delivery systems.
More
3 specifically, the present invention provides the capability to perform
conference calls, preferably
4 video conference calls, utilizing an enhanced electronic book viewer and/or
electronic book delivery
system.
6 The present invention provides an electronic book viewer that can be
utilized in a
7 conference call. In one embodiment, the electronic book viewer comprises a
memory, a video
8 display, a speaker, a microphone, a transmitter, and a receiver. Data
representing an electronic
9 book can be stored in the memory and displayed on the video display. The
transmitter is
connected to the microphone, while the speaker is connected to the receiver.
The transmitter and
11 the receiver have sufficient bandwidth to accommodate a conference call. In
another embodiment,
12 the electronic book viewer comprises a memory, a video display, a speaker,
a microphone, a
13 transmitter, a receiver, a processor, and conference calling software. The
processor is connected
14 to the other components, and the conference calling software executes on
the processor, enabling
the electronic book viewer to be utilized in a conference call. Optionally,
the conference call
16 includes video received by the electronic book viewer and displayed on the
video display. As
17 another option, the electronic book viewer further comprises a camera
connected to the transmitter,
18 and the conference call includes video transmitted from the electronic book
viewer.
19 The present invention also provides a conferencing system comprising at
least one electronic
book viewer, at least one end equipment; and an interconnection network
capable of linking two
21 or more of the at least one electronic book viewer and the at least one end
equipment in a
22 conference call.
23 The present invention also provides a method of conferencing using an
electronic book
24 viewer. The method comprises displaying an electronic book on the
electronic book viewer;
participating in a conference call while viewing the electronic book; and
communicating information
26 content of the conference call with the electronic book viewer.
Participation in the conference call
27 may involve, for example, initiating or receiving a conference call.
28 The present invention also provides a method of distance learning using an
electronic book
29 viewer. The distance learning method comprises displaying learning material
as an electronic book
on the electronic book viewer; participating in a conference call while
displaying the electronic
CA 02382650 2011-03-02
3
book; and communicating information content of the conference call with the
electronic book
viewer. For example, the learning material may include a textbook, and the
conference call may
be an interactive lecture concerning a subject of the textbook.
As used herein, the term "conference call," or sometimes more simply "call,"
denotes a
traditional two-party call as well as a multi-party call or a single party
call with storage and
future access by others, includes calls utilizing only one or a combination of
several types of
media (e. g., voice only or voice plus video and/or other media such as data,
programs, etc.),
and includes one-way broadcast calls (i.e., view only) as well as full duplex
calls or mixed
media combinations of one-way broadcast and full duplex (e.g., broadcast video
from one party
and shared return audio from all parties).
In one aspect, the present invention provides a conferencing system,
comprising: at least
one electronic book home system, comprising: a portable electronic book viewer
comprising: a
feature for selecting an electronic book from a list of available books
displayed in the electronic
book viewer, a feature for ordering a selected electronic book; a feature for
receiving an
electronic transmission of the selected electronic book; a feature for
establishing and managing
a conference call while viewing the selected electronic book; a feature for
participating in the
conference call while viewing the selected electronic book; a memory in which
data
representing an electronic book can be stored, a video display, connected to
the memory for
displaying the selected electronic book, a speaker, a microphone, a
transmitter connected to the
microphone, and a receiver connected to the speaker, wherein the transmitter
and the receiver
have sufficient bandwidth to accommodate a conference call, whereby the
electronic book
viewer can be utilized in a conference call; and a library unit, locally
coupled to the electronic
book viewer, configured to download from a remote source and store a plurality
of electronic
books and to deliver the selected electronic book to the electronic book
viewer upon request, the
library unit comprising: local connections between the library unit and the
electronic book
viewer and the library unit and an interconnection network, and a controlling
mechanism for
controlling communications involved in the conference call; and at least one
end equipment;
wherein the interconnection network is capable of linking two or more of the
at least one
electronic book viewer and the at least one end equipment in the conference
call during the
conference call; wherein the conference call is initiated by selecting a
portion of a text of the
displayed selected electronic book.
In a further aspect, the present invention provides an electronic book
conferencing
system, comprising: at least one electronic book home system, comprising: an
electronic book
viewer comprising: a feature for selecting and ordering an electronic book
from a list of
CA 02382650 2011-03-02
3a
available books displayed in the electronic book viewer; a feature for
receiving an electronic
transmission of the selected electronic book; a feature for establishing and
managing a
conference call while viewing the selected electronic book; a feature for
participating the
conference call while viewing the selected electronic book; a memory in which
data
representing an electronic book can be stored, a video display, connected to
the memory for
displaying the selected electronic book, a speaker, a microphone, a
transmitter connected to the
microphone, and a receiver connected to the speaker, wherein the transmitter
and the receiver
have sufficient bandwidth to accommodate a conference call, whereby the
electronic book
viewer can be utilized in a conference call; and a library unit configured to
receive a plurality of
electronic books from a remote operations center, and store the plurality of
electronic books for
local and secure access by the electronic book viewer, the library unit
comprising: a local
connection between the library unit and the electronic book viewer, and a
controlling
mechanism for controlling communications involved in the conference call; a
feature for
charging the recipient a price for the ordered electronic book; at least one
end equipment; and
an interconnection network capable of linking two or more of the at least one
electronic book
viewer and the at least one end equipment in a conference call, wherein the
conference call is
initiated by selecting a portion of a text of the displayed selected
electronic book.
Certain embodiments of the present invention achieve desirable advantages.
Because an
electronic book viewer is a daily use household appliance that has a high tech
aura while being
very practical, portable, and easy to use, it is highly advantageous to
enhance the functionality
of an electronic book viewer to support video conferencing. There are
synergistic effects from
the combined convenience of viewing electronic books in association with
participation in a
video conference call. For example, a conference call can be initiated while
viewing an
electronic book, which is a directory containing call connection numbers or
addresses. As
another example, a conference call can be initiated while viewing an
electronic book, which
may be a topic of the call or which can be transmitted wholly or partially as
part of or in
conjunction with the call. In yet another example, a user engaged in
conference call that the user
finds boring or distracting can conveniently view an electronic book while
maintaining the
conference call connection. Other advantages and benefits of the present
invention will be
apparent from the following Detailed Description of a Preferred Embodiment,
with references to
the Drawings.
CA 02382650 2011-03-02
3b
Brief Description Of The Drawings
Figure 1 is a schematic showing an overview of the electronic book selection
and
delivery system.
Figure 2 is a block diagram of a hardware configuration for a four component
home
subsystem.
Figure 3 is a block diagram of a hardware configuration for a two unit home
subsystem.
CA 02382650 2002-02-22
WO 01/18665 PCTIUSOO/24819
4
1 Figure 4 is a block diagram for an example of a library unit.
2 Figure 5 is a block diagram of an electronic book viewer.
3 Figure 6 is a block diagram of an electronic book viewer with video
conferencing
4 capabilities.
Figure 7 is a diagram of various video conferencing network connections
according to a
6 preferred embodiment of the present invention.
7 Figure 8 is a diagram of a mini-network without a central node, according to
a preferred
8 embodiment of the present invention.
9 Figure 9 is a diagram of a mini-network having a library central node,
according to a
preferred embodiment of the present invention.
11 Figure 10 is a connection diagram of a conference call among the three
viewers.
12 Figure 11 is a block diagram of a video conferencing central node.
13 Detailed Description Of A Preferred Embodiment
14 I. Overview of An Electronic Book Delivery System
Electronic book selection and delivery systems distribute books to bookstores,
public
16 libraries, schools and consumers, providing a secure electronic system for
both delivering selected
17 books and receiving payments. The electronic book system of the present
invention has an unusual
18 combination of features that provides the consumer with a daily use
household appliance that has
19 a high tech aura while being very practical, portable, and easy to use.
An advantage of electronic books is the elimination of the distribution of any
physical object
21 such as a paper book or computer memory device from any book or text
distribution system. The
22 purchase of a book becomes a PAY-PER-READTM event avoiding the overhead,
"middle-men,"
23 printing costs, and time delay associated with the current book
distribution system. Published
24 material and text such as the President's speech, a new law, a court
decision on abortion, or O.J.
Simpson's testimony can be made immediately available to the consumer at a
nominal fee.
26 The primary components of an electronic book delivery system are the
subsystem for
27 preparing the text for delivery and the subsystem for receiving and
selecting text that was delivered.
28 The system may be configured for use by bookstores, public libraries,
schools and consumers. The
29 system for consumer use is made up of four subsystems, namely: (1) an
operations center, (2) a
CA 02382650 2010-07-12
distribution system, (3) a home subsystem including reception, selection,
viewing, transacting and
transmission capabilities, and (4) a billing and collection system.
The operations center performs several primary functions: manipulating text
data (including
receiving, formatting and storing of text data), security encoding of text,
cataloging of books,
providing a messaging center capability, and performing uplink functions. In
one embodiment, the
system delivers the text from the operations center to consumer homes by
inserting text data within
analog video signals. The insertion of text is generally performed with an
encoder at an uplink site
that is within or near the operations center. The system can use several lines
of the vertical blanking
interval (VBI), all the lines of the analog video signal, a digital video
signal or unused portions of
bandwidth to transmit text data. Using the VBI delivery method, the top ten or
twenty book titles
may be transmitted with video during normal programming utilizing existing
cable or broadcast
transmission capability without disruption to the subscriber's video
reception. Using the entire video
signal, thousands of books may be transmitted within just one hour of air
time. Nearly any analog
or digital video or data distribution system may be used to deliver the text
data.
The home subsystem performs at least four functions: connecting to the
distribution system,
selecting text, storing text, and transacting through a communicating
mechanism. The components of
the home subsystem may be configured in a variety of hardware configurations.
Each function may
be performed by a separate component, the components may be integrated, or the
capability of
existing cable set top converter boxes, computers, and televisions may be
utilized. A connector,
library unit and viewer unit may be used. In one embodiment, the connector
portion of the home
subsystem receives an analog video signal and strips or extracts the text from
the video. The home
library stores the text signal, provides a user fi-iendly software interface
to the system and processes
the transactions at the consumer home. The viewer provides a screen for
viewing text or menus and
novel user friendly controls.
The viewing device may be a portable book shaped viewer which stores one or
more
books for viewing and provides a screen for interacting with the home library
unit. In one
embodiment, a high resolution LCD (liquid crystal display) is used to both
read the books and to
interact with the home library software. In one embodiment, an optional phone
connector or return
path cable connection initiates the telephone calls and, with the aid of the
library, transmits the
CA 02382650 2002-02-22
WO 01/18665 PCT/USOO/24819
6
1 necessary data to complete the ordering and billing portion of the consumer
transaction. The billing
2 and collection system performs transaction management, authorizations,
collections and publisher
3 payments automatically.
4 Figure 1 shows one embodiment of an electronic book selection and delivery
system 200.
The delivery system 200 includes: an operations center 250 including an uplink
site 254; a video
6 distribution system 208; an electronic book device, or home system 258
including a video
7 connector 212, a library 262, a viewer 266, and a phone connector 270;
telephone system 274;
8 an Internet web site 279; and a billing and collection system 278. Also as
shown in Figure 1, the
9 home system 258 may include connections to a television 259, a printer 318
and a personal
computer 261 may be used to display menu screens, electronic books, electronic
files, or any other
11 information associated with the electronic book delivery system 200. In
addition, the television 259
12 and the personal computer 261 may provide control functions that replicate
and supplement those
13 of the viewer 266.
14 The operations center 250 receives textual material from outside sources
282 such as
publishers, newspapers, and on-line services. Alternately, the outside sources
may maintain
16 electronic books at the Internet web site 279. The outside sources 282 may
convert textual and
17 graphical material to digital format, or may contract with another vendor
to provide this service.
18 The operations center 250 may receive the textual and graphical material in
various digital formats
19 and may convert the textual material to a standard compressed format for
storage. In so doing, the
operations center 250 may create a pool of textual material that is available
to be delivered to the
21 home system 258. The textual material may be grouped by books or titles for
easy access.
22 As used herein, "book" means textual or graphical information such as
contained in any
23 novels, encyclopedias, articles, magazines, newspapers, catalogues,
periodicals, or manuals. The
24 term "title" may represent the actual title assigned by an author to a
book, or any other designation
indicating a particular group, portion, or category of textual information.
The title may refer to a
26 series of related textual information, a grouping of textual information,
or a portion of textual data.
27 For example, "Latest Harlequin Romance", "Four Child Reading Books (Ages 10-
12),"
28 "Encyclopedia BRTTANNICA'TM," "President's Speech," "Instruction Manual,"
"Schedule of 4th
29 of July Events," "Pet Handbooks," "Roe v. Wade," and "The Joy of Cooking,"
are suitable titles.
Also, the title may be a graphical symbol or icon. Thus, a picture of a wrench
may be a title for a
CA 02382650 2010-07-12
7
repair book, a picture of a computer a title for a computer book, a graphical
symbol of a telephone
a title for a telephone book, a drawing of a dagger a title for a mystery
book, a picture of a bat and
ball a title for a sports book, and a picture of tickertape a title for a
business book.
The operations center 250 includes the uplink site 254 for placing the text
onto a
telecommunications signal and sending the telecommunications signal into a
distribution system. The
uplink site 254 would generally include an encoder (not shown in Figure 1) to
encode the text onto
the telecommunications signal.
Many analog and digital video distribution systems may be used with the
electronic book
delivery system 200, such as cable television distribution systems, broadcast
television distribution
systems, video distributed over telephone systems, direct satellite broadcast
distribution systems,
and other wired and wireless video distribution systems. Nearly any
distribution system which can
deliver a telecommunications signal, including a video signal, will work with
the electronic book
delivery system 200. It is also possible to distribute the electronic book
without using a
telecommunications signal.
The home system 258 performs five functions related to the reception and
viewing of
electronic books: (1) connecting with a video distribution system; (2)
selecting data; (3) storing
data; (4) displaying data, and (5) handling transactions. An important
optional function of the home
system 258 is communicating using, in one embodiment, a telephone
communication system 274.
The home system 258 may be made up of four parts: a video connector 212 or
similar type of
connector for connecting with the distribution system 208, a library 262 for
storing and processing,
a viewer 266 for viewing menus and text and a telephone connector 270 for
connecting with a
telephone communications system 274.
The billing and collection system 278 may be co-located with the operations
center 250 or
located remote from the operations center 250. The billing and collection
system 278 may be in
communication with the home system 258 using telephone-type communication
systems (for
example 274). Any of a number of communication systems, such as a cellular
system or the
Internet, will operate with the billing and collection system 278. The billing
and collection
system 278 records the electronic books or portions of text that are selected
or ordered by the
subscriber. The collection system can charge a subscriber's
CA 02382650 2010-07-12
8
credit account or bill the subscriber. In addition, the billing and collection
system 278 may monitor
the amount due to publishers or other outside sources 282 who have provided
textual data or other
services such as air time to enable the text delivery system 200 to operate.
Also within the scope of the present invention is an intranet configuration of
the
embodiments illustrated in Figure 1 that may be used as a part of a private
distribution network for
distributing and circulating electronic books. For example, a university
library or bookstore may
use such an intranet to circulate electronic books to university students and
professors.
Additional details regarding the electronic book selection and delivery system
200 as well
as alternatives can be found in the copending U.S. Patent No. 5,986,690,
already cited above.
11. Electronic Book Home System
The hardware configuration for a four component home system 258 is shown in
Figure 2.
Figure 3 shows a hardware configuration for a two component home subsystem.
The home system
258 performs several functions, such as receiving data and video
transmissions, stripping (or
extracting) the data from the video signal, screening and storing the data,
providing user friendly
interface controls and software, displaying menus and text, processing
transactions, initiating
telephone calls and transmitting billing data. Various hardware configurations
may be utilized to
achieve the desired functions of the home system 258. For example, as shown in
Figure 3, the
home system 258 can be configured to utilize the reception and channel tuning
capability of the
current installed subscriber base of cable converter boxes and televisions 601
and networked
computers. The home system 258 can also be designed as an advanced set top
terminal with menu
generation capability, electronic memory and a telephone modem.
The electronic components that make up the home system 258 can be arranged in
a variety
of ways. In the four unit subsystem of Figure 2 the viewer 266 and library 262
are wired together
while the remaining components communicate through RF (radio frequency)
transceivers 604. In
a simple version of the home system 258 there are only two units, a library
262 and a viewer 266.
Figure 3 shows a two unit home system 258 with certain optional features.
The viewer 266 is generally equipped with a high resolution viewing area 602,
digital logic
(including a key 605, security 606, and a microprocessor 621), video graphics
control and memory
607, power supply circuitry (not shown in Figure 3), an optional battery 603
and an optional RF
WO 01/18665 CA 02382650 2002-02-22 PCT/USOO/24819
9
1 transceiver 604. In a two unit arrangement, the library 262 contains the
connector function to the
2 electronic book distribution system, connector function to a public
telephone communications
3 system, and memory 600, which may be removable and portable memory 600'
(Figure 4). More
4 specifically, the library 262 would include data stripping functions 617,
digital logic 609, memory
storage 600, power circuitry 610, optional connections 611, including cellular
or PCN (personal
6 communications network) phone 611', optional battery (not shown), optional
tuner module 613 and
7 an optional RF transceiver 604. The connector 212 and the public telephone
system connection
8 270, as well as the removable portable memory unit 600' of the library 262
may be broken out into
9 separate components. (Figure 3 shows a removable portable hard disk memory
600' with
removable cartridges 614.) Finally, the home system 258 may include an
attached keyboard 267
11 or a wireless keyboard 268. Both the attached keyboard 267 and the wireless
keyboard 268 may
12 be used to communicate with the viewer 266 (not shown) or the library unit
262. The wireless
13 keyboard 268 may communicate using RF signaling, for example.
14 In an alternate arrangement, all functions of the home system 258 maybe
incorporated into
a single unit. The functions of the library 262, for example, may be carried
out by a card or chipset
16 in the viewer 266. All the communications devices needed to couple the home
system 258 to
17 various telecommunications networks may also be incorporated into the
viewer. All interfaces
18 between the home system 258 and the subscriber may be included with the
viewer 266. In this
19 embodiment, the viewer 266 may include a communication device for receiving
inputs from a
separate keyboard.
21 Therefore, the home system 258 may have as many as five separate
components, which
22 communicate with each other. The two, three, four or five separate
components which make up
23 the home subsystem can communicate with each other in a variety of ways,
including hardwired
24 connection 615, RF transceiver 604 and other wireless methods.
RF communications may be used in the home, allowing separate components to be
located
26 throughout the home without restriction. The data communicated between the
units may be secure
27 data. In addition, the library 262 may provide power to the viewer 266
through the hard wire
28 communication link 615.
29 To receive and strip data from a video signal at the consumer's home, a
device such as a
cable interface device or cable connector 212, 21' is used. The cable
connector device 212'
WO 01/18665 CA 02382650 2002-02-22 PCT/USOO/24819
1 includes a tuner 613, while the cable interface device 212 makes use of
existing tuning equipment
2 in the home. In either configuration, data is stripped from the video signal
and stored at the
3 subscribers location in the library 262. The phone connector 270, optional
connector 611, and a
4 modular connector initiate communications and transmit ordering and billing
information to the
5 operations center 250 or billing and collection system 278. A digital
connector 619 is provided to
6 communicate digital information with the set top terminal 601. The library
262 is the intelligent
7 component of the home subsystem, incorporating the hardware and software
necessary to store the
8 text data, generate menus and effect the purchase transactions. In addition
to an RF transceiver
9 604, the home library 262 also includes the necessaryjacks and connections
(e.g., communications
10 connector 270) to allow the system to be connected to the viewer 266. As
shown in Figure 3, the
11 library 262 communicates the text data to the viewer 266 in a secure
format, which requires a key
12 605 for decryption. The text may be decrypted page by page just before
viewing.
13 A. Video Connector
14 The connection to the video distribution system may be a cable connector to
a cable
television delivery system, as shown in Figure 3. The cable connector includes
a data stripper
16 circuit 617, which accepts video input from either a set top terminal 601,
a TV or a VCR, or an
17 optional tuner block 613 that receives the CATV signal through the cable
connector 212'. The
18 data stripper circuit 617 strips data out of the video, and outputs a
digital bit stream to the digital
19 logic portion 609 of the library unit 262. The data is embedded in the
video signal either in the
vertical blanking interval or the active video portion in an encrypted and
compressed format. The
21 data stripper circuit 617 can be placed inside the set top terminal 601,
TV, or in the library unit.
22 The data stripper circuit 617 outputs the digital bit stream to be used by
the library digital logic 609.
23 The video connector 212 may also contain a channel tuner module 613 that
can tune to the
24 video channel and provide access to the video that contains the data to be
stripped. Using the
optional tuner module 613, a set top terminal, VCR, or TV tuner is not needed
in the home
26 subsystem. The optional tuner module 613 would instead receive the CATV
signal directly through
27 the cable connector 212'. Additional connector options, which allow for the
receipt of electronic
28 book content using alternative delivery methods, are possible. This
ubiquitous access is provided
29 using the modular connector as depicted in Figure 3.
WO 01/18665 CA 02382650 2002-02-22 PCTIUSOO/24819
11
1 B. Library
2 An embodiment of the library 262 for a two unit home subsystem is shown in
both Figure
3 3 and Figure 4. The embodiment shown includes the following optional parts:
the video connector
4 212, phone connector 270, RF transceiver 604, and battery pack 624 in
addition to a removal
portable memory 600', microprocessor 628, instruction memory unit 632, digital
logic 636, and
6 power unit 640.
7 The library 262 contains a digital logic section 609 (not shown in Figure 4)
which includes
8 the microprocessor 628, the digital logic 636 and the instruction memory
unit 632. The
9 microprocessor 628 may be a secure microprocessor such as the Mot SC21
device sold by
Motorola. The digital logic section 609 receives the serial digital bit stream
from the data stripper
11 circuit 617 and process the data. Error correction is also performed by the
digital logic section 609
12 and the data is checked for a proper address. If the address of the data is
correct and the library
13 262 is authorized to receive the data, the data is transferred to the
memory storage unit 600, 600'.
14 Authorization to receive the data is provided by the cable headend or
another distribution point.
An authorization code may be sent in the serial digital bit stream. The
digital logic section 609 sends
16 appropriate text and graphical data to the memory storage unit 600, 600'.
It transfers this data in
17 a compressed and encrypted format and the data remains stored in a
compressed and encrypted
18 format.
19 1. Memory Storage Unit
The memory storage unit of the library may be a removable portable memory unit
600' (as
21 shown in Figures 2-4). A variety of options are available for memory
storage: a hard disk drive,
22 such as one having an 80 megabyte or 200 megabyte capacity; a hard disk
with removable platters;
23 and a CD ROM. Referring to Figure 3, a hard disk drive unit 600', which
contains removable
24 platters, may also be used. This would provide virtually unlimited library
storage capacity. Data
will be stored in the memory storage unit in a compressed and encrypted
format. As is also shown
26 in Figure 3, the data may also contain a key or unique ID number that
matches the ID or key of the
27 viewer 266. This matching of a unique key or ID number prevents
unauthorized transfer of text
28 data from the memory storage unit to an unauthorized viewer. Small memory
devices such as smart
29 cards, electronic memory cards or PCM CIA cards (personal computer memory
card industry
association) may also be used to store the data.
WO 01/18665 CA 02382650 2002-02-22 PCTIUSOO/24819
12
1 2. Power Circuitry
2 As shown in Figures 3 and 4, the library 262 will accept power from AC wall
power 610,
3 DC power 640, or optional battery power 624. The power circuitry 610,640 may
provide all the
4 voltage necessary from either the battery 624 or AC unit for the various
circuitry in the library. The
power circuitry 610, 640 may also provide power to the viewer through a single
data cable when
6 connected to the viewer. The power circuitry 610,640 will recharge the
battery using AC power
7 when in operation. With the optional battery unit 624 installed, the library
262 becomes a portable
8 unit and can still provide power to the viewer 266. In order to extend
battery life, power
9 conservation measures maybe utilized, such as shutting down the memory
system when not in use.
When the viewer unit 266 is being utilized and the library circuitry is not
being utilized, virtually all
11 power may be shut down to the library 262.
12 3. Connection to the Public Telephone System
13 In an embodiment, the connection to the telephone system may be provided by
a connector
14 device 611, which consists of a modem. Various available modems may be used
to perform this
function. As shown in Figure 3, cellular phone or PCN phone connections 611'
may also be
16 provided. When the home system 258 is first initialized, the modem may be
used to transfer the
17 name and credit card information of the consumer to the billing and
collection system 278. The
18 telephone connection 270 may be utilized each time an electronic book is
purchased by a consumer
19 to complete and record the transaction. The telephone connection 270 may
also be used to receive
the text data from the operations center 250, by-passing the video
distribution system 208.
21 Additionally, the telephone connection 270 may be used to send and/or
receive video and/or audio
22 associated with a conference call. The telephone connection 270 may be a
separate unit as shown
23 in Figure 3. However, alternate means exist to connect the home system 258
to the billing and
24 collection system 278 or the operation center 250. The modular connector
(shown in Figures 3 and
4) provides access to each communication network to provide a path from the
home system 258
26 to the billing and collection system 278 or the operations center 250.
27 Additional details regarding the home system 258 as well as alternatives
can be found in the
28 copending U.S. Application Serial Nos. 09/289,956 and 08/336,247, cited
above.
WO 01/18665 CA 02382650 2002-02-22 PCT/US00/24819
13
1 III. Electronic Book Viewer
2 Figure 5 is a block diagram of a viewer 266 showing its internal components.
The viewer
3 266 of Figure 5 is similar to the viewer 266 depicted in Figure 3. The
viewer 266 is preferably
4 designed to physically resemble a bound book. The viewer 266 is made up of
five primary
components and several optional components: (1) LCD display 602; (2) digital
circuitry (not
6 shown); (3) video graphics controller 607'; (4) input controls 740'; (5)
book memory 728; (6)
7 optional power supply circuitry 736; (7) optional battery 603'; (8) optional
RF transceiver 604;
8 and (9) optional alternative communication interface devices, such as
cellular or mobile connector
9 611; (10) optional keyboards 267 and 268 (not shown); and (11) an optional
speaker/microphone
608'.
11 (1) A high resolution LCD screen 602, of VGA quality, may be used by the
viewer 266
12 to display text and graphic images. Alternatively, flat panel CRT
technology can be employed for
13 the screen 602. The screen 602 maybe the size of one page of an electronic
book. A two page
14 screen or two screens may also be used with the viewer 266.
(2) Digital circuitry that includes a secure microprocessor 621, instruction
memory 732,
16 and digital logic. Data is transferred to the viewer 266 in compressed and
encrypted format. The
17 secure microprocessor 621 compares the ID number of the viewer 266 with the
incoming data
18 stream and only stores the text data if the ID number of the viewer 266
matches that within the
19 incoming data stream. The viewer 266 may be configured not to output text
data or other data, and
the data can be decompressed and decrypted only at the moment of viewing and
only for the
21 current page being viewed. These measures provide additional security
against unauthorized access
22 to data.
23 (3) A video graphics controller 607' that is capable of assisting and
displaying VGA quality
24 text and graphic images is included in the viewer 266. The graphics
controller 607' is controlled
by the digital circuitry described above. Text may be displayed in multiple
font sizes.
26 (4) The viewer 266 of Figure 5 has touch panel controls. These unique and
novel controls
27 allow the consumer to select stored electronic books and electronic books
from catalogues, move
28 a cursor, and turn pages in an electronic book. Typically, the controls
include forward and reverse
29 page buttons, a ball for cursor movement, one or more selection buttons, a
current book button and
a bookmark button.
WO 01/18665 CA 02382650 2002-02-22 PCTIUSOO/24819
14
1 (5) Book memory 728 for at least one electronic book or other data is
included in the
2 viewer 266. The memory 728 stores text and any graphics, which represent
pictures in a book.
3 The memory 728 can also store menu graphics data. Two different memory 728
devices may be
4 used in the viewer 266, one for the instructions for the microprocessor 621
in the digital circuitry
and a second type of memory may be used for the book memory 728 (and
graphics). Various
6 memory devices available on the market may be used such as, ROM, RAM or a
small hard disk.
7 Since an electronic book requires approximately 0.6 megabytes of storage, a
small hard disk
8 providing approximately 60 MBytes of storage provides memory to store
approximately 100
9 electronic books.
Text for electronic books may be displayed in various font sizes. To
accommodate various
11 fonts for display, a variety of fonts are stored in instruction 732 or book
memory 728. Thus larger
12 or smaller fonts may be recalled from memory 621, 728 to create displays
desired by the
13 subscriber.
14 (6) Power supply circuitry 736 in the viewer 266 accepts power from either
an AC power
source or from an optional battery 603' or the library 262. The power supply
circuitry 736
16 provides the necessary voltages to accommodate the various systems within
the viewer 266.
17 (7) An optional battery 603' is provided in one embodiment. The battery
603' is
18 automatically recharged when AC power is available.
19 (8) An optional RF transceiver 604 which provides a two-way data link
between the
viewer 266 and other components of the home subsystem can also be included in
the viewer 266.
21 (9) The optional alternative communication interface devices allow the
viewer 266 to make
22 use of a variety of communication paths. For instance, the viewer 266 may
include a cellular or
23 PCN transceiver 611 for mobile communications.
24 (10) The optional wired (attached) keyboard 267 and wireless (e.g., RF)
keyboard 268
(see Figure 2) may be used with the viewer 266 to provide communications
between the subscriber
26 and the viewer 266.
27 (11) The optional speaker and microphone 608' allow the viewer 266 to
provide audio
28 signals to the subscriber, and allow the subscriber to provide an audio
input. The speaker and
29 microphone 608' may be used in conjunction with the cellular transceiver
608 or other
WO 01/18665 CA 02382650 2002-02-22 PCTIUSOO/24819
1 telecommunications equipment to provide for reception and transmission of
telephony and
2 data.
3 The viewer 266 of Figure 5 has parts available for providing connections to:
a library 744,
4 electronic card memory 748, CD ROM units 752, and a portable memory unit 756
(such as that
5 shown in Figure 3 as 600'). Various electronic memory cards such as PCMCIA
can be used with
6 the viewer 266 to supply and store electronic books.
7 Security, low power consumption and excellent display technology are desired
features of
8 the viewer 266 design. The viewer 266 should be lightweight and portable.
The viewer 266
9 contains a software operating system that allows electronic books to be
stored, read and erased
10 and includes the capability to order electronic books and retain them in
memory 728 for a
11 predefined period of time determined by the system operator. The software
can be configured to
12 allow the electronic book to be read during a period of time (i.e., two
weeks) and then
13 automatically erased, read once and erased, or held in memory permanently.
Each viewer 266 may
14 have a unique key 605. All of the data storage may be encrypted with the
key 605 for an individual
15 viewer 266 to prevent more than one viewer 266 accessing the text file or
electronic book file.
16 To support video conferencing, the viewer 266 can be augmented with
additional features,
17 as shown in Figure 5. The additional features can be integrated into an
enhanced electronic book
18 viewer 266 or provided as one or more upgrades to a more basic electronic
book viewer 266. In
19 Figure 5, a camera 608" and an input device 740' are included in the viewer
266. The camera
608"is preferably a small video camera located and oriented on the viewer 266
so that it is typically
21 directed at the face, torso, or entire body of one viewing the display 602.
The direction of pointing
22 is manually adjustable by the user. Alternatively or additionally, the
camera 608" can be
23 electronically steerable, focusable and otherwise controllable.
24 The input device 740'may take many forms. The input device 740'performs the
function
of accepting user input in order to establish and manage a conference call,
e.g., entering party
26 identifiers, hanging up, etc. Exemplary forms of the input device 740'
include a telephone style
27 number key pad; cursor/pointer movement device, such as a track ball,
joystick or touch pad; click-
28 type buttons; and a wired or wireless interface to a separate keyboard.
Alternatively or
29 additionally, the display 602 and the input device 740' can be combined in
the form of a
WO 01/18665 CA 02382650 2002-02-22 PCT/US00/24819
16
1 touchscreen. The input device 740' may be a combinations of several
different forms, including the
2 illustrative ones just mentioned.
3 Conference calling also requires an upstream (or reverse path or
transmission)
4 communication capability of sufficient bandwidth. The upstream communication
capability for video
conferencing is in addition to the downstream communication capability by
which textual book data
6 can be received, and the upstream communication capability requires
significantly more bandwidth
7 than that minimally required for ordering or billing related to electronic
books.
8 To support conference calling, the instruction memory 732 of the viewer 266
preferably
9 contains programs for call establishment and management. The programs
provide menus and a
graphical user interface for call initiation, adding a party to a call,
accepting an incoming call,
11 terminating a call, and setting up call characteristics as well as display
and other output
12 characteristics.
13 Figure 6 is a block diagram of videoconferencing transmission and reception
functions of
14 the electronic book viewer 266. In regards to transmission, a video signal
from the camera 608"
is subjected to source encoding (i.e., conversion to spectrally efficient or
compressed digital form)
16 by a video source encoder 705, and an audio signal from the
speaker/microphone 608'is subjected
17 to source encoding by an audio source encoder 710. The source encoded video
and audio signals
18 are synchronized and multiplexed together by a synchronizer multiplexer
715. The video source
19 encoder 705, audio source encoder 710 and the synchronizer multiplexer 715
together can function
to perform standard audio/visual encoding algorithms such as MPEG (motion
picture expert
21 group) or ITU standard H.261 or H.263, which are well known in the art.
Other media signals or
22 signals containing signaling information can also be multiplexed with the
audio and video signals, as
23 described in greater detail below. The multiplexed audio/visual signal
output from the synchronizer
24 multiplexer 715 is optionally encrypted by an encryptor 720 and channel
encoded (e.g., forward
error correction encoding and /or interleaving) by a channel encoder 725
before being transmitted
26 by a transmitter 730 upstream into a video conferencing connection network.
Those skilled in the
27 art will appreciate that the transmission functions illustrated in Figure 6
can be implemented in an
28 order different from that illustrated.
29 In regards to reception, an incoming (i.e., downstream) video conferencing
signal is
received by a receiver 750, decoded by a channel decoder 755, decrypted by a
decryptor 760,
WO 01/18665 CA 02382650 2002-02-22 PCTIUSOO/24819
17
1 and demutliplexed with proper relative timing among video, audio and other
data components by
2 a demultiplexer synchronizer 765. Video signals are decompressed and output
for display by a
3 video source decoder 770. Audio signals are decompressed and output for
playing by an audio
4 source decoder 775. Again, the order of processing can be different from
that illustrated.
The other signals that can be multiplexed with the audio and video conference
call signals
6 include, for example, data files, electronic books, or HTML (hyper-text
markup language) based
7 content according to the ATVEF (advanced television enhancement forum)
specification. The
8 HTML content may be displayed on the viewer 266, may be stored in the home
system for later
9 use, or may be passed to devices such as televisions or computers attached
to the home system
components for immediate or later display. Broadcast data triggers,
announcements, and content
11 resources can be sent in the VBI of the video conference video signal on in
an auxiliary data channel
12 delivered with the video conference signal. Announcements are used to
indicate to the viewer 266
13 that interactive programming content is available for a given video
conference or segment of a video
14 conference. Content resources contain the actual content or programming to
be run on the
receiving device. Such content includes Web pages, scripts and images. The
broadcast data
16 trigger is used to synchronize the initiation of the interactive
programming content to the ongoing
17 video conference.
18 IV. Network for Video Conferencing
19 Figure 7 illustrates a video conferencing network 1000 and various
configurations for
connecting viewers 266 (or other video conferencing end equipment 1005) to the
network 1000.
21 The network 1000 provides full duplex interconnectivity between viewers 266
involved in a given
22 video conference call. The network 1000 can support many simultaneous
calls. One typical
23 configuration for connecting a viewer to the network 1000 is via a library,
such as shown in the
24 lower central area of Figure 7, where a viewer 266a is connected to a
library 262a, which in turn
is connected to the network 1000. A variation of this configuration is shown
in the upper left area
26 of Figure 7, where a set top terminal 601 is an intermediary between the
network 1000 and a
27 library 262b linked to a viewer 266b. Another typical connection
configuration is a direct
28 connection between the network 1000 and a viewer 266, such as the viewer
266c provided with
29 an optional communicator, or the viewer 266d, which is provided with
integrated library
functionality.
CA 02382650 2010-07-12
18
A. Mini-Networks
Figure 7 also illustrates a mini-network 1010. The mini-network 1010 can
provide full
duplex interconnectivity between viewers 266 in a limited area (e. g., within
a home, building, or
campus) so as to support video conference calls without involvement of the
network 1000. For
example, in the case of a video conference call between two viewers 266e and
266f within the mini-
network 1010, the connection between the viewers 266e and 266f can be a direct
connection or
it can be a connection through the library unit 262c. Figures 8 and 9 further
illustrate these two
connection options, respectively.
1. Direct Links Between Viewers
In Figure 8, the viewers 266e and 266f, as well as a third viewer 266g,
communicate with
each other without an intermediary. The communication paths between the
viewers 266e-266g
are preferably wireless links, and in one embodiment the wireless links are RF
links that utilize the
RF transceiver 604 in each of the viewers 266e-266g. Wireless infrared links
are also possible. In
alternate embodiments, the communication paths can be provided by wired
networks, such as an
existing data or computer network (e. g., LAN), existing television cabling,
or existing home/building
power wiring (as disclosed, for example, in U.S. Patent No. 5,319,634. In any
case, the channel
linking the viewers 266e-266g is multiplexed to accommodate six unidirectional
links that connect
the three viewers 266e-266g in a fully meshed network. Although one skilled in
the art will readily
appreciate that many multiplexing schemes are possible, including TDMA (time
division multiple
access) and FDMA (fi-equency division multiple access), the multiplexing
scheme preferably utilizes
CDMA (code division multiple access), also termed SSMA (spread spectrum
multiple access),
techniques. Advantages of CDMA in this instance include security of
transmissions and the fact that
FCC (Federal Communication Commission) regulations might not apply to these
transmissions.
In one embodiment, an asynchronous direct sequence CDMA system with a common
spreading code is utilized. Similar CDMA systems are well known and discussed,
for example, in
U.S. Patent No. 5,103,459 and U.S. Patent No. 5,546,381. In such a system, the
transceiver 604
of each of the viewers 266e-266g includes an omnidirectional or essentially
omnidirectional
antenna, so that each viewer 266e-266f can receive the transmission of the
other two viewers.
Each of the viewers 266e-266g encodes its
WO 01/18665 CA 02382650 2002-02-22 PCT/US00/24819
19
1 transmissions using the same m-sequence as a direct sequence spreading code
but at different
2 phases. For example, the viewer 266e may transmit at a relative code phase
of zero; the viewer
3 266f may transmit at a relative code phase of -3.2 chips; and the viewer
266g may transmit at a
4 relative code phase of +38.0 chips. Actually, each of the viewers 266e-266g
can transmit at a
random code phase while the receiver circuitry includes a search/selection
algorithm as well as
6 some contention protocol to resolve code phase collisions. The transceiver
604 of each of the
7 viewers 266e-266g in this embodiment is a CDMA receiver that suppresses self-
reception
8 interference, despreads the received signals at the various incoming codes
phases, tracks those
9 phases, and possibly mitigates multipath and/or interuser interference.
Because the mini-network 1010 of Figure 8 is fully meshed, it can operate
under
11 distributed control. Preferably, a receiver-oriented control protocol is
utilized. Each of the viewers
12 participating in a video conference call broadcasts its audio/video
signal(s), and any viewer that
13 receives the signal(s) performs the necessary processing to output the
audio and video components
14 in combination with those of other received signal(s). Preferably, the
video portions of the incoming
signals are scaled and displayed together in some arrangement on the display
602, while the audio
16 portions of the incoming signals are summed, synchronized to the video, if
necessary, so that a
17 summed, synchronized audio signal is sounded from the speaker/microphone
608'.
18 2. Using a Library Unit as a Central Node
19 In Figure 9, the library unit 262c is used as a central node in the mini-
network 1010, and
connections 1200, 1205, 1210 link the library unit 262c to the viewers 266e-
266g, respectively.
21 The connections 1200, 1205, 12 10 can be hardwired or wireless. Although
Figure 9 depicts the
22 connections 1200, 1205, 1210 in a star arrangement, other arrangements,
such as a ring, for
23 example, are possible. In cases where the connections 1200, 1205, 12 10
share a common medium
24 (e.g., a hardwired ring, or free space in the case of wireless
connections), then a multiplexing
scheme is necessary. Again, many well known multiplexing schemes are possible,
including TDMA
26 and FDMA, but the multiplexing scheme preferably utilizes CDMA techniques
for wireless
27 connections. In this case, the centralized node (i.e., the library unit
262c) allows a synchronous,
28 orthogonal CDMA system to be utilized.
29 Specifically, synchronous orthogonal CDMA spreading codes, such as Walsh-
Hadamard
codes, are employed on the uplinks from each of the viewers 266e-266g to the
library unit 262c;
CA 02382650 2010-07-12
meanwhile, a broadcast downlink is transmitted from the library unit 262c and
received by each
of the viewers 266e-266g. In one embodiment, the broadcast downlink is spread
using one of the
Walsh-Hadamard codes unused by the uplinks and includes control data for
uplink timing and
transmission power control feedback as well as call data comprising combined
audio/video signal (s)
from each viewer participating in the conference call.
In one embodiment, the library unit 262c provides centralized control of
communications
involved in a video conference call. Such centralized control may include call
set-up, audio
summing, and video combining. Alternatively, the library unit 262c can perform
only switching
and connectivity functions, while each of the viewers 266e-266g performs the
control functions,
just like in the case of a directly connected mini-network where there is no
library unit 262c.
The access control of the library unit 262c can be modified to accommodate
multiple viewer
connections during a conference call. During typical use for delivery of
electronic books, only a
single viewer holds the encryption key for the library unit 262c. This is done
in order to prevent
unauthorized distribution of electronic books. As depicted in Figure 9,
however, the library unit
262c may communicate with several viewers 266e-266g during a conference call.
This can be
accomplished by operating the library unit 262c in a "clear" mode during a
conference call. Clear
mode transmissions from the library unit 262c may be unencrypted or,
preferably, clear mode
transmissions from the library unit 262c utilize a common key to which all of
the viewers 266e-266g
have access.
B. Large Scale Network
The mini-network just described is capable of supporting calls between viewers
in a
relatively small geographical area, without involvement of the network 1000.
For calls of greater
range the network 1000 is necessary. National or worldwide connectivity is
possible using the
network 1000, which may take various forms, as described below, including
hybrids of the
following.
1. Cable Television Backbone
In one embodiment, the network 1000 utilizes a cable television transmission
network. A
cable television distribution network is described in U.S. Patent No.
5,559,549. Generally, a cable
television transmission network comprises headends, each of which serves a
number of subscribers.
The connection between a headend and a
CA 02382650 2010-07-12
21
subscriber may be metallic coaxial cable, optical fiber, or a hybrid, e. g.,
hybrid fiber-coax (HFC)
systems. The most common cable networks today are HFC networks, but 100% fiber
optic
networks, i.e., fiber to the curb (FTTC), will be more prevalent in the
future.
To support conference calling, the cable television network and the home
interface to the
cable network must support two-way communications. This can be accomplished by
having
separate physical fibers/cables or directional multiplexing on the same
fibers/cables or some
combination of both. For example, the frequency range between 5 MHZ and 42 MHZ
is allocated
for upstream (also called "uplink") transmissions (i.e., from a subscriber to
a headend). In a typical
HFC layout, there are separate optical fibers for upstream and downstream
transmissions between
a headend and each of several neighborhood fiber nodes, where signals are
converted between
optical and electrical forms. However, upstream and downstream transmissions
are multiplexed
together onto the same coaxial cable that links a fiber node to subscriber
sites (e.g., homes, offices).
Bidirectional amplifiers employing diplexers are typically utilized in the
coaxial portion between a
fiber node and subscriber sites. Further details of a bidirectional cable
television system are
contained in U.S. Patent No. 7,590,993, filed July 29,1998, entitled METHOD
AND APPARATUS
FOR USING PROGRAMS WATCHED DATA. An alternative use of the cable plant is via
cable
modem pairs between a viewer or library unit and a headend.
Further upstream from the cable headends are one or more operations centers.
The general
downstream functionality of an operations center is collecting programs and/or
data, packaging the
collected programs and/or data, and transmitting the packaged programs and/or
data to headends.
The link between an operations center and a headend may involve fixed
terrestrial medium (e.g.,
cable/fiber), terrestrial microwave, satellite, or some combination of
channels, although satellite
channels are the most common presently. The link between a headend and an
operations center
may be bidirectional to support duplex calling between electronic viewers.
Operations centers may
be linked together to provide support over an even larger geographical area.
In direct broadcast satellite (DBS) systems, the headend is absent or
bypassed, and the
operations center communicates directly with the subscriber via satellite. To
the extent that a DBS
satellite link is bidirectional with sufficient upstream bandwidth, a DBS
system may also be utilized
to support calling between electronic viewers. As an alternative to the DBS
system, a separate,
WO 01/18665 CA 02382650 2002-02-22 PCT/US00/24819
22
1 dedicated or proprietary system of satellites may be utilized with the
present invention to provide
2 a satellite backbone for the network 1000. Alternatively, diverse
communication channels, such
3 as telephony or the Internet, can be used to provide upstream communications
in conjunction with
4 a downstream satellite transmission. For example, a downstream satellite
broadcast of audio and
video from a teacher or professor along with upstream audio only transmissions
from students via
6 a diverse communication channel is well suited to distance learning
applications using the conference
7 calling capabilities of the present invention.
8 The extent to which bidirectional communication capability is present in the
cable television
9 system determines the support for calling between electronic viewers 266. In
one embodiment, a
central node within the network 1000 performs signal processing and
coordination to establish and
11 manage a multiparty video conference call, as described in greater detail
below. A video
12 conferencing central node may be combined or collocated with other
equipment in the cable
13 television network. For example, a central node may be associated with an
operations center to
14 provide national or regional coverage. A smaller scope of coverage may be
supported by a central
node associated with a headend. An even smaller scope of coverage may be
supported by a
16 central node associated with a fiber node. Continuing one step further, the
mini-network, as
17 described above, results when a central node is associated with a library
unit.
18 2. Cellular Telephony Backbone
19 The cellular telephony system is a bidirectional communication network that
can be utilized
for calls between electronic book viewers 266 with cellular transceivers.
Analog and digital cellular
21 telephony systems are well known in the art and easily support two-party
conference calls without
22 video. Digital cellular systems are preferred for use with the present
invention, because they offer
23 greater bandwidth, which is necessary for transmission of video. A central
node for video
24 conferencing in a cellular network may be a cellular base station (i.e., at
the center of a cell) or
mobile switching station, which links many base stations together.
26 3. PCS Backbone
27 A wireless PCS (personal communication system) is another bidirectional
communication
28 network that is well known in the art and can be utilized for calls between
electronic book viewers
29 having PCS transceivers. The PCS may be, for example, a PCN.
WO 01/18665 CA 02382650 2002-02-22 PCT/US00/24819
23
1 4. PSTN Backbone
2 The PSTN (public switched telephone network) can be utilized with the
present invention
3 to communicate video conferencing calls among viewers 266 having a PSTN
interface. The PSTN
4 reaches worldwide and encompasses land lines (both metallic and fiber
optic), terrestrial microwave
links, and satellite links, as well as a large number of switching centers and
exchanges. The PSTN
6 is well known in the art, and in particular, multiparty conference calling
in the PSTN is well known.
7 Furthermore, schemes for video transmission via the PSTN are known in the
art. For example,
8 U.S. Pat. No. 5,563,882 (the '882 patent), which is hereby incorporated by
reference, discloses
9 a video conferencing system that utilizes ISDN (integrated services digital
network) and the H.320
video telephone protocol. The'882 patent discloses a "multipoint control unit
(MCU)," which is
11 an example of a video conferencing central node, as used herein.
12 5. Hybrids
13 Hybrids of cable television networks, cellular telephony networks, PCNs,
the PSTN, and
14 other networks, such as the Internet, are possible. By way of example, some
cable television
networks presently provide interconnection to the PSTN and the Internet, so
that a television cable
16 can be the single communications conduit for a home or office. Further
integration and
17 interconnections to PCNs and/or cellular networks is technically
straightforward and likely in the
18 future.
19 V. Conferencing Processes and Methods
A. Call Establishment
21 A video conferencing call can be established in a variety of manners. One
or more channels
22 for combined audio, video, and possibly other data can be set up at the
initialization of the call.
23 Alternatively, video and possibly other data communication can be added to
an existing call that
24 initially carries less media or fewer media channels, such as an audio-only
call, for example.
In any case, channels for the various media are typically requested by one of
the call
26 participants. Requests are communicated to a video conferencing central
node via a signaling
27 network that maybe inband or out-of-band with respect to the media content
of the call. In a
28 circuit switched or connection oriented network, a request is processed by
one or more video
29 conferencing central node(s) so as to route the requested media channels,
reserve adequate switch
ports and/or configure various switches between the call participants as
necessary to support the
CA 02382650 2010-07-12
24
requested media. For example, a request for video communication midstream in
an audio-only call
that is already established would prompt the video conferencing central node
to configure its video
switch to establish one or more connections of sufficient bandwidth to
accommodate duplex video
transmissions and possibly perform other processing to mark, join, or
synchronize the video
transmissions to the pre-existing audio communications. Further examples of
requests schemes in
a video conferencing system are disclosed in the `882 patent and U.S. Patent
No. 5,896,128 (the
128 patent). In a packet switched or connectionless network, requests as just
described may not be
necessary. In an asynchronous transfer mode (ATM) network, which has some
qualities of both
circuit switching and packet switching, use of requests would be appropriately
modified in
accordance with well known understanding in the art.
It is also possible to add a party to an ongoing video conferencing call. For
example, a
point-to-point two-party call can be converted to a bridged three-party call.
A technique for doing
so without disruption to the call is disclosed in the `882 patent.
A particular participant may select which, if any, other participants of a
conference call will
be able to receive all or some of its signals. This selection may be made
during initial call
establishment or "on the fly." For example, if a party A to a three-way
conference call among
parties A-C wishes to disclose to party B something secretly from party C,
party A may temporarily
turn off the ability of party C to hear or view the signals of party A.
B. Viewer Signal Processing and Functions
A viewer 266 performs signal processing associated with the transmission and
reception
of call content. In regards to transmission, the audio signal sensed by the
speaker/microphone 608'
is converted to digital form and compressed. For voice or speech audio, the
following families of
algorithms are particularly suitable: pulse code modulation (PCM), delta
modulation (DM) and
linear predictive coding (LPC). Each of the above families of algorithms
includes variations, such
as differential and adaptive variations. Similarly, the video signal sensed by
the camera 608" is
converted to digital form and compressed. The fundamental techniques for image
compression
include vector quantization, discrete cosine transforms, and run-length
encoding. Combinations of
these techniques are also possible. For video sequences of images, interframe
encoding based on
motion prediction can be applied to provide further compression gains. These
techniques as well as
others are utilized in standard video compression algorithms, including low-
rate MPEG and ITU
CA 02382650 2010-07-12
standard H.261 and H.263 for video conferencing, which are well known in the
art. Any
compression algorithm that produces acceptably low bit rates for video and/or
audio transmission
through the network and can be implemented in real time with acceptably small
delay is suitable
for use with the present invention. Encoding of video and audio may be
distinct or interrelated.
By way of example, MPEG is a combined standard for both video and audio. These
and other
features are disclosed in additional detail in the `128 patent.
In regards to reception, a viewer 266 executes a decompression algorithm
corresponding
to the compression algorithm. In addition, the viewer 266 may perform
additional processing of
the received signals. For example, the viewer 266 can deselect some or all
other conference call
participants for output on that particular viewer 266, scale video images,
overlap video images, and
otherwise customize the display and output characteristics. Furthermore, the
viewer 266 may be
programmed to automatically select the dominant speaker on the basis of the
audio components of
the conference call and enlarge the size and/or display resolution of the
corresponding video output.
These and other reception signal processing features are disclosed in the `
128 patent cited above
as well as U.S. Patent No. 5,801,756.
Optionally, the video conferencing system can support more sophisticated
remote
participant controls of video and camera functions. For example, the viewer
camera 608" may be
electronically controllable (e. g., electronically steerable, focusing, zoom,
pan, etc.) locally or
remotely, and one or more other participating viewers 266 can command these
electronically
controllable aspects of the camera 608" to customize and dynamically alter the
characteristics of
the video image sensed by the camera 608". Alternatively, the same "camera"
video effects may
be achieved using signal processing techniques. Camera commands and related
video manipulation
commands can be communicated from the remote (controlling) viewer to the
subject (controlled)
viewer by a low rate signaling channel.
Other conferencing display controls are possible, including, for example: (1)
round robin
display in a predetermined sequence; (2) visual token passing, wherein
everybody can see the
participant who holds the token; (3) dominant voice activation, wherein
whoever has the loudest
audio signal is displayed; and (4) "Hollywood Squares" style display of all
participants jointly in an
arrangement such as a checkerboard.
CA 02382650 2010-07-12
26
Additional output manipulations are possible for non-video media. Examples of
non-video
media (besides audio) include electronic books, data files, documents,
spreadsheets, graphics,
programs, text streams, web pages, and interactive whiteboards. Each of the
above can be
contained within a window on the display 602 and sized, positioned and layered
to suit the user's
preferences. Alternately, each of the above can be provided as a picture-in-
picture. According to
one process of the present invention, a new object for display, when it is
received at a viewer 266,
is initially displayed as a small icon. When the user provides input to open
the icon (e.g., by
clicking it), the icon transforms into a window of default size and position.
The user can then
adjust the display characteristics of the window. In the case of an
interactive whiteboard, each
participant in the conference call may view and potentially edit the contents
of the whiteboard,
which is preferably a window on which simple drawings, graphics, and text may
be entered. Each
call participant sees the same version of the whiteboard, which is globally
updated periodically or
as necessary.
A receiving viewer 266 with sufficient memory can record a call in its
entirety or selected
components of a call, including one's own audio, video, or other components.
The ability to store
and later retrieve calls or call components is useful for record keeping,
recollection or message
delivery. The record function can be programmed to operate without a human
user present at the
receiving viewer 266. In this case, the record function provides the viewer
266 with the capability
of an answering machine or voice mail system with video and/or other media
embodiments. By way
of illustration of this capability, an unattended viewer 266 may receive a
video call from a calling
party. The unattended viewer 266 answers the call automatically and transmits
an audio/video
greeting to the calling party. In response to the greeting, the calling party
transmits an audio/video
message to the unattended viewer 266, which records the message.
A receiving viewer 266 can also convert an audio message to text in accordance
with
algorithms (e. g., voice recognition) that are well known in the art. The
resulting text file can be
displayed on the display 602 of the viewer 266 or stored in memory, such as
book memory 728,
within the viewer 266. This feature is useful to produce a transcript of a
call or to "listen" to a call
or message silently. The same speech to text conversion capability can be
utilized with outgoing
calls or messages as well. By way of example, a user may choose to transmit
her audio signal as a
text stream in place of or in addition to her audio signal. The viewer 266 can
also be programmed
CA 02382650 2010-07-12
27
to convert text to speech in accordance with well known algorithms. Such a
feature is useful for
users who have difficulty seeing or reading. Additional details of viewer
conversion between
speech and text are provided in the above-cited U.S. Patent No. 6,261,564.
C. Central Unit Functions
Some or all of the viewer functionality, as described in the preceding section
can be
performed by a video conferencing central node instead (or additionally). For
example, the record
and storage function can be easily, and in cases advantageously, located in a
central storage node
rather than distributed among viewers. As further examples, a central node can
perform speech-
text conversion and can store and manage an interactive whiteboard. Central
unit recording,
storage and management are desirable when the content of the video conference
call concerns a
lecture or presentation.
In a preferred embodiment, a video conferencing central node performs central
combining
for a conference call. Central combining simplifies the communication among
the viewers 266
participating in a conference call by establishing full duplex connections
having fixed or bounded
bandwidth, one between each participating viewer 266 and the central node. The
central node
manages all aspects of the call and is ultimately responsible for the
selection of information
received by each participating viewer on its single channel. The central node
receives all uplink
transmissions of video, audio and possibly other media. The central node then
makes various
combinations of the uplink data, one combination being transmitted to each
participating viewer
266. Each participating viewer 266 can customize its downlink combination and
output display, as
described above by communicating configurations commands to the central node,
which effects the
requested configuration. Further details of central combining for video
conferencing is disclosed in
the `128 patent already cited and U.S. Patent No. 5,657,096.
Figure 10 illustrates central combining for a conference call among the three
viewers 266a,
266b and 266c. A node 1025 is the central node for this call. Nodes 1030 and
1035 are utilized
for connection of the viewer 266a. In this example, the network 1000 initially
determines that the
node 1025 would act as the central node. Upon establishment of the conference
call, signaling
within the network 1000 occurs to set up the node 1025 as the central node and
to configure
switches at the nodes 1030 and 1035 so as to make the connections as shown, in
the case of circuit
switching. In the case of packet switching, the illustrated routes may be ones
or the most likely ones
WO 01/18665 CA 02382650 2002-02-22 PCT/USOO/24819
28
1 of many possible routes for given packets. Signals such as video, audio,
and/or other media
2 generated or originating at the viewer 266a are transmitted to the central
node 1025. Likewise,
3 each of the viewers 266b and 266c transmit uplink signals to the central
node 1025. In one
4 embodiment, each of the viewers 266a-266c also transmits via signaling
connections configuration
information pertaining to its display setup and other functions. Based on this
configuration
6 information, the central node 1025 performs combining of data from some or
all of the three uplinks
7 to create three customized downlink signals -- one for each of the viewers
266a-266c -- and
8 transmits the downlink signals along the paths shown. One skilled in the art
will appreciate that
9 uplink and downlink transmissions to/from a given viewer need not share the
same path physically
or logically.
11 In one embodiment, the central node 1025 performs audio combining.
Typically, the audio
12 uplink components from every other participating viewer 266 are summed to
create a sum or
13 combined audio signal, which is then transmitted to a participating viewer
as part or all of its
14 downlink transmission. Audio summation combining has the effect of allowing
every call participant
to hear all other participants, as occurs in a face to face conference.
Optionally, a viewer's own
16 uplink audio signal can be added to the sum downlink audio it receives,
thus providing a side tone.
17 The summation can be a weighted sum and the weighting can be altered
dynamically. For example,
18 relative audio weights, and thus amplification, can be in proportion to the
size of that speaker's
19 corresponding video image on a particular viewer screen. As another
example, the relative audio
weight of the dominant speaker at any time may be increased. The time
relationship between the
21 component audio signals and their corresponding video signals is maintained
and synchronized as
22 necessary. Those skilled in the all will appreciate these and many other
variations and features from
23 well known techniques of audio conferencing.
24 In an embodiment of the present invention, the central node 1025 performs
video
processing to combine various video components of a video conferencing call.
In the simplest case,
26 the central node 1025 switches video signals from one or more uplinks to a
given downlink without
27 processing the content of the video signals. In this case, each viewer
performs any video content
28 processing. In a more active role, the central node 1025 performs
substantive processing of the
29 uplink video signals to create a composite video signal for each
participating viewer 266. In this
more active role, the central node 1025 can select from the available uplink
video images (still or
WO 01/18665 CA 02382650 2002-02-22 PCT/US00/24819
29
1 moving) as well as other media, position and scale the images. Downward
scaling (shrinking) of
2 images may result in a reduction of resolution compared to the corresponding
video uplink signal.
3 Upward scaling (enlargement or magnification) of images may produce a
coarser image whose
4 resolution can be enhanced using well known techniques, such as, for
example, image interpolation
and edge enhancement. The composite image for each participating viewer 266 is
then encoded
6 for transmission. Standard digital encoding formats for moving video are
preferred, such as, for
7 example, an MPEG, H.261 or H.263 format.
8 Figure 11 is a block diagram of the video conferencing central node 1025.
The central
9 node 1025 contains circuitry to perform the functions just described,
including a video combiner
1410, audio summation combiner 1415, and a switch 1420. The central node 1025
also contains
11 a synchronizer 1425 to adjust relative timing between audio and video
signals as necessary and a
12 memory 1430 in which signals can be recorded for future retrieval and
playback. A signaling
13 module 1435 transmits and receives signaling information to/from viewers
and other nodes. Finally,
14 a controller 1440 controls the operation of the other circuitry. Those
skilled in the art will recognize
that other arrangements than the one illustrated in Figure 11 are possible and
that another node or
16 central node may contain less than all of the illustrated circuitry.
17 VI. Applications
18 The present invention provides synergistic effects from the combined
convenience of
19 viewing electronic books in association with participating in a video
conference call.
At the most basic level, the viewer 266 of the present invention provides the
advantages
21 and convenience of a single device that can perform the dual functions of
viewing electronic books
22 and participating in conference calls. The combination takes advantage of
common components,
23 such as the display 602 and communication capability, that can be utilized
for both functions. The
24 viewer 266 is simpler to transport and utilize than two or more separate
devices. Furthermore, the
portability of the viewer 266, especially when utilizing wireless
communications, enables it to be
26 used in a variety of settings, including at the home and office, while
traveling, even at a
27 beach.
28 Another example of the synergistic advantages of the present invention is
the ability to
29 initiate a conference call while viewing an electronic book. The electronic
book may be directly
pertinent to the conference call. For example, the electronic book may be a
directory or listing of
WO 01/18665 CA 02382650 2002-02-22 PCTIUSOO/24819
1 call connection numbers, addresses, or the like, such as a "White Pages",
"Yellow Pages" or a
2 personal contact booklet. While or soon after viewing the electronic book
directory on a viewer
3 266 to look up one or more parties to call, a conference call can be
established conveniently using
4 the same viewer 266. The viewer 266 can be programmed to mask the connection
number or
5 address from the user, so that a user need only deal with the called parties
name or "handle" that
6 is more familiar. In particular, a user can simply highlight, click or
otherwise select a name from the
7 directory, and the viewer will automatically capture the appropriate
connection information and
8 initiate the call. A directory can also be used in this manner to add a new
party to an existing call.
9 As another example, the electronic book may be a merchandise catalog. While
reading the
10 catalog, a user can initiate a call to a store or seller to order an item
in the catalog or to obtain
11 additional information. The electronic book catalog can be written to
accommodate easy and
12 automated calling in relation to the catalog contents. For example, a user
may highlight, click or
13 otherwise select an item in the catalog and press a single button (e.g., a
"dial" or "connect' 'button)
14 to place the call. In certain embodiments, the call may be a conference
call in which the selected
15 items are auctioned and the callers are live participants in the auction.
More generally, the
16 electronic book need not be a catalog, but can be any publication
containing an advertisement for
17 stores, merchandise or special information services (e.g., sports results,
weather information,
18 astrology, dealer locator services, or adult entertainment) that can be
called.
19 Other synergistic effects occur when the electronic book is not directly
pertinent to the
20 conference call. A user engaged in viewing an electronic book can initiate
or receive a call with
21 minimal disruptive effect because the call and the electronic book are seen
and/or heard on the
22 same viewer 266. Furthermore, the user can automatically mark an active
electronic book at the
23 current page or position when a call is initiated or received, to further
ease resumption of reading
24 the electronic book after conclusion of the call. A user participating in a
conference call that the
25 user finds boring or disruptive can inconspicuously and conveniently read
an electronic book on the
26 same viewer 266 while feigning interest in the conference call. When the
display 602 of the
27 electronic book viewer 266 is actually two separate screens, one screen can
display the electronic
28 book or text while the other screen can be utilized to display the video
conference. For incoming
29 calls, caller identification ("caller ID") can be performed in accordance
with well known techniques
30 in the art, and the caller identification can be conveniently displayed on
the display 602.
WO 01/18665 CA 02382650 2002-02-22 PCT/US00/24819
31
1 Perhaps most importantly, the present invention finds advantageous use in
distance learning
2 applications. Textbooks and lecture or presentation materials (e.g., an
outline or slide show of the
3 lecture or presentation) can take the form of electronic books, and
lectures, presentations or
4 discussions can be conducted as video conference calls. In one preferred
arrangement, video is
broadcast from the professor or presenter to all participants in the call
while audio only from all call
6 participants (including the professor or presenter) is combined so that all
can hear all others' audio,
7 though only the professor or presenter can be seen. In this way, questions
and answers can be
8 exchanged. In another preferred arrangement, a visual token passing scheme
is employed, wherein
9 the professor or presenter is the default token holder, but the professor or
presenter can pass the
token to a student or audience member so that everyone can sees the student or
audience member's
11 camera 608" video when, for example, the student or audience member asks a
question.
12 Interactive whiteboards are desirably incorporated into the conference call
to facilitate student
13 questions or input that is more naturally expressed non-verbally (such as
sketches or mathematical
14 formula). Text files, text banner, graphics files, video clips, computer
programs, and other data is
also advantageously incorporated into the conference call to aid in effective
communication and
16 learning. For example, the professor or presenter may play a video
recording or run a computer
17 simulation as part of a lecture or presentation.
18 The present invention provides many other benefits and advantages when used
in distance
19 learning applications. For example, students need only have a single,
lightweight viewer 266, rather
than a large number of bulky and heavy books. A viewer 266 with sufficient
memory capacity can
21 easily store all the books used by a busy student over the course of a
semester. As another
22 example, the speech-to-text capabilities of the present invention simplify
note-taking by students.
23 As a further example, the ability to store some or all components of a
distance learning conference
24 call lecture enables students who were "absent" from the lecture to easily
view the missed lecture.
Also, a student wishing to review a particular topic can easily access the
lecture covering the topic
26 for retrieval and replay, without having to meet a professor during office
hours just to have the
27 professor repeat what was said in the lecture. When a student does require
consultation with a
28 professor, the consultation can take the form of a conference call, and
relevant sections of a
29 textbook (e.g. an assigned homework problem) can be displayed and discussed
conveniently.
Professor-student consultations can also advantageously utilize the
whiteboards provided by the
W001/18665 CA 02382650 2002-02-22 PCT/USOO/24819
32
1 present invention. Schools and universities can utilize the present
invention to offer distance learning
2 classes to a very large number of students economically, without the
traditional physical constraints
3 of room size and capacity of physical facilities.
4 In short, the present invention can revolutionize campuses and the
educational process,
sparking a new renaissance of learning, much like Gutenberg's moveable typeset
printing did over
6 500 years ago.
7 The terms and descriptions used herein are set forth by way of illustration
only and are not
8 meant as limitations. Those skilled in the art will recognize that numerous
variations are possible
9 within the spirit and scope of the invention as defined in the following
claims and their equivalents.
