Note: Descriptions are shown in the official language in which they were submitted.
CA 02712620 2010-09-03
Multi-Platform Computing Device
Controllable by Heterogeneous Mobile Devices
Field of Invention
[00011 The invention provides a configuration for executing software
applications designed
for a mobile device, such as a mobile smart phone, on an application console,
controlled by a
control device, which may be another mobile smart phone of the same brand,
different brand,
or some other remote control device.
Background of Invention
[00021 Mobile games are software applications designed to entertain their
players and are
to be played on mobile devices, such as a mobile smart phone, personal
entertainment devices
or other suitable portable computing devices. Mobile games have grown rapidly
as part of a
growing trend of casual gaming. Mobile games currently enjoy the advantage of
a very large
installed base, estimated at over 2 billion users worldwide.
[00031 However, mobile games often meet only some simple entertainment
requirement of
a single user. The relative low computing power provided by a mobile device
such as a mobile
phone and its small form factor often limit how a mobile game can be played,
thus placing a
limit on how a mobile game can be enjoyed. For example, mobile games played on
a mobile
phone may not be as responsive as desired, or may not provide as exciting a
gaming experience
as would be possible with a game console, which generally has its own or can
be connected to
sophisticated output devices.
[00041 Due to the popularity of virtual social activity, more entertainment
features of
mobile games and the natural extension of the mobile phone's connectivity,
multi-player
mobile games are quickly finding an audience. However, in general, mobile
games are
designed to run on a particular brand of mobile smart phone and often are not
compatible with,
i.e., unable to run on, another brand of mobile smart phone, or even a
different model or
version of the mobile smart phone from the same manufacturer. This makes
playing multi-
player mobile games a challenging task. The limited computing power provided
by a mobile
device also tends to make playing a multi-user game on a mobile device not
satisfying.
22020001.3
CA 02712620 2010-09-03
-2-
[0005] Due to the limitations of mobile phone games, video game is still the
main form in
the game market where a game console is playing the key role. Nintendo, Sony,
and Microsoft
are three top contenders with their own popular consoles WiiTM, PlayStationTM,
and XboxTM,
respectively. Some console games are multi-player games. But they generally
require
controllers for the particular brand of console. For example, controllers
designed for Wii
generally are not compatible with controllers designed for Xbox, and vice
versa. Further, the
adoption of game consoles depends heavily on the content available for them
and add-on home
entertainment functions such as playing music and movies as well as connecting
to the Internet.
While it may be desirable to play mobile games on a game console, to take
advantage of its
ability to drive sophisticated output devices and its multi-player capability,
in general, mobile
games are not designed to run on any game console.
[0006] While mobile games are used as an example in the foregoing, it will be
appreciated
that the challenges, shortcomings and limitations discussed herein are not
limited to playing
mobile games or multi-player mobile games. One would find the same or similar
desires and
limitations with respect to a software application designed for mobile
computing devices or
using heterogeneous mobile devices to control the execution of a software
application in a
collaborative session, whether the software application is one designed for
one of the mobile
devices, for several different mobile devices, or for none of the mobile
devices.
[0007] The forgoing creates challenges and constraints for enabling
heterogeneous mobile
devices to control the execution of a software application in a collaborative
session, such as
playing a multi-player mobile game. It is an object of the present invention
to mitigate or
obviate at least one of the above mentioned disadvantages.
Summary of Invention
[0008] The present invention is directed to apparatus and methods of enabling
heterogeneous mobile devices to control the execution of a software
application in a
collaborative session or in a single user mode. A broad aspect of the present
invention involves
a configuration for executing software applications designed for a mobile
device, such as a
mobile smart phone, on an application console, controlled by a control device,
which may be
22020001.3
CA 02712620 2010-09-03
-3-
another mobile smart phone of the same brand, different brand, or some other
remote control
device.
[0009] In one practical application of the present invention, there is a game
unit that is
enabled to load any of multiple virtual machines for supported smart mobile
phone platforms,
through which games designed for different mobile phone platforms can be run
on a single
game unit. Multiple virtual machine images are stored in the game unit storage
media. When
the game to be played is selected, the corresponding virtual machine is
dynamically loaded into
the game unit memory so that the selected game is able to be executed. A
universal mobile
phone game control action set may be provided and installed on both the game
unit and all
participant mobile phones, through which any mobile phone can be used as a
controller of any
mobile phone game running on the game unit, even though the game may be
designed for a
totally different mobile phone platform. Through such a multi-virtual machine
dynamic loading
architecture and the universal mobile phone game control action set, there may
be configured a
universal multi-player game system for playing mobile phone games, including a
game unit and
multiple heterogeneous mobile phones acting as game controllers.
[0010] In a first aspect of the invention, there is provided a computing
device for executing
software applications designed for a mobile device. The mobile device has a
microprocessor,
one or more input devices and a set of action codes corresponding to user
actions sensed by
said one or more input devices. The computing device is configured to be
responsive to a
control device, the control device having one or more controller input devices
and a set of
controller action codes corresponding to user actions sensed by said one or
more controller
input devices, the computing device comprising a processor for executing a
kernel process;
communication circuitry controlled by the kernel process for establishing a
signal
communication channel connecting the control device to said computing device;
a storage unit
having one or more storage devices, said storage unit having stored thereon
one or more
software applications, said one or more software applications being executable
in an execution
environment native to said mobile device, an action code translator coupled to
the kernel
process and configured to map a controller action code received from said
control device to a
translated action code that is native to said mobile device, said kernel
process being configured
to be responsive to request received from said control device for executing a
software
22020001.3
CA 02712620 2010-09-03
-4-
application selected from said one or more software applications and, upon
said software
application being selected, determine an execution environment suitable for
said selected
software application and establishing said execution environment, execute said
selected
software application in said execution environment; and forward any translated
action code to
said selected software application.
[0011] In a feature of this aspect of the invention, the computing device
further comprises a
consolidated set of universal action codes, said set of action codes forming a
portion of said
consolidated set of universal action codes, wherein the action code translator
is configured to
map the action code to one of the universal action codes, which is then mapped
to the translated
action code. In another feature of this aspect of the invention, the action
code translator is
configured to cause an action code received from the computing device to be
transmitted to the
one or more control devices. In yet another feature of this aspect of the
invention, the kernel
process is configured to be responsive to request to add a second mobile
device to jointly
control the execution of said selected software application with said control
device.
[0012] In another feature of this aspect of the invention, said one or more
software
applications include a game designed for a mobile telephone handset, and said
software
application is said game. Optionally, said execution environment suitable for
said mobile
device is an operating system of said mobile telephone handset. As another
option, said game
is responsive to input received from any one of multiple mobile control
devices, thus enabling a
multi-player mobile phone game.
[0013] In another aspect of the invention, there is provided a control device
for controlling
execution of a software application on a computing device, said software
application being
designed for a mobile device, said control device having a microprocessor
which has a device
kernel executed thereon, a data storage unit, controller input devices and
communication
circuitry. The control device comprises a client application executing on said
microprocessor,
said client application comprising: a universal action code translator for
translating a controller
action code generated by said control device in response to user action
applied to one or more
controller input devices to a corresponding universal action code, a
communication unit
coupled to said universal action code translator for exchanging control
messages over a
communication channel between the control device and the computing device
established
22020001.3
CA 02712620 2010-09-03
-5-
through said communication circuitry, and an application console control unit
for
communicating with the computing device to direct selection and execution of
the software
application at the computing device.
[0014] In yet another aspect of the present invention, there is provided a
computer readable
storage medium comprising computer executable instructions, when executed by a
computing
device having a processor, causing said computing device to wait for user
connection request
from a control device, cause said computing device to establish data
communication with said
control device and provide user interface to said control device for user
selection of an action to
be performed by said computing device upon connection with said control
device, upon
receiving selection of a software application from said control device,
determine an execution
environment suitable for said selected software application and establish said
execution
environment on said computing device, execute said selected software
application in said
execution environment, and convert any user control signal received from said
control device to
an action code native to said selected software application and forward said
converted action
code to said selected software application.
[0015] In other aspects the invention provides various combinations and
subsets of the
aspects described above.
Brief Description of Drawings
[0016] For the purposes of description, but not of limitation, the foregoing
and other
aspects of the invention are explained in greater detail with reference to the
accompanying
drawings, in which:
[0017] Figure 1 illustrates a configuration for executing software
applications on a multi-
user controlled application console controlled by one or more users using
their respective
control devices;
[0018] Figure 2 is a block diagram illustrates several components of an
application console
that is used in the configuration shown in Figure 1;
22020001.3
CA 02712620 2010-09-03
-6-
[0019] Figure 3 is a block diagram illustrates two control devices, one
running in controller
mode and another in synchronous mode, connected to the application console
shown in Figure
2;
[0020] Figures 4A illustrates in a flow chart form steps performed by the
application
console of Figure 2 to execute a software application as requested and
controlled by a control
device;
[0021] Figures 4B illustrates in a flow chart form steps performed by the
application
console of Figure 2 in response to a user request to join a session already
started by a user;
[0022] Figure 4C illustrates in a flow chart form steps performed by the
application console
of Figure 2 to enable a control device to run in synchronous mode;
[0023] Figure 5 illustrates a game unit control console user interface on a
mobile phone;
[0024] Figure 6 illustrates in a flow chart form steps performed by a control
device
collaborating with the application console of Figure 2 to execute a software
application in
either controller mode or synchronous mode;
[0025] Figure 7 illustrates a block diagram of a universal multi-player game
system for
playing mobile phone games adapted from a configuration shown in Figure 1;
[0026] Figure 8 illustrates a flowchart for connecting the game client on a
player's mobile
phone to a game unit, playing a single-player or multi-player game, joining a
multi-player
game, uploading or downloading a game, and disconnecting from the game unit;
and
[0027] Figure 9 illustrates a flowchart for a game unit handling incoming
signals from a
mobile phone.
Detailed Description of Embodiments
[0028] The description which follows and the embodiments described therein are
provided
by way of illustration of an example, or examples, of particular embodiments
of the principles
of the present invention. These examples are provided for the purposes of
explanation, and not
limitation, of those principles and of the invention. In the description which
follows, like parts
are marked throughout the specification and the drawings with the same
respective reference
numerals.
22020001.3
CA 02712620 2010-09-03
-7-
[0029] The invention provides a configuration for executing software
applications designed
for a mobile device, such as a mobile smart phone, on an application console,
controlled by a
control device, which may be another mobile smart phone of the same brand,
different brand
adapted for this purpose, or some other remote control device.
[0030] By way of a general overview, Figure 1 illustrates a configuration for
executing
software applications on an application console controlled by one or more
users using their
respective control devices. The application console is a computing device 100,
controlled by
one or more users using their respective mobile devices 200, such as modified
mobile
telephone handsets 200a, 200b. The users 202 connect their respective mobile
phones 200a,
200b to the computing device 100 through signal connections 302, such as
wireless
connections 302a and 302b, respectively. If a mobile phone 200a or 200b is in
physical
vicinity of computing device 100, the wireless connection 302a or 302b can be
designed
according to but not limited to a Bluetooth or Wi-Fi specification. Signal
connection 302a or
302b may also be a wide-area-network ("WAN") connection. In general, signal
connections
302, for transmitting data or control signal, may be wired or wireless, or any
combination
thereof.
[0031] The computing device 100 is typically connected to various output
devices. For
example, video output may be sent through a video connection cable 304a or
304b to a video
output device, such as a video projector 306 or a PC monitor 308 as shown in
Figure 1. When
a video projector 306 is used, the video output is in turn projected on a
surface, such as a
projector screen 310. Other video output devices may include a television set
or a portal DVD
system that includes a display screen. There may be other types of output
devices. For
example, there may be audio output devices, such as speakers 312, a television
set, or a home
theater system, connected to the computing device 100 through audio cables 314
or a wireless
connection. There may also be vibration or lighting control devices connected
to computing
device 100 for generating movement and lighting effects of a game, among
others.
[0032] A user 202 uses control device 200 to communicate with the application
console
100 and select a software application to execute on the application console
100. A software
application generally is designed to run in its own native execution
environment and typically
cannot be executed in a non-native environment. For example, a game for a
smart phone is
22020001.3
CA 02712620 2010-09-03
-8-
often designed to run in the operating system of that particular brand of
smart phone and is
generally not compatible with, i.e., unable to execute in, any other operating
systems. Often, a
software application designed for a smart phone is not compatible with the
application console,
either. Once a software application, such as a game, is selected, the
application console 100
determines what execution environment is required or capable of supporting the
execution of
the selected software application. One example of such an execution
environment is the native
operating system for which the software application is designed. Another
example is a virtual
machine for execution on the application console to simulate the native
operating system.
When the required execution environment is determined, the application console
establishes the
required execution environment, for example, by loading a corresponding
virtual machine
image dynamically on demand. This establishes a native execution environment.
Non-native
but compatible execution environment may also be established to provide the
required
operating system support. Multiple virtual machines for major smart mobile
phone operating
systems may be stored on a storage device so that the application console 100
can support
multiple smart mobile phone operating systems. Upon the required execution
environment
being established, for example, after loading of a corresponding virtual
machine image, the
application console initiates the execution of the selected software
application in the native
execution environment. The user controls the execution of the software
application, such as a
phone game, using the mobile device 200, which communicates with the kernel of
the
application console. The kernel in turn forwards the user inputs to the loaded
virtual machine
and the software application.
[00331 In general, each mobile device 200, such as a smart phone, has one or
more input
devices. The input devices may be keypad, track ball or track wheel, motion
sensor, touch
screen, voice command system, among others. A collection of action codes
generally
correspond to (or are assigned to) user actions sensed by the one or more
input devices. A
smart phone's action codes are native to the smart phone's own native
execution environment
and any software application designed for its native execution environment.
Another control
device, with its own input device or devices, also has its own collection of
controller action
codes corresponding to user actions sensed by its input device or devices.
Unless the other
control device and the mobile smart phone are of the same brand and same
model, these two
collections of action codes usually are not the same. As will be described in
detail below, an
22020001.3
CA 02712620 2010-09-03
-9-
action code translator is provided to map an action code from one collection
to the other, i.e., to
obtain a translated action code. The action code translator bridges the
communication between
the mobile device and the software application executing on the application
console. This
allows a mobile device to be a control device whether a smart phone capable of
executing the
software application itself, or an incompatible smart phone, or merely a
remote control mobile
device, not a telephone handset.
[0034] Referring to Figure 2, the construction of an application console that
is suitable for
use in the exemplary configuration illustrated in Figure 1 is described in
more detail. An
application console is a computing device that includes both hardware
components and
software components executing thereon. Figure 2 shows an application console
100 that
includes, but not necessarily limited to, a processor 102, a memory device
104, a data storage
unit 106, signal/data communication port 108 or transceiver 110, and output
ports 112. A
console unit kernel 114, which can be any suitable operating system designed
for the
architecture of the processor 102 and the memory device 104, is loaded (or
partially loaded) to
the memory device 104 and executes on the processor 102. Communication
circuitry, together
with their controlling firmware or software modules, such as signal/data
communication port
108 or transceiver 110 establishes a communication channel 302 connecting a
control device
200 to the application console 100 so that the application console and the
control device can
exchange control signal messages or data over the communication channel 302.
Typically, the
operation of the communication circuitry is controlled by the console unit
kernel 114. As noted
earlier, the communication channel 302 may be wired, wireless or any
combination thereof.
For example, there may be separate signal transceivers and communication port
for supporting
both wired and wireless communication channels. Or, there may be provided only
a single
transceiver for both data and control signal communication. Storage unit 106
may have one or
more storage devices, for storing thereon copies of software code for
establishing any
supported native execution environments, such as code of supported virtual
machines or images
of virtual machines, and copies of one or more software applications
executable in each
supported native execution environments. Output ports 112 may include video
port 116, audio
port 118 or any other ports, for connecting the application console to any
supported output
devices, such as video display devices 306,308,310, audio devices 312, among
others.
Dedicated output modules 120, such as video module or audio module, may be
provided for
22020001.3
CA 02712620 2010-09-03
-10-
processing output signals sent over and driving those output ports.
Application console may
also have its own display screen 122 or console input devices 124.
[0035] The storage unit 106 may be logically divided into several units, if
desired. For
example, Figure 2 shows a storage unit 106 that includes a console unit
virtual machine
repository 126 for storing copies of data of supported execution environments,
such as stored
virtual machine images 128 of native operating systems, a software application
repository 130
for storing copies of supported software applications 132, and an action code
storage 134.
[0036] The console unit kernel 114 is configured to be responsive to request
received from
a mobile device 200 for connection and executing a software application
selected from the one
or more software applications stored on the storage device 106. A user
interface, such as a user
selectable menu, may be provided to the mobile device 200 for user selection.
Upon a software
application being selected, console unit kernel 114 determines, e.g., from
meta data associated
with the selected software application, an execution environment suitable for
(e.g., native to)
the selected software application and establishes the execution environment
(e.g., by loading
the corresponding virtual machine image from the console unit virtual machine
repository 126).
Console unit kernel 114 also causes the loaded virtual machine 136 to load the
selected
software application and execute the loaded software application 138 on top of
the loaded
virtual machine 136.
[0037] The console unit kernel 114 is configured so that the execution of the
loaded
software application 138 can be controlled by one or more control devices 200.
In one
implementation, the console unit kernel 114 includes an action code translator
140, which is in
data communication with the signal transceiver 110. Action code translator 140
maps control
data exchanged through the communication circuitry. Any action code received
via signal
transceiver 110 from a mobile control device 200 is translated, if required,
by the action code
translator 140, to an action code native to the loaded software application
138 (and the loaded
virtual machine 136). The action code translator 140 is also configured to
perform reverse
translation, when required, i.e., to translate an action code native to the
loaded software
application 138 to a corresponding action code of the control device 200
communicating with
the application console 100.
22020001.3
CA 02712620 2010-09-03
-11-
[0038] There may be further provided a consolidated collection of universal
action codes.
Any action code from a smart phone's collection of action codes is uniquely
mapped to a
universal action code in the consolidated collection of universal action
codes, and so is any
code from a control device's collection of controller action codes. The action
code translator
140 may be configured to map any action code from any supported collection to
a
corresponding universal action code and then map from the universal action
code to a translated
action code of the destination collection, i.e., the collection native to the
execution environment
of the software application. Action code storage 134 is utilized to store a
collection of
universal action codes 142 and mappings for converting between the collection
of universal
action codes and any set of action codes of supported mobile device 200. The
action code
translator 140 is also configured to provide the reverse conversion, i.e., to
map the translated
action code to the corresponding universal action code and from the universal
action code to
the original action code, using the collection of universal action codes 142
stored in action code
storage 134.
[0039] A control device may be adapted from a mobile smart phone, e.g., by
installing an
application client to modify it for use in the configuration of Figure 1. It
may also be any other
mobile control device suitably configured. Figure 3 illustrates two controller
devices 200,
200b connected to an application console. Some of the elements of application
console 100 are
removed in Figure 3 for clarity. A control device 200, or controller,
typically includes: (i) one
or more controller input devices 204 for sensing user action and generating a
controller action
code corresponding to sensed user action; (ii) a control device processor 206
that runs a
controller kernel 208, i.e., an execution environment native to the control
device (the
"controller execution environment"); (iii) communication circuitry, such as a
communication
port or controller signal transceiver 210 for establishing a signal
communication channel
connecting the control device 200 to the computing device 100; (vi) a
controller screen 212 for
displaying information to the user of the control device; and (v) an
application repository 214
for local storage. It will be appreciated that not all of these components
will always be found in
a single unit. Only those required for the configured purpose will need to be
present. For
example, it is not necessary to always include an application repository if
the control device is
not required to execute the application on itself.
22020001.3
CA 02712620 2010-09-03
-12-
[0040] The control device 200 includes an application client 216 executing in
the
controller execution environment. Such an application client 216 typically
includes a controller
action code translator 218 coupled to the controller signal transceiver 210
and controller's copy
220 of a collection of universal action codes, for mapping a controller action
code generated by
any of its input devices 204 to a corresponding universal action code. The
controller action
code translator 218 is also configured to perform reverse conversion, i.e., to
map a received
universal action code to a corresponding controller action code. The
application client 216 also
has an application host control console 222. The application host control
console 222 is
configured to enable the control device to establish connection with the
console unit kernel 114
so a user can select a software application for execution on the application
console, to upload
software applications to the application console, to download software
applications from the
application console, to enquire status of application console, among others.
[0041] Referring to Figure 3 again, if a selected software application is able
to run on a
control device 200b, a user has the option to run the selected software
application on the
control device 200b at the same time, having the execution state of the local
application
synchronized with the remote application executing on the application console.
This is referred
to running the application in "synchronous mode", as the execution states of
the local
application and the remote application are kept synchronized. The application
host control
console 222, after having requested a software application to be executed on
the application
console, also causes the controller kernel 208 to load the software
application from the local
application repository 214 and execute the loaded application locally on the
control device
200b. If the mobile device 200b does not have a locally stored copy of the
selected software
application, the application host control console downloads a copy to local
application
repository 214, from storage unit 106 of the application console. Likewise, if
the application
console does not have the selected software application, a copy is uploaded.
The application
client 216 is configured to enable the locally loaded application 224 to be
controlled by inputs
received from both local input devices 204 and action codes received from
controller
transceiver 210 and translated by the controller action code translator 218.
For example, if the
application console is connected to multiple control devices 200, the copy of
the application
executing remotely on the application console is controlled by the control
device 200b, and
other control devices 200. To synchronize the execution of the copy of the
application
22020001.3
CA 02712620 2010-09-03
-13-
executing remotely on the application console 100 and the copy of the
application executing
locally on the control device 200b, the application console 100 broadcasts
current state of the
remote application to all connected control devices 200 (or broadcasts the
command to alter the
current state), thus enabling the control device 200b to synchronize the state
of the locally
loaded application 224 in synchronous mode with that running on the
application console. For
any control device 200, this may be achieved, for example, by executing the
sequence of action
codes received by the application console and broadcasted to all control
devices, except those
control codes sent by the control device 200 itself. When running in
synchronous mode, any
visual output of the loaded application may be sent to controller screen 212
for displaying to
the user.
100421 Figures 4A illustrates in a flow chart form steps performed by the
application
console 100, for executing a software application on the application console
and controlling the
execution of the software application. In operation, to execute a software
application on
application console 100, in a collaboration session including multiple users
202, a first user
202a (as a "master user") requests connection (step 402) with application
console 100. The
master user 202a has a mobile control device. This may be one adapted from a
smart phone, by
installing thereon first the application client 216. In response to the user
request and further
interaction with the user, the application console 100 directs or causes the
control device 200a
to display a user interface (step 404), such as a window control 510, which is
displayed on a
mobile phone screen. Window control may be a menu, selectable by user 202a.
Figure 5
shows an example of such a window control, adapted for user to select and play
a phone game
on a game unit. Top level actions which can be selected from the menu 510
include Connect to
Game Unit 520, Start a Game 530, Join a Game 540, Upload a Game 550, Download
a Game
560, and Disconnect from Game Unit 570. When a top level action is selected,
the window
control 510 will be updated to display the next level actions that can be
executed for the
selected top level action. These selectable menu items correspond to steps of
action in the
process to be described in detail later. As will be appreciated, more action
items can be added
or some may be removed or modified to match functionalities actually
implemented.
[00431 As a master user, the user 202a may select a software application to
start, for
example, by selecting menu item Start a Game 530, but may not select Join a
Game as there is
22020001.3
CA 02712620 2010-09-03
-14-
no game to join at this time. When Start a Game 530 is selected, the
application console 100
causes the control device 200a to display a list of one or more software
applications, e.g.,
mobile phone games, for user selection. Upon receiving user selection of a
software
application (step 406) from the list of software applications, the application
console determines
the required execution environment, e.g., the native operating system of the
selected software
application (step 408), loads the virtual machine corresponding to the native
operating system
(step 410), and causes the selected software application to be loaded from its
software
application repository and executed in the virtual machine environment
provided (step 412).
[0044] After the software application is loaded, the application console waits
for user
control signal from the control device 200. Any user control signal, such as
action code
received from the control device 200a (step 414) is mapped to a corresponding
action code
native to the selected software application (step 416). This is done by the
action code translator
140 of the application console. A client action code translator executing on
the control device
200a may have already mapped a controller action code to a corresponding
universal action
code, in which case, the corresponding universal action code will be
translated, i.e., converted,
to the native action code. The translated action code is forwarded to the
software application to
control its execution (step 418).
[0045] Any subsequent users 202b may select to join the collaborative session,
for
example, by selecting the menu item Join a Game 540. This allows a multi-user
collaborative
session to be executed on the application console 100. This is illustrated in
Figure 4B in a flow
chart form. After a master control device 202a has started a software
application, a second
control device requests connection with the application console, which in
response, provides
the option for joining a collaborative session. Upon user selection of joining
the collaborative
session (step 420), the application console 100 optionally adds the subsequent
user 202b to a
pool of users participating in the control of the selected software
application (step 422). The
application console 100 may add the subsequent user 202b by adding an
identification of the
subsequent smart phone 200b to the pool, for example. When the application
console 100
receives an action code from the subsequent smart phone 200b (step 424), it
may optionally
first verify that the second smart phone is included in the pool of users
(step 426) and then
22020001.3
CA 02712620 2010-09-03
- 15-
maps the action code (step 428) and transmits the mapped action code (step
430) to the loaded
software application. Thus, the execution state of the software application is
advanced.
[0046] If any control device 200, whether a master control device 200a or a
subsequent
control device 200b, is able to execute the selected software application, for
example, being a
smart phone for which the loaded game is designed, a user can control and
execute a selected
software application in "synchronous mode". This may be achieved by
configuring the
application console to execute the following additional steps (Figure 4C)
after completion of
the step of determining the required execution environment (step 408):
(1) determine whether the controller device (master or second smart phone) is
enabled to execute the selected software application (step 440)
(2) if it is enabled, start a local copy of the software application on the
controller
device (step 442)
(3) if a user requests to join a multi-user session (step 444) after it is
already started
by the master user, send to the second phone all necessary data to advance the
application on the second smart phone to the current execution state (step
446) (e.g., by
sending to the second phone the sequence of all action codes received from all
controller devices since the start of the selected application on the console
unit) and
then wait for further user input to advance the execution state of the local
copy (step
448)
(4) each time when a new user input is received (step 414), in addition to
transmitting the corresponding action code to the selected software
application
executing on the console unit, also transmit the universal action code to the
controller
device (step 450), unless the universal action code is received from it
[0047] As briefly mentioned above, both an application console 100 and an
application
client 216 installed on a mobile telephone 200 can be optionally configured to
enable a user to
download a software application from the software application repository 130
of the application
console to the mobile telephone's local application repository 214 (by
selecting, for example,
Download a Game 560), upload a software application from the mobile phone's
local
application repository 214 to the software application repository 130 (by
selecting, for example,
22020001.3
CA 02712620 2010-09-03
-16-
Upload a Game 550), or pause/resume a software application to allow a mobile
telephone to
execute a task not related to the software application, such as answering or
making a telephone
call or reviewing or sending a text message.
[00481 Referring to Figure 6, steps performed by a mobile device in
collaboration with
application console in establishing a session and controlling the execution of
a software
application are described and summarized below. As already described, a mobile
device 200
(or its application host control console 222) first requests and establishes
connection with the
application console. Once connected, the application console 100 directs or
causes the mobile
device to display a user interface (see Figure 4A, step 404), such as a menu
500, for the user to
provide request to the application console. A user selects an action (step
602) to be performed
by the application console. A user may choose to start a session (step 604) by
selecting, for
example, Start a Game 530. The control device (or its controller kernel 208)
next determines
whether controller kernel is enabled to execute the user selected software
application (step
606). If it is enabled, a local copy of the selected software application is
loaded and executed
on the mobile device (step 608). In addition, whether the local kernel is
enabled or not, the
mobile device always request the application console 100 to continue the steps
from step 406 as
illustrated in Figure 4A to start a session of the selected software
application (step 610), i.e., to
load and execute the selected software application on the application console.
The mobile
device then waits for user input. When it receives a user control signal (step
612), in the nature
of an action code sensed by its controller input devices 204, the controller
checks again whether
the kernel is enabled to execute the software application locally. If it is,
the action code is
forwarded to the local copy of the software application to advance its
execution state (step
616). Meanwhile, the controller action code translator 218 translates, i.e.,
converts, the
controller action code to its corresponding universal action code (step 618)
and transmits the
translated universal action code to the application console (step 620) to
advance the execution
state of the console copy of the software application. If the kernel is not
enabled, the mobile
device simply translates the action code and forwards it to the application
console.
[00491 When a user is presented with a menu 500 at step 602, a user may also
choose to
join a session (step 622) already started by another user, by selecting, for
example, Join a Game
540. Again, the control device (or its controller kernel 208) next determines
whether controller
22020001.3
CA 02712620 2010-09-03
-17-
kernel is enabled to execute the user selected software application (step
624). If it is enabled, a
local copy of the selected software application is loaded and executed on the
mobile device
(step 626). When a user joins a session, the session executing on the
application console
usually has advanced to a state from the initial starting state. Therefore,
the local copy of the
software application needs to be advanced to the same execution state (step
628). Then the
mobile device waits for further user control signal (step 612) and will
continue in the same way
as in the scenario described above in which a user starts a session on both
the local mobile
device and the application console. If it is determined at step 624 that the
mobile device is not
enabled, no local copy of the software application will be started and the
mobile device will
simply start waiting for user input, i.e., wait to receive user control signal
(step 612) and
continue from there in the same manner as when a user selects to start a
session.
[0050] During a session, whether started by a single user or participated by
multiple users
who have joined the session, all mobile devices that execute in synchronous
mode, i.e., those
that are locally enabled to execute the selected software application (step
630), wait for
broadcast signal from the application console (step 632). As described above,
when a control
device sends an action code to the application console, the application
console 100 also sends
the action code to all mobile devices known to the application console to be
executing in
synchronous mode. Each of these mobile devices executing in synchronous mode
(or, its
controller translator 218) translates the action code received from the
application console to a
corresponding action code in the local set of action codes (step 634) so that
it can use the
corresponding local action code to advance the execution state of the local
copy 224 of the
software application (step 616). Of course, if the action code is originated
from a mobile
device executing in synchronous mode, the local copy 224 of the software
application has
already processed the local action code received directly from its own input
device 204. The
broadcasted action code will simply be ignored, i.e., not be acted on. As an
alterative to
broadcasting an action code to all mobile devices executing in synchronous
mode, the
application console may also broadcast the action code to all mobile devices
participating in a
session, whether in synchronous mode or controller mode, and let the mobile
device to decide
to translate the action code and act on it if it is executing in synchronous
mode and simply
ignore the broadcasted action code if in controller mode.
22020001.3
CA 02712620 2010-09-03
-18-
[00511 As will be appreciated, the configuration depicted in Figure 1 can be
adapted for
many different applications. As already described in great detail, the
configuration can be
adapted to execute on the application console 100 a software application that
is native to one of
the control devices, such as a mobile smart phone. However, the software
application to be
executed on the application console does not need to be native to any of the
mobile devices
200. It is also contemplated that the application console may be configured to
execute any
software application, which may be foreign to any mobile smart phone, or may
not even be
designed for a mobile smart phone. As long as a suitable virtual machine image
is provided for
enabling the software application to be executed therein, a user may use a
mobile control
device 200 to select to execute the software application, by causing the
application console 100
to load dynamically a virtual machine image for the selected software
application and execute
the selected software application in the loaded virtual machine environment.
[00521 As a special application, the configuration utilizing an application
console and
multiple control devices described herein may be adapted for providing a
universal multi-player
game system for playing mobile phone games. Figure 7 illustrates in greater
detail a block
diagram of such a universal multi-player game system 700 for playing mobile
phone games.
Such a system may be implemented from the configuration illustrated in Figure
1 by adapting
the application console 100 as a game unit 710 and mobile smart phones 740,770
as control
devices 200. Through multi-virtual machine dynamic loading architecture and a
universal
mobile phone game control action set, multiple players are able to join a
mobile phone game
hosted on the game unit using their respective heterogeneous mobile phones.
Although playing
a mobile phone game is described in this example for illustration, it will be
appreciated that the
description is generally applicable to executing a multi-user controlled
collaborative session of
a software application on an application console.
100531 Referring to Figure 7, the game unit 710 runs on its own kernel 712,
which can be
any common mobile phone operating system, e.g., AndroidTM, or a customized
game unit
kernel. On top of the kernel 712 are two components that enable the universal
multi-player
game system for playing mobile phone games: an action code translator 716 to
convert action
codes to and from a universal mobile phone game control action set and a
matching virtual
machine 714 for a loaded game 718. The virtual machine 714 is loaded
dynamically from the
22020001.3
CA 02712620 2010-09-03
-19-
virtual machine repository 722, in which a farm of virtual machine images 724
is kept.
Additional virtual machine images, when available, can be downloaded and saved
in the virtual
machine repository 722, therefore, new mobile phone platforms may be
supported.
[0054] Action code translator 716 is responsible for interpreting incoming
game control
signals received from mobile phones and broadcasting outgoing control signals
to mobile
phones, through a game control signal transmitter 720 for both directions. The
game control
action set is distinct and universal in that any action events originated from
any input devices or
combinations and from any mobile phones can be mapped to a uniquely defined
game action in
the set, thus enabling the controlling of a game using a different mobile
phone. Mobile phone
input devices include but not limited to classic keypad, touch screen,
pointing device, tracking
ball, scrolling device, touch pen, and motion detection sensor.
[0055] A virtual machine image 724 is loaded dynamically as required. When the
game
unit is in the standby mode, the game unit runs on its own kernel and no
virtual machine runs
on top of the kernel. Only when a game is selected to play, will the game unit
710 look up
meta data associated with the selected game to determine the target virtual
machine for the
game. Then, the target virtual machine 724 is loaded and initialized to run on
top of the game
unit kernel 712 to provide a virtual platform for the game to run. The game
unit kernel
communicates with controllers and acts as a bridge between controllers and the
virtual
machine, with suitable modifications and adaptations as necessary.
[0056] To play a game on the game unit 710, the game 718 is loaded from a game
repository 726 of the game unit 710 into the proper virtual machine 714. New
games may be
uploaded to the game repository 726 through a card reader 728, a wireless
network link 761
and 787 from mobile phones 740 and 770, a wired connection, or any other
suitable means.
Downloading games through Internet from a mobile phone game provider to the
game
repository 726 may be another alternative, not shown in Figure 7.
[0057] Game unit 710 is a computing device, on which any smart mobile phone
platform
may run as a virtual device. Game unit 710 may, but does not have to, include
a human
interactive input device or a screen. Whether the game unit includes its own
input devices and
display screen or not, it may receive input signals such as device control
signals or game action
codes through communication channels, such as a wireless network (755, 759,
781, and 785)
22020001.3
CA 02712620 2010-09-03
-20-
and generates output via a video output module 732. Generated video output
signal can be sent
to a projector 790 or a PC monitor (see Figure 1) through a video cable 733.
[0058] Also shown in Figure 7 are mobile phones 740 and 770 which participate
in the
universal multi-player game system for playing mobile phone games by
connecting to a game
unit 710. To enable a mobile phone to connect to a game unit 710, an instance
of game client
needs to be installed on the mobile phone. Figure 7 shows game clients 754 and
780 installed
on mobile phones 740 and 770, respectively. A game client 754 or 780 comprises
a universal
mobile phone game control action set 750 or 776 (which may be the same as and
is compatible
with the universal game control action set 716 installed on the game unit
710), a game control
signal transmitter 752 or 778, and a game unit control console 758 or 784.
Instances of
universal game control action set and game control signal transmitter running
on the game unit
and all mobile phones work together to enable a universal game control across
mobile phone
platform. A game unit control console 758 or 784 running on each mobile phone
740 or 770
sends device control signal to the game unit 710, checks the game unit status,
or receives other
information from the game unit.
[0059] A user having a mobile phone 740 or 770 can participate in a game
either in the
synchronous mode or controller mode, depending on whether the selected game
can run on the
specific mobile phone.
[0060] The mobile phone 740 participates in a game in the synchronous mode,
which
means when playing a game in the shown game system 700, regardless single-
player game or
multi-player game, the same game is run on both the mobile phone 740 and the
game unit 710
and state of two game instances is always synchronized. One way to achieve
game state
synchronization is to parse any game action originated from the mobile phone
740, either
triggered by a keypad 748 or other input device (not shown), and transmit the
action code
parsed by the universal game control action set 750 to the game unit 710
through the game
control signal transmitter 752 and wireless link 755. The transmitted game
action signal will be
received by the counterpart game control signal transmitter 720 on the game
unit 710 and then
passed to the universal game control action set 716 for interpretation,
thereafter, a valid game
action will be executed against the game instance 718 on the game unit 710 to
synchronize
game state so that the video output device is refreshed accordingly. At the
same time, the game
22020001.3
CA 02712620 2010-09-03
-21-
instance 744 running on the mobile phone 740 will take the same game action
initiated from the
local input device and reflect game state on a mobile phone screen 746. In
another direction, if
the game state on the game unit 710 is advanced due to a game action received
from another
mobile phone 770 participating in the game, the game unit 710 will broadcast
the received
game action to all mobile phones running in the synchronous mode through an
opposite
direction described above. As a result, state of the game instance running on
the mobile phone
740 is advanced and reflected on the mobile phone screen 746.
[0061] When a mobile phone 770 participates in a game in the controller mode,
no game
instance runs on the mobile phone 770. Rather, the mobile phone 770 is acting
as a remote
game controller for the game running on the game unit 710. When a keypad 774
or other input
device (not shown) triggers a game action, it is parsed by the universal game
control action set
776 and transmitted to the game unit 710 through the game control signal
transmitter 778 and
wireless link 781. The transmitted game action signal is handled by the game
unit the same
way as a game action signal received from a mobile phone 740 running in the
synchronous
mode. As a result, state of the game running on the game unit 710 is updated
according to the
original action initiated from the mobile phone 770 and reflected on the video
output device.
The game unit 710 does not send any control signal to a mobile phone 770
running in the
controller mode; alternatively, a mobile phone 770 running in the controller
mode can simply
ignore any game action control signal received from the game unit. However,
the game unit
710 still needs to broadcast any received game action, even if from a
controller mode mobile
phone 770, to all synchronous mode mobile phones, such as the mobile phone
740, for game
state synchronization.
[0062] Mobile phones so configured can be used to play a multi-player game all
in the
same mode (synchronous or controller mode) or some in synchronous mode and
others in
controller mode.
[0063] During a game play, if a mobile phone player needs to switch to another
task, such
as sending a text message, or handling some higher priority task, such as an
incoming call, but
does not want to quit the current game, the game client 754 or 780 on the
mobile phone 740 or
770 is programmed to be informed of such an event and then automatically send
a game pause
signal to the game unit 710 so that the current game instance on the game unit
710 can be
22020001.3
CA 02712620 2010-09-03
-22-
suspended, i.e., the execution of the game is paused. If the mobile phone is
running in the
synchronous mode, the local game instance will be suspended, too. Through the
game signal
broadcasting mechanism described above, the game unit broadcasts game pause
signal to all
game instances. When the other task is finished, the player can switch back to
the suspended
game, and then a game resume signal is sent to the game unit 710 so that the
suspended game
instances on the game unit 710 can be resumed. Again, the game unit 710 will
also broadcast a
game resume signal to all other games instances executing in synchronous mode.
In addition to
the automatic suspension and resumption of loaded game, a user may also
utilize a menu
provided (Figure 5) to manually suspend or resume the loaded game.
[00641 The game unit 710 is also configured to detect signal connection
between the game
unit and each of all connected mobile devices and the connection quality
thereof. During a
game play, if the connection between a mobile phone 740 or 770 and the game
unit 710 is
temporarily unavailable or of a quality deemed unacceptable as detected by the
game unit 710,
the game unit 710 can automatically put the game into suspension and broadcast
a game pause
signal to all game instances. When this happens, both the mobile phone that
lost the connection
and the game unit will try to reestablish the connection for a pre-determined
time period. The
game unit can be configured to resume all game instances if connection is
reestablished
successfully within this period or terminate the game if no connection is
reestablished within
this period.
[00651 The game unit 710 can also support sleep mode for energy saving. During
a game
play, if the game unit 710 has not received any signals from any connected
mobile phones for
some predefined period, it will switch itself into sleep mode with the game
suspended. If later
any connection becomes active again, the game unit 710 will "wake up" from the
sleep mode
and resume the suspended game instance.
[00661 At anytime during a game play, one mobile phone is designated as a
"master"
controller, which is responsible for starting a game and configuring the game
unit 710. Other
mobile phones act as "subordinate" controllers, which may join a game started
by a "master"
mobile phone. Any mobile phone can become the "master" controller as long as
it is the first
mobile phone connecting to the game unit 710. When there are multiple mobile
phones
connected to a game unit, the game unit may designate another mobile phone as
a "master"
22020001.3
CA 02712620 2010-09-03
-23-
phone by, for example, associating a "master" designation with the other
mobile phone's
identification label. In general, such a re-designation is initiated by a
request from a "master"
phone, though it may be configured to allow one of other connected mobile
phones to request
such a re-designation. This allows a "master" mobile phone to pass the
"master" controller role
to another mobile phone.
[0067] Games can be uploaded from a local game repository 742 or 772 on a
mobile phone
740 or 770 to the game unit 710 via communication connection such as wireless
connection
761 or 787. Game downloading from the game unit 710 to a mobile phone 740 or
770 can also
be similarly supported.
[0068] If the mobile phone is in the vicinity of the game unit, a mobile phone
740 or 770
may be able to connect to a game unit 710 through a short range wireless
network designed
according to but not limited to a Bluetooth or Wi-Fi specification, if so
equipped and
configured. If a mobile phone 740 or 770 is located at a place remote from the
game unit 710
at a greater distance, the mobile phone may connect to the game unit through a
WAN
connection, whether wired or wireless or in combination.
[0069] For security consideration, authentication of users may be implemented.
For
example, a simple pass code set on the game unit 710 or a registration and
logon method can be
required for a successful connection between a mobile phone 740 or 770 and the
game unit
710. More advanced security mechanisms may be adopted as well. Alternatively
or in
addition, different permission levels may be set up on the game unit 710 so
that each mobile
phone may be associated with a permission level, to enable more refined user
authentication
and access control.
[0070] Figure 8 shows a flowchart 800 that illustrates interaction processes
between a
mobile phone 740 or 770 and a game unit 710, including connecting the game
client on a
mobile phone to a game unit, playing a single-player or multi-player game,
joining a multi-
player game, uploading or downloading a game, and disconnecting from the game
unit.
[0071] A game client 754 or 780 is started from the mobile phone home screen
810.
Before a game client can be started, it must be installed on each mobile phone
as shown in step
812 and 814. In step 816, a player selects "Connect to Game Unit" after a game
client is
22020001.3
CA 02712620 2010-09-03
-24-
launched. If the connection to a game unit can be established, a game unit
control console user
interface will be displayed in step 818 and 830. Otherwise, the player would
be limited to
playing games on the mobile phone itself (step 820).
[00721 From the game unit control console as shown in step 830, the user is
able to initiate
one of the following processes: uploading a game, downloading a game, starting
a game,
joining a game, and disconnecting from a game unit. Steps 832, 842, 852, 878,
and 892
represent the entry points of these processes, respectively.
100731 The process of uploading a game is illustrated in steps 832, 834, and
836, in which a
selected game package is transmitted from a mobile phone local game repository
742 or 772 to
the game repository 726 of a game unit 710. A game uploader 756 or 782 reads
game package
from the local game repository 742 or 772 and transmits data over the wireless
link 761 or 787.
It may also append meta data for the uploaded game, including but not limited
to game title,
target platform and version number, and update time, so that the game can be
identified and
loaded properly. In order to play a game in a universal multi-player mobile
phone game
system, the game package is preferably first loaded in the game unit
repository 726.
[00741 The process of downloading a game is illustrated in steps 842, 844,
846, and 848, in
which a selected game package is transmitted from the game repository 726 of a
game unit 710
to a mobile phone local game repository 742 or 772. The game is assumed to be
compatible
with the target mobile phone platform though not necessary if only for
storage. For a mobile
phone 740 to play a game in the synchronous mode, the game is preferably
downloaded to the
local game repository 742 first.
[00751 The process of starting a game begins with steps 852 and 854. If the
game selected
in step 854 is compatible with the mobile phone platform, the game will be
played in the
synchronous mode, shown in steps 858 and 860; otherwise, the game will be
played in the
controller mode, shown in step 862. After that, the player is able to choose
playing a single-
player game or a multi-player game, given that the selected game supports
multi-player mode.
If the player chooses the multi-player mode, the player would have to wait for
other players to
join the game before the game can be started. Steps 864, 866, 868, and 870
illustrate this.
After the game is over, the game unit control console will be displayed.
22020001.3
CA 02712620 2010-09-03
-25-
[0076] The process of joining a game begins with step 878 when a player wants
to join a
game initiated by another player. Again, if the game is compatible with the
mobile phone
platform, the game will be played in the synchronous mode, shown in steps 882
and 884,
otherwise, the game will be played in the controller mode, shown in step 886.
After all players
have joined, the game will be started in step 888. In a multi-player game, the
mode in which a
mobile phone is running (synchronous or controller) is independent of the mode
in which other
mobile phones are running. In other word, a game player does not need to know
what mobile
phone platforms other players are using. After the game is finished, the game
unit control
console will be displayed.
[0077] The process of disconnecting from a game unit happens in steps 892,
894, and 896.
During the disconnection process, the connection between a mobile phone 740 or
770 and
game unit 710 is terminated.
[0078] Figure 9 shows a request handler 900 that illustrates how a game unit
710 handles
incoming requests from a mobile phone 740 or 770. The request handler 900
begins when a
request is received in step 905. The incoming request may be one of the
followings: a
connection request, a disconnection request, a game download request, a game
upload request,
a game start request, a game joining request, or a game control action. The
request type is
checked in steps 910, 920, 930, 940, 950, 970, and 980 to determine the flow
path to be taken.
Additional request types can be added to enhance the functionality of a
universal multi-player
game system for playing mobile phone games.
[0079] If the incoming request is a connection request, a connection between a
mobile
phone 740 or 770 and game unit 710 will be established in step 914. If this is
the first
connection from any mobile phones to the game unit, game unit 710 needs to be
activated from
standby mode in step 912 before the connection is established, in which some
preloading
process can be executed, including but not limited to loading the universal
game control action
set into memory.
[0080] If the incoming request is a disconnection request, the existing
connection between
a mobile phone 740 or 770 and the game unit 710 will be terminated in step
924. If the mobile
phone is a participant of the current game while a disconnection request is
received, the game
will be terminated first in step 922. If this is the last connection from any
mobile phones to the
22020001.3
CA 02712620 2010-09-03
-26-
game unit, the game unit 710 will be deactivated and go back to standby mode
in step 926 after
the connection is terminated, in which some cleanup process can be executed,
including but not
limited to unloading the universal game control action set from memory.
[0081] If the incoming request is a game download request, the game unit 710
will first
send a list of compatible games back to the mobile phone 740 or 770 in step
932. Then after
the player makes the selection, the selected game package will be sent to the
mobile phone via
a communication link in step 934. The downloaded game will be saved in the
mobile phone
local game repository.
[0082] If the incoming request is a game upload request, the game unit 710
will make itself
ready to receive the incoming game package over a communication link 761 or
787 in step 942.
The uploaded game package with its meta data, including but not limited to
game title, target
platform and version number, and update time, will be saved in the game
repository 726 in step
944.
[0083] If the incoming request is a game start request, the game unit 710 will
look up the
corresponding virtual machine for the selected game in the virtual machine
repository 722 and
load the virtual machine in step 952, then the selected game will be loaded
from the game
repository 726 and started in step 954. If the mobile phone initiating the
game is running in the
synchronous mode, the game unit 710 will send a synchronization command back
to the
requesting mobile phone to get the game started on the mobile phone and ensure
the game state
is in sync as shown in steps 956 and 958. Then the game control is given back
to the game
initiator, who will select single-player or multi-player mode, assuming that
the selected game
supports multiple players. If the player chooses multi-player mode, the player
would have to
wait for other players to join the game before the game can be started.
Otherwise, the game
will be started immediately in single-player mode. Steps 960, 962, 964, and
966 illustrate this
process.
[0084] If the incoming request is a game joining request, the game unit 710
will determine
whether the requesting mobile phone will run in the synchronous mode in step
972. If it is to
run in synchronous mode, the game unit 710 sends a synchronization command
back to the
requesting mobile phone to get the game started on the mobile phone and ensure
the game state
is in sync as shown in step 974. Then the game unit 710 sends a game joining
notice to the
22020001.3
CA 02712620 2011-10-25
-27-
game initiating mobile phone in step 976. After all players have joined, the
game initiator can
start the game and synchronize state of all game instances.
[0085] During a game play, game control actions are typically the most common
incoming
requests. After receiving a game control action, the universal game control
action set 716 will
interpret it and refresh state of the game running on the game unit 710
accordingly in step 982.
Then, if this is a multi-player game, the game unit 710 will broadcast the
game control action to
all participant mobile phones running in the synchronous mode in step 986,
through which state
of all game instances will be synchronized in step 988 relying on the
universal game control
action set installed on all mobile phones.
[0086] If the game unit 710 determines the incoming request is an invalid
request in step
990, it will simply ignore the request in step 992.
[0087] Various embodiments of the invention have now been described in detail.
Those
skilled in the art will appreciate that numerous modifications, adaptations
and variations may be
made to the embodiments without departing from the scope of the invention. The
scope of the
claims should not be limited by the embodiments set forth in the examples, but
should be given the
broadest interpretation consistent with the description as a whole.
22020001.4
