Note: Descriptions are shown in the official language in which they were submitted.
CA 02615877 2008-01-18
WO 2007/018377 PCT/KR2006/003074
Description
APPARATUS FOR PROVIDING MULTIPLE SCREENS AND
METHOD OF DYNAMICALLY CONFIGURING MULTIPLE
SCREENS
Technical Field
[1] Apparatuses and methods consistent with the present invention relate to
configuring multiple screens, and more particularly, to dynamically
configuring
multiple screens which provide multiple contents on a single physical display
device
and an apparatus for providing the multiple screens.
Background Art
[2] Related art broadcast receivers such as digital televisions (TVs) or
digital set-top
boxes provide only one content element on a single physical display device or
simul-
taneously display a main screen and a sub-screen on a single physical display
device.
[3] Even though related art broadcast receivers can simultaneously display
both the
main screen and the sub-screen on the same display screen, they can only
arrange the
main screen and the sub-screen in a limited number of manners. In the case of
a
content displayed within the main screen, all elements of the content, i.e.,
video data,
audio data, and other data, are displayed. On the other hand, in the case of a
content
displayed within the sub-screen, only some of the elements of the content are
displayed.
[4] Content sources include a broadcast service such as a satellite
broadcaster, a
terrestrial broadcaster, or a cable broadcaster, a storage medium such as
digital
versatile discs (DVDs), or an external device connected to an input terminal.
However,
it is quite difficult to display contents provided by such various content
sources on a
display screen using the existing broadcast receivers.
[5] In an interactive TV application environment such as the Multimedia Home
Platform (MHP), the Advanced Common Application (ACAP), and the Open Cable
Application Platform (OCAP), it is assumed that only one screen is output on a
physical display device.
[6] In the interactive TV application environment, for example, a Home
Audio/Video
Interoperability (HAVi)-based user interface (UI) is adopted. According to the
HAVi
UI standard, even though no restriction is imposed on the number of screens
displayed
on a physical display device, only one screen is generally displayed on a
physical
display device.
Disclosure of Invention
Technical Problem
2
WO 2007/018377 PCT/KR2006/003074
[7] In such an environment, it is difficult to perform operations, such as
decoding,
digital signal processing, user interaction processing, etc. with respect to
one among
multimedia contents displayed on a screen while displaying the multimedia
contents on
independent screens. In addition, it is also difficult to dynamically control
the life
cycles of application and the use of resources in the units of the screens.
[8] Accordingly, there exists a need for a method of displaying a variety of
contents on
a dynamically configured screen.
Technical Solution
[9] The present invention provides an apparatus and method for dynamically
configuring multiple screens which provide a plurality of contents on a
physical
display device.
[10] Further, the present invention provides an overlay screen that is
disposed at the
outmost side of the display screen.
[11] According to an aspect of the present invention, there is provided an
apparatus for
providing multiple screens, the apparatus including a digital signal
processing module
which receives video information, audio information, or data information and
restoring
a service based on the video information, the audio information or the data in-
formation, a service processing module generating a plurality of display
screens,
logical screens and an overlay screen for displaying the restored service, and
an output
module mapping the plurality of logical screens generated by the service
processing
module to different locations on a display screen and allowing the overlay
screen to
overlay the display screen.
[12] According to another aspect of the present invention, there is provided
an apparatus
for providing multiple screens, the apparatus including a service processing
module
generating a display screen, a logical screen, and an overlay screen
displaying a service
and associating the produced logical screen and the overlay screen with the
service,
and an output module mapping the produced logical screen to a certain area on
the
produced display screen and allowing the overlay screen to overlay the display
screen.
[13] According to still another aspect of the present invention, there is
provided an
apparatus for providing multiple screens, the apparatus including a digital
signal
processing module receiving video information, audio information, or data
information
and restoring a service, a service processing module generating a display
screen,
logical screens, and an overlay screen for displaying the restored service,
and an output
module displaying the logical screens generated by the service processing
module on
an arbitrary area of a display screen and disposing the overlay screen so as
to overlay
the display screen.
[14] According to a further aspect of the present invention, there is provided
a method
of dynamically configuring multiple screens, the method includes receiving
video in-
CA 02615877 2008-01-18
3
WO 2007/018377 PCT/KR2006/003074
formation, audio information, or data information and restoring a service,
generating a
display screen, logical screens and an overlay screen for displaying the
restored
service, mapping the logical screens to different locations on a display
screen, and
overlaying the overlay screen on the display screen.
[15] According to yet another aspect of the present invention there is
provided a method
of dynamically configuring multiple screens, the method includes generating a
display
screen, logical screens, and an overlay screen displaying a service, and
mapping the
logical screens to arbitrary areas on the display screen and overlaying the
overlay
screen on the display screen.
Description of Drawings
[16] The above and other aspects of the present invention will become more
apparent by
describing in detail exemplary embodiments thereof with reference to the
attached
drawings in which:
[17] FIGS. 1A to 1F are diagrams illustrating arrangements of a main screen
and a sub-
screen on a physical display device of according to a related art;
[18] FIG. 2 is a diagram illustrating the relationship between a logical
screen, an
overlay screen, and a display screen according to an exemplary embodiment of
the
present invention;
[19] FIGS. 3A to 3E are diagrams illustrating a configuration of a screen
including a
mapper according to an exemplary embodiment of the present invention;
[20] FIGS. 4A to 4C are diagrams illustrating a method of independently
overlaying
various graphic contents on a display screen according to an exemplary
embodiment of
the present invention;
[21] FIGS. 5A to 5C are diagrams illustrating a method of using a plurality of
graphic
surfaces for one overlay screen according to an exemplary embodiment of the
present
invention;
[22] FIGS. 6A and 6B are diagrams illustrating applied examples of an overlay
screen
according to an exemplary embodiment of the present invention;
[23] FIG. 7 is a block diagram illustrating service sources according to an
exemplary
embodiment of the present invention;
[24] FIGS. 8A and 8B are diagrams illustrating a non-abstract service and an
abstract
service according to an exemplary embodiment of the present invention;
[25] FIGS. 9A and 9B are diagrams illustrating attribute information and
interfaces of a
logical screen, and an overlay screen;
[26] FIG. 10 is a diagram illustrating an attribute 'z-order' of a logical
screen and an
overlay screen according to an exemplary embodiment of the present invention;
[27] FIGS. 11A and 11B are diagrams each illustrating an attribute
'Display_Area' of a
logical screen according to exemplary embodiments of the present invention;
CA 02615877 2008-01-18
CA 02615877 2008-01-18
4
WO 2007/018377 PCT/KR2006/003074
[28] FIG. 12 is a diagram illustrating a method of mapping three services to a
display
screen according to an exemplary embodiment of the present invention;
[29] FIG. 13 is a block diagram illustrating a configuration of an apparatus
for providing
multiple screens according to an exemplary embodiment of the present
invention;
[30] FIG. 14 is a flowchart illustrating a method of dynamically configuring
multiple
screens according to an exemplary embodiment of the present invention;
[31] FIG. 15 is a diagram illustrating a software architecture for providing
multiple
screens according to an exemplary embodiment of the present invention;
[32] FIG. 16 is a diagram illustrating the relationships among modules
constituting an
application programming interface (API) layer according to an exemplary
embodiment
of the present invention;
[33] FIG. 17 is a flowchart illustrating a method of displaying a plurality of
services that
are displayed on logical screens and an overlay screens on a display screen by
the
modules illustrated in FIG. 16 according to an exemplary embodiment of the
present
invention; and
[34] FIG. 18 is a flowchart illustrating a method of exchanging services
displayed on
logical screens between the modules illustrated in FIG. 16 according to an
exemplary
embodiment of the present invention.
Mode for Invention
[35] Advantages and features of the present invention and methods of
accomplishing
the same may be understood more readily by reference to the following detailed
de-
scription of exemplary embodiments and the accompanying drawings. The present
invention may, however, be embodied in many different forms and should not be
construed as being limited to the exemplary embodiments set forth herein.
Rather,
these exemplary embodiments are provided so that this disclosure will be
thorough and
complete and will fully convey the concept of the invention to those skilled
in the art,
and the present invention will only be defined by the appended claims. Like
reference
numerals refer to like elements throughout the specification.
[36] The present invention is described hereinafter with reference to
flowchart il-
lustrations of user interfaces, methods, and computer program products
according to
exemplary embodiments of the invention. It will be understood that each block
of the
flowchart illustrations, and combinations of blocks in the flowchart
illustrations, can be
implemented by computer program instructions. These computer program
instructions
can be provided to a processor of a general purpose computer, special purpose
computer, or other programmable data processing apparatus to produce a
machine,
such that the instructions, which are executed via the processor of the
computer or
other programmable data processing apparatus, create means for implementing
the
functions specified in the flowchart block or blocks.
5
WO 2007/018377 PCT/KR2006/003074
[37] These computer program instructions may also be stored in a computer
usable or
computer-readable memory that can direct a computer or other programmable data
processing apparatus to function in a particular manner, such that the
instructions
stored in the computer usable or computer-readable memory produce an article
of
manufacture including instruction means that implement the function specified
in the
flowchart block or blocks.
[38] The computer program instructions may also be loaded onto a computer or
other
programmable data processing apparatus to cause a series of operational steps
to be
performed on the computer or other programmable apparatus to produce a
computer
implemented process such that the instructions that are executed on the
computer or
other programmable apparatus provide steps for implementing the functions
specified
in the flowchart block or blocks.
[39] Each block of the flowchart illustrations may represent a module,
segment, or
portion of code, which comprises one or more executable instructions for im-
plementing the specified logical function(s). It should also be noted that in
some al-
ternative implementations, the functions noted in the blocks may occur out of
the
order. For example, two blocks illustrated in succession may in fact be
executed sub-
stantially concurrently or the blocks may sometimes be executed in the reverse
order,
depending upon the functionality involved.
[40] For a better understanding of the present invention, the terms used in
this disclosure
will now be defined.
[41] The term 'service' indicates a group of multimedia contents displayed
together, i.e.,
a group of service components.
[42] Service components are elements of a service and include a video
component, an
audio component, and a data component. A data component is an application in a
service.
[43] Further, the application is classified into an unbound application and a
service
bound application. Since the unbound application has a high priority, the
resource is
smoothly allocated. A monitor application corresponds to a specific unbound ap-
plication that has the highest priority.
[44] The service bound application is associated to a transport stream, and
allows a
lower priority than that of the unbound application. The service bound
application does
not function as a critical system. When a competition for resources occurs,
the service
bound application has a larger possibility of abandoning the resource
allocation than
the unbound application. The service bound application includes a conjunction
type
that operates in conjunction with a stream that is being transported and a non-
conjunction type that operates independently from the stream.
[45] The term'service context' indicates an object which can control the
executing of a
CA 02615877 2008-01-18
6
WO 2007/018377 PCT/KR2006/003074
service and includes various resources, devices, and execution state
information
needed for providing a service.
[46] The term 'physical display device' indicates a physical device which
actually
displays the content of a service, an output port which outputs the content of
a service
or a storage medium which stores a service.
[47] The term 'display screen' indicates a screen actually displayed on a
physical display
device. An arbitrary service may be directly set in the display screen, and
the display
screen may be displayed on a physical display device. Alternatively, at least
one
logical screen which is mapped to a certain area of the display screen may be
displayed
on the physical display device.
[48] The term 'logical screen' indicates a space in which an arbitrary service
is displayed.
A logical screen is a virtual screen before being mapped to a display screen
and thus is
not displayed on a physical display device.
[49] The logical screen and the display screen may be a combination of a
background
still image, a video raster, and a graphic raster. The graphic raster may be a
combination of text, lines, colors, and images or a mixture of video frames.
[501 The term 'overlay screen' indicates a space in which subtitles and
various functions
including volume selection and mute selection) are displayed and the overlay
screen
overlaps the display screen. In detail, the overlay screen is disposed at the
outmost side
of the display screen.
[51] A plurality of overlay screens can be displayed on the display screen to
overlap
different locations on the display screen.
[52] The overlay screens display data component of services transmitted from a
terrestrial broadcast, a cable broadcast, a personal video recorder (PVR).
[53] For example, the overlay screen stores the transmitted data component in
a storage
medium and then maps to the display screen.
[54] Further, the overlay screen overlays the display screen on the basis of
dis-
crimination information of the overlay screens or an attribute 'z-Order' that
determines
the order of arranging the overlay screens on a z-axis of the display screen.
In here, the
discrimination information of the overlay screen indicates that a screen is an
overlay
screen.
[55] For example, when the overlay screen is mapped to the display screen on
the basis
of the attribute 'z-Order', the value of the attribute 'z-Order' of the
overlay screen is
larger than those of logical screens that are applicable to be mapped to the
display
screen. Further, when the overlay screen is positioned on the display screen
on the
basis of the discrimination information, a screen having the discrimination
information
of the overlay screen is set to be absolutely positioned at the foremost of
the display
screen.
CA 02615877 2008-01-18
7
WO 2007/018377 PCT/KR2006/003074
[56] One or more overlay screens may be provided. In this case, the order of
the overlay
screens on the z-axis is determined on the basis of an attribute value of 'z-
Order' of the
overlay screen.
[57] The overlay screen is suitable for displaying the unbound application,
rather than
the service bound application, because the unbound application is easily
displayed on
the entire display screen regardless of the configuration of the display
screen.
[58] For example, it is easy for a user that an electronic program guide (EPG)
appli
cation that is a representative unbound application is displayed on the
foremost of the
screen on the z-axis regardless of the number of the logical screens on the
current
physical display device and the configuration.
[59] The term 'main service' indicates a service that is selected as a main
service
through a menu displayed on the physical display device or a remote controller
by a
user or through an API by an application, and the screen on which the main
service is
displayed is referred to as a'main screen'.
[60] The term 'Picture-in-Picture service' (PiP service) indicates a service
that is selected
as a sub-service in the main service through a menu displayed on a physical
display
device or a remote controller by a user via an API by an application, and the
PiP
service may be displayed on a picture-in-picture screen (PiP screen) or a main
screen.
[61] The PiP screen includes a screen that occupies a part of another screen
as illustrated
in FIGS. 1A to 1D and a screen that is simultaneously displayed with another
screen
without overlapping the other screen as illustrated in FIGS. 1E to 1F.
[62] FIG. 2 is a diagram illustrating the relationship between a logical
screen, an
overlay screen, and a display screen according to an exemplary embodiment of
the
present invention.
[63] Referring to FIG. 2, a service may be displayed using logical screens
210, 212, and
214, and an overlay screen 216. The logical screens 210, 212, and 214 are
mapped to
display screens 220, 222, and 224 through a mapping block 230. The overlay
screen
216 overlaps the entire display screens 220, 222, 224 separately from the
mapping
block 230. The overlay screen 216 must overlap a display screen to which a
plurality
of logical screens are mapped. Further, a plurality of overlay screens may
overlap the
display screen.
[64] In detail, the logical screens 210 and 212 and the overlay screen 216 are
mapped to
the display screen 220 and overlap the display screen 220, the logical screens
212 and
214 are mapped to the display screen 222, and the logical screens 210, 212,
and 214
and the overlay screen 216 are mapped to the display screen 224 and overlap
the
display screen 224.
[65] In short, at least one logical screen which displays a service is mapped
to an
arbitrary area of a display screen by the mapping block 230. The overlay
screen 216
CA 02615877 2008-01-18
8
WO 2007/018377 PCT/KR2006/003074
may overlap an arbitrary location of the display screen separately from the
mapping
block 230.
[66] The mapping block 230 is a group of various pieces of information needed
for
mapping a logical screen to a display screen. Examples of the various pieces
of in-
formation include coordinate information of a predetermined area on a display
screen
to which each of a plurality of logical screens is mapped, discrimination
information of
the logical screens and the display screen, and information specifying in what
order the
logical screens are displayed on the display screen.
[67] The mapping block 230 can change the size of the logical screen so to be
allocated
in an arbitrary area of the display screen. That is, the mapping block 230 can
perform
scaling of the logical screen and allocating of the position thereof, and
FIGS. 3A to 3E
are diagrams illustrating a configuration of the screen including a mapper as
the
mapping block.
[68] Referring to FIG. 3A, the main screen including a combination of a
background
still image B, a video raster V, and a graphic raster G is mapped to the
entire display
screen by a mapper with a normal size. The PiP screen including only video
components is mapped to the entire display screen by the mapper with a reduced
size.
In this case, the mapped PiP screen is displayed on the main screen, which is
determined depending on a Z value. The reference character Z refers to z-order
value
which will be described later. The overlay screen may be combined with the
display
screen. The overlay screen is a specific screen disposed at the outmost side,
and may
be used when providing a caption function. The PiP screen may have only a
video
component as illustrated in FIG. 3A, or may have a combination of the
background
still image B, the video raster V, and the graphic rater G as illustrated in
FIG. 3B.
[69] Referring to FIG. 3C, the main screen including the combination of the
background
still image B, the video raster V, and the graphic rater G is mapped to the
entire display
screen by the mapper with a normal size. Two PiP screens #1 and #2 having only
video
component is mapped to an arbitrary area of the display screen by the mapper
with a
reduced size. In this case, the mapped PiP screen is disposed on the main
screen and
the Z value can be constantly maintained. Further, the overlay screen may be
combined
with the display screen. The configuration of the screen may have a plurality
of PiP
screens including only video components as illustrated in FIG. 3C or a
plurality of PiP
screens including a combination of the background still image B, the video
raster V,
and the graphic rater G as shown in FIG. 3D.
[70] Picture-outside-picture (POP) screens are illustrated in FIG. 3E. It can
be
understood that the conventional PiP screen is displayed inside the main
screen and the
POP screen is displayed outside the main screen. Referring to FIG. 3E, the
plurality of
PiP screens #1 and #2 including a combination of the background still image B,
the
CA 02615877 2008-01-18
9
WO 2007/018377 PCT/KR2006/003074
video raster V, and the graphic rater G are mapped to arbitrary areas of the
display
screen by the mapper with a reduced size. In this case, the Z value of the
mapped POP
screens #1 and #2 may be constantly maintained. Further, the overlay screen
may be
combined with the display screen.
[71] The mapping block 230 may be realized by interfaces or functions prepared
by
various computer program languages to be executed and create or change the re-
lationship between the logical screen and the display screen by using the
above in-
formation as parameters.
[72] Also, the mapping block 230 may be realized by a hardware which has a
mapping
function between a logical screen and a display screen.
[73] Further, services provided by various service sources may be displayed on
a
display screen, and the display screen may be displayed on a physical display
device,
as illustrated in FIGS. 4A to 4C.
[74] FIGS. 4A to 4C are diagrams illustrating a method of independently
overlaying
various graphic contents on a display screen according to an exemplary
embodiment of
the present invention, that is, a method of overlaying a plurality of overlay
screens on
the display screen. In here, the graphic content of the service must be
directly
connected to the graphic surface included in the respective overlay screen.
[75] Referring to FIG. 4A, a main screen configured by a combination of a
background
still image B, a video raster V, and a graphic raster G is mapped to the
entire display
screen without changing the size. A PiP screen configured by a combination of
the
background still image B, the video raster V, and the graphic raster G is
mapped to an
arbitrary location of the display screen with a reduced size. In this case,
the mapped
PiP screen is disposed on the main screen, which is determined by a z-value.
The
character 'z' refers to a value of z-order. Further, overlay screens only
having graphic
component overlay the display screen. In this case, the z-value of the overlay
is
maximum, and thus the overlay screen overlaps an outermost side of the display
screen
screens. In this case, a mapper is not used for the overlay screens.
[76] Referring to FIG. 4B, a main screen configured by a combination of a
background
still image B, a video raster V, and a graphic raster G is mapped to the
entire display
screen without changing the size. A PiP screen configured by a combination of
the
background still image B, the video raster V, and the graphic raster G is
mapped to an
arbitrary location of the display screen with a reduced size. Further, two
overlay
screens #1, #2 only having graphic component overlay the display screen. In
this case,
z-values of the overlay screens may be same or different from each other.
[77] Referring to FIG. 4C, a main screen configured by a combination of a
background
still image B, a video raster V, and a graphic raster G is mapped to the
entire display
screen without changing the size. A PiP screen configured by a combination of
the
CA 02615877 2008-01-18
10
WO 2007/018377 PCT/KR2006/003074
background still image B, the video raster V, and the graphic raster G is
mapped to an
arbitrary location of the display screen with a reduced size. Further, a
plurality of
overlay screens #1, and #N only having graphic component overlay the display
screen.
In this case, z-values of the overlay screens may be same or different from
each other.
[78] FIGS. 5A to 5C are diagrams illustrating a method of using a plurality of
graphic
surfaces for one overlay screen according to an exemplary embodiment of the
present
invention. In this case, by associating a service context with one overlay
screen, it is
possible to display graphic contents of the service associated with the
overlay screen.
[79] FIG. 5A is a diagram illustrating a single plane overlay screen. The
single graphic
surface is used for closed captioning or for jointly using the closed
captioning and
another function (for example, volume selection, mute selection, a user
interface for
setting function). In this case, whenever some of the functions in the jointly
used
scenario changes, the entire contents must be latently renewed. This renewal
can be
reduced by updating subsets (for example, the subsets in the changed area) of
the
visual pixels.
[80] However, the latent renewal and these optimizing methods cause complex
coding
and increased debugging. Further, operation and memory cycles are increased.
In
contrast, a minimum hardware resource is needed in the setting. For example,
the
hardware compositor or multiple graphic surfaces are not needed.
[81] FIG. 5B illustrates a dual plane overlay screen. A compositor and a
second graphic
surface are added to separate the function from the other functions that are
jointly used
in a user interface for closed captioning and a second graphic surface (Gs).
The added
graphic surface is jointly used in an application environment including a
native en-
vironment embodying an equipment setting function. However, this is
independent of
the PiP and PoP setting. An unbound application may preferably be used to
display the
overlay screen separately from a graphic of the current service bound
application.
[82] When the multiple environment is actively displayed on the co-used
surface, the
co-operation is performed by requesting the renewal of the function by a
mutual
exclusion basis or for the displaying timing.
[83] As a result, the previous setting of the overlay screen is generalized by
providing a
distinct graphic surface for closed captioning and the respective application
en-
vironment.
[84] FIG. 5C illustrates N plane overlay screen. The abovementioned
generalization
provides the simplest system architecture during requesting a basic resource,
that is, an
exclusive graphic surface for the respective environment.
[85] The shape and the resolution of the overlay screen are expected to be
constantly
maintained and conformed to the shape of the entire display device.
[86] In the OCAP structure, the exclusive graphic surface in the overlay
screen is
CA 02615877 2008-01-18
11
WO 2007/018377 PCT/KR2006/003074
preferably used by the unbound application, not by the service bound
application. It is
because the exclusive graphic surface is independent of the PiP and POP
setting, and
the exclusive graphic surface can use the entire display graphic having a
constant
resolution separately from the graphic of the service bound application.
[87] For example, the above use supplies a visual mark for a user that makes
the ap-
plication of the screen be in focus for the shake of unreserved input event
dispatch,
permits outline drawing without any limitations by an application, and
provides a title
for the PiP and POP screen.
[88] FIGS. 6A and 6B are diagrams illustrating an applied example of an
overlay screen
according to an exemplary embodiment of the present invention.
[89] Referring to FIG. 6A, another example of the overlay screen can be used
to
indicate which screen displayed on the display screen is currently in focus.
In this case,
in focus screen refers a screen including an application that obtains an event
when a
user transfers an input event by using an interface such as a remote
controller.
[90] Referring to FIG. 6B, the overlay screen can be used to indicate a border
of the
screen or a title of the service displayed on the screen.
[91] FIG. 7 is a diagram illustrating examples of service sources according to
an
exemplary embodiment of the present invention
[92] There are service sources which provide broadcast services such as a
terrestrial
broadcaster 320 and a cable broadcaster 330, service sources which provide
services
stored in a storage medium such as a PVR 340, and service sources (not
illustrated in
FIG. 7) which provide services via a wired network or a wireless network.
[93] A broadcast receiver 310 receives services from the service sources and
produces
logical screens and overlay screens displaying each of the received services.
[94] Then, an arbitrary service is directly set on the display screen to be
displayed on a
physical display device using a predefined method or a method set by a user or
an ap-
plication. Otherwise, at least one logical screen that is mapped to an
arbitrary area on
the display screen and the overlay screen that overlaps the display screen is
displayed
on a physical display device 350. In short, services provided by the
terrestrial
broadcaster 320, the cable broadcaster 330, and the PVR are displayed on the
physical
display device 350.
[95] The terrestrial broadcaster 320, the cable broadcaster 330, and the PVR
340 are il-
lustrated in FIG. 7 as being service sources, but the present invention is not
limited to
it. Any type of multimedia content source which provides multimedia contents
that can
be displayed together can be a service source according to an exemplary
embodiment
of the present invention.
[96] Services according to an exemplary embodiment of the present invention
can be
classified into abstract services and non-abstract services, as illustrated in
FIGS. 8A
CA 02615877 2008-01-18
12
WO 2007/018377 PCT/KR2006/003074
and 8B.
[97] The abstract services are not services provided by broadcast signals
transmitted in
real time but services independent of broadcast channels. The abstract
services include
only data components, i.e., application, without video components and audio
components. Examples of the abstract services include services having unbound
ap-
plications based on the OCAP standard.
[98] The non-abstract services are understood as services other than abstract
services.
[99] According to the current exemplary embodiment of the present invention,
both
abstract services and non-abstract services are independent. For example,
abstract
services may be directly set on the physical display device not through
logical screens
and non-abstract services may be displayed on the logical screens and the
overlay
screen. Then, the logical screens may be mapped to the display screen in which
the
abstract services are set. Thereafter, the display screen may be output
through the
physical display device. By doing so, the abstract services can be displayed
on the
display screen independently of the non-abstract services. In addition, the
abstract
services and non-abstract services may be mapped to different logical screens.
Thereafter, the logical screens may be mapped to a single display screen. In
other
words, the abstract services can be displayed on the display screen
independently of
non-abstract services.
[100] According to the current exemplary embodiment of the present invention,
the
logical screen, the overlay screen, and the display screen may be categorized
as being
different objects. Alternatively, a screen may serve as a logic screen, an
overlay screen,
or a display screen according to attribute information of one screen object.
[101] In detail, whether a screen object is a logical screen or a display
screen or an
overlay screen depends on the 'type' information among screen object
attributes.
[102] Attribute information of the screen object includes a plurality of
attributes 'z-
Order', 'Display_Area', 'Visibility', 'Associated_Display_Screen',
'Associated_Service_Contexts', 'OutputPort' and 'Overlay_information'.
[103] FIG. 9A and 9B illustrates attribute information of a screen object and
interfaces
for processing the attributes of the screen.
[104] An attribute 'Type' 510 is for determining a screen type - a logical
screen or a
display screen.
[105] An attribute 'z-Order' 520 is for determining in an order in which a
plurality of
logical screens and overlay screens are arranged along the z-axis. The overlay
screen
may not include the attribute 'z-Order' 520. FIG. 10 illustrates different
configurations
of logical screens on a physical display device for different combinations of
the values
of attributes 'z-Order' of the logical screens and the overlay screen.
[106] Referring to FIG. 10, first and second logical screens 620 and 630 are
respectively
CA 02615877 2008-01-18
13
WO 2007/018377 PCT/KR2006/003074
mapped to predetermined areas of a display screen 610 and the overlay screen
640
overlays the display screen 610. In detail, the first logical screen 620 is
displayed on
the display screen 610, the second logical screen 630 is displayed on the
first logical
screen 620, and the overlay screen 640 is displayed on the second logical
screen 630.
In other words, the display screen 610, the first logical screen 620, the
second logical
screen 630, and the overlay screen are sequentially arranged in the direction
of the z-
axis. In this case, an attribute 'z-Order' of the first logical screen 620 may
be set to a
value of 1, an attribute 'z-Order' of the second logical screen 630 may be set
to a value
of 2, and an attribute 'z-Order' of the overlay screen 640 may be set to a
value of 3. The
attributes 'z-Order' of the first and second logical screens 620 and 630 and
the overlay
screen 640 may be set to any numbers or characters as long as they can
represent a
certain order in which the first and second logical screens 620 and 630 and
the overlay
screen 640 are to be arranged along the z-axis.
[107] An attribute 'Display_Area' 530 is information regarding a display
screen area of a
logical screen, as to be illustrated in FIGS. 1 1A and 11B.
[108] FIG. 11A illustrates that a logical screen 710 is mapped to an entire
area of the
display screen 720, and FIG. 11B illustrates that a logical screen 730 is
mapped to a
partial area of the display screen 740.
[109] The attribute 'Display_Area' may include information specifying the two-
dimensional coordinates of a predetermined portion of a display screen to
which the
logical screen is to be mapped or may include information specifying a
predetermined
location on the display screen and an offset value indicating how much the
logical
screen deviates from the predetermined location on the display screen.
[110] An attribute 'Visibility' 540 determines whether a logical screen is to
be visibly or
invisibly displayed on a display screen. It is possible to make a logical
screen appear
on or disappear from a display screen by altering the value of the attribute
'Visibility'
530.
[111] An attribute'Associated_Display_Screen' 550 is information regarding
display
screens associated with a logical screen. A logical screen which is not
associated with
any display screens may not be displayed on a physical display device nor be
transmitted to external output devices.
[112] An attribute 'Associated_Service_Contexts' 560 is information regarding
service
contexts connected to a logical screen, an overlay screen, or a display
screen. Services
set in such service contexts may be displayed on a logical screen, an overlay
screen, or
a display screen.
[113] An attribute 'OutputPort' 570 is information regarding devices by which
a display
screen is to be output, and such devices include display screens,
wired/wireless com-
munication media, and various storage media.
CA 02615877 2008-01-18
14
WO 2007/018377 PCT/KR2006/003074
[114] An attribute 'Overlay-Information' 580 includes discrimination
information of
overlay screen indicating that a screen is an overlay screen, and positional
information
indicating that the overlay screen overlays an arbitrary location of the
display screen.
The overlay screen can be positioned at the foremost of the display screen
through the
attribute 'Overlay-Information'.
[115] Interfaces for identifying or altering the values of the attributes
illustrated in FIGS.
9A and 9B may be provided. Referring to FIGS. 9A and 9B, the interfaces may
include
an interface 'SET' for setting attribute values or connecting a logical screen
to a display
screen, an interface 'ADD' for adding attribute values or connecting a logical
screen to
a service, an interface 'GET' for identifying attribute values, and an
interface
'REMOVE' for deleting attribute values. These interfaces may include
processes,
functions, procedures, or methods that perform their functions, respectively.
[116] For example, a method 'getDisplayScreen(void)' returns a display screen
associated
with the current screen. In detail, if the current screen is a logical screen,
the method
'getDisplayScreen(void)' returns the associated display screen. If the current
screen is
display screen, the method 'getDisplayScreen(void)' returns reference
information
regarding the current screen. Further, if the current screen is a logical
screen, but there
is no associated screen, the method 'getDisplayScreen(void)' returns a value
of 'NULL'.
[117] According to another example, a method 'public void set-
DisplayArea(HScreenRectangle rect) throws SecurityException,
I11ega1StateException'
provides a function for mapping the current logical screen to a predetermined
area of
the associated display screen. An instance that is provided as a parameter is
of a class
'HScreenRectangle' of a package 'org.havi.ui', and has two-dimensional
position in-
formation. The execution of the methods 'SecurityException' and 'I11ega1State-
Exception' may be conducted as an exceptional operation for the method 'setDis-
playScreen(HScreen screen)'. The method 'I11ega1StateException' may be
executed
when the current screen is a logical screen or when a portion of a display
screen
associated with a current logical screen cannot change due to the
characteristics of a
host platform.
[118] According to still another example, a method 'getOutputArea(void)'
returns
regional information of a current screen as HScreenRectangle information. If
the
current screen corresponds to a display screen, the method
'getOutputArea(void)'
returns HScreenRectangle information having the same value as HScreenRectangle
(0,0,1,1). If the current screen is a logical screen, the method
'getOutputArea(void)'
returns information regarding an area on a display screen occupied by the
current
screen. If the current screen is a logical screen but is not associated with
any display
screen, the method 'getOutputArea(void)' returns a value 'NULL'.
[119] Certain terms are used throughout the following description to refer to
particular
CA 02615877 2008-01-18
15
WO 2007/018377 PCT/KR2006/003074
interfaces. However, one skilled in the art will appreciate that a particular
function is
named simply to indicate its functionality. This detailed description of the
exemplary
embodiments does not intend to distinguish between functions that differ in
name but
not function.
[120] FIG. 12 is a diagram illustrating a process that three services are set
on three logical
screens to be mapped to a single display screen.
[1211 Referring to FIG. 12, a first service includes all the three service
components, i.e.,
video, audio, and data components, a second service includes only video and
audio
components, and a third service includes only data component. However, the
present
invention does not impose any restrictions on service components, and the
first and
second services illustrated in FIG. 12 are exemplary.
[122] As illustrated in FIG. 12, the first, second and third services are
displayed on a
physical display device in almost the same manner as in the related art.
According to
the current exemplary embodiment of the present invention, it is possible to
display a
plurality of services on a physical display device independently of one
another without
imposing any restrictions on the number of services that can be displayed on a
single
display screen. In this case, the logical screen displays services by being
mapped to the
display screen, but the overlay screen displays services without being mapped.
[123] FIG. 13 is a block diagram of an apparatus for providing multiple
screens
according to an exemplary embodiment of the present invention.
[124] Referring to FIG. 13, an apparatus 900 for providing multiple screens
includes a
digital signal processing module 940, a service processing module 950, an
output
module 960, and a user/application interface module 965.
[125] Also, the apparatus 900 includes a broadcast signal reception module
910, a storage
medium 920, and an external input module 930 as service sources, and includes
a
physical display device 970, a storage medium 980, and an external output
module 990
as service output media.
[126] The term 'module', as used herein, means, but is not limited to, a
software or
hardware component, such as a Field Programmable Gate Array (FPGA) or an Ap-
plication Specific Integrated Circuit (ASIC), which performs certain tasks. A
module
may advantageously be configured to reside on the addressable storage medium
and
configured to execute on one or more processors. Thus, a module may include,
by way
of example, components, such as software components, object-oriented software
components, class components and task components, processes, functions,
attributes,
procedures, subroutines, segments of program code, drivers, firmware,
microcode,
circuitry, data, databases, data structures, tables, arrays, and variables.
The func-
tionality provided for in the components and modules may be combined into
fewer
components and modules or further separated into additional components and
modules.
CA 02615877 2008-01-18
16
WO 2007/018377 PCT/KR2006/003074
[127] The digital signal processing module 940 receives various information of
a service
such as a multimedia content, e.g., video information, audio information, or
data in-
formation, from the broadcast signal reception module 910, the storage medium
920,
or the external input module 930.
[128] The broadcast signal reception module 910 receives a satellite,
terrestrial, or cable
broadcast signal and transmits the received broadcast signal, the storage
medium 920
stores video information, audio information, or data information of a service,
and the
external input module 930 receives video information, audio information, or
data in-
formation of a service from an external device such as a network interface
module
connected to a network.
[129] The digital signal processing module 940 restores a plurality of
services using
received service components. The restored services include abstract or non-
abstract
services.
[130] Here, the phrase 'a plurality of services' refers to two or more
services transmitted
by the broadcast signal reception module 910 or two or more services
respectively
transmitted by the broadcast signal reception module 910 and the storage
medium 920.
[131] The digital signal processing module 940 may restore services according
to
selection by a user or an application with the aid of the user/application
interface
module 965. In this case, the user or the application may select the
connection between
an arbitrary service and a screen.
[132] The service processing module 950 produces a logical screen, an overlay
screen,
and a display screen to display a service restored by the digital signal
processing
module 940.
[133] The output module 960 maps a plurality of logical screens produced by
the service
processing module 950 to the display screen and overlays the overlay screen on
the
display screen. The mapping of the logical screens to the display screen or
overlapping
of the overlay screen on the display screen may be conducted using a
predefined
method or a method set by the user with the aid of the user/application
interface
module 965.
[134] A service restored by the digital signal processing module 940 may not
be
processed by the service module 950. Instead, a service restored by the
digital signal
processing module 940 may be directly mapped to a certain portion of a display
screen
produced by the output module 960.
[135] A display screen provided by the output module 960 may be displayed on
the
physical display device 970 or may be stored in the storage medium 980.
Examples of
the storage medium 980 include computer readable floppy discs, hard discs, CD-
ROM.
DVD, DVD-ROM, BD (Blu-ray Disc), and semiconductor memories.
[136] Also, a display screen provided by the output module 960 may be
transmitted to an
CA 02615877 2008-01-18
17
WO 2007/018377 PCT/KR2006/003074
external device connected to a network via the external output module 990.
[137] For this, the output module 960 may include a plurality of output ports
via which a
display screen can be provided. In this case, a display screen can be provided
via an
output port set in advance as a default or an output port chosen by the user
with the aid
of the user/application interface module 965.
[138] The user or the application can choose one of a plurality of services or
restore
desired services using the user/application interface module 965. Also, the
user can
choose one of a plurality of display screens using the user/application
interface module
965.
[139] Since the modules illustrated in FIG. 13 are divided according to their
functions, it
is possible to be connected to the other modules.
[140] FIG. 14 is a flowchart illustrating a method of dynamically configuring
multiple
screens according to an exemplary embodiment of the present invention.
[1411 In general, video information, audio information, and data information
constituting
a multimedia content are transmitted in a predetermined format, for example,
an
MPEG stream format. In operation S 1010, an apparatus for providing a service
such as
a multimedia content service receives video information, audio information,
and data
information and restores a service based on the video information, the audio
in-
formation, and the data information. Here, the service restored in operation S
1010 may
be selected or previously determined by a user or an application. The user may
use a
menu displayed on the display device or a remote controller to select the
connections
between an arbitrary screen and a screen. The application may select the
connections
using an API.
[142] Further, data information includes application information regarding
application
for a service, and these application information includes signal information
indicating
whether the application can be executed on a PiP screen. Examples of the
application
information include an application information table (AIT) based on the MHP
standard
and an eXtended application information table (XAIT) based on the OCAP
standard.
The signal information may be added to the application information.
[143] Thereafter, in operation S 1020, the restored service is set such that
it can be
displayed on a logical screen or an overlay screen. In operation S 1030, the
logical
screen is mapped to a display screen and the overlay screen overlays the
display
screen. In operation S 1040, the display screen is provided to the user using
a display
screen, a storage medium, or a network.
[144] The restored service is illustrated in FIG. 14 as being displayed on a
physical
display device via a logical screen. However, the restored service may be
directly
displayed on a physical display device without passing through the logical
screen.
[145] When the user selects the PiP service, the PiP service is realized in
two modes. In
CA 02615877 2008-01-18
18
WO 2007/018377 PCT/KR2006/003074
the first mode, only video component for PiP service selected on the main
screen is
provided without creating a separate logical screen for PiP service, that is,
PiP screen.
In the second mode, a separate logical screen for PiP service is created to
provide the
PiP service selected on the created PiP screen.
[146] FIG. 14 illustrates a method of mapping and overlaying only one service
to a
display screen for simplicity. However, a plurality of services may be mapped
to or
overlay a display screen with or without passing through a plurality of
logical screens
or overlay screens.
[147] When a display screen is provided to the user in this manner, the user
can perform a
plurality of services.
[148] FIG. 15 is a diagram illustrating a software architecture for providing
multiple
screens according to an exemplary embodiment of the present invention.
[149] Referring to FIG. 15, a software architecture 1100 includes a device
driver layer
1110, an API layer 1120, and an application layer 1130.
[150] The device driver layer 1110 receives service components from various
multimedia
content sources and decodes the received service components. Examples of the
received service components include video information, audio information, and
data
information.
[151] The API layer 1120 generates a logical screen, an overlay screen, and a
display
screen and maps a service, the logical screen, the overlay screen, and the
display
screen to one another.
[152] The application layer 1130 provides a user interface so that a user can
dynamically
configure the logical screen and the overlay screen which display a service or
transmits
a user command to the API layer 1120 so that the API layer 1120 can execute
the user
command.
[153] The user enables the device driver layer 1110 with the aid of the
application layer
1130 to provide a display screen via a physical display device or to store the
display
screen in a storage medium. In addition, the user can enable the device driver
layer
1110 to transmit a display screen to an external device via a network.
[154] For this, the device driver layer 1110 may include a plurality of output
ports which
can provide a display screen. Otherwise, API layer 1120 may include the
plurality of
output ports.
[155] In order to dynamically configure a plurality of logical screens on a
display screen,
the API layer 1120 may include a plurality of software modules, e.g., a
multiscreen
manager module ' MultiScreenManager ' 1210, a multiscreen context module'
Multi-
ScreenContext' 1230, a multiscreen context listener module' MultiScreenCon-
textListener' 1250, and a multiscreen context event module
'MultiScreenContextEvent'
1240, as illustrated in FIG. 16.
CA 02615877 2008-01-18
19
WO 2007/018377 PCT/KR2006/003074
[156] The multiscreen manager module 1210 manages the multiscreen context
module
1230, searches for a desired screen, displays information specifying what
devices are
shared by screens, registers the multiscreen context listener module 1250, or
cancels
the registration of the screen context listener module 1250.
[157] The multiscreen context module 1230 is an interface object associated
with a
screen object 1220 and determines whether the screen object 1220 is to become
a
logical screen, an overlay screen, and a display screen according to an
interface
operation performed by the multiscreen context module 1230. Various attributes
such
as the attributes 510 through 570 illustrated in FIGS. 9A and 9B may be set in
the
multiscreen context module 1230. The multiscreen context module 1230 can
provide
the functions 'SET', 'ADD', 'GET', and 'REMOVE' described above with reference
to
FIGS. 9A and 9B.
[158] When attribute information of the screen object 1220 is altered by the
multiscreen
context module 1230, the multiscreen context event module 1240 serves as an
event
class announcing that the attribute information of the screen object 1220 has
been
changed, and the multiscreen context listener module 1250 serves as a listener
interface object which can be realized in a predetermined application class
which
attempts to receive an event prompted by the multiscreen context event module
1240.
[159] An application 1260 is a module which is driven on the application layer
1130. The
application 1260 allows the user to choose a desired service and to freely
arrange a
plurality of logical screens or overlay screens on a display screen.
[160] In detail, the application 1260 transmits various commands which allow
the user to
dynamically configure and manage logical screens or overlay screens to the
multiscreen manager module 1210, and the multiscreen manager module 1210
controls
operations corresponding to the various commands to be executed through the
multiscreen context module 1230.
[161] The multiscreen context module 1230 is associated with the screen object
1220 and
manages the attribute information of the screen object 1220 illustrated in
FIGS. 9A and
9B. In order to manage the attribute information of the screen object 1220,
the
multiscreen context module 1230 may include a variety of functions or methods.
[162] The multiscreen manager module 1210 may be performed so as to receive
various
service components supplied from various sources of a device driver layer 1110
to
display on the logical screens, the overlays screen, or the display screen.
These
functions may be performed by an additional module (not illustrated).
[163] FIG. 17 is a flowchart illustrating a method of displaying a plurality
of services
displayed on logical screens and overlay screens by the modules illustrated in
FIG. 16
on a display screen according to an exemplary embodiment of the present
invention.
[164] Referring to FIG. 17, in operation S 1310, the multiscreen manager
module 1210
CA 02615877 2008-01-18
20
WO 2007/018377 PCT/KR2006/003074
produces a display screen and a number of logical screens, corresponding to
the
number of services to be performed, and overlay screens.
[165] In operation S 1320, the multiscreen manager module 1210 connects the
logical
screens and the overlay screen to respective corresponding services received
from the
device driver layer 1110. The multiscreen manager module 1210 may call a
method
'addServiceContext' for each of the logical screens and a method
'addServiceContext'
of overlay screen by setting service context objects of the received services
as
parameters for the logical screens and the overlay screen. The method 'addSer-
viceContext' connects a logical screen and an overlay screen to a service and
may be
provided by the multiscreen context module 1230.
[166] A method 'addServiceContext' may be represented by a format 'voidaddSer-
viceContext(javax.tv.service.selection.ServiceContext context, booleasshowCC)
throws java.lang.SecurityException, java.lang.IllegalStateException'. A
parameter
'showCC' is a boolean type parameter for determining a use of the overlay
screen. If
the parameter 'showCC' is true, the method 'addServiceContext' displays
subtitles
through the overlay screen. If the parameter 'showCC' is false, the method
'addSer-
viceContext' conducts an application through the overlay screen.
[167] In operation S 1330, once the logical screens and the overlay screen are
connected
to the respective services, the multiscreen manager module 1210 connects the
logical
screens and the overlay screen to the display screen. At this time, the
multiscreen
manager module 1210 may call a method 'setDisplayScreen' for each of the
logical
screens by setting a display screen object to which the logical screens are
connected as
a parameter. The method 'setDisplayScreen' connects a logical screen to a
display
screen and may be provided by the multiscreen context module 1230. Further,
the
overlay screen can call the method 'setOverlaylnformation' of the overlay
screen
object. The method 'setOverlaylnformation' functions as a method for notifying
that
the screen is an overlay screen, and is supplied from the multiscreen context
module
1230.
[168] A method 'setDisplayScreen' may be set to 'public void setDisplayScreen
(HScreen
screen) throws SecurityException, Illegal StateException', and this method
allows an
instance 'HScreen' that is provided as a parameter to be associated with the
current
logical screen. In this case, the instance 'HScreen' is preferably a display
screen.
[169] A parameter of the method 'setDisplayScreen(HScreen screen)' may include
a
value of 'NULL'. In this case, when the method 'setDisplayScreen(HScreen
screen)' is
executed without exception handling, the current logical screen is no longer
associated
with the display screen.
[170] The execution of the methods 'SecurityException' and
'I11ega]StateException' may
be conducted as an exceptional operation for the method
'setDisplayScreen(HScreen
CA 02615877 2008-01-18
21
WO 2007/018377 PCT/KR2006/003074
screen)'.
[1711 The method 'I11ega]StateException' may be executed when a current screen
is a
logical screen or when a portion of a display screen associated with a current
logical
screen cannot change due to the characteristics of a host platform.
[172] In operation S 1340, when the logical screens and the overlay screen are
connected
to the service and a plurality of logical screens and the overlay screen is
connect to a
single display screen, areas on the display screen to which the logical
screens and the
overlay screen are to be respectively mapped are determined. At this time, a
pre-
determined method provided by the multiscreen context module 1230 can be
called to
determine an area on the display screen where the logical screens and the
overlay
screen are to be displayed.
[173] FIG. 18 is a flowchart illustrating a method of exchanging services
displayed on
logical screens between the modules illustrated in FIG. 16 according to an
exemplary
embodiment of the present invention.
[174] In operation S 1410, the multiscreen manager module 1210 temporarily
terminates
a service to be exchanged between two logical screens.
[175] Thereafter, in operation S 1420, information regarding the service is
exchanged
between the two logical screens. At this time, a method 'removeServiceContext'
and a
method 'addServiceContext' are called for each of the two logical screens,
thereby
exchanging service contexts set in the two logical screens between the two
logical
screens. The method 'removeContext' removes a service context connected to a
logical
screen, and the method 'addServiceContext' adds a new service context to a
logical
screen. The methods 'removeServiceContext' and 'addServiceContext' may be
provided
by the multiscreen context module 1230. According to the current exemplary
embodiment of the present invention, service information regarding a service
connected to logical services is exchanged between the logical services,
thereby
obtaining the effect of exchanging a main screen and a sub-screen. In
addition,
according to the current exemplary embodiment of the present invention, even
when
three or more services are performed on a display screen, they can be
exchanged
between an arbitrary number of logical screens.
[176] In operation S 1430, occurrence of an event in which the connection
between the
service and the two logical screens has been changed is generated. Thereafter,
in
operation S 1440, the multiscreen context event module 1240 transmits the
event to the
multiscreen context listener module 1250, and a service newly connected to the
two
logical screens begins to be performed.
Industrial Applicability
[177] According to the present invention, it is possible to perform a
plurality of services
provided by various sources such as cable broadcasts, terrestrial broadcasts,
various
CA 02615877 2008-01-18
22
WO 2007/018377 PCT/KR2006/003074
storage media, and external inputs, in various manners using a single physical
display
screen.
[178] While the present invention has been particularly illustrated and
described with
reference to exemplary embodiments thereof, it will be understood by those of
ordinary skill in the art that various changes in form and details may be made
therein
without departing from the spirit and scope of the present invention as
defined by the
following claims. Therefore, it is to be understood that the above-described
exemplary
embodiments have been provided only in a descriptive sense and will not be
construed
as placing any limitation on the scope of the invention.
CA 02615877 2008-01-18
