Note: Descriptions are shown in the official language in which they were submitted.
CA 02356232 2001-08-27
DYNAMICALLY RESIZABLE DISPLAY ELEMENTS
Inventor:
George Hansen
Background of the Invention
Field of fhe Invention
The present invention relates to customization of user interfaces, and
more particularly to generating, specifying, and applying dynamically
resizable
display elements in visual presentations such as user interfaces.
so Description of the Background Art
User interfaces for computer software are often customizable in various
ways. It is well known in the art to provide functionality that allows users
to
specify attributes of a user interface's appearance, such as: fonts; layouts;
back-
ground color, image, or pattern; sounds; positioning of menu items and task
15 bars; and the like. Such functionality is available, for example, in the
Microsoft
Windows 98 operating system from Microsoft Corporation, as well as in most
other operating systems. Similar functionality is also available in many
software
applications, allowing a user to customize certain aspects of the user
interface
-1 -
Case 4759
16319/04759/DOCS/1058603.3
CA 02356232 2001-08-27
associated with a particular application, without necessarily affecting user
inter-
face components of other applications or of the operating system.
One example of the type of customization that is available in operating
systems such as Microsoft Windows 98 is the ability to change the colors, pat-
s terns, and font attributes for various elements of windows generated by the
op-
erating system. These elements generally reside in the "non-client" area of
the
window, which includes all regions outside the area in which content is pre-
sented to the user by the application (or operating system) controlling the
win-
dow. For example, referring now to Fig.1, depicting a customizable non-client
2o area according to the prior art, a user may specify that title bars 101 of
windows
should be white, and the text 102 printed thereon should be black, in a large
sans
serif font. Accordingly, window 100, and all other windows generated by the
operating system, carry the specified appearance. Such windows include win-
dows associated with particular applications, though in general the
customizable
15 elements are limited to the non-client area of the window. Referring now to
Fig.
2, depicting an example of an alternative customization according to the prior
art, another user may specify that title bars 101 should be black, and the
text 102
printed thereon should be white, in an italic serif font. It is known to
provide
stored profiles containing customization schemes for one or more users of a
2o computer.
-2-
Case 4759
16319/04759/DOCS/1058603.3
CA 02356232 2001-08-27
One known technique for providing customizability is to provide a num-
ber of user-selectable backgrounds for various non-client user interface
areas.
Thus, in addition to being given the option of a selectable background color,
us-
ers can choose a distinctive appearance embodied in a background image. For
example the title bar, status bar, or other areas of windows can be assigned a
background resembling a chain-link fence, or a wood-grain texture. Conven-
tionally, such backgrounds are provided as stored image tiles, which may be re-
trieved, replicated as needed, and applied to fill out the desired non-client
user
interface area.
1o In some cases, a user interface may provide a number of distinct "skins,"
each including a set of image tiles for various non-client areas, a particular
font
specification and color, and the like, designed to embody a particular theme.
Skins generally include a unified set of consistent graphic elements for
buttons,
menus, title bars, backgrounds, and the like, all related to the defined
theme. For
is example, stony textures for various elements, combined with an old-style
font,
may be provided as a medieval-themed skin, while futuristic elements and fonts
may be provided as a science-fiction-themed skin. Referring now to Figs. 3 and
4, there are shown two examples 300 and 400 of skins as may be provided by the
prior art. Skins may be downloadable from an Internet website, such as those
20 offered by NeoPlanet of Phoenix, Arizona, which provides skins suitable for
use
with an Internet browser.
-3-
Case 4759
16319/ 04759/ ROCS/ 1058603.3
CA 02356232 2001-08-27
Conventionally, skins are implemented by providing small graphic ele-
ments, or tiles, which are replicated as needed to fill the appropriate non-
client
area of the user interface window. If a large area is to be filled, a greater
number
of replications of the tile are generated. In the Microsoft Windows operating
system, non-client areas may be painted by calling routines such as
WM NCPAINT and WM_NCSIZE, which are documented in the Microsoft De-
veloper Network and at www.microsoft.com.
Such a technique for implementing skins limits the flexibility and com-
plexity of skins that may be provided, since simple replication of a fixed-
size tile
so does not always allow the background image to be scaled appropriately for
the
size of the non-client area. This limitation is particularly evident when
skins are
applied to resizable windows. In such applications, the failure of the skin to
scale to the size of the window as it is being resized or adjusted becomes
evident
to the user. If tiles are made too small, the repetition of tiles as the
window is
is made larger becomes overly apparent. Conversely, if tiles are made larger,
they
may be disproportionate to the overall size of the non-client area as the
window
is made smaller. In addition, if tiles are made especially large (to provide
for
greater complexity or detail in the background image, for example), part of
the
tile may even be cut off if the window is made so small that the non-client
area
2o cannot display a single tile in its entirety. In some cases, user resizing
of win-
dows may cause particular artistic elements to not appear, or to appear in
inap-
-4-
Case 4759
16319/04759/ DOGS/ 1058603.3
CA 02356232 2001-08-27
propriate locations with respect to particular portions of the window, so that
the
desired customization effect is not properly achieved. Additionally, changes
to
user interface elements such as font sizes are not generally performed, so
that
such user interface elements may not appear in proportion to the overall
window
s when the window is resized, or may even overlap other areas of the window or
screen in an inappropriate manner.
Similar problems exist in other domains where the size and/ or shape, and
in particular the aspect ratio, of an image area may be dynamically changed.
For
example, when displaying an image or animated motion picture in an image area
2o having an aspect ratio differing from the original aspect ratio, certain
elements of
the image or motion picture may be cut off or may appear in an inappropriate
location. Conventionally, distortions and resizing may be applied to the image
as a whole; however such techniques fail to appropriately scale and reposition
image elements in a manner that preserves the original artistic intent of the
crea-
is for of the image or motion picture.
What is needed is a technique that provides greater flexibility in the appli-
cation of background images to user interface elements and areas. What are fur-
ther needed are techniques for adjusting the size of background image elements
and other user interface elements in order to suit the size of a user
interface ele-
2o ment or area, for preserving desired artistic elements for user interface
skins,
even when windows are resized and repositioned by the user, and for
facilitating
-5-
Case 4759
16319/ 04759/ DOCS/ 1058603.3
CA 02356232 2001-08-27
various parameters and techniques for mapping background images to user in-
terface elements and other display regions.
Summary of the Invention
The present invention provides improved flexibility in the application of
skins or other image elements to user interface areas such as non-client
areas.
The invention facilitates mapping of arbitrarily complex patterns onto such ar-
eas, by performing replication, stretching, and other distortions as
appropriate.
Given an image tile, such as a bitmap, any number of transformations and re-
gions can be defined and specified. These include, for example stretch
regions,
To logic for changing font size, and the like. Given a particular size of a
display
area, the defined transformations are performed as needed to ensure that the
im-
age is applied in a manner that maintains the desired proportions and level of
detail. Placement of unique artistic elements in their appropriate locations
is as-
sured.
15 The present invention ensures that artwork, fonts, and other image ele-
ments are appropriately placed and sized in relation to the non-client areas
of a
window. This is accomplished by mapping particular elements to particular an-
chor points within the various non-client areas, and by adjusting image charac-
teristics in accordance with specified parameters and according to the current
20 size of the window. Thus, appropriate resolution settings, font sizes,
scaling fac-
tors, and the like are selected, based on parameters that may be specified in
ad-
-6-
Case 4759
16319/ 04759/ DOCS/ 1058603.3
CA 02356232 2001-08-27
vance by the artist creating the skin, and further based on current window
size or
other factors. The present invention generalizes this technique so that
elements
of the artwork, as well as text elements and the like, may be mapped onto
virtu-
ally any size window, with aesthetically appealing results. Accordingly, the
in-
s vention may be applied in any context where dynamic resizing of image ele-
ments is desirable.
The present invention takes into account relationships among various im-
age elements, and makes appropriate adjustments. For example, font size may
be scaled in order to maintain proportionality with overall window size. Such
a
1o change to font size may in turn result in resizing of non-client areas
containing
text. Backgrounds for such non-client areas are adjusted by appropriate
scaling
and/or replication of the artistic elements associated with the backgrounds.
Particular techniques and parameters for scaling, resizing, distorting, and
replicating image elements are configurable by the artist creating the image
15 and/or by the end user. A data file or other source of stored parameters
may be
employed in order to define the logic and methodology for these operations for
a
particular image element or for a skin.
By providing operations and parameters for mapping image elements to
dynamically resizable windows, the present invention facilitates application
of
2o complex skins to regions of on-screen windows, and further ensures that
such
Case 4759
16319/ 04759/ DOGS/ 1058603.3
CA 02356232 2001-08-27
skins are appropriately presented, even after user operations which may alter
the
size and/or placement of such windows.
The present invention is applicable to other domains as well, such as dy-
namically altering the positioning and sizing of elements in any image or
series
of images, responsive to the size and/ or shape of the image area. For
example,
individual elements of an animated motion picture may be positioned and/or
sized appropriately responsive to changes in aspect ratio of the image area.
Pa-
rameters may be defined as to the posidonal relationship and "stretchable'
char-
acteristics of various on-screen image elements and portions of elements. Re-
to sponsive to the size and/or shape of the image area, the defined parameters
are
retrieved and used to determine new positions, shapes and distortions for the
various image elements.
Brief Description of the Drawings
I5 Fig.1 is a screen shot depicting an example of customization of a non-
client area according to the prior art.
Fig. 2 is a screen shot depicting a second example of customization of a
non-client area according to the prior art.
Fig. 3 is a screen shot showing application of a themed skin to an onscreen
2o window, according to the prior art.
-g-
Case 4759
16319/04759/DOCS/1058603.3
CA 02356232 2001-08-27
Fig. 4 is a screen shot showing application of a second themed skin to an
onscreen window, according to the prior art.
Figs. 5A, 5B, and 5C are screen shots depicting modification of sizing and
placement of user interface elements responsive to dynamic resizing of an on-
screen window.
Fig. 6 is a block diagram of an architecture for practicing the present in-
vention.
Figs. 7A, 7B, 7C, 7D, and 7E depict examples of scaling a user interface
element according to various techniques.
1o Figs. 8A, 8B, 8C, 8D, and 8E depict a detailed example of scaling and resiz-
ing a user interface element including several subcomponent elements.
Figs. 9A, 9B, 9C, 9D, and 9E depict examples of modifying an image to fit
a frame having a changed aspect ratio.
Fig.10 depicts an example of a set of configurable parameters for user in-
is terface elements.
Fig.11 is a screen shot of a user interface employing dynamically resizable
user interface window according to the techniques of the present invention.
Fig.12 is an example of application of a stretch region.
-9-
Case 4759
16319/04759/DCX'S/1058603.3
CA 02356232 2001-08-27
Detailed Description of the Preferred Embodiments
The following description of preferred embodiments of the invention il-
lustrates the invention in the context of the display of a user interface for
a per-
sonal computer. However, one skilled in the art will recognize that the tech-
niques and methods of the present invention may be applied to other domains,
environments, and contexts, such as any system in which an image, animation,
motion picture, or the like is resized or reshaped. Accordingly, the following
de-
scription is presented for illustrative purposes only, and is not intended to
limit
or define the scope of the invention in any way, which scope is defined solely
by
Io the claims presented below.
In one embodiment, the invention operates on a personal computer run-
ning, for example, the Microsoft Windows 98 operating system from Microsoft
Corporation. The invention may be implemented as a feature of the operating
system, or as an add-in or plug-in for such an operating system.
Alternatively,
I5 the invention may be implemented as a component of a software application
that
may be run on a personal computer.
It is well known in the art for operating systems to include a graphical
user interface (GUI) that facilitates ease of user interaction with the
system. Mi-
crosoft Windows 98 is an example of an operating system providing such func-
2o tionality. In particular, such GUIs, including that of Microsoft Windows
98, typi-
cally display information to the user in one or more on-screen windows that
may
-10-
Case 4759
16319/ 04759/ DCX:S/1058603.3
CA 02356232 2001-08-27
overlap one another, and that may be resized, repositioned, and reconfigured
at
will by the user. In general, users interact with the operating system via a
com-
bination of a keyboard and a pointing device such as a mouse; an on-screen cur-
sor moves in response to the user's manipulation of the pointing device.
Reposi-
tinning and resizing of on-screen windows may take place in response to the
user's commands which may be specified by, for example, "dragging and drop-
ping' the on-screen cursor by moving the mouse accordingly. Techniques for
detecting user input in such a manner, and for displaying, resizing, and
reposi-
tinning on-screen windows accordingly, are well known in the art of user inter-
zo faces for personal computers.
Referring now to Fig. 6, there is shown a block diagram of an architecture
for practicing the invention according to one embodiment. User input 601 in-
eludes commands for resizing, repositioning, and reconfiguring on-screen win-
dows and other user interface elements. As described above, such commands
~s may be provided to the system via manipulation of a pointing device (not
shown) such as a mouse, trackball, or any other suitable device (e.g.
touchpad,
joystick, etc.). Operating system 602 accepts user input 601 and modifies win-
dow size, shape, and position 603 for any number of on-screen windows. Win-
dow size, shape, and position 603 are stored in, for example, random access
2o memory (RAM; not shown) for use in rendering on-screen images according to
the specified parameters.
-11-
Case 4759
16319/04759/DOCS/1058603.3
CA 02356232 2001-08-27
Scaling and resizing module 607 accepts window size, shape, and position
603 in determining appropriate scaling, resizing, and positioning operations
to
be applied to various elements of windows. Module 607 also obtains skin ele-
ments 604 containing bitmaps and/or other image elements, such as tiles, to be
used as source material in rendering on-screen windows. In accordance with
stored parameters and rules 605 for manipulating, distorting, scaling, and/ or
resizing image elements, module 607 outputs resized image elements for the
window. In one embodiment, skin elements 604 and parameters and rules 605
are stored in various files in memory or storage 606, which may be implemented
1o for example on a hard drive (not shown) or RAM.
Rendering engine 608 draws the window including the resized image
elements generated by module 607. As is known in the art, engine 608 writes to
memory locations in frame buffer 609. Display 610, which may be a conventional
cathode-ray tube (CRT), liquid crystal display (LCD), or the like, outputs the
re-
sultant image based on the contents of frame buffer 609.
Many of the components of Fig. 6 are well known in the art, and their op-
eration is not described in detail here. The present invention, in one embodi-
ment, is implemented in the context of module 607 and the associated skin ele-
menu 604 and parameters and rules 605 for drawing on-screen windows. Ac-
2o cordingly, the following description provides additional details on the
operation
-12-
Case 4759
16319/04759/DCXS/1058603.3
CA 02356232 2001-08-27
of the dynamically sizing and scaling components, as they may apply for exam-
ple in the context of the architecture of Fig. 6.
Referring now to Figs. 5A, 5B, and 5C, there is shown an example of ap-
plying a skin and scaling a user interface window 500 according to the tech-
s piques of the present invention. For illustrative purposes, the example
depicted
in Figs. 5A, 5B, and 5C present a simplified implementation of the invention;
one
skilled in the art will recognize that more complex implementations and opera-
tions may be contemplated in the context of the present invention.
Fig. 5A depicts window 500 in an initial size and shape, as may be pre-
1o sented on display 610 in accordance with conventional display techniques.
Win-
dow 500 is a resizable, repositionable user interface window as is familiar to
us-
ers of GUI-based operating systems such as Microsoft Windows 98. Client area
501 contains content that may be generated by an application running on the
computer, or that may be generated by the operating system itself. Other por-
15 tions of window 500 represent the non-client area of the window, including
menu bar 502, title bar 504/505, and standard controls 506 for resizing window
500. These non-client areas contain various backgrounds and other elements
that
embody the skin for the user interface. For clarity of illustration, other
features
and elements which are commonly provided in such on-screen windows are
20 omitted from the depiction of Fig. 5A, such as for example scroll bars,
borders,
tool bars, status bars, and the like.
-13-
Case 4759
16319/04759/DOCS/1058603.3
CA 02356232 2001-08-27
In the example of Fig. 5A, the title bar contains two regions 504 and 505.
Region 504 contains text representing the title of the window, and region 505
represents the remainder of the title bar, which does not contain text. In
accor-
dance with the present invention, region 504 contains a particular background
s that is well suited to the display of text overlaying the background. Region
505
contains a different background, which may be thematically related to the back-
ground shown in region 504 but which may be more visually complex and per-
haps less well-suited to the display of overlaying text. Thus, a first image
ele-
ment, or tile, may be provided for display in region 504, while a second image
io element may be provided for display in region 505. The respective
backgrounds
may be selected by the end user, and may be associated with a particular theme
or design motif.
According to rules and parameters that may be specified by the artist cre
ating the skin, image elements for each of the display regions may be scaled,
rep
Is licated, or otherwise processed in order to fill the respective regions.
For exam
ple, the image element for region 504 may be scaled to fit region 504, while
the
image element for region 505 may be independently replicated (as shown by
multiple instances 507 of an image element) in order to fit region 505. The
ability
to scale, replicate, resize, and/ or otherwise distort image elements independ-
2o ently of one another to fit various regions ensures that the skin will be
mapped
-14-
Case 4759
16319/04759/ ROCS/ 1058603.3
CA 02356232 2001-08-27
appropriately to window 500 even when the user resizes or changes the shape of
window 500.
Referring now to Fig. 5B, there is shown window 500 after it has been
resized by the user. Region 505 is now wider, due to the increased width of
win-
dow 500, while region 504 remains the same size, since the text size has not
changed. Accordingly, the background for region 504 is unchanged. However,
region 505 now contains additional instances 507 of the image element
specified
for the background of that region. By including additional instances 507,
while
leaving region 504 unchanged, the present invention maintains the overall
stylis-
Io tic and artistic intent of the creator of the interface.
Referring now to Fig. 5C, there is shown window 500 at substantially the
same size as depicted in Fig.. 5B. However, in Fig. 5C, region 504 is widened
in
order to accommodate a longer title, and region 505 is shortened accordingly
(by
reducing the number of instances 507 of the image element). Thus, the present
is invention determines the relative sizes of regions 504 and 505 of the title
bar,
based on the current text data being displayed in region 504. In this manner,
the
present invention ensures that the displayed text does not extend into the
back-
ground for region 505 which would render the text more difficult to read.
Figs. 7A through 7E depict a simplified example of various techniques for
2o mapping an image element to a window, in response to a user changing the
size
and/ or shape of the window. By showing a single image element and the vari-
-15-
Case 4759
16319/04759/DOCS/1058603.3
CA 02356232 2001-08-27
ous distortions, and replications that may be applied thereto, Figs. 7A
through 7E
illustrate the various techniques provided by the present invention, as they
may
be specified and applied in accordance with the user s wishes and the artist's
specifications. It will be appreciated, by one skilled in the art, that this
technique
s may be applied to multiple regions, or parts of the non-client area using
different
image tiles and distortion/ replication schemes.
Referring now to Fig. 7A, there is shown a region 700 of an onscreen win-
dow in an original (first) size and shape. The image element 705 displayed
within region 700 may be provided, for example, in the form of a bit map that
zo may be stored in memory or storage 606 and retrieved and applied when
render-
ing an on-screen window. If region 700 is resized or if its shape is changed,
im-
age element 705 may be mapped to the new size and shape in any of a number of
ways. Some examples of the techniques that may be used for mapping image
element 705 are displayed in Figs. 7B through 7E.
I5 In Fig. 7B, image element 705 is replicated as many times as is necessary
to
fill newly-sized region 701. This technique is commonly used in the prior art
for
applying skins to dynamically resizable windows. As can be seen from the ex-
ample of Fig. 7B, this replication technique may not be appropriate for many
types of image elements. For relatively simple textures, replication may be a
2o suitable technique for applying the image element to a larger region.
However,
for distinctive image elements 705, such as shown in the example, replication
-16-
Case 4759
16319/04759/ DOCS/ 1058603.3
CA 02356232 2001-08-27
yields a result that is aesthetically very different from the element as
presented in
the original size, and does not adequately preserve the intent of the artist,
who
may have intended that the image element's 705 distinctive features appear
only
once in the image area. Thus, the technique shown in Fig. 7B, while often used
in
s the prior art, is unsuitable for many types of images.
In Fig. 7C, image element 705 is stretched horizontally so as to fit newly-
sized region 702. Though such a technique is relatively trivial to implement,
it
may result in unwanted distortions of particular distinctive features of image
element 705. In the example of Fig. 7C, the stretching operation creates
notice-
~o able distortion in the spiral portions of the artwork, causing them to be
flattened
horizontally rather than maintaining their original rounded appearance. Thus,
for certain types of artwork where such distortion undesirably alters the
appear-
ance of the image element, such stretching (or scaling) of image element 705
is
not a suitable solution.
is In Fig. 7D, image element 705 remains unchanged from its original shape
and size. Accordingly, blank areas appear on either side of image element 705,
since region 703 is much larger than the originally-sized region 700 as shown
in
Fig. 7A. This solution is often unsuitable, since it fails to maintain the
desired
proportion between image element 705 and the overall window. In addition, the
2o blank areas are an undesired introduction to the interface window, which
was
not intended by the artist.
-17-
Case 4759
16319/04759/DOCS/1058603.3
CA 02356232 2001-08-27
In Fig. 7E, image element 705 is mapped to resized region 704 by replica-
tion of the center portion, while the end portions containing the distinctive
spiral
design are preserved in their original form. This mapping preserves the
original
design most effectively, while distorting image element 705 in a relatively
unob-
s trusive manner. In accordance with the present invention, the artist
specifies
which portions of image element 705 are to be replicated or scaled, and which
portions are to remain unchanged. Thus, in the example shown, the artist may
specify that the central portion may be replicated (or scaled), while the
spiral de-
sign remains intact. Alternatively, the artist may specify that the spiral
portions
io may be scaled only in such a manner that no distortion results (i.e., the
same scal-
ing factor applies to both the x and y axes). Such specifications may be
provided,
for example, in the form of stored parameters accompanying the stored image
elements as described below.
Figs. 8A through SE show details of resizing operations in accordance
i5 with the example of Fig. 7E. In order to implement resizing of various
compo-
nents of the image according to different rules and parameters, several compo-
nents of image region 700 are identified. Referring now to Fig. 8A, there is
shown an example of identification of various components of the image region,
including top edge 801, bottom edge 805, left edge 806, right edge 804, left
image
2o region 803, central image region 809, and right image region 802. Spiral
portions
-18-
Case 4759
16319/04759/DOCS/1058603.3
CA 02356232 2001-08-27
807 and 810 are located in left image region 803 and right image region 802,
re-
spectively. Central portion 808 is located in central image region 809.
As described, the artist may define the various regions of the image ele-
ment as appropriate, and may specify what type of distortion, replication, or
scaling is to be applied to each region. In the example shown the artist may
spec-
ify, for example, that regions 803 and 802 are not to be distorted or
enlarged,
though they may be reduced in size if necessary, while region 809 may be repli-
Gated along the horizontal axis only. Thus, as shown in Fig. 8B, when image re-
gion 811 is widened, regions 803 and 802 are unchanged, as are spiral portions
so 807 and 810 located therein, while region 809 contains replicated
instances~of cen-
tral portion 808 as needed to fill the expanded width.
Referring now to Fig. SC, there is shown an example in which image re-
gion 812 is enlarged along both the horizontal and vertical axes. Since in
this ex-
ample the artist specified that replication of region 809 is permissible only
along
~s the horizontal axis, no vertical replication takes place, which would be
inappro-
priate given the nature of this particular image. Since in this example the
artist
specified that regions 803 and 802 are not to be distorted or enlarged, they
re-
main the same size as before, even though there is room for those regions to
be
enlarged.
2o Referring now to Fig. 8D, there is shown an example in which image re-
gion 813 is reduced in size along both the horizontal and vertical axes. Here,
all
-19-
Case 4759
16319/04759/DOCS/1058603.3
CA 02356232 2001-08-27
regions are reduced in size in the appropriate proportions, in accordance with
the parameters and rules specified by the artist.
Referring now to Fig. 8E, there is shown an example in which image re-
gion 814 is reduced in size along the horizontal axis, but its vertical
dimension
s remains the same. Here, regions 803 and 802 cannot be reduced in size
without
causing distortion, which would contravene the artist's specified rules. Thus,
re-
gions 803 and 802 remain the same size as in the original image, and region
809 is
reduced along the horizontal axis so that the image fits within the smaller
overall
space. Image element 808 in region 809 is either scaled down in size, or
cropped,
to as may be specified by the artist.
One skilled in the art will recognize that the examples shown herein are
merely illustrative, and that many other techniques for practicing the present
in-
vention may be employed. For example, in one embodiment, the invention may
perform analysis of the image in order to determine the best way to distort,
repli-
Ts cate, crop, or scale various portions of the image so that it maps
appropriately to
a given area. Such analysis may be performed by automated detection of fore-
ground elements or other image characteristics, and may be used when an artist
fails to provide adequate guidance as to how the mapping should take place. Al-
ternadvely, such analysis may be used to supplement or augment rules and pa-
2o rameters specified by the artist. In an alternative embodiment, the end
user may
-20-
Case 4759
16319/04759/DOCS/1058603.3
CA 02356232 2001-08-27
have an opportunity to specify customizable parameters for how such analysis
may be applied to the process of mapping images.
Referring now to Figs. 9A through 9E, there is shown another example of
application of the techniques of the present invention, in which an image is
s mapped from one aspect ratio to another. The techniques of the present inven-
tion are thus applicable, for example, in transferring an animated motion
picture
from one medium to another, in which the two media have different aspect ra-
tios. Commonly, theatrical releases of motion pictures use a widescreen format
(16:9 ratio) that does not translate directly to the 4:3 ratio commonly
employed in
io television sets. Thus, when the motion picture is transferred to videotape
or
videodisc (such as DVD), reformatting of the image is required in order to fit
the
4:3 aspect ratio.
Fig. 9A depicts the widescreen version 900 of the image, which may be a
frame from an animated motion picture. Several techniques are known in the art
is for translating version 900 to a medium having a different aspect ratio.
Referring now to Fig. 9B, there is shown pan-and-scan version 901, in
which only a portion of the original image is shown at any given time. The re-
maining areas of the image are not shown. Such a technique is deficient in
that it
fails to preserve the original composition of the image, and often results in
im-
2o portant information being omitted from the transferred image.
-21-
Case 4759
16319/04759/DOCS/1058603.3
CA 02356232 2001-08-27
Referring now to Fig. 9C, there is shown letterboxed version 902, in which
the entire original version 900 is scaled, in proportion, to fit the new
medium.
Since the aspect ratios differ, blank areas 908 exist on the transferred
image.
Such a technique preserves the original composition, but often results in
dimin-
fished resolution, since substantial portions of the screen are left unused.
In addi-
tion, many viewers find the presence of blank areas 908 distracting.
Referring now to Fig. 9D, there is shown a distorted version 903, in which
the proportions of the image are distorted (in this case, scaled along the
vertical
axis) so as to map onto the new aspect ratio. Such a technique is clearly
deficient
io in that such distortions of the image are usually unacceptable to viewers.
Referring now to Fig. 9E, there is shown a version 904 that is constructed
in accordance with the techniques of the present invention. Areas 905 and 907
are left unchanged from the original, and area 906, containing relatively unim-
portant details of the image, is scaled or cropped down to fit the new aspect
ra-
15 tio. Thus, version 904 avoids the problems and limitations of versions 901,
902,
and 903, as described above, and presents an image that most closely retains
the
original composition of the artist without sacrificing resolution. In one
embodi-
ment, the original artist may specify which areas are to be untouched and
which
are to be scaled or otherwise altered. In another embodiment, the system of
the
2o present invention may analyze original image 900 in order to determine
those
-22-
Case 4759
16319/04759/DOCS/1058603.3
CA 02356232 2001-08-27
areas that are main subjects, and thereby determine what types of alterations
are
most suitable in modifying the image.
Though the example of Figs. 9A through 9E is discussed in terms of mo-
tion pictures, the same techniques may be applied in the context of still
images
being transferred from one medium to another, when the media have differing
aspect ratios.
In one embodiment, the present invention provides functionality for
specifying various parameters, including spacing, sizing, relative
positioning,
rotation, color manipulations, and permissible alterations, for various image
1o elements and regions. The various regions and their relationships to one
another
are defined and specified by storing values for various parameters. Such
values
may include, for example, maximum and minimum sizes, stretch regions, scaling
axes, maximum scale factors, and the like.
For an implementation of the invention that operates in the context of a
is user interface, these parameter values define characteristics with respect
to user
interface components such as borders, title bars, and the like. An artist
generat-
ing a user interface layout may specify some or all of these parameters in
order to
define how the various elements of the user interface are to be modified when
the size or shape of the window is changed. Default values for some or all of
the
2o parameters may be provided by the system, if appropriate. Source material
for
image artwork to be used in generating the user interface window may be pro-
-23-
Case 4759
16319/04759/DOCS/1058603.3
CA 02356232 2001-08-27
vided in the form of a bitmap that is given a predetermined file name and that
resides in a particular predefined directory. In general, the bitmap is of
appro-
priate size with respect to the expected size range of the window; however,
the
bitmap may be scaled and/ or replicated as needed to fill available image
regions.
Referring now to Fig.10, there is shown an example of a user interface
layout that defines various configurable parameters that may be specified in
ac-
cordance with an implementation of the present invention. This example is pro-
vided for illustrative purposes; one skilled in the art will recognize that
many
other types of parameters and interrelationships among elements may be speci-
1o fied and defined. The lines shown on Fig.10 are used only to illustrate the
boundaries of the various areas. The actual image need not contain border
lines.
The parameters shown in the example of Fig.10 are as follows:
1000: Top-left corner of bitmap (0, 0). Position values for other user inter-
face parameters are provided according to coordinates relative to
15 this point.
1001: Right edge of left border (LBORD_RIGHT).
1002: Bottom edge of top border (TBORD_BOTTOM).
1003: Left edge of right border (RBORD_LEFT).
1004: Top edge of bottom border (BBORD TOP).
zo 1005: Right edge of right border. Defined by width of bitrnap.
1006: Bottom edge of bottom border. Defined by height of bitmap.
-24-
Case 4759
16319/ 04759/ ROCS/ 1058603.3
CA 02356232 2001-08-27
1007: Left border vertical stretch point/region (LBORD VSTR).
1008: Top border horizontal stretch point/region (TBORD HSTR).
1009: Right border vertical stretch point/ region (RBORD VSTR).
1010: Bottom border horizontal stretch point/region (BBORD HSTR).
1011: Right edge of left title area (LTITLE_RIGHT).
1012: Bottom edge of title area (TITLE BOTTOM).
1013: Left edge of right title area (RTITLE_LEFT).
1014: Center title area horizontal stretch point/region (CTITLE_HSTR).
1015: Left title area vertical stretch point/region (LTITLE_VSTR).
1016: Right title area vertical stretch point/region (RTITLE_VSTR).
1017: Center title area vertical stretch point/region (CTITLE_VSTR).
1018: Right edge of system menu button (SYSMENU RIGHT).
1019: Bottom edge of system menu button (SYSMENU BOTTOM).
1020: Left edge of minimize button (MIN LEFT).
s5 1021: Left edge of maximize button (MAX_LEFT).
1022: Left edge of close button (CLOSE LEFT).
1023: Bottom edge of min/max/close buttons (CLOSE BOTTOM).
1024: Left edge of title text area (TITLETEXT_LEFT). This value need not
be to the right of LTITLE_RIGHT as shown in Fig.10; it can extend
2o into the left title area.
-25-
Case 4759
16319/04759/DOCS/1058603.3
CA 02356232 2001-08-27
1025: Right edge of title text area (TITLETEXT_RIGHT). This value need
not be to the left of RTITLE_LEFT as shown in the Fig.10; it can ex-
tend into the right title area.
1026: Left edge of menu text area (MENUTEXT_LEFT). This value need
not be to the right of LTITLE_RIGHT as shown in Fig.10; it can ex-
tend into the left title area.
1027: Right edge of menu text area (MENUTEXT_RIGHT). This value
need not be to the left of RTITLE_LEFT as shown in Fig.10; it can
extend into the right title area.
zo -1028: Top of title text area (TITLETEXT_TOP).
1029: Bottom of title text area (TITLETEXT_BOTTOM).
1030: Top of menu text area (MENUTEXT_TOP).
1031: Bottom of menu text area (MENUTEXT_BOTTOM).
1032: Client area. Height may be zero, if appropriate.
15 Stretch point parameters 1007-1010 define locations where the bitmap will
stretch, either as points or regions. Regions are used, for example, when an
im-
age pattern, such as a checkerboard or texture, is to be replicated. Regions
may
be specified by providing starting and ending pixel locations defining a span
of
an image element. For example "TBORD HSTR=100,103" would indicate that
2o the specified four-pixel-wide region be replicated as many times as
necessary to
fill the top border to its actual width. The replicated pixels are inserted
directly
-26-
Case 4759
16319/04759/ DUCS/1058603.3
CA 02356232 2001-08-27
after the stretch point/region. Region replication is done in integral chunks
where possible.
Referring momentarily to Fig.12, there is shown an example of applica-
tion of a stretch region. Region 1008 defines an area 1201 of a bitmap that is
to be
s replicated. In window 1202, area 1201 is displayed once, in accordance with
the
available space in the image. In window 1203, area 1201 is replicated several
times as needed to fill the available space in the enlarged window.
Additional parameters that may be employed in connection with a resiz-
able user interface window such as that shown in Fig.10 include:
~ Border for title text area (TITLETEXT_BORDER). Defines the
boundary of the text drawing area within the title text area. In one
embodiment, these values are specified as offsets inward from the
title text rectangle. The border area will not be overdrawn by the ti-
tle text. This parameter is primarily used if a title bitmap is sup-
zs plied. In one embodiment, the fonts and line-heights used for the
title text area are defined by settings within the operating system,
such as the Control Panel in Microsoft Windows 98.
~ Border for menu text area (MENLTTEXT_BORDER). Defines the
boundary of the text drawing area within the menu text area. In
one embodiment, these values are specified as offsets inward from
the menu text rectangle. The border area will not be overdrawn by
-27-
Case 4759
16319/04759/DOCS/1058603.3
CA 02356232 2001-08-27
the menu text. This parameter is primarily used if a menu bitmap
is supplied. In one embodiment, the fonts and line-heights used for
the menu text area are defined by settings within the operating sys-
tem, such as the Control Panel in Microsoft Windows 98.
~ Text justification for the title text (TITLETEXT~JUST). May be top
(0), center (1), or bottom (2) justified. Specifies where the text is
drawn within the title text area when the design size of the area ex-
ceeds the font size.
~ Text justification for the menu text (MENUTEXT~UST). May be
1o top (0), center (1), or bottom (2) justified. Specifies where the text is
drawn within the menu text area when the design size of the area
exceeds the font size.
~ Internal leading for title text (TITLETEXT_LEADING). May be off
(0) or on (1). Indicates whether to draw title text using internal
leading.
~ Internal leading for menu text (MENUTEXT LEADING). May be
off (0) or on (1). Default is on. Indicates whether to draw menu
text using internal leading.
~ Minimum heights for the three horizontal gutters that separate the
2o text areas from each other and from the rest of the bitmap
(GUTTERS).
-28-
Case 4759
16319/04759/DOCS/1058603.3
CA 02356232 2001-08-27
~ Title text area vertical stretch point/ region (TITLETEXT_VSTR).
~ Title text area horizontal stretch point region (TITLETEXT_HSTR).
~ Menu text area vertical stretch point/ region (MENUTEXT_VSTR).
~ Menu text area horizontal stretch point region
s (MENUTEXT_HSTR).
In one embodiment, the system of the present invention employs the
above parameters to generate an image appropriate to the current size of the
user
interface window. One example of the logic that is employed in applying the pa-
rameters to a resized window is as follows. When text areas are to be
expanded,
zo such as when a large amount of text is to be displayed or when a large font
is
used, the system of the present invention performs scaling on the background
bitmap in an upward direction, downward direction, or both, depending on the
text justification setting for the area. If a specified gutter limit prevents
an area
from being sufficiently expanded the desired direction, the system expands the
Is area in the opposite direction. If this expansion also reaches its limit,
the system
expands the area in the direction of another text area, pushing that area as
neces-
sary. If there is still not room for the text, the center title area is
stretched at
CTITLE _ -VSTR between LTITLE_RIGHT and RTITLE LEFT to make room for the
text. The left title area is stretched at LTITLE_VSTR between LBORD RIGHT
20 _and LTITLE_RIGHT to match the new height of the center title area.
Likewise,
-29-
Case 4759
16319/04759/ DOGS/1058603.3
CA 02356232 2001-08-27
the right title area is stretched at RTITLE_ _VSTR between RTITLE_LEFT and
RBORD LEFT.
The artist provides bitmaps for the title text area and for the menu text
area, each containing graphics to be associated with the title text area, such
as a
decorative border or background image. Each bitmap is given a predefined
name, and in one embodiment resides in a predefined directory. In one em-
bodiment, position values for the above-described parameters are provided as
pixel coordinates relative to the upper-left corner of each bitmap. If the
title text
area in the resized window is larger than the supplied bitmap, the system of
the
so present invention replicates that portion of the defined by TITLETEXT_VSTR
and/ or TITLETEXT_ _ -HSTR (or MENUTEXT_VSTR and/ or MENUTEXT_HSTR)
as necessary to fill the corresponding areas of the user interface window.
In one embodiment, each supplied bitmap includes a one-pixel frame
around the actual design; the color of this frame defines the transparency
color
is for the bitmap. As is known in the art, areas having a corresponding color
defi-
nition within the bitmap are rendered as transparent areas, so that any
underly-
ing color or image is allowed to show through these areas.
The above-described techniques can further be generalized to apply to
specifying parameters for font scaling and similar resizable elements. Accord-
2o ingly, element sizes, orientations, and shapes can be made dependent on
current
font size for text being displayed in a given image region. Conversely, font
sizes
-30-
Case 4759
16319/04759/DOCS/1058603.3
CA 02356232 2001-08-27
for various text areas may be made dependent upon available space within a
given image region, which may in turn be dependent upon the overall size of a
window as specified by the end user.
Referring now to Fig.11, there is shown a screen shot 1100 of an example
s of an application of the present invention in generating a dynamically
resizable
user interface window. Background artwork 1103 is replicated as appropriate to
fill the image area. Distinctive visual elements, such as rough edges
1101,1102
and 1104, are placed at particular locations with respect to edges of the
interface
window, using parameters as described above. By employing the dynamic scal-
zo ing and replicating operations described herein, the present invention
ensures
that such distinctive visual elements are scaled and positioned appropriately
given the various graphical and text elements presented in the user interface
window.
From the above description, it will be apparent that the invention dis-
zs closed herein provides a novel and advantageous system of specifying dynami-
cally resizable display elements in visual presentations such as user
interfaces.
The foregoing discussion discloses and describes merely exemplary methods and
embodiments of the present invention. As will be understood by those familiar
with the art, the invention may be embodied in other specific forms without de-
zo parting from the spirit or essential characteristics thereof. For example,
the tech-
niques of the invention may be applied in any graphical or visual display
context
-31-
Case 4759
16319/04759/DOCS/1058603.3
CA 02356232 2001-08-27
wherein resizing of the image area may take place. One skilled in the art will
recognize that the particular parameters and descriptive elements specified
herein are merely exemplary of the techniques of the present invention. Accord-
ingly, the disclosure provided herein is intended to be illustrative, but not
lim-
iting, of the scope of the invention, which is set forth in the following
claims.
-32-
Case 4759
16319/04759/DOCS/1058603.3
