Note: Descriptions are shown in the official language in which they were submitted.
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
INTERACTIVE DIGITAL FLYER SYSTEM
TECHNICAL FIELD
[0001] The present disclosure relates to digital flyers and, in particular,
to an interactive
digital flyer.
BACKGROUND
[0002] Marketing is an integral part of doing business that allows
businesses to identify and
attract new and potential customers to the business. It is also an important
strategy to ensure
that past customers become repeat customers.
[0003] There are many forms of marketing. However, one of the oldest forms
of marketing
is the use of information pamphlets to inform the public about the wares and
services that the
business offers. Such pamphlets can be generally referred to as "flyers", but
which can also be
referred to as a circular, handbill, leaflet, or ad. A more comprehensive
pamphlet about wares
carried by the business may also be referred to as a catalogue, parts book,
and trade literature.
Collectively, such forms of information pamphlets will be described hereonin
as a "flyer".
[0004] Flyers are often used not only to inform the public about the wares
and services
offered by the business but also to convey information about prices and
special deals, often for
a limited time. For example, a grocery store may circulate a printed flyer
advertising produce
and meats on sale. In addition, the printed flyer may contain coupons that
entitle the purchaser
to a certain discount. This coupon not only entices the customer to visit the
store but also
provides feedback to the store about the effectiveness of the printed flyer in
generating
business.
[0005] To distribute such printed flyers, a popular distribution medium has
been to insert
the flyers into newspapers in order to piggyback the distribution channels of
widely read
newspapers. Another often used medium is direct home distribution ¨ a practice
homeowners
and renters often call "junk mail". However, distribution of printed flyers
often came at great
expense due to the number of flyers that had to be printed for distribution.
Furthermore,
environmentally conscious consumers often balked at the waste generated by the
printed flyers.
[0006] With the digital revolution, businesses began to embrace the concept
of "digital
flyers". Digital flyers (also referred to as electronic flyers, online flyers,
and Internet flyers) offer
1
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
a significant cost saving over the traditional printed flyers and are
"greener" than traditional
printed flyers. Thus, the digital flyers industry has steadily grown, with
many businesses, having
an online presence, using digital flyers to market their products and
services.
[0007] Presently, digital flyers are created by scanning each page of the
printed flyers. In
order to obtain high quality images, scanning must be done at a sufficiently
high resolution.
Alternatively, digital flyers may be provided by the retailers, which would
also be used to print
the paper version of the flyers. With the digital images, the digital flyer is
created by stitching
together the various images in a sequence that corresponds to the paper
version of the same
flyer. However, because of the size of the digital image, there is often
associated a slow
browsing time. Moreover, because the flyer is simply an image, there are
little additional
functions and information associated with the digital flyer.
[0008] More advanced systems for providing digital flyers do exist. Such
systems allow for
a more fluid browser experience such as the ability to zoom-in and zoom-out of
the image and
have mechanisms to provide additional information about each item in the flyer
(e.g. product
description). However, the advanced systems rely on browser plug-in such as
Adobe Flash or
other proprietary framework that requires a bulky initial download and lengthy
installation.
Moreover, the proprietary digital flyer system that runs on such proprietary
framework are
complex and requires significant resources on the client's device (such as a
PC, Mac,
smartphones, tablets and any other communications devices) to run smoothly.
[0009] In addition to the burden imposed on the client's device, there is a
high operational
burden for the businesses preparing the digital flyer. On existing systems, in
order to provide
any additional information on the flyer, the image had to be manually labelled
and tagged. For
example, to provide information about products presented on the flyer, each
product had to be
tagged. This is a time consuming process.
[0010] Lastly, existing digital flyers (like traditional paper flyers) do
not provide detailed
insights and feedback to the businesses as to how and where the viewers were
interested and
interacted with the flyer. At most, digital and printed flyers may provide
feedback about viewer's
interest in a particular product through coupons (in printed flyers) or click-
throughs of manually
tagged items (in digital flyers).
[0011] Therefore, a new generation of interactive digital flyers that
addresses at least some
of these shortcomings would be highly desirable.
2
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
SUMMARY
[0012] According to an embodiment in the present disclosure, a
communication device for
displaying a digital flyer is disclosed. The communication device includes a
user interface for
receiving a request for the digital flyer, the digital flyer comprising one or
more flyer image; a
communication module configured to request for the digital flyer at a zoom
level, the request
including information related to a viewport of the communication device; and
receive a flyer
image tile within a flyer image tile set of the requested digital flyer
associated with the zoom
level, the set including multiple copies of the digital flyer segmented into
tiles with each copy
representing the digital flyer at one of a plurality of zoom levels. The
communication device also
includes a display for displaying the flyer image tile.
[0013] According to another embodiment in the present disclosure, a method
of displaying
a digital flyer on a communication device is disclosed. The method includes
requesting at the
communication device for a digital flyer at a zoom level, the digital flyer
comprising one or more
flyer image, the request including information related to a viewport of the
communication device;
receiving a flyer image tile within a flyer image tile set of the requested
digital flyer associated
with the zoom level, the set including multiple copies of the digital flyer
segmented into tiles with
each copy representing the digital flyer at one of a plurality of zoom levels;
and displaying the
flyer image tile on the communication device.
[0014] According to a further embodiment, a computer-readable storage
medium
comprising instructions in code which when loaded into a memory and executed
by a processor
of a communication device causes the communication device to perform the
method of
displaying a digital flyer as specified above is disclosed.
[0015] According to another embodiment in the present disclosure, an
interactive digital
flyer system is disclosed. The interactive digital flyer system includes a
flyer image tiler for
creating a flyer image tile set for a digital flyer, the digital flyer
comprising one or more flyer
image, the flyer image tile set comprising multiple copies of the digital
flyer, with each copy of
the digital flyer representing the digital flyer at one of a plurality of zoom
levels and being
segmented into a plurality of flyer image tiles; and a flyer image tile store
for storing the flyer
image tile set.
[0016] According to a further embodiment in the present disclosure, a
method of creating a
flyer image tile set using a flyer image is disclosed. The method includes
determining a first
3
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
dimension associated with a resolution of a viewport of a communication
device; determining a
first scaling factor for scaling the flyer image to fit within the first
dimension of the viewport;
determining a second dimension associated with a resolution required to
display the flyer image
at sufficient detail; determining a second scaling factor for scaling the
flyer image to fit within the
second dimension; determining a range between the first and second scaling
factors, the range
being used to determine a plurality of zoom levels; and generating a copy of
the flyer image at
each of the plurality of zoom levels, the copy being segmented into tiles.
[0017] According to an even further embodiment, a computer-readable storage
medium
comprising instructions in code which when loaded into a memory and executed
by a processor
of a communication device causes the communication device to perform the
method of creating
a flyer image tile set using a flyer image as specified above is disclosed.
[0018] According to another embodiment in the present disclosure, a method
of processing
a digital flyer is disclosed. The method includes receiving a request for a
digital flyer at a zoom
level, the digital flyer comprising one or more flyer image, the request
including information
related to a viewport of the communication device; retrieving from a flyer
image tile store a flyer
image tile within a flyer image tile set of the requested digital flyer, the
set including multiple
copies of the digital flyer segmented into tiles with each copy representing
the digital flyer at one
of a plurality of zoom levels; and sending the retrieved flyer image tile to
the communication
device.
[0019] According to a further embodiment, a computer-readable storage
medium
comprising instructions in code which when loaded into a memory and executed
by a processor
of a communication device causes the communication device to perform the
method of
processing a digital flyer as specified above is disclosed.
[0020] According to an embodiment in the present disclosure, a method of
displaying a
digital flyer with contextual information in a viewport of a communication
device is disclosed.
The method includes receiving a source flyer image and polygon mapping
information
associated with a digital flyer from a server, the polygon mapping information
comprising a
polygon for defining a polygonal area on the source flyer image, the polygon
being tagged with
contextual information; and overlaying the source flyer image with the polygon
mapping
information.
4
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
[0021] According to another embodiment, a computer-readable storage medium
comprising
instructions in code which when loaded into a memory and executed by a
processor of a
communication device causes the communication device to perform the method of
displaying a
digital flyer with contextual information in a viewport of a communication
device as specified
above is disclosed.
[0022] According to a further embodiment in the present disclosure, a
system for
contextualizing a digital flyer is disclosed. The system includes a polygon
mapping module for
incorporating contextual information in the digital flyer using polygon
mapping information, the
polygon mapping information comprising a polygon for defining a polygonal area
on a source
flyer image of the digital flyer, the polygon being tagged with the contextual
information; and a
flyer data store for storing the tagged polygon.
[0023] According to another embodiment in the present disclosure, a method
for displaying
a digital flyer comprising a flyer image tile set on a communication device is
disclosed. The
method includes retrieving a flyer image tile from the flyer image tile set,
the flyer image tile
being associated with the most zoomed-out state of a viewport of the
communication device, the
retrieved flyer image tile being associated with an underlay viewport;
retrieving a flyer image tile
from the flyer image tile set, the flyer image tile being associated with the
current zoom level of
the viewport, the retrieved flyer image tile being associated with a current
viewport; displaying in
the viewport, a portion of the flyer image tile associated with the underlay
viewport that is visible
in the viewport; and overlaying onto the portion of the flyer image tile
associated with the
underlay viewport that is displayed in the viewport, a portion of the flyer
image tile associated
with the current viewport that is visible in the viewport.
[0024] According to another embodiment, a computer-readable storage medium
comprising
instructions in code which when loaded into a memory and executed by a
processor of a
communication device causes the communication device to perform the method of
displaying a
digital flyer comprising a flyer image tile set on a communication device as
specified above is
disclosed.
[0025] According to a further embodiment in the present disclosure, a
communication
device is disclosed. The communication device includes a pre-loading module
configured to
retrieve a first and a second set of flyer image tiles from a flyer image tile
set stored in memory,
the first set of flyer image tiles being associated with the most zoomed-out
state and the second
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
set of flyer image tiles being associated with the current zoom level of a
viewport of the
communication device, the pre-loading module further configured to associate
the first set of
flyer image tiles with an underlay viewport and the second set of flyer image
tiles with a current
viewport; and a display configured to display the first set of flyer image
tiles that are visible in
the viewport and overlay the second set of flyer image tiles visible in the
viewport onto the first
set of flyer image tiles.
[0026] According to an even further embodiment in the present disclosure, a
method of
managing memory of an interactive flyer system is disclosed. The method
includes determining
that the memory requires management; ordering flyer image tiles stored in the
memory in z-
order; and for each top z-order flyer image tile, determining portion of the
flyer image tiles stored
in the memory that is covered by the top z-order flyer image tile, and
removing from the memory
the determined portion of the flyer image tiles.
[0027] According to another embodiment in the present disclosure, a
communication device
is disclosed. The communication device includes memory for storing one or more
flyer image
tiles associated with a digital flyer and memory management module configured
to execute on
one or more processors. The memory management module includes instructions to
determine
that the memory requires management; order the one or more flyer image tiles
stored in the
memory in z-order; and for each top z-order flyer image tile, determine
portion of the one or
more flyer image tiles stored in the memory that is covered by the top z-order
flyer image tile,
and remove from the memory the determined portion of the one or more flyer
image tiles.
[0028] According to a further embodiment in the present disclosure, a
computer-readable
storage medium comprising instructions in code which when loaded into a memory
and
executed by a processor of a communication device causes the communication
device to
perform the method of managing memory of an interactive flyer system as
specified above is
disclosed.
BRIEF DESCRIPTION OF THE DRAWINGS
[0029] These and other features and advantages of the disclosure will
become apparent
from the following detailed description, taken in combination with the
appended drawings, in
which:
6
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
[0030] FIG. 1 depicts a block diagram of a communication system in which an
embodiment
of the present technology may be applied;
[0031] FIG. 2 shows a block diagram of an interactive flyer system
according to an
embodiment of the present technology;
[0032] FIG. 3 illustrates the layers of an interactive flyer according to
an embodiment of the
present technology;
[0033] FIG. 4A illustrates an interactive flyer at the most zoomed-out
level configured to fit
within a viewport of a display of a communication device;
[0034] FIG. 46 illustrates the interactive flyer of FIG. 4A at the next
zoom level;
[0035] FIG. 5 shows a process for generating the interactive flyer
according to an
embodiment of the present technology;
[0036] FIG. 6 illustrates a process for preparing a flyer image for use in
the base layer as
shown in FIG. 3 in generating an interactive flyer;
[0037] FIG. 7 illustrates a process for segmenting the flyer image
according to an
embodiment of the present technology;
[0038] FIG. 8 illustrates an exemplary flyer image tile set segmented using
a binary zoom
ratio;
[0039] FIG. 9 illustrates another exemplary flyer image tile set segmented
using a flexible
zoom ratio;
[0040] FIG. 10A depicts a process for a communication device for requesting
and receiving
a flyer image tile set at an initial zoom level;
[0041] FIG. 10B depicts a process for an interactive flyer system for
responding to a
request as described in FIG. 10A;
[0042] FIG. 11A depicts a process for a communication device for requesting
and receiving
a flyer image tile based on viewer action;
7
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
[0043] FIG. 11B depicts a process for an interactive flyer system for
responding to a
request as described in FIG. 11A;
[0044] FIG. 12 shows a flyer image contextualized through polygon mapping
according to
an embodiment of the present technology;
[0045] FIG. 13 illustrates a process for contextualizing a flyer image
using polygons;
[0046] FIG. 14 illustrates a process for displaying a digital flyer with
contextual information;
[0047] FIG. 15 illustrates an embodiment of navigational information
generated using
polygon mapping;
[0048] FIG. 16 shows a process for generating the embodiment as shown in
FIG. 15 using
the polygon mapping information;
[0049] FIG. 17 illustrates another embodiment of navigation information
generated using
polygon mapping;
[0050] FIG. 18 shows a process for generating one of the embodiments as
shown in FIG.
17 using the polygon mapping information;
[0051] FIG. 19A shows a process for generating one of the embodiments as
shown in FIG.
17 using the polygon mapping information;
[0052] FIG. 19B shows the visual representation of the process in FIG. 19A;
[0053] FIG. 20 shows a process for generating one of the embodiments as
shown in FIG.
17 using the polygon mapping information;
[0054] FIG. 21 illustrates a process for generating related information
using contextual
information contained in another related flyer;
[0055] FIG. 22 illustrates the related information generated using the
process in FIG. 21;
[0056] FIG. 23A illustrates a snapshot of a viewer's viewport constructed
using device
information and polygon mapping information according to an embodiment of the
present
technology;
8
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
[0057] FIG. 23B illustrates the snapshot of FIG. 23A with an altered
viewport size;
[0058] FIG. 24 illustrates a process for generating feedback using the
device information
and polygon mapping information according to an embodiment of the present
technology;
[0059] FIG. 25 depicts an embodiment of a feedback report generated using
feedback data
stored in the feedback system;
[0060] FIG. 26 depicts another embodiment of a feedback report generated
using feedback
data stored in the feedback system;
[0061] FIG. 27 depicts a process implementing a feedback system according
to another
embodiment of the present technology;
[0062] FIG. 28 depicts another embodiment of a feedback report generated
using feedback
data stored in the feedback system;
[0063] FIG. 29 illustrates an embodiment for predictively caching the flyer
image tiles by
using the feedback data stored in the feedback system;
[0064] FIG. 30 illustrates another embodiment for predictively caching the
flyer image tiles
by using contextual information conveyed through polygon mapping information;
[0065] FIG. 31 illustrates a process for pre-loading flyer image tiles
according to an
embodiment of the present disclosure;
[0066] FIG. 32 illustrates an embodiment for pre-loading flyer image tiles
by using an
underlay viewport;
[0067] FIG. 33 illustrates how the underlay viewport is used with the
current viewport;
[0068] FIG. 34A illustrates another embodiment of the pre-loading method;
[0069] FIG. 34B illustrates an exemplary situation the pre-loading method
according to an
embodiment of the present disclosure may address;
[0070] FIG. 35 illustrates how the exemplary situation shown in FIG. 33B
may be
addressed by an embodiment of the pre-loading method disclosed in this
disclosure;
9
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
[0071] FIG. 36A illustrates two sets of flyer image tiles, one having
flexible zoom levels and
the other having inflexible zoom levels;
[0072] FIG. 36B illustrates the two sets of flyer images tiles in FIG. 35A
with one of the four
tiles loaded;
[0073] FIG. 37 illustrates a method for managing memory according to an
embodiment of
the present disclosure; and
[0074] FIGs 38A-38E illustrates how the memory management in FIG. 36
proceeds
visually.
[0075] It will be noted that throughout the appended drawings, like
features are identified by
like reference numerals.
DETAILED DESCRIPTION
[0076] In this specification and the appended claims, the singular forms
"a," "an," and "the"
include plural references unless the context clearly dictates otherwise.
Unless defined
otherwise, all technical and scientific terms used herein have the same
meaning as commonly
understood to one of ordinary skill in the art to which this disclosure
belongs.
[0077] It will be further understood that the terms "comprises" or
"comprising", or both when
used in this specification, specify the presence of stated features, integers,
steps, operations,
elements, and/or components, but do not preclude the presence or addition of
one or more
other features, integers, steps, operations, elements, components, and/or
groups thereof.
[0078] The term "flyer", as described above, will be used generally to
refer to forms of
marketing that includes information about the wares and services that
businesses offer. "Flyer"
may also be known as circulars, pamphlets, handbills, leaflets, ads,
catalogues, parts books,
and trade literatures. In this specification, the term "flyer" will be used to
collectively refer to the
aforementioned forms of marketing.
[0079] The terms "interactive flyer", "digital flyer", and "interactive
digital flyer" in this
disclosure may be used interchangeably to refer to a flyer generated by an
interactive flyer
system according to an embodiment of the present technology.
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
[0080] The terms "left", "right", "up", and "bottom" in the present
disclosure refers generally
to the direction with respect to the viewport of the communication device. It
will be understood
that these terms are relative to the orientation of the viewport.
[0081] The terms "verticalTheight"ry-axis" and "horizontal"/"width"/"x-
axis" in the present
disclosure are used with respect to the viewport of the communication device.
It will be
understood that these terms are relative to the orientation of the viewport.
[0082] The term "device information" refers to information about the
communication device.
For example, the device information may include viewport information, timezone
of the device,
date and time at which the communication device is being operated, IP address,
geographic
locale, operating system, Internet browser information and other information.
The viewport
information is information related to the viewport of the communication
device, including
information about the display of the communication device.
[0083] The term "feedback information" refers to information used in
generating a feedback
report. In one embodiment, the feedback information may include device
information sent from
the communication device to determine, in conjunction with polygon mapping
information,
pageviews to generate the feedback report. In another implementation, the
feedback
information may include viewscore of each flyer image tile, which may be used
to generate the
feedback report.
[0084] The term "user" in the present disclosure is used to refer to any
user of the
interactive flyer system including publishers, vendors and viewers.
[0085] The term "publisher" refers to the user of the interactive flyer
system responsible for
preparing the interactive digital flyer according to an embodiment of the
present technology.
[0086] The term "vendor" or "merchant" in this disclosure may be used
interchangeably to
refer to the business using the interactive digital flyer for marketing and
business purposes.
[0087] The term "viewer" refers to the user of the interactive flyer system
viewing the
interactive digital flyer according to an embodiment of the present
technology.
[0088] The term "wireless communication technology" is intended to describe
any
communication protocol that enables a communication device to communicate with
a server
without wires. By way of example, but not limitation, wireless communication
protocol may
11
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
include IEEE 802.11x standards (sometimes referred to as Wi-Fi) such as, for
example, the
IEEE 802.11a, 802.11b, 802.11g, and/or 802.11n standard. Other communication
protocols
include: IEEE 802.16e (also referred to as Worldwide Interoperability for
Microwave Access or
"WiMAX"); IEEE 802.20 (also referred to as Mobile Wireless Broadband Access);
Mobitex Radio
Network, DataTAC; GPRS (General Packet Radio System); TDMA (Time Division
Multiple
Access) including GSM (Global System for Mobile Communication); CDMA (Code
Division
Multiple Access) including CDMA 2000 and UTRA (Universal Terrestrial Radio
Access); FDMA
(Frequency Division Multiple Access); OFDMA (Orthogonal FDMA), SC-FDMA (Single-
Carrier
FDMA); CDPD (Cellular Digital Packet Data); iDEN (integrated Digital Enhanced
Network);
EvD0 (Evolution-Data Optimized); EDGE (Enhanced Data rates for GSM Evolution);
UMTS
(Universal Mobile Telecommunication Systems); HSPDA (High-Speed Downlink
Packet
Access); and other various network technologies, including LTE-advanced
technologies. An
OFDMA system utilizes Orthogonal Frequency Division Multiplexing (OFDM) and
sends
modulation symbols in the frequency domain on orthogonal subcarriers, and may
be
implemented in technologies such as E-UTRA or E-UTRAN.
[0089] The details and particulars of these aspects of the technology will
now be described
below, by way of example, with reference to the attached drawings.
[0090] Referring to FIG. 1, there is shown a block diagram of a
communication system 100
in which an embodiment of the present technology can be applied. It should be
expressly
understood that this figure is intentionally simplified to show only certain
main components. The
communication system 100 may include other components beyond what is
illustrated in FIG. 1.
[0091] The communication system 100 includes a number of communication
devices
102,104, 106 which may be connected to the communication system 100 in any of
several
different ways. Accordingly, three different types of communication devices
102, 104, 106 are
depicted in FIG. 1 employing different exemplary ways of connecting to the
communication
system 100. It should be expressly understood that communication devices other
than those
shown in FIG. 1 may be used to connect to the communication system 100.
Communication
devices 102, 104, 106 may be connected to the server 160 using wireless
technologies or wired
communication technologies.
[0092] In one embodiment, communication devices 102, 104 may connect to the
server 160
using wireless communication technology 110. To facilitate the provision of
wireless technology
12
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
110, there may be provided one or more wireless RF Access Points (AP) 112 and
base station
114.
[0093] In another embodiment, communication device 106 may connect to the
server 160
using wired communication technology 120 such as IEEE 802.3 standard. Other
wired
communication technologies may be contemplated by a skilled person in the art.
[0094] Using wireless or wired technologies, communication devices 102,
104, 106 connect
to the network gateway 130. The internal network 150 is typically behind a
firewall 140, which
serves to safeguard the internal network 150 from unauthorized access. Only
authorized
communication devices 102, 104, 106 are granted access to the server 160.
[0095] Now turning to FIG. 2, a block diagram of an interactive flyer
system 200 is shown.
It should be expressly understood that this figure is intentionally simplified
to show only certain
main components. The interactive flyer system 200 may include other components
beyond
what is illustrated in FIG. 2. In one implementation, the interactive flyer
system 200 may be
implemented in the server 160. In another implementation, the interactive
flyer system 200 may
be implemented in a distributed environment.
[0096] As shown in FIG. 2, the interactive flyer system 200 includes a
flyer image tiler 210
and a flyer image tile store 220. Additionally, the interactive flyer system
200 may include
components for contextualizing the interactive flyer. The components may
include a polygon
mapping module 230 and a flyer data store 240. Furthermore, the interactive
flyer system 200
may also include a feedback data store 250 and a feedback data analyzer 260.
While the
components of the interactive flyer system 200 are shown in FIG. 2 as distinct
components, it
will be understood that they may be implemented as a single component or
grouped in any
combination of components. Moreover, some of the components may reside
externally. For
example, the flyer image tile store 220 may be located remotely and accessed
when generating
the interactive flyer. The flyer image tile store 220, flyer data store 240,
and feedback data store
250 may be implemented in any form of memory such as, but not limited to,
random access
memory (RAM), flash memory, magnetic disk, solid state drives or other forms
of storage
format.
[0097] Thus, as it will be described below, an interactive digital flyer
comprises a base
layer, a geometry layer and, optionally, a data layer. The base layer includes
a source flyer
image. However, according to an embodiment of the present technology, the
source flyer
13
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
image is a flyer image tile set having at least one flyer image tile. The
flyer image tile set
includes multiple copies of the flyer image at various zoom levels, with a
copy of the flyer image
at the most zoomed-out level being sized and formatted to fit within the
resolution of the
viewport of the communication device. Each copy of the flyer image at the
varying zoom levels
is segmented into tiles. The geometry layer contains co-ordinate and scaling
information
associated with the flyer image tiles. Where the interactive flyer system 200
includes a flyer
data store 240, the interactive digital flyer may also include a data layer.
Flyer data may include
polygon mapping information, product data, point of interest information,
feedback data and
other information that will be discussed below. It will be understood that the
interactive flyer
may include a single data layer containing all the flyer data or may include
multiple data layers
with each data layer containing a type of flyer data. For example, first data
layer may include
polygon mapping data, second data layer may include product data, and so on.
[0098] The combination of these layers thus provides for an advanced
interactive flyer. The
base layer comprising of at least one set of flyer image tiles and the
geometry layer enables the
interactive flyer system 200 to provide advanced features such as zoom and pan
without any
additional proprietary software such as Adobe Flash or Microsoft Silverlight
when used in a
web browser. Moreover, the data layer provides contextual information that can
be used in
variety of ways to provide for advanced features. Where interactive flyer
system 200 includes a
feedback system, the data layer may be dynamically updated according to viewer
behaviour.
Thus the interactive flyer can change with viewer actions. These and other
features of the
interactive flyer system 200 will now be described in detail.
Interactive Flyer Structure
[0099] The interactive flyer according to the present technology comprises
of several layers
of information. Turning to FIG. 3, at its base, the interactive flyer includes
the source flyer
image. However, instead of a single image, the base layer 300 comprises
multiple sets (e.g.
300a, 300b, 300c) of the flyer image for various screen resolutions, with each
set including
multiple copies (e.g. 300a1, 300a2, 300a3) of the flyer image and with each
copy being the flyer
image at a particular zoom level.
[00100] In FIG. 3, the base layer 300 includes three sets (300a, 300b, and
300c) of the flyer
image, each set for different resolution of a viewport of a communication
device. In each set,
there contains multiple copies (e.g. 300a1, 300a2, 300a3) of the flyer image,
with each copy
representing a zoom level. In FIG. 3, each set (300a, 300b and 300c) is shown
with three zoom
14
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
levels (or two zoom steps) (e.g. 300a1, 300a2, and 300a3), with each copy of
the flyer image in
the set being segmented into tiles. In other words, there are at least one set
of flyer images
(e.g. 300a, 300b, 300c) where the most zoomed-out level of the flyer image in
the set (e.g.
300a1, 300b1 or 300c1) is configured such that the entire flyer image fits
within the viewport.
Thus, in FIG. 3, the most zoomed-out level flyer image in the set (e.g. 300a1,
300b1, 300c1) is
configured to fit three different resolutions of a viewport. For example,
300a1, 300b1 and 300c1
may be the flyer images for a communication device having resolutions 1024 by
768, 1280 by
800, and 1920 by 1080. From that zoomed-out level, each copy thereafter
represents the flyer
image at an increasing zoom level.
[00101] While FIG. 3 depicts three sets of the flyer image, it will be
understood that there
may be more or less than three sets of the flyer image. The details of
segmenting the flyer
image will be described in further details below.
[00102] The interactive flyer further includes a geometry layer 302 that
provides a reference
system in the digital space for the tiled flyer images. For example, the
geometry layer 302 may
include a 2D X-Y coordinate system and scaling system to allow the digital
flyer to seamlessly
pan and zoom using the tiled images. Referring to FIG. 4A, a screen 400 is
shown with its
viewport 402 occupying a majority of the screen space. In the viewport 402,
there is shown a
flyer image 404 configured to fit the entire flyer image 404 within the
viewport 402. In other
words, the most zoomed-out view of the flyer image 404 in the tile set is
configured to fit within
the viewport 402.
[00103] To implement seamless panning and zooming of the flyer images using
the tile set,
a coordinate and scaling system may be used. In this embodiment, the
coordinate system may
designate the bottom left corner of the flyer image 404 as the starting
coordinate (0,0). This
allows the interactive flyer system 200 to determine where and how the viewer
interacted with
the flyer image 404. For example, if the viewer hovers using a mouse pointer
406 as shown in
FIG. 4A and requests a zoom-in action (e.g. using a scroll wheel), the
interactive flyer system
200 may determine which flyer image tile to retrieve as follows. First, using
the coordinate
system, the system is able to determine where in the flyer image the mouse
pointer 406 is
hovering. This can be done by converting the coordinate where the mouse
pointer 406 is
located to pixel equivalent of the flyer image 404. In one embodiment, the
system may utilize
the resolution (e.g. DPI (dots per inch)) of the flyer image. Thereafter, the
system may retrieve
the tiled images in the next zoom level that is displayable within the
viewport 402. For example,
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
FIG. 413 depicts the flyer image 410 at the next zoom level. As it can be
seen, the flyer image
410 at the next zoom level is greater in resolution and thus, larger than the
flyer image 404 at
the previous zoom level. At this zoom level, the system retrieves the shaded
flyer image tiles
412 that are displayable within the viewport 402. Thus, as the viewer pans
using the pointer
414, the appropriate flyer image tiles are loaded and shown in the viewport
402.
[00104] The interactive flyer may also include a data layer 304 that
provides flyer-related
information such as context (i.e. polygon mapping information as it will
described below),
product data, points of interest and feedback information.
[00105] Thus, according to an embodiment of the present technology, an
interactive flyer is
generated by creating a base layer comprising at least one flyer image tile
within a flyer image
tile set (i.e. 500 in FIG. 5). Each flyer image tile set represents the flyer
image configured for
common resolutions of a display of a communication device. For example, there
may be two
sets of flyer image tiles with one set configured for a display having 1920 by
1080 resolution and
the other set configured for a display having 1024 by 768 resolution. In each
set, there contains
multiple copies of the flyer image, with each copy representing the flyer
image at a particular
zoom level. Furthermore, each copy of the flyer image is segmented into tiles.
Thereafter, a
geometry layer is defined with the geometry layer containing co-ordinate and
scaling information
about the flyer image tiles (i.e. 502 in FIG. 5). The interactive flyer may
further include creating
a data layer having flyer-related information (i.e. 504 in FIG. 5). The data
layer may further
include context, points of interest, product specific data and feedback
information. As discussed
above, there may be one or more data layers incorporating the flyer-related
information.
[00106] While FIG. 5 depicts a process comprising process elements 500, 502
and 504 in
order, it will be understood that the process elements 500, 502 and 504 may be
carried out in
other orders. It will also be understood that other process depicted in other
figures may also be
carried out in other orders as reasonably contemplated.
Tiled Flyer Image
[00107] Referring to FIG. 6, preparation of the flyer image tile set for
use in the base layer of
the interactive flyer begins with obtaining the flyer image. The flyer image
may be provided by
the retailer or may be scanned from the printed-version of the flyer. The
flyer image is then
normalized (i.e. 600 in FIG. 6). Normalization refers to the process of
manipulating the flyer
16
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
image to conform the flyer image to a standard format. This may entail
applying some form of
image processing, such as crop, image alignment, image adjustment and image
scaling.
[00108] The normalized flyer image is then segmented into tiles (i.e. 602
in FIG. 6). Tiles
are portions of the flyer image at varying resolutions optimized for different
screen sizes and
resolutions. Because of the bounded nature of a flyer, the flyer image is
optimized such that, at
the most zoomed-out level, the full image of the flyer fits within the
dimensions of the viewport,
while at the most zoomed-in level, the flyer page includes the necessary level
of details. For
example, if the flyer page includes fine prints, the most zoomed-in level
should include sufficient
level of detail to allow the viewer to read the fine prints.
[00109] Using this range (i.e. the most zoomed-out to zoomed-in level) as
reference, the
flyer image tiler 210 generates multiple copies of the flyer image at
different resolutions, with
each copy representing the flyer image at increasing zoom level. In other
words, with the range
determined, the flyer image tiler 210 generates a copy of the flyer image for
each zoom level,
with each copy containing more resolutions than the previous copy to provide
greater level of
detail. These multiple copies represent one set of the flyer image tiles (e.g.
300a in FIG. 3) and
the process is repeated for other commonly known resolutions of display
screens (e.g. 300b and
300c in FIG. 3). The generated tiles, the normalized flyer image, and the
information related to
the flyer image (i.e. coordinate system and zoom levels) may then be stored in
the flyer image
tile store 220 (i.e. 604 in FIG. 6).
[00110] Turning to FIGs. 7-9, an example embodiment of segmentation of the
flyer image
will be described in detail (i.e. 602 in FIG. 6). The flyer image tiler 210
determines the
dimension associated with the resolution of the viewport (i.e. 700 in FIG. 7).
Using the
dimension, the flyer image tiler 210 determines a first scaling factor ("131")
for scaling the flyer
image so that the entire flyer image fits within the determined dimension
(i.e. 702 in FIG. 7).
[00111] The flyer image tiler 210 then determines the dimension associated
with a resolution
required to display the flyer image at sufficient detail (i.e. 704 in FIG. 7).
For example, the flyer
may contain fine prints that require a particular resolution for the viewer to
be able to read the
fine prints. Using this dimension, the flyer image tiler 210 is able to
determine a second scaling
factor ("132") for scaling the flyer image such that the image includes the
requisite details at the
most zoomed-in (i.e. blown up) state (i.e. 706 in FIG. 7).
17
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
[00112] The flyer image tiler 210 then determines the range ("R") between
the first and
second scaling factor (i.e. 708 in FIG. 7). For example, if p1=1 and p2=12,
R=12 (i.e. 32/31). In
other words, the range is a factor of the available dimension of the viewport
and the dimension
of the original flyer image where sufficient level of detail can be seen.
[00113] The flyer image tiler 210 then generates a copy of the flyer image
at each zoom
level between the range, with each copy being segmented into tiles (i.e. 710
in FIG. 7). In one
implementation, each zoom level is 2 times the previous zoom level (i.e. FIG.
8). In another
implementation, the zoom level is flexible to allow the system to choose the
most appropriate
zoom ratio given the range and the number of zoom steps required (i.e. FIG.
9).
[00114] In the first implementation, the number of zoom steps is inflexible
because each
zoom level is 2 times the previous zoom level. In other words, the zoom ratio
is 2 and the
number of zoom steps is determined by this zoom ratio. To determine the number
of zoom
steps required, the system determines the smallest power of 2 that is equal to
or greater than
the range, which will be hereinafter referred to as Rp. For example, if the
range required is 12,
the system determines Rp to be 16 (24). This means that 4 zoom steps (i.e.
transition steps
between lx, 2x, 4x, 8x and 16x) are required to achieve a range of 12. Because
12 is not a
power of 2, the system determines Rp, which serves to ensure that the system
provides equal
or greater level of detail. In this example, to achieve a range of 12 using a
zoom level of 2, the
system determines that minimum of 4 zoom steps are required.
[00115] Referring to FIG. 8, a tile set 800 for a given screen resolution
is shown. With the
given screen resolution, the flyer image tiler 210 determines the available
screen dimension (i.e.
height and width) and determines the scaling factor (i.e. 31) required to fit
the entire flyer image
within the screen of the communication device.
[00116] The flyer image tiler 210 then determines the resolution that is
required to display
the flyer image at sufficient detail. This resolution allows the flyer image
tiler 210 to determine
the required dimension to show the flyer at the required level of detail, for
example to read the
fine prints. Using the required dimension, the flyer image tiler 210
determines the scaling factor
(i.e. 132) required to scale the flyer image to show sufficient detail.
Thereafter, the flyer image
tiler 210 determines the range R between 131 and 132, and Rp. Using the
numbers from the
above example, 131=1 and 132=12, the system determines R=12 and Rp=24=16.
Therefore, the
18
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
tile set 800 includes 4 steps or 5 zoom levels (i.e. 4 transition steps
between lx, 2x, 4x, 8x and
16x zoom levels).
[00117] Alternatively, or additionally, the flyer image tiler 210 may
retrieve or receive a list of
common resolutions of viewer's screens (e.g. 1920 by 1080, 1280 by 800 and
1024 by 768) and
determine the scaling factor (i.e. 131) to fit the flyer image within each of
the common resolution
in the list. Thereafter, for each common resolution in the list, the system
may generate multiple
copies of the flyer image at each zoom level, with each copies of the flyer
image being
segmented into tiles. The flyer image tiles for each common resolution makes
up a tile set. For
example, in FIG. 3, there are three sets of flyer image tiles for three
different screen resolutions.
[00118] In the second implementation, the number of zoom steps is flexible,
with each zoom
level not being necessarily twice the previous zoom level. In other words, the
zoom ratio is not
necessarily 2. Using the above example, using flexible zoom level, the system
is able to
provide exactly the required range of 12. After determining the required
range, the system
determines the appropriate zoom steps ("S") to transition from viewing the
full flyer within the
viewport to viewing the flyer at the finest detail level. In this example,
assume that the system
determines S to be 3. Thus, the zoom ratio may be calculated using the
following formula:
Logx(R)=S, where X is the zoom ratio. Using the numbers from the current
example, with
Logx(12)=3, the system determines the zoom ratio to be 2.29. The resulting
zoom levels for the
3 zoom steps are then lx, 2.29x, 5.24x and 12x, as shown in FIG. 9. Similarly
to the first
implementation, the system may generate a plurality of tile sets 910 of the
flyer image 912
where the system retrieves or receives a list of common resolutions.
[00119] Moreover, in both the first and second implementation, after
determining the zoom
ratio or zoom steps, there may be a review process to add or eliminate one or
more zoom
levels. The review process analyzes the generated copies of the tiled flyer
image at the plurality
of zoom levels to ensure that the zoom steps between the zoom levels are
appropriate for the
viewer. Referring to the first implementation above, after the review process,
it may be
determined that zoom level 8x does not add much value to the range in the
flyer image tile set,
and thus, the flyer image tiles at zoom level 8x may be eliminated. Thus, only
3 zoom steps or
4 zoom levels would exist ¨ i.e. lx, 2x, 4x, and 16x. The same can be applied
to the second
implementation, or any other implementation. In the second implementation, it
may be
determined that because of the content in the flyer image, zoom level 2.29x is
redundant or
unnecessary. In such a case, the flyer image tiles in the zoom level 2.29x may
be eliminated,
19
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
resulting in only 2 zoom steps or 3 zoom levels: lx, 5.24x and 12x.
Conversely, it may be
determined that the jump from 5.24x to 12x may be too steep after the review
process. In such
a case, an intermediate zoom level may be added. For example, a halfway zoom
level between
5.24x and 12x (i.e. 8.62x) may be added to improve the viewer's browsing
experience. Thus,
after the analysis, the system may remove one of the copies of the tiled flyer
image from the
flyer image tile set that corresponds to one of the plurality of zoom levels.
Conversely, the
system may add a new copy of the tiled flyer image into the flyer image tile
set corresponding to
a new zoom level that is absent from the plurality of zoom levels.
[00120] Thus, the communication device (e.g. 106 in FIG. 1) communicates
with the
interactive flyer system 200 to display an interactive digital flyer.
Referring to FIGs. 10 and 11,
the communication device requests for an interactive digital flyer at an
initial zoom level, with the
request comprising device information of the communication device (i.e. 1000
in FIG. 10A). For
example, the device information may include viewport information such as
viewport size and
resolution. It will be understood that device information may include more
than viewport
information such as information related to the device such as locale
information, IP address,
and other device related information that may be used by the interactive flyer
system 200. The
interactive flyer system 200 receives the request for the digital flyer at the
initial zoom level (i.e.
1010 in FIG. 10B) and retrieves from a flyer image tile store (e.g. 220 in
FIG. 2) a flyer image tile
within a flyer image tile set (i.e. 1012 in FIG. 10B). As discussed in the
disclosure, the flyer
image tile set includes multiple copies of the flyer image segmented into
tiles with each copy
representing the flyer image at a zoom level. Since the flyer image tile at
the initial zoom level
was requested, the flyer image tile set associated with the resolution of the
viewport of the
communication device is selected. Thereafter, the flyer image tile, within the
flyer image tile set,
representing the most zoomed-out view configured to fit within the viewport of
the
communication device is retrieved. After retrieval, the flyer image tile
retrieved is sent and
received by the communication device (e.g. 1002 and 1014 in FIGs. 10A and 10B,
respectively).
The received flyer image tile is then displayed in the viewport of the
communication device (i.e.
1004 in FIG. 10A).
[00121] In addition to retrieving the flyer image tile, the system 200 may
retrieve flyer-related
information associated with the flyer. For example, the flyer-related
information may be
contextual information defined through polygon mapping, as it will be
discussed below.
Moreover, flyer-related information may be point of interest, product specific
or feedback
information. Details relating to flyer-related information will be further
described below.
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
[00122] After the communication device displays the digital flyer at the
initial zoom level, the
viewer may perform a viewer action, such as zoom or pan. Based on the viewer
action, the
communication device may further request for a new flyer image tile to be
displayed in the
viewport of the communication (i.e. 1100 in FIG. 11A), with the request
comprising device
information. In one implementation, the device information may include
viewport information
such as the location coordinates and the zoom level of the viewport (i.e.
information from the
geometry layer). Upon receiving the request for the new flyer image tile (i.e.
1110 in FIG. 11B),
the system 200 retrieves from the flyer image tile store 220 flyer image tile
that is associated
with the viewer action and displayable within the viewport of the
communication device (i.e.
1112 in FIG. 11B). The received new flyer image tile is then displayed in the
viewport of the
communication device (i.e. 1104 in FIG. 11A).
[00123] Additionally, the system may retrieve a flyer image tile that is
not yet displayable in
the viewport of the communication device but to be cached at the communication
device for
future retrieval. By pre-emptively caching flyer image tile(s) not yet
displayable in the viewport
but may soon be requested, performance is improved at both the communication
device and the
interactive flyer system. Given the structure of the interactive flyer,
caching may be based on
the information included therein to increase the hit rate of the cache. For
example, caching may
be based on the feedback information included in a feedback store, the device
information
included in the request, or flyer-related information included in a flyer data
store. After retrieval,
the flyer image tile(s) is sent and received by the communication device to be
displayed or
cached at the communication (i.e. 1102 and 1114 in FIGs. 11A and 11B).
Context Information ¨ Polygon Mapping
[00124] Within the interactive flyer system 200, a system for
contextualizing the interactive
flyer is disclosed. The system may comprise of a polygon mapping module 230
and a flyer data
store 240 as depicted in FIG. 2. While the present disclosure describes the
polygon mapping
module 230 and flyer data store 240 as being a subsystem of the interactive
flyer system 200, it
will be understood that it may be implemented as a single system or as a
separate and distinct
system (e.g. external to the interactive flyer system 200). It will also be
understood that the
system for contextualizing the interactive flyer may be distributed across
several systems. For
example, the flyer data store 240 may be part of the interactive flyer system
200 while the
polygon mapping module 230 may be separately implemented.
21
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
[00125]
Now also referring to FIGs. 12 and 13, a system for contextualizing an
interactive
flyer is disclosed. The system includes a polygon mapping module 230 for
contextualizing the
interactive flyer. To incorporate contextual information into the interactive
flyer, the polygon
mapping module 230 may use polygon mapping information that includes at least
one polygon
for defining a polygonal area on the source flyer image representing one or
more pages of the
digital flyer. Thereafter, the polygon(s) is tagged with the contextual
information. In other
words, the polygon mapping module 230 defines the polygonal area on a flyer
page using a
polygon and tags the polygon with contextual information (i.e. 1300 and 1302
in FIG. 13). It
will be understood that source flyer image generally refers to the underlying
image source for
the interactive digital flyer. In one embodiment, the source flyer image may
be a non-tiled
image representing the digital flyer. In another embodiment, the source flyer
image may include
a flyer image tile set as discussed above and exemplified in FIG. 3.
[00126]
In one embodiment, the polygon mapping module 230 may include an interface
that
allows a publisher, vendor or viewer to define the polygons and tag the
polygons with the
contextual information. For example, the publisher may prepare the interactive
digital flyer
using the information given from the vendor, including incorporating
contextual information
through the polygon mapping module 230. In another embodiment, after the
publisher prepares
the interactive digital flyer, the vendor may then incorporate the contextual
information through a
vendor-only portal. In a further embodiment, viewers may be crowdsourced to
define the
polygons using the polygon mapping module 230. In an even further embodiment,
the polygon
mapping module 230 may automatically define and tag the polygons based on
predetermined
information. This predetermined information may be provided by the vendor
based on an
agreed format of the digital flyer or may be based on feedback information
from a feedback data
store (e.g. 250 in FIG. 2).
[00127]
Additionally, the interface of the polygon mapping module 230 may allow the
publisher, vendor or viewer (e.g. crowdsourced) to define a point of interest
on the digital flyer.
A point of interest is an indication on the digital flyer intended to draw the
viewer's attention. For
example, it may point out popular items, severely discounted products or a
product promoted by
the vendor. Alternatively, or additionally, the point of interest may also be
defined by the
polygon mapping module 230 based on predetermined information as discussed
above.
[00128]
With polygon mapping information, the interactive flyer may be contextualized
without individually tagging or labelling the items in the interactive flyer.
Rather, polygonal area
22
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
may be used on the flyer page to convey contextual information that conveys
categorical or
sectional information such as "Electronics" and "Computers" as shown in FIG.
12. This greatly
reduces the time it takes to give meaning to the content included in a digital
flyer. However, it
will be understood, and will be further described below, that the polygons may
be used to define
polygonal areas at finer granular level, such as product or item level. This
polygon mapping
information may be stored in the flyer data store 240.
[00129] In FIG. 12, there is shown a flyer page 1200 that has been
contextualized using the
polygon mapping module 230. As shown, there are "top-level" polygons 1202,
1204, 1206 and
1208. The reference to "top-level" polygon refers to polygons that are not
contained in any
other polygon. While the polygons in FIG. 12 are generally rectangular in
shape (i.e. four
sides), it will be understood that polygons may take other forms having any
number of sides, so
long as the sides form a closed circuit to define an area (i.e. a polygonal
area). For ease of
illustration, polygon mapping in the present disclosure will be described with
polygons having
four sides.
[00130] Each of the "top-level" polygons defines a polygonal area 1202a,
1204a, 1206a and
1208a, which demarcates a section of the flyer page 1200 as having a
particular contextual
relationship. In FIG. 12, each of the "top-level" polygons have been tagged as
"Electronics",
"Sales", "Garden" and "Computer", which signifies that the items contained
within those
polygonal area all relate to "Electronics", "Sales", "Garden" or "Computers".
Furthermore, sub-
relationships may be defined by using overlapping polygons, such as polygons
1210, 1212 and
1214. In this particular example, the sub-level polygons 1210, 1212 and 1214
have been
tagged with the labels "TV", "Laptops" and "Memory". Thus, within the
polygonal area 1210a,
1212a and 1214a, the items contained therein will be one of "TV", "Laptops" or
"Memory".
[00131] In addition to the polygons, the polygon mapping module 230 may
define points of
interests, such as points of interests 1220 and 1222. These points may be
anything that may be
of interest to the viewer such as special discounts, popular items, new items,
items like-minded
viewers found interesting, and other interesting information.
[00132] Once the polygons are defined using the polygon mapping module 230,
the system
is able to provide feature-rich information including hierarchical
information, enhanced
navigation aids, improved caching, comprehensive feedback and other
information previously
not available in a digital flyer. In one embodiment, the interactive flyer
system may generate
23
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
navigational information using the polygons. The navigational information may
be in the form of
a table of content, or a navigational strip, or both. In other embodiments,
the polygons may be
used to generate feedback information, to better predict caching, and to
interlink between
related flyers. Some of the embodiments will be described below.
[00133] Thus, the communication device communicates with the interactive
flyer system 200
to display a digital flyer with contextual information. Referring to FIG. 14,
the communication
device receives a source flyer image and polygon mapping information
associated with a digital
flyer from a server (i.e. 1400 in FIG. 14). As previously mentioned, the
source flyer image may
be a non-tiled flyer image or a flyer image tile set as shown in FIG. 3. The
polygon mapping
information refers to polygons used to define a polygonal area on the source
flyer image, with
each polygon being tagged with contextual information. This polygon mapping
information is
overlaid on the source flyer image (i.e. 1402 in FIG. 14). For example, the
polygon mapping
information may be included in a data layer (e.g. 304 in FIG. 3) of the
digital flyer. Generally,
the polygon mapping information is not shown on the flyer page but rather used
to enrich the
viewer's experience by generating, for example, navigational information or
related information.
However, it will be understood that the polygon mapping information may be
shown to the user
overlaid on top of the flyer page. In such a case, the flyer page may appear
similar to flyer page
1200 as shown in FIG. 12.
[00134] In addition to receiving the polygon mapping information, the
communication device
may further receive point of interest information associated with the digital
flyer from the server.
Similarly to the polygon mapping information, point of interest information
may be overlaid over
the source flyer image and shown or not shown in the viewport of the
communication device.
[00135] With the polygon mapping information and point of interest
information, the
communication device is able to display the digital flyer with feature-rich
information. Such
feature-rich information may include navigational information and related
information as it will be
described below. Moreover, the polygon mapping information and the point of
interest
information may be used in conjunction with feedback information to improve
the efficiency and
data accuracy by the interactive flyer system.
Navigational Information ¨ Table of Contents
[00136] One embodiment of navigational information that may be generated
using the
polygons is a table of content 1500 as shown in FIG. 15. Instead of manually
creating a table of
24
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
content by linking an entry in the table of content with a particular flyer
page, the interactive flyer
system according to an embodiment of the present technology is able to auto-
generate the table
of content using the polygon mapping information. This is achieved by
analyzing the spatial
position and inclusion relationship of the polygons to understand the
hierarchical relationship of
the polygonal areas defined by the polygons.
[00137] In FIG. 16, there is shown one exemplary method for auto-generating
the table of
content. To generate the table of content 1500, the interactive flyer system
looks through the
inclusion relationship of the polygons. It first finds all the top-level
polygons not contained within
any other polygons (i.e. 1600 in FIG. 16). In FIG. 12, the top-level polygons
would include
polygons tagged as "Electronics", "Sales", "Computers", and "Garden".
Thereafter, the system
finds all the next-level polygons contained in the top-level polygons found at
1600 but otherwise
not contained within any other polygon (i.e. 1602 in FIG. 16). Referring again
to FIG. 12, these
next-level polygons would include polygons tagged as "W", "Laptops", and
"Memory". The
system then sets the next-level polygons as the new top-level polygons (i.e.
1604 in FIG. 16)
and repeats locating all the next-level polygons until all the polygons are
found (i.e. 1606 in FIG.
16).
[00138] With the generated table of content, the flyer viewer would be able
to quickly jump to
the flyer page containing the polygon. Various implementations are possible
for showing the
appropriate section of the flyer page. In one embodiment, the selected entry
of the table of
content may be shown by zooming into the associated polygonal area. For
example, when the
viewer clicks on "TV" in the table of contents 1500, the system may serve the
flyer page zoomed
into the "TV" polygon. In another embodiment, the selected entry of the table
of content may be
shown with the associated polygonal area highlighted. Continuing with the
above example,
instead of zooming into the "TV" polygon, the whole flyer page may be shown
with the "TV"
polygonal area highlighted.
Navigational Information ¨ Navigation Strip
[00139] Another embodiment of the navigational information may be in the
form of a
navigation strip, several types of which are shown in FIG. 17. The navigation
strip is an aid that
helps the viewer navigate through the entire flyer. However, unlike a
geographic mapping
application, a mini-map type of navigation aid that consists of small zoomed-
out versions of the
flyer page is not beneficial to a viewer of digital flyers. The reason is that
a flyer page typically
contains numerous items with pictures and text that are hard to read when
scaled down. A
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
navigation strip (or may also be referred to as a "legend") that consists of
scaled down versions
of the full flyer page would thus fail to convey any meaningful information to
the viewer. Rather,
in the present technology, the interactive flyer system uses the polygon
mapping information to
generate a navigation strip that contains contextual information in compact
form. Three
possible implementations of the navigation strip will now be described.
[00140] In FIG. 17, there is shown a flyer page 1700 simplified to show
only the top-level
polygons tagged with the labels "Electronics", "Sales", "Computers", and
"Garden". In the
polygons labelled "Electronics", "Computers" and "Garden", there is further
included a point of
interest 1702, 1704, 1706. The point of interest may convey information about
a particular item
such as discounts at 50% off or $50 off. It may also serve to highlight
information that may be
of particular interest to the viewer. To better illustrate the functioning of
the navigation strip
1710, 1720 and 1730, the current viewport 1708 of the communication device
being used to
browse the flyer is also shown in FIG. 17.
[00141] The first implementation of the navigation strip is a directly
proportional navigation
strip 1710 in FIG. 17, which comprises of to-scale rendering of the flyer
sections. However,
instead of the strip including scaled down versions of the flyer page, each
section may be
replaced with a solid colour (in FIG. 17, the solid colour is white for
clarity; however, other
colours may be used) and may include the title of each section (e.g. the
contextual information
used to tag the polygon). Where the flyer page includes points of interest,
the navigation strip
may also show the approximate locations in the navigation strip. In effect,
the navigation strip
1710 conveys contextual information in the most true to position and to-scale
format without
simply using scaled-down images of the flyer page 1700.
[00142] One method to create the directly proportional navigation strip
1710 is illustrated in
FIG. 18. To create the strip, the polygons are first scaled down by applying a
scaling factor (i.e.
1800 in FIG. 18). While the same scaling factor may be applied to x and y axis
of the polygons,
it is to be understood that different scaling factors may be used. This has
the advantage of
optimizing the screen space occupied by the navigation strip 1710. In one
embodiment, the
scaling factor for the x-axis may depend on the width of the polygon and the
length of the
associated label, and the scaling factor for the y-axis may depend on the
height of the polygon.
Optionally, prior to applying the scaling factor, the polygons may be aligned
along a grid to
produce a more organized appearance. The scaled down polygons are then
labelled with the
contextual information corresponding to the contextual information of the
polygon (i.e. 1802 in
26
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
FIG. 18). For example, where the polygon is tagged with the label
"Electronics", the system
extracts this contextual information and labels the scaled polygon
accordingly. Thereafter, the
scaled down polygons are displayed in the navigation interface (i.e. 1804 in
FIG. 18). In one
implementation, the navigation interface may be located below the flyer page
1700, which may
be hidden when not in use. In another implementation, the navigation interface
may be located
to the side of the flyer page 1700, for example to display the table of
content.
[00143] Additionally, where the flyer page includes points of interest,
they may be plotted
using the scaling factor as reference. In a further embodiment, the current
viewport information
may be displayed on the navigation strip 1710 using the same scaling factor.
Additionally, as
the viewer changes the viewport, the navigation strip 1710 may change position
(e.g. scroll) to
maintain perspective and context of the flyer page.
[00144] A second implementation of the navigation strip is a text-only
proportional navigation
strip 1720 in FIG. 17. This navigation strip is a variation of the directly
proportional navigation
strip 1710 but one that requires less vertical space. In summary, the text-
only proportional
navigation strip 1720 is a vertically compressed version of the directly
proportional navigation
strip 1710 that compresses the y-axis while maintaining the proportion along
the x-axis.
[00145] One method of creating the text-only proportional navigation strip
1720 is an
extension of the method for creating the directly proportional navigation
strip 1710 as discussed
above and depicted in FIG. 18. Referring to FIGs. 19A and 19B, a scaling
factor is applied to
the polygon (i.e. 1900 and 1912 in FIGs. 19A and 19B, respectively). However,
unlike FIG. 18,
the scaling factor for the x and y-axis need not be different to optimize the
screen space
occupied by the navigation strip. This is because the vertical space will be
compressed
subsequently and only the proportion along the x-axis will be maintained.
Optionally, as
discussed above, prior to applying the scaling factor, the polygons may be
aligned along a grid
to produce a more organized appearance.
[00146] The scaled polygons are then labelled with the contextual
information corresponding
to the polygon, with the labels being positioned at a reference point of the
polygons (i.e. 1902
and 1914 in FIGs. 19A and 19B, respectively). The reference point may be
anywhere within the
polygonal area defined by the polygon. Where the system language is English,
the reference
point may be the top left corner as shown in 1914 of FIG. 19B. Thereafter, a
bounding box is
created around the label (i.e. 1904 and 1916 in FIGs. 19A and 19B,
respectively) and the scaled
27
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
polygons are removed (i.e. 1906 in FIG. 19A). In text-only proportional
navigation strip, the
polygons are used to maintain proportion of the contextual information along
the x-axis. The y-
axis is compressed to reduce the vertical space occupied by the navigation
strip. Finally, each
label (and the bounding box) for each polygon is displaced in a reference
direction, stopping if
the bounding box of the labeling text encounters a bounding box of another
labeling text or an
edge of the navigation strip 1720 (i.e. 1908 and 1918 in FIGs. 19A and 19B,
respectively).
Again, the reference direction may in any direction; however, where the system
language is
English, the reference direction may be up as shown in FIG. 19B. By this
displacement, the
vertical space is compressed while still maintaining horizontal relationships
among the flyer
sections.
[00147] To complete the navigation strip, the interactive flyer system may
add the points of
interest. However, because of the lack of vertical space, the interactive
flyer system may
instead use a vertical line to indicate the approximate location of the points
of interest. In
another embodiment, parentheses may be inserted after the labeling text. For
example, if the
points of interest referred to a discount, the parenthesis may display "2
discounts", signifying
that there are 2 discounts in the particular section. Finally, similarly to
the directly proportional
navigation strip 1710, the interactive flyer system may insert the current
viewport information
appropriately scaled.
[00148] The third implementation is the vertically most compact navigation
strip, hereinafter
referred to as area proportional navigation strip 1730 in FIG. 17. It involves
transforming the 2D
polygonal areas defined by the polygons into a 1D navigation strip. FIG. 20
illustrates one
implementation of performing such transformation.
[00149] In the exemplary implementation, the system determines the scaling
factor that
relates the polygonal area defined by the polygon to the distance on the
navigation strip (i.e.
2000 in FIG. 20). In this particular example, the area proportional navigation
strip 1730 is
horizontal in direction; however, it is to be understood that the 1D
navigation strip may be
vertical in direction, or any other direction, in other implementations. Thus,
the scaling factor
translates the polygonal area occupied by the polygons into horizontal
distance on the
navigation strip 1730.
[00150] Thereafter, the polygons are ordered (i.e. 2002 in FIG. 20). There
are various ways
to order the polygons; however, where the system language is English, the
following method
28
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
may be used to order the polygons. Prior to ordering, the polygons may first
be aligned along a
grid to produce a more organized appearance. The interactive flyer system then
creates a list of
coordinates of each polygon, with the coordinate representing the top left
most vertex of the
polygon. The list is then ordered using the x-axis coordinate. Where there are
two polygons
with the same x-coordinate, the coordinates are ordered from top to bottom.
For example, in
the flyer page 1700 of FIG. 17, the polygons having labels "Electronics" and
"Computers" have
the same x-coordinate. In such a case, preference is given to "Electronics".
When the polygons
in FIG. 17 are ordered, the ordered list would appear as follows: electronics,
computers, sales
and garden.
[00151] To create the navigation strip 1730 (i.e. 2004 in FIG. 20), the
interactive flyer system
starts with the first polygon in the ordered list. To determine the amount of
horizontal distance
that should be allocated to the first polygon, the scaling factor is applied
to the polygonal area of
the first polygon. This process is repeated until all the polygons in the
ordered list are
converted. Finally, the viewport information may also be included in the
navigation strip 1730
using the scaling factor.
[00152] While the navigation strip was described with three specific
implementations, it will
be understood that various other implementations may be possible.
Interlinking Related-Flyers and Integrating Related-Data
[00153] Contextual information conveyed through polygon mapping enabled the
viewer to
overview the entire flyer at a glance and to navigate to different sections of
the flyer using the
contextual information. For example, the table of contents generated by
analyzing the
hierarchical structure of the polygons allowed the interactive flyer system to
quickly generate a
flyer summary (e.g. table of contents). Further, the polygon mapping enabled
the generation of
navigational aids (e.g. contextual navigation strip) that conveyed meaningful
and useful
contextual information. In addition to these advantages, the polygon mapping
may also allow
the system to relate the contextual information of the flyer to information
not contained in the
flyer. For example, the related information may be from a flyer at a competing
store, allowing
the viewer to comparison shop. In a further example, the related information
may be
information that would be desirable to informed shoppers. Such information may
include price
history, product reviews, price comparison data not contained in other flyers,
cross border
prices, targeted advertising and offers, and many more.
29
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
[00154] The first type of related information derives from other flyers. By
using the
information from the polygon mapping, related information contained in other
flyers can be
easily associated. This allows flyer viewers to comparison shop between flyers
very easily.
One implementation for providing such related information will be discussed
using FIGs. 21 and
22.
[00155] At 2100, the interactive flyer system retrieves all related digital
flyers. Relevance
may be based on a single factor or a combination of several factors. For
example, the
interactive flyer system may retrieve all the flyers within the same
geographical area, or flyers
from stores with online shopping. This "initial set" of flyers is then ordered
based on relevancy,
which may also be based on one or many factors (i.e. 2102 in FIG. 21). For
example, the
system may take into consideration the following: physical distance between
the stores of the
related flyers and the store of the current flyer or the viewer's location;
brand awareness of the
related flyers; previous preference or history of the viewer; behaviour
history of other viewers;
business relationships between the publisher of the flyer and the vendor;
contextual similarities
between the related flyers and the current flyer; store relationships (e.g.
known competitors or
affiliates); or any other factors.
[00156] Once the retrieved flyers are ordered according to relevancy, the
interactive flyer
system displays the retrieved flyers in the related information interface to
provide access to the
relevant digital flyers in the order of relevancy (i.e. 2104 in FIG. 21). The
interface may be a
simple link to allow the viewer to switch to the most relevant flyer. A more
advanced
implementation is depicted in FIG. 22, which is an extension of the area
proportional navigation
strip (i.e. 1730 in FIG. 17) as described above. It will be understood that
the interface may also
be implemented using extensions of the directly and text-only proportional
navigation strip (i.e.
1710 and 1720 in FIG. 17). In the particular implementation shown in FIG. 22,
in addition to
generating the navigation strip for the current flyer, the interactive flyer
system also generates
the navigation strip for the next most relevant flyer. This allows the viewer
to jump from one
flyer to another seamlessly. In this particular implementation, intra-flyer
navigation (i.e. within
the flyer) is achieved by moving horizontally (i.e. x-axis) and inter-flyer
navigation (i.e. between
flyers) is achieved by moving vertically (i.e. y-axis). The generated
navigation strip is kept in
memory so that inter-flyer navigation is possible not only to the next flyer
but also to previously
viewed flyers. Additionally, the relative position of the navigation strip of
the next flyer may be
changed depending on the position of the viewport. For example, if the center
of the current
viewport is within the polygon tagged as "Electronics", the position of the
next flyer in the
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
navigation strip may be changed such that its "Electronics" section is aligned
with the current
flyer, as shown in FIG. 22. The same may be applied to the related flyer
previously shown.
This dynamic positioning of the navigation strip of the next flyer and the
previous flyer allows the
viewer to immediately jump into the most relevant section.
[00157] The second type of related data stems from other data sources,
outside of those
contained in other flyers. Other sources may include public information (such
as those available
from the Internet), private information (such as those provided by partners)
and sources other
than flyers. This involves determining the interest of the viewer by analyzing
different clues.
For example, the interactive flyer system may infer the viewer's interest by
determining how the
current viewport intersects with the polygons, the position of the mouse
cursor, zoom level or
any other clues. Depending on the clues, the interactive flyer system may
retrieve related data
such as price comparison data, historical market pricing, cross-border
comparisons, product
details such as reviews, specifications and descriptions, and targeted
advertisements and
offers. The advantage is that such integration with other data sources is
enabled without
explicitly tagging individual items that appear on a flyer. Rather, the
publisher only needs to
convey contextual information using the polygons.
[00158] Moreover, depending on the extent of other data available, the
interactive flyer
system may provide the viewer with the functionality to filter the interactive
flyer based on data
available not only within the flyer but also based on the other data. For
example, consider the
scenario where a shopper is searching for good deals on flat screen TVs. The
shopper opens
the digital interactive flyer and places a filter to search for "TV" with
discounts greater than 25%
off. The interactive flyer system proceeds to highlight the sections of the
interactive flyer
according to the filter. According to a further embodiment of the present
technology, the
interactive flyer system may provide the viewer with the option to set filters
that also takes into
account data from other flyers and other non-flyer sources. For example, the
interactive flyer
system may retrieve other TV discounts from other sources (e.g. cross-border)
that may be offer
greater discount than those contained in the present flyer. Alternatively, or
additionally, the
interactive flyer system may retrieve price history of the products contained
in the TV section by
consulting its or other database. In effect, the interactive flyer system
allows the viewer to set
search filters on digital image-based flyers that are normally only reserved
for text based search
engines.
31
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
[00159] While the different types of navigational information and related
information have
been described separately, it will be understood that the interactive digital
flyer system may
incorporate one or more of such information within the same system. For
example, interactive
digital flyer may incorporate a table of content in a navigation interface and
a navigational strip
in another navigation interface. In another implementation, the interactive
digital flyer may
incorporate a table of content in a navigation interface and a modified text-
only proportional
navigational strip in a related information interface. It will be understood
that numerous
combinations of the navigational information and related information may be
generated using
the contextual information conferred by the polygon mapping information and
other flyer-related
information as discussed above.
Feedback system
[00160] Feedback systems may be generally categorized into explicit and
implicit feedback
systems. Explicit feedback refers to resulting actions of the viewer, while
implicit feedback
refers to the viewer behaviour leading to the resulting actions. For example,
the viewer is
presented with items 1-10. In coming to the decision of clicking on items 1
and 5, the viewer
looks through items 1-5. An explicit feedback system would report back that
the viewer clicked
on items 1 and 5, while the implicit feedback system would report back that
the viewer viewed
items 1-5. Thus, an explicit feedback system is result oriented while an
implicit feedback
system is behaviour oriented. According to an embodiment of the present
technology, implicit
feedback is made possible for image based digital flyers. Using the feedback
information, a
host of features are made possible including predictive caching, dynamic
generation of points of
interests, and viewer interest maps.
[00161] One embodiment of a feedback system may be implemented by using the
polygon
mapping information stored in the feedback data store 250 as discussed above.
Polygon
mapping information includes one or more polygons that are used to define
polygonal area(s),
with each polygon being tagged with contextual information. The information
conveyed by the
tagged contextual information is dependent on the granularity of the polygon
mapping
information. In addition to the polygon mapping information stored in the
feedback data store
250, the feedback data store 250 may also receive device information from the
communication
device, such as those shown in FIG. 1, periodically. The device information
conveys
information about the communication device and the viewer. For example, the
device
information may include information such as viewport information including
viewport
coordinates, date, time, session information, viewer information (such as IP
address and locale
32
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
information), information about the communication device (such as operating
system and
Internet browser information), or any other information. Using the device
information, such as
viewport coordinates (xl , yl) and (x2, y2) in FIG. 23A of viewport (e.g.
2300A in FIG. 23A), and
the polygon mapping information, the system is able to generate detailed
feedback report.
[00162] Referring to FIGs. 23A and 23B, an exemplary implementation for
using feedback
information stored in the feedback data store 250 and polygon mapping
information stored in
the flyer data store 240 to generate a feedback report will be discussed. It
will be understood
that a digital flyer may include one or more sets of polygon mapping
information. One set may
be used to generate navigational and related information as discussed above.
Another set may
be used to generate a feedback report. This has the advantage of conveying one
set of
information for the flyer viewer, while collecting and comparing another set
of information for the
vendor.
[00163] FIG. 23A shows the viewport information of viewport 2300A for a
viewer at a
particular point in time. This snapshot of the viewport is generated by
referencing the device
information (contained in the device information stored in the feedback data
store 250) of the
viewer at the particular point in time with the polygon mapping information
stored in the flyer
data store 240. The device information may be sent by the communication
device, such as on a
periodic basis, to the interactive flyer system 200 and stored in the feedback
data store 250.
Thus, for a given viewer, during a given session, there may be one or more
such device
information entries in the feedback data store 250, which can be used to
construct the snapshot
of the viewer's viewport at a particular point in time when the device
information was sent.
Using this information and other information contained in the device
information, various
feedback report may be generated using the device information from a single
viewer, all the
viewers or a selected group of viewers. For example, the device information
may contain user-
persona information derived from the HTTP header, which can identify
information such as
screen resolution, operating system, and browser type and version. Using this
information,
viewers may be grouped as "high-tech" viewers (e.g. viewers with large screen
resolution, latest
operating system and browser versions) or "office" viewers (e.g. viewers with
standard business
versions such as typical screen sizes and browsers as determined by the
system, and viewing
the digital flyer at business hours).
[00164] In one embodiment, the concept of pageview is used to generate a
feedback report.
Pageview refers to the number of views that a polygon received. As described
previously, the
33
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
polygon demarcates a polygonal area that conveys contextual information about
products and
items contained within the polygonal area. Furthermore, the user may, for the
purpose of
generating feedback information, provide a separate set of polygon mapping
information that
contains more detailed information to generate a more detailed and insightful
feedback report.
Thus, depending on the granularity of the polygon mapping information,
pageview represents
viewer's interests in the content contained in the polygonal area of a
polygon. In one extreme,
where the viewport does not include any polygon, none of the pageview of the
polygons will be
varied. In the other extreme, where the viewport displays a single polygon,
only the pageview of
that polygon will be varied. In one implementation, the pageview will be
incremented by 1. In
between the two extremes, the viewport may intersect one or more polygons and
the pageview
of each polygons may be varied depending on various factors including relative
viewport
position, weighing factors and typical screen sizes of viewers.
[00165] Referring to FIG. 23A, an example viewport 2300A intersecting three
polygons 2302,
2304 and 2306 is shown. Using the device information, such as the viewport
information
containing viewport coordinates (x1, y1), and (x2, y2), the feedback data
analyzer 260
determines the area of each polygons 2302, 2304 and 2306 proportional to the
viewport 2300A.
In this example, the feedback data analyzer 260 may determine that polygons
2302, 2304 and
2306 occupy 0.2, 0.5 and 0.3 of the available area in the viewport 2300A. In
one
implementation, the pageview of each polygon may be incremented by the
proportion of the
viewport area occupied (i.e. 0.2, 0.5 and 0.3). In another implementation, the
proportional area
(i.e. proportion of the viewport area occupied by the polygon) determined may
be varied by a
multiple.
[00166] In a more advanced implementation, the feedback data analyzer 260
may consider
other factors. For example, because of the varying screen sizes of viewers,
the feedback data
analyzer 260 may normalize the feedback information using a typical viewport
2310. In this
context, normalization refers to the process of averaging out the feedback
information to remove
the skewing effects of extreme data sets. Because viewport 2300A may be
changed by the
viewer, the pageview for the polygon may be skewed. Referring to FIG. 23B, the
viewer may
have decreased the viewport 2300B to occupy only a small portion of the
available screen real
estate. In this configuration, the viewport 2300B will generate multiple
pageviews for polygon
2304. Thus, one implementation would be to vary the proportional area by a
weighting factor,
such as a ratio between the viewport size and the typical viewport 2310. So,
if viewport 2300B
was determined to have an area of 1, while the area of the typical viewport
2310 is 100, the
34
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
pageview generated by the viewport 2300B would have a weighing factor of only
0.01. Thus,
the pageviews generated by the viewport 2300B would be made marginal. The same
normalization process may be applied to large screens.
[00167]
In another implementation, the weighting factor may be dependent on the
differential
area between the viewport and the typical viewport 2310. For example, in FIG.
23A, the area
within the typical viewport 2310, may be weighed at 100% while the area
between the typical
viewport 2310 and the viewport 2300A may be weighed at 50%, or another reduced
percentage.
For smaller viewport 2300B, as shown in FIG. 2300B, the area between the
viewport 2300B and
the typical viewport 2310 may be weighed at a negative value (e.g. -50%) to
account for the
multiple pageviews generated by such small viewport 2300B.
[00168]
Thus, the system generates a feedback report by retrieving device information
from
a feedback data store 250 (i.e. 2400 in FIG. 24). The device information may
include viewport
information associated with a viewport of a communication device and may
include a variety
information such as, but not limited to, viewport coordinates, date, time,
session information,
viewer information (such as IP address or geographic locale information),
information about the
communication device (such as operating system used by the communication
device, or
Internet browser used by the communication device), or any other information.
The system also
retrieves polygon mapping information from a flyer data store 240 (i.e. 2402
in FIG. 24). As
previously described above, the polygon mapping information includes one or
more polygons
that each define a polygonal area with each polygon being tagged with
contextual information.
Using the device information and the polygon mapping information, the system
determines, for
each polygon in the polygon mapping information, a proportion of the viewport
area that it
occupies (i.e. 2404 in FIG. 24). With the determined proportional area
occupied by the polygon,
the pageview associated with each polygon is varied (i.e. 2406 in FIG. 24).
In one
implementation, the pageview is varied by a multiple of the proportional area.
In another
implementation, the pageview is varied by the proportional area and corrected
by a weighting
factor. An exemplary weighting factor may be a ratio between the viewport area
and a typical
viewport 2310. Another exemplary weighting factor may depend on the
differential area
between the viewport and the typical viewport 2310. Using the pageview
information, the
system then generates a feedback report (i.e. 2408 in FIG. 24).
[00169]
While the feedback report generated may be a simple output of the pageview
associated with each of the polygon, a more advanced implementation may be
possible by
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
using the device information and the polygon information stored in the
feedback data store 250
and the flyer data store 240. One implementation is shown in FIG. 25, which
illustrates a
feedback report 2500 in the form of a frequency map. It will be understood
that a frequency
map may also be referred to as a heat map or impression report. In this
implementation, the
polygon mapping information used to convey contextual information and used to
generate
navigational information and related information as discussed above is
overlaid with the
feedback information in the form of colouration to produce the feedback report
2500. As
indicated in the legend, the intensity of colouration depicts a higher viewing
activity by the
viewers. While the colouration in FIG. 25 depicts the number of pageviews
determined by the
feedback data analyzer 260, the colouration may also depict other information.
For example,
the colouration may depict the average time spent by viewers in the given
polygon.
[00170] In another implementation of the feedback report, a different set
of polygon mapping
information may be used to generate the feedback report. For example, the
publisher may have
defined a second set of polygon mapping information that contains more
granular information.
As shown in FIG. 26, the "Electronics" and the "W" sections are further broken
down to the
product level using a second set of polygon mapping information. For
simplicity, only the
"Electronics" and the W sections are shown with more granular information.
Thus, using the
device information and the polygon mapping information (i.e. the second set
and the initial set),
the vendor is able to obtain a feedback report 2600 that is more detailed than
the feedback
report 2500 in FIG. 25.
[00171] In a further implementation, date, time, elapsed time and session
information
contained in the device information may be used to create a feedback report
that reflects the
viewer's viewing activity over a course of a session. This may include
information aggregating a
plurality of viewers, who may be grouped according to the device information.
For example,
viewers in a particular geographic locale may be grouped. In another
implementation, the
device information may include information related to the communication device
(e.g. browser or
operating system information extracted from HTTP header) to group users into
different
categories as discussed above. By relating viewer's viewing activity (i.e.
pageview of the
polygon) to device information such as date, time, and elapsed time, a series
of feedback report
representing viewer's activity at different points in time may be generated.
The series of
generated feedback report may be used as frames in a time-elapsed frequency
map.
36
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
[00172] Now referring to FIG. 27, another embodiment for implementing a
feedback system
is shown. In this particular embodiment, feedback information is generated by
taking advantage
of the flyer image tile set as described above. While this embodiment will be
described
separately from the feedback system using polygon mapping information as
discussed referring
to FIGs. 23-26, it will be understood that the two embodiments may be
implemented as a single
system by leveraging a combination of information conveyed through polygon
mapping
information, flyer image tile sets served, and device information.
[00173] In FIG. 27, the interactive flyer system 200 provides an initial
viewscore to each tile
(i.e. 2700 in FIG. 27). For example, each tile may be assigned an initial
viewscore of 0.
Subsequently, each time the viewer zooms into the flyer page or pans across
the flyer page and
a new tile image is required, the system 200 not only retrieves the tile but
also varies the
viewscore (i.e. 2702 in FIG. 27). In one implementation, the magnitude of
variation may depend
on the relative position of the tile in the viewport. For example, an image
tile at the centre of the
viewport may be assigned a higher magnitude of variation than an image tile at
the edge of the
viewport. In another implementation, the magnitude of variation may depend on
the zoom level
of the tile. For example, image tiles at deeper zoom level may be assigned a
higher magnitude
of variation since serving such tiles signifies greater interest from the
viewer. In a further
implementation, the magnitude of variation may depend on viewer events such as
zoom-in,
click, and time spent. For clarity, it will be noted that variation refers to
both increase and
decrease of the viewscore. Optionally, the system 200 may also note the date
and time when
the viewscore was varied. The viewscore is then stored in the feedback data
store 250 (i.e.
2704 in FIG. 27). Storing of the viewscore may be initiated periodically at
pre-defined intervals.
[001741 The feedback data store 250 contains the viewscore associated with
the flyer image
tiles, which provides information related to the behaviour of the viewer. Such
implicit feedback
information may be valuable information to both the vendor and the publisher,
which was not
possible with previous digital flyer system. In a simple implementation, the
collected information
may be exported in a text format or displayed in a textual representation such
as a table. In
another implementation, the viewscore may be used to produce a frequency map
that gives a
visual representation of the interest level of the viewer. FIG. 28 shows an
example of such
frequency map, which may also be referred to as coloured frequency map, heat
map, or
impression report. The feedback report 2800 of FIG. 28 shows viewer interests
depicted by
different colour schemes. In the report 2800, area 2802 and 2804 received 50-
100 hits from the
viewers, while area 2806 received 100-500 hits from the viewers. The area of
the flyer page
37
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
that received the most attention is area 2808, which received 500+ hits. This
type of report
illustrates the behaviour of the viewers and is made possible by plotting the
viewscore of each
tile.
[00175] Different implementations of the map are possible. For example, the
feedback data
store 250 may also keep the date and time at which the viewscores were
incremented or
decremented. By storing the date and time at which the viewscores changed, a
timeline-based
feedback report may be provided. In a further embodiment, the feedback data
store 250 may
keep the IP address of the viewer to distinguish between unique/repeat
viewers, and locales of
the viewers. By varying the type of data collected along with the viewscore,
various data may
be made available on the feedback report.
[00176] In a further embodiment, the system 200 may leverage the polygon
mapping
information stored in the flyer data store 240 to provide a feedback report
such as the one
shown in FIGs. 25 or 26. In generating the feedback report 2500 or 2600, the
system 200 may
summate the viewscore of the tiles in each polygonal area and provide a
feedback report 2500
or 2600 that describes the relative interest level of each section.
[00177] Additionally, the information stored in the feedback data store 250
may be used to
improve the quality of the data. In one implementation, information in the
feedback data store
250 may be used to validate the points of interests (such as 1702, 1704 and
1706 in FIG. 17)
stored in the flyer data store 240. After the initial creation of the points
of interests, the system
200 may analyze the behaviour of the viewer using the information stored in
the feedback data
store 250. With this analysis, the system 200 may remove or keep the existing
points of
interests. Moreover, if the flyer is generating a lot of interests in an area
that was not initially
tagged as a point of interest, the system 200 may dynamically tag the area as
a point of interest.
In effect, the interactive flyer system 200 is able to react to the behaviour
of the viewers in real
time. Consequently, the quality of the data managed by the system is vastly
improved.
[00178] In a further embodiment, the information stored in the feedback
data store 250 may
be provided to the polygon mapping module 230. In one embodiment, the feedback
information
may be provided to the user using the interface for defining the polygon
mapping information.
This may be in the form of an indication that informs the user, at the time of
the polygon
mapping, including defining points of interests, which sections or products
have previously
attracted much attention from viewers. Through such information, vendors would
be able to
38
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
accurately predict viewer's interests and highlight information knowing that
it would attract much
attention from the viewers. In another embodiment, the polygon mapping module
230 may use
the feedback information to define the polygon mapping information, including
points of
interests, based on predetermined information, such as vendor-published agreed
format, as
previously discussed.
Pre-Loading
[00179] According to another embodiment of the present disclosure, the
interactive flyer
system 200 may include a pre-loading module that efficiently pre-loads the
flyer image tiles.
Due to limited memory space on communication devices (e.g. communication
devices 102, 104
and 106 in FIG. 1) on which the digital flyer may be displayed, flyer image
tiles should be
efficiently stored. For example, a single flyer image tile may be 256 pixels
by 256 pixels, with
each pixel being 4 bytes. This means that each image tile would require 256KB
of memory. In
a digital flyer having multiple flyer images, with each flyer image
representing each page of the
digital flyer, the flyer image tiler 210 may create hundreds of flyer image
tiles for the full digital
flyer (i.e. the flyer image tile set includes multiple copies of the flyer
image at different zoom
level, with each copy segmented into flyer image tiles). Thus, even with
modern communication
devices having large memory capacity, there is a real likelihood of
communication devices
running out of memory.
[00180] In one embodiment according to the present disclosure, feedback
information may
be used to improve the efficiency of the interactive flyer system 200. In a
traditional non-tiled
image-based flyer system, a high quality version of the flyer was downloaded
either in full or
page-by-page. In more advanced systems, a lower quality of the image may have
been initially
downloaded to provide a preview before downloading the full version. However,
because such
systems could not take implicit feedback information into consideration, the
full image was
downloaded even when the viewers never in fact looked at the entire image.
Moreover, caching
prediction did not change with viewer's behaviour. According to an embodiment
of the present
technology, predictive caching is based on implicit feedback information.
Additionally, or
alternatively, predictive caching is based on the contextual information
conferred through
polygon mapping information.
[00181] Generally, retrieving graphical information over a network can be a
performance
bottleneck. By storing the information locally, performance of the system can
be dramatically
improved. However, caching may also be wasteful if the cached information is
never utilized. In
39
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
the present technology, caching prediction is made highly accurate by
incorporating the
information from viewer's behaviour.
[00182] In one embodiment, caching prediction may be based on the
viewscores assigned to
the image tiles. Since high viewscores signify high level of viewing activity,
the interactive flyer
system 200 may preemptively retrieve those image tiles. This is under the
assumption that by
caching those image tiles, a high cache hit rate would be realized. In another
embodiment,
caching prediction may be based on the pageview assigned to each polygon and
pre-emptively
caching flyer image tiles associated with the polygons having high pageviews.
Moreover, since
viewscores and pageviews are updated with viewer's behaviour, the predictive
caching would
also dynamically vary with the viewer's behaviour. Thus, the interactive flyer
system can stay
nimble and efficient by constantly changing to the viewer's interests.
[00183] In another embodiment, the interactive flyer system 200 may
retrieve flyer image
tiles (for caching) under the assumption that the viewer is likely to browse
to the surrounding
area of the flyer. For example, referring to FIG. 29, a tiled flyer image 2900
with viewport 2902
is shown. With the current position of the viewport 2902, the interactive
flyer system 200 would
retrieve the tiled flyer images as shown in the shaded portion 2904. However,
after retrieval of
the shaded portion 2904, the interactive flyer system 200 may further retrieve
tiled flyer images
in the surrounding portion 2906 under the assumption that the viewer is likely
to pan to the
surrounding area. The interactive flyer system 200 may also determine the
direction or speed of
travel of the viewport 2902, or both. Using this information, the interactive
flyer system 200 may
retrieve the appropriate shaded portion. For example, if the viewport 2902 was
moving to the
right (e.g. due to a panning action) at velocity A, the interactive flyer
system 200 may retrieve
two columns of flyer tiles to the right of the viewport 2902. In another
example, if the viewport
2902 was moving to the right at velocity B (B>A), the interactive flyer system
200 may retrieve
four columns of flyer tiles to the right of the viewport 2902 since flyer
tiles would need to be
retrieved at a faster rate. Additionally, or alternatively, the interactive
flyer system 200 may also
consider the zooming rate to retrieve one or more extra deeper levels of zoom
contained within
the viewport 2902.
[00184] In a further embodiment, the interactive flyer system 200 may
incorporate the
contextual information provided by the polygon mapping information. For
example, referring to
FIG. 30, flyer page 3000 with polygons 3002, 3004, 3006 and 3008 is shown. By
considering
the viewer's current viewport 3010, the interactive flyer system 200 may
retrieve all the tiled flyer
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
images within the polygon 3006. Under this predictive caching scheme, the
interactive flyer
system 200 may assume that the viewer will continue to view items that belong
to the same
context (i.e. within the polygon 3006). For example, if the viewer's viewport
is currently within
the "Electronics" section, it is likely that the viewer will continue to
peruse through the
"Electronics" section or, perhaps, continue to a related section. It may also
be likely that when
the viewport 3010 approaches closer to the edges of polygon 3006, the flyer
image processor
may further retrieve tiled flyer data images of nearby polygons. For example,
if the viewport
3010 is closer to the top edge of polygon 3006, the interactive flyer system
200 may retrieve
tiled flyer images in the polygon 3002. Thus, through the use of contextual
information enabled
by the polygon mapping information according to an embodiment of the present
technology,
caching is improved.
[00185] In an interactive flyer system 200 that incorporates both polygon
mapping
information and tiled flyer images, the system may consider both contextual
and implicit
feedback information to determine the order in which the tile images are
cached. In one
exemplary implementation, predictive caching may be associated with the
relative position and
movement of the viewport (i.e. image tiles and zoom level), the polygon
mapping and the
aggregate popularity derived from the implicit feedback. In this
implementation, tiles that are
closer to the centre of the viewport and at the current zoom level may be
assigned a higher
caching score. From thereon, tiles at the same zoom level but further from the
centre of the
viewport may be assigned a lower caching score. The system may also assign
caching scores
to tiles that are located deeper in the zoom levels, with the scores being
varied depending on
their proximity to the centre of the viewport. In addition to the relative
position, the system may
also consider the polygon mapping and the implicit feedback information to
arrive at a holistic
predictive scheme that is not only dependent on the current viewer but also
the aggregate
popularity of the flyer based on other viewers.
[00186] Now turning to FIGs. 31 and 32, a method for pre-loading flyer
image tiles according
to an embodiment of the present disclosure is disclosed. The interactive flyer
system 200
includes a pre-loading module that retrieves one or more flyer image tiles
that are associated
with the most zoomed-out state (i.e. 3100 in FIG. 31). In one embodiment, the
retrieved tiles
may have been received from a server (e.g. server 160 in FIG. 1 or any other
server that may
implement an embodiment of the technology disclosed in the present disclosure)
and loaded
into the memory. The retrieved tiles are associated with an underlay viewport
3200 that
contains the flyer image tiles at the lowest resolution (i.e. the most zoomed-
out state) and forms
41
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
the base image on which the flyer image tiles in the current viewport 3202 are
loaded. As
discussed in this disclosure, the digital flyer comprises at least one set of
flyer image tiles for the
different viewport resolutions. Each set includes multiple copies of the flyer
image at different
resolutions for the varying zoom levels, with each copy of the flyer image
being segmented into
flyer image tiles. As the viewer interacts with the user interface, the flyer
image shown in the
current viewport 3202 changes. For example, the viewer may pan or zoom and the
current
viewport 3202 may retrieve and display the corresponding copy of the flyer
image tiles (e.g. flyer
image tile 3210 in FIG. 32) from the flyer image tile set. Based on the action
of the viewer, the
flyer image tiles visible in the current viewport 3202 may change. These flyer
image tiles that
are visible in the current viewport 3202 are retrieved from the memory (i.e.
3102 in FIG. 31). As
in the flyer image tiles associated with the underlay viewport, the flyer
image tiles visible in the
current viewport 3202 may have been received from the server. The interactive
flyer system
200 then displays a portion of the flyer image tiles in the underlay viewport
3200 that is visible in
the current viewport 3202 and overlays onto that portion, flyer image tiles
for the zoom level
associated with the current viewport 3202 and visible in the current viewport
3202 (i.e. 3104 and
3106 in FIG. 31). The advantage of this method will be described using FIGs.
32 and 33.
[00187] Referring to FIGs. 32 and 33, as the viewer zooms into the flyer
image tile 3210, the
flyer image tiles at the lower zoom level are retrieved (e.g. from the flyer
image tile store 220)
and displayed in the current viewport 3202. If the viewer decides to pan right
(as shown by
arrow 3300), the interactive flyer system 200 would soon have to retrieve
flyer image tiles in
columns 3312 and 3314. Without pre-loading the flyer image tiles in columns
3312 and 3314,
there may be a momentary lapse in flyer image being shown in the viewport 3202
until the flyer
image tiles in columns 3312 and 3314 are retrieved. However, using the present
embodiment,
because the underlay viewport 3200 contains the flyer image at the lowest
resolution and
serves as the base image on which the flyer image tiles in the current
viewport 3202 are
displayed, there is no interruption in the flyer image being shown. While the
flyer image tiles
contained in the underlay viewport 3200 are lower resolution of the required
flyer image, the
flyer image tiles provide a good representation of the flyer image tiles being
loaded. This is
made possible since there is no difference in content between the multiple
copies of the flyer
image tiles in the flyer image tile set; rather, the image tiles for the
different zoom levels are at
different resolutions. Thus, as the viewer pans to the right, the viewer first
sees the flyer image
of the low resolution flyer image tile 3210, while the interactive flyer
system 200 retrieves the
flyer image tiles in columns 3312 and 3314. When the flyer image tiles in
columns 3312 and
42
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
3314 are retrieved, they are drawn on top of the flyer image tile 3210 as
shown in FIG. 33. The
result is an interactive flyer system 200 that provides for a more refined
experience for the
viewer while giving the illusion of fast load time for the flyer image tiles.
While FIG. 32 has
been described using the viewport 3202 moving in the right direction, it will
be understood that
the viewport 3202 may move in a variety of direction. Moreover, while the
foregoing has been
described with retrieving flyer image tiles in two columns 3312 and 3314, it
will be understood
that flyer image tiles may be retrieved in a variety of different ways. For
example, the flyer
image tiles may be retrieved in one or more than one columns or rows, or may
be retrieved
selectively.
[00188] Now referring to FIG. 34A, there is shown another implementation of
a method for
pre-loading flyer image tiles according to the present disclosure. As in FIG.
32, the pre-loading
module contains an underlay viewport 3200 that contains the flyer image tiles
of the viewport at
the most zoomed-out position. As discussed in this disclosure, the flyer image
tile set includes
multiple copies of the flyer image, the copy associated with the most zoomed-
out position being
configured to fit within the viewport. Where the digital flyer is comprised of
many flyer images
(e.g. each flyer image comprising one or more pages of the digital flyer), one
or more of such
flyer images are configured to fit within the viewport at the most zoomed-out
view. For example,
in a digital flyer having ten pages, in the most zoomed-out view, four out of
the ten pages may
be configured to fit within the viewport. The number of pages that fit within
the viewport at the
most zoomed-out view depends on the digital flyer and the implementation of
the interactive
flyer system 200. In FIG. 34A, the underlay viewport 3200 contains the flyer
image tiles for the
most zoomed-out view of the flyer image while the current viewport 3202
displays the flyer
image tiles depending on the viewer's action. FIG. 34A further displays the
flyer image tiles
relevant to the current viewport 3202. In the embodiment discussed above, as
the viewer pans
to the right, the interactive flyer system 200 need not pre-load the flyer
image tiles in columns
3312 and 3314 since the flyer image tile 3210 is included in the pre-loading
module through the
underlay viewport 3200. While the flyer image tiles in columns 3312 and 3314
are loaded into
the current viewport 3202, the flyer image tile 3210 is shown in the current
viewport 3202, which
eliminates any lapse in flyer image being shown in the current viewport 3202.
However, if the
viewer does an action to pan to the right at an extreme velocity, there may be
a situation when
the current viewport 3202 crosses an area outside of the underlay viewport
3200 as shown in
FIG. 34B. In this situation, while the flyer image tiles in columns 3312 and
3314 are loading,
there will be a momentary lapse in flyer image in the current viewport 3202
since the underlay
43
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
viewport 3200 has not yet loaded the flyer image tile 3404. Thus, according to
an embodiment
of the present disclosure, the pre-loading module may pre-load flyer image
tiles based on the
action of the viewer. In this embodiment, the pre-loading module may load the
flyer image tiles
in columns 3312 and 3314, the flyer image tile 3404, or the flyer images tiles
in columns 3400,
3402, or a combination thereof. The pre-loading module may consider the
direction or speed of
travel of the viewport, or the zooming rate of the viewport in determining
which flyer image tiles
to pre-load.
[00189] In a further embodiment, the interactive flyer system 200 may
establish a
relationship between underlay viewport 3200 and the current viewport 3202. For
example,
based on the zoom level ratio between the underlay viewport 3200 and the
current viewport
3202, the viewer's panning action may be translated to the underlay viewport
3200. Referring
back to the zoom levels discussed in FIG. 8 or 9, if the current viewport 3202
is at 12x zoom
level and the underlay viewport 3200 is at lx zoom level, the interactive
flyer system 200 may
translate the viewer's action at 1/12 the rate to the underlay viewport 3200.
The translation ratio
may be any value appropriate for the underlay and current viewport 3200 and
3202. Thus,
when the viewer pans to the right, both the current viewport 3202 and the
underlay viewport
3200 would move at the same time, except that the underlay viewport 3200 would
move at a
slower rate (e.g. 1/12th the rate). As shown in FIG. 35, as the current
viewport 3202 moves
right, the underlay viewport 3200 moves with the current viewport 3202 based
on the
determined relationship. Thus, the flyer image tiles contained in the underlay
viewport 3200 is
updated based on the viewer's action.
[00190] While the pre-loading module has been described with a specific
embodiment, it will
be understood that any combination of the disclosed embodiments may be used.
Memory Management
[00191] Communication devices (e.g. 102, 104 and 106 in FIG. 1) often have
limited memory
space, particularly in mobile communication devices. Given the plurality of
the flyer image tiles
required for the different zoom levels, memory becomes a valuable resource
that must be
carefully managed. Even with efficient and intelligent pre-loading as
discussed in the present
disclosure, communication devices may run out of memory quickly and unused or
lower priority
flyer image tiles stored in memory will need to be released.
44
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
[00192] In FIG. 36A, two sets of flyer image tiles are shown. Flyer image
tile set 3600
represents a set of flyer image tiles for flexible zoom levels and flyer image
tile set 3610
represents a set of flyer image tiles for inflexible zoom levels. Using the
zoom levels as shown
in FIGs. 8 and 9 and explained in the present disclosure, the flyer image tile
set 3600 includes a
copy of the flyer image at lx resolution and 2.29x resolution and the flyer
image tile set 3610
includes a copy of the flyer image at lx resolution and 2x resolution. For
simplicity, other zoom
levels have been omitted. In the flyer image tile set 3610, because the flyer
image tile set
comprises of power of 2 tiles, tiles in each zoom level overlap the tiles in
the next zoom level.
As illustrated in FIG. 36B, when the shaded flyer tile from the tile set 3610
at lx zoom level is
loaded and shown in the viewport, the tile from the tile set 3610 at 2x zoom
level is fully
overlapped. Thus, an interactive flyer system 200 including flyer image tile
set as in tile set
3610 (e.g. power of 2 tiles), memory management is made simple since a flyer
image tile not
visible in the viewport can be safely removed from the memory. However, for
the flyer image
tile set 3600, since flyer image tiles from one zoom level are not necessarily
overlapped by the
next zoom level, memory management is significantly more difficult. As shown
in FIG. 36B,
when the shaded flyer tile from the tile set 3600 at lx zoom level is loaded
and shown in the
viewport, the flyer tile at the 2.29x zoom level is not necessarily covered.
Thus, a system and
method for managing memory are disclosed that determines the portion of flyer
image tiles that
may be released from memory.
[00193] Referring to FIG. 37, a method for managing memory is disclosed.
The memory
management module determines that memory management is required (i.e. 3700 in
FIG. 37).
In one embodiment, the memory management module may determine whether a
trigger for
memory management has been satisfied with the trigger being based on a number
of criteria,
for example, a pre-determined memory capacity threshold or a predetermined
elapsed time
since last memory management. The memory management module then orders the
flyer image
tiles stored in memory in z-order (i.e. 3702 in FIG. 37). As illustrated in
FIG. 3, the interactive
flyer includes a geometry layer that contains spatial information. In one
embodiment, the spatial
information is based on a Cartesian coordinate system relative to the
viewport. The Cartesian
coordinate system may also define a z-space that is orthogonal to the
viewport. In other words,
the z-axis goes into or out of the viewport. In block 3702 of FIG. 37, the
memory management
module orders the flyer image tiles stored in memory according to the z-order
relative to the
viewport as seen by the viewer. For example, the flyer image tiles may be
ordered from 1st to
nth z-order layer, where the 1st z-order layer includes flyer image tiles at
the top of the z-order
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
and the nth z-order layer includes flyer image tiles at the bottom of the z-
order. In other words,
the flyer image tiles would be ordered from the perspective of the viewer as
seen through the
current viewport - i.e. first layer seen by the viewer is represented by 1st z-
order layer.
Conversely, the z-order may be based on Internet layering standards, such as
those in HTML,
wherein higher z-order numbers overlap the lower z-order numbers. Embodiments
in this
disclosure will be described using the first standard, with the z-order
numbering being based on
the viewer's perspective.
[00194] Using the flyer image tiles at the top of the z-order (herein
referred to as "top order
flyer image tiles"), the memory management module determines the flyer image
tiles in memory
covered by the top order flyer image tiles (i.e. 3704 in FIG. 37). In one
embodiment, the memory
management module determines the covered portion by determining portion of the
flyer images
tiles intersected by the top order flyer image tiles. The covered portion of
the flyer image tiles
are removed from the memory since they are not visible to the viewer as the
viewer sees the top
order flyer image tiles (i.e. 3706 in FIG. 37).
[00195] Now turning to FIG. 38A, a display 3800 having a viewport 3802 is
shown. In the
viewport 3802, a partially loaded tiled flyer image 3804 is shown. For purpose
of illustration, the
flyer image tiles shown in FIGs. 38A-38E have been simplified. The interactive
flyer system 200
implementing one or more embodiments in the present disclosure may contain and
display
more complex sets of flyer images tiles and varying viewport sizes.
[00196] As illustrated, three out of four flyer image tiles have been
loaded and displayed in
the viewport 3802. While the fourth tile of the tiled flyer image 3804 is
being loaded, the flyer
image tiles from another zoom level 3806 is visible. In this example, the
tiled flyer image 3804
includes the tiles from flyer image at lx zoom level and the tiled flyer image
3806 includes the
tiles from the flyer image at 2.29x zoom level. Since other tiles from the
tiled flyer image 3806
are no longer visible, the memory management module may remove the other tiles
from the
memory. The method for such memory management will now be discussed using
FIGs. 38B-
38E.
[00197] In FIG. 38B, the tiled flyer image 3804 is shown with one of the
four tiles loaded in
memory. The tiles from the tiled flyer image 3804 are the top order image
tiles and any flyer
image tiles below (i.e. lower in z-order) the tiled flyer image 3804 would not
be visible to the
viewer. Since the portion of the tiles from the tiled flyer image 3806 in the
shaded portion would
46
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
not be visible to the viewer, the memory management module removes that
portion from
memory. This is visually shown in FIG. 38B.
[00198] Turning to FIG. 38C, the second tile from the tiled flyer image
3804 is loaded and
any tiles below the second tile would no longer be visible in the viewport.
Thus, the memory
management module removes the covered portion of the tiled flyer image 3806 as
depicted in
FIG. 38C. Similarly, in FIG. 38D, the third tile from the tiled flyer image
3804 is loaded and the
portion of the tiled flyer image 3806 covered by the loaded third tile is
removed from memory.
For purpose of illustration, if the fourth tile from the tiled flyer image
3804 is not required or is
currently being loaded, the only portion of the tiled flyer image 3806
remaining in memory would
be that as shown in FIG. 38E. The shaded portion 3808 that was initially in
memory is released
from the memory and made available to the interactive flyer system 200. Thus,
as the viewer
interacts with the digital flyer, the memory management module works to
release portions of the
memory that is no longer needed. While the foregoing was discussed in
isolation, it will be
understood that the memory management module may work in conjunction with the
pre-loading
module to efficiently manage valuable memory space in communication devices by
pre-loading
flyer image tiles and removing unnecessary flyer image tiles.
[00199] While the patent disclosure is described in conjunction with the
specific
embodiments, it will be understood that it is not intended to limit the patent
disclosure to the
described embodiments. On the contrary, it is intended to cover alternatives,
combinations,
modifications, and equivalents as may be included within the scope of the
patent disclosure as
defined by the appended claims. In the description below, numerous specific
details are set
forth in order to provide a thorough understanding of the present patent
disclosure. The present
patent disclosure may be practiced without some or all of these specific
details. In other
instances, well-known process operations have not been described in detail in
order not to
unnecessarily obscure the present patent disclosure.
[00200] It is further understood that the use of relational terms such as
first and second, and
the like, if any, are used solely to distinguish one from another entity,
item, or action without
necessarily requiring or implying any actual such relationship or order
between such entities,
items or actions.
[00201] The flowchart and block diagrams in the Figures illustrate the
architecture,
functionality, and operation of possible implementations of systems, methods
and computer
47
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
program products according to various embodiments of the present disclosure.
In this regard,
each block in the flowchart or block diagrams may represent a module, segment,
or portion of
code, which comprises one or more executable instructions for implementing the
specified
logical function(s). It should also be noted that, in some alternative
implementations, the
functions noted in the block may occur out of the order noted in the figures.
For example, two
blocks shown in succession may, in fact, be executed substantially
concurrently, or the blocks
may sometimes be executed in the reverse order, depending upon the
functionality involved. It
will also be noted that each block of the block diagrams and/or flowchart
illustration, and
combinations of blocks in the block diagrams and/or flowchart illustration,
can be implemented
by special purpose hardware-based systems that perform the specified functions
or acts, or
combinations of special purpose hardware and computer instructions.
[00202] Some portions of the detailed description in the above are
presented in terms of
algorithms and symbolic representations of operations on data bits or binary
digital signals
within a computer memory. These algorithmic descriptions and representations
may be the
techniques used by those skilled in the data processing arts to convey the
substance of their
work to others skilled in the art.
[00203] An algorithm is generally, considered to be a self-consistent
sequence of acts or
operations leading to a desired result. These include physical manipulations
of physical
quantities. Usually, though not necessarily, these quantities take the form of
electrical or
magnetic signals capable of being stored, transferred, combined, compared, and
otherwise
manipulated. It has proven convenient at times, principally for reasons of
common usage, to
refer to these signals as bits, values, elements, symbols, characters, terms,
numbers or the like.
It should be understood, however, that all of these and similar terms are to
be associated with
the appropriate physical quantities and are merely convenient labels applied
to these quantities.
[00204] Unless specifically stated otherwise, as apparent from the above
discussions, it is
appreciated that throughout the specification discussions utilizing terms such
as "processing,"
"computing," "calculating," "determining," or the like, refer to the action
and/or processes of a
computer or computing system, or similar electronic computing media player
device, that
manipulate and/or transform data represented as physical, such as electronic,
quantities within
the computing system's registers and/or memories into other data similarly
represented as
physical quantities within the computing system's memories, registers or other
such information
storage, transmit session or display devices.
48
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
[00205] Embodiments within the scope of the present disclosure can be
implemented in
digital electronic circuitry, or in computer hardware, firmware, software, or
in combinations
thereof. Apparatus within the scope of the present disclosure can be
implemented in a computer
program product tangibly embodied in a machine-readable storage medium for
execution by a
programmable processor; and method actions within the scope of the present
disclosure can be
performed by a programmable processor executing a program of instructions to
perform
functions of the present disclosure by operating on input data and generating
output.
Embodiments within the scope of the present disclosure may be implemented
advantageously
in one or more computer programs that are executable on a programmable system
including at
least one programmable processor coupled to receive data and instructions
from, and to
transmit data and instructions to, a data storage system, at least one input
device, and at least
one output device. Each computer program can be implemented in a high-level
procedural or
object oriented programming language, or in assembly or machine language if
desired; and in
any case, the language can be a compiled or interpreted language. Suitable
processors include,
by way of example, both general and special purpose microprocessors.
Generally, a processor
will receive instructions and data from a read-only memory and/or a random
access memory.
Generally, a computer will include one or more mass storage devices for
storing data files.
Embodiments within the scope of the present disclosure include computer-
readable media for
carrying or having computer-executable instructions, computer-readable
instructions, or data
structures stored thereon. Such computer-readable media may be any available
media, which is
accessible by a general-purpose or special-purpose computer system. Examples
of computer-
readable media may include physical storage media such as RAM, ROM, EPROM, CD-
ROM or
other optical disk storage, magnetic disk storage or other magnetic storage
devices, or any
other media which can be used to carry or store desired program code means in
the form of
computer-executable instructions, computer-readable instructions, or data
structures and which
may be accessed by a general-purpose or special-purpose computer system. Any
of the
foregoing can be supplemented by, or incorporated in, ASICs (application-
specific integrated
circuits). It should be understood that embodiments of the present disclosure
may be used in a
variety of applications. Although the present disclosure is not limited in
this respect, the methods
disclosed herein may be used in many apparatuses such as in the transmitters,
receivers and
transceivers of a radio system. Radio systems intended to be included within
the scope of the
present disclosure include, by way of example only, cellular radiotelephone
communication
systems, satellite communication systems, two-way radio communication systems,
one-way
pagers, two-way pagers, personal communication systems (PCS), personal digital
assistants
49
CA 02825074 2013-07-18
WO 2012/097438 PCT/CA2012/000042
(PDA's), notebook computers in wireless local area networks (WLAN), wireless
metropolitan
area networks (WMAN), wireless wide area networks (VVVVAN), or wireless
personal area
networks (WPAN, and the like).
