Note: Descriptions are shown in the official language in which they were submitted.
CA 02606372 2007-10-26
WO 2006/108190 PCT/US2006/013756
Computing Online Advertising Statistics and Implementing
Computer-Generated Modifications to an Online Advertising
Program in Real Time
TECHNICAL FIELD
[0001] The disclosed embodiments relate generally to online advertising. More
particularly, the disclosed embodiments relate to methods and systems to
generate statistics,
alerts, and other information regarding online advertising in real time, and
to automatically
modify an online advertising program in real time.
BACKGROUND
[0002] Online advertising is an important advertising mediuin that continues
to grow
rapidly as use of the Internet expands. A key concern for advertisers is that
the money that
they spend on online advertisements be well spent. Consequently, online
advertisers want
feedback on the effectiveness of their advertisements so that they can adjust
their advertising
programs accordingly.
[0003] To date, because of the vast amount of advertising data (e.g., ad
impression
and click through logs) that must be evaluated, large-scale online ad
placement providers
(e.g., Google and Yahoo!) have only been able to generate statistics regarding
online
advertising on a daily basis. This delay can be costly for both the advertiser
and the ad
placement provider. For example, advertisers may continue to show unproductive
ads or
reach their daily spending limits too soon. Ad placement providers may lose
revenue by
displaying unproductive ads with few click throughs or by displaying ads that
have run out of
budget.
[0004] Advertisers generally don't find out about advertising problems until
they take
the initiative to review and analyze the statistics for their advertisements.
This delay can be
costly for both the advertiser and the ad placement provider, too.
-1-
CA 02606372 2007-10-26
WO 2006/108190 PCT/US2006/013756
SUMMARY
[0005] The present invention overcomes the limitations and disadvantages
described
above by generating statistics, alerts, and other information regarding online
advertising in
real time. As used herein, "real time" means within minutes, rather than hours
or days. For
example, cumulative statistics concerning online advertising are updated in
real time, within
minutes of new data being received. In this example, the real-time online
advertising
statistics are cumulative statistics that are based in part on data that was
received within
minutes of the statistics being generated or updated. Similarly, real-time
alerts/notifications
are based at least in part on data that was received within minutes of the
alerts/notifications
being generated, although clearly such alerts/notifications could also be
based in part on data
that was received hours or days before. Likewise, real-time modifications to
an online
advertising program mean changes in an online advertising program that are
made within
minutes of receiving an alert or other notification.
[0006] One aspect of the invention involves a metllod of receiving one or more
data
streams containing real-time data concerning online advertisements for an
advertiser;
applying a continuous query to the one or more data streams; generating an
ongoing response
to the continuous query; and sending at least a portion of the ongoing
response to a computer
associated with the advertiser.
[0007] Another aspect of the invention involves a computer system with at
least one
computer that is configured to receive one or more data streams containing
real-time data
concerning online advertisements for an advertiser; apply a continuous query
to the one or
more data streams; generate an ongoing response to the continuous query; and
send at least a
portion of the ongoing response to a computer associated with the advertiser.
[0008] Another aspect of the invention involves a machine readable medium
having
stored thereon data representing sequences of instructions, which when
executed by a
computer system with at least one computer, cause the computer system to
receive one or
more data streams containing real-time data concerning online advertisements
for an
advertiser; apply a continuous query to the one or more data streams; generate
an ongoing
response to the continuous query; and send at least a portion of the ongoing
response to a
computer associated with the advertiser.
-2-
CA 02606372 2007-10-26
WO 2006/108190 PCT/US2006/013756
[0009] Another aspect of the invention involves a computer system that
includes
means for receiving one or more data streams containing real-time data
concerning online
advertisements for an advertiser; means for applying a continuous query to the
one or more
data streams; means for generating an ongoing response to the continuous
query; and means
for sending at least a portion of the ongoing response to a computer
associated with the
advertiser.
[0010] Another aspect of the invention involves a method at a computer
associated
with an advertiser. The method involves receiving at least a portion of an
ongoing response to
a coiltinuous query. The continuous query is applied by a remote computer to
one or more
data streams containing real-time data concerning online advertisements for
the advertiser.
[0011] Another aspect of the invention involves a system that includes a
computer
associated with an advertiser. The computer is configured to receive at least
a portion of an
ongoing response to a continuous query. The continuous query is applied by a
reinote
computer to one or more data streams containing real-time data concerning
online
advertisements for the advertiser.
[0012] Another aspect of the invention involves a machine readable medium
having
stored thereon data representing sequences of instructions, which when
executed by a
computer associated with an advertiser, cause the computer to receive at least
a portion of an
ongoing response to a continuous query. The continuous query is applied by a
remote
computer to one or more data streams containing real-time data concerning
online
advertisements for the advertiser.
[0013] Another aspect of the invention involves a method at a computer
associated
with an ad placeinent provider. The method involves receiving a plurality of
data streams
containing real-time data concerning online advertisements in an online
advertising program
for an advertiser; applying a first continuous query to the plurality of data
streams; generating
a first ongoing response to the first continuous query; applying a second
continuous query to
the first ongoing response; generating a second response to the second
continuous query; and
in response to an event associated with generating the first or second
response, implementing
a computer-generated change in the advertising program in real time.
[0014] Another aspect of the invention involves a computer system comprising
at
least one computer associated with an ad placement provider. The computer
system is
-3-
CA 02606372 2007-10-26
WO 2006/108190 PCT/US2006/013756
configured to receive a plurality of data streams containing real-time data
concerning online
advertisements in an online advertising program for an advertiser; apply a
first continuous
query to the plurality of data streams; generate a first ongoing response to
the first continuous
query; apply a second continuous query to the first ongoing response; generate
a second
response to the second continuous query; and in response to an event
associated with
generating the first or second response, implenlent a computer-generated
change in the
advertising program in real time.
[0015] Another aspect of the invention involves a machine readable medium
having
stored thereon data representing sequences of instructions, which when
executed by a
computer system associated with an ad placement provider, cause the computer
system to
receive a plurality of data streams containing real-time data concerning
online advertisements
in an online advertising program for an advertiser; apply a first continuous
query to the
plurality of data streams; generate a first ongoing response to the first
continuous query;
apply a second continuous query to the first ongoing response; generate a
second response to
the second continuous query; and in response to an event associated with
generating the first
or second response, implement a computer-generated change in the advertising
program in
real time.
[0016] Another aspect of the invention involves a computer system that
includes
means for receiving a plurality of data streams containing real-time data
concerning online
advertisements in an online advertising program for an advertiser; means for
applying a first
continuous query to the plurality of data streams; means for generating a
first ongoing
response to the first continuous query; means for applying a second continuous
query to the
first ongoing response; means for generating a second response to the second
continuous
query; and in response to an event associated with generating the first or
second response,
means for implementing a computer-generated change in the advertising program
in real
time.
BRIEF DESCRIPTION OF THE DRAWINGS
[0017] For a better understanding of the aforementioned aspects of the
invention as
well as additional aspects and embodiments thereof, reference should be made
to the
Description of Embodiments below, in conjunction with the following drawings
in which like
reference numerals refer to corresponding parts throughout the figures.
-4-
CA 02606372 2007-10-26
WO 2006/108190 PCT/US2006/013756
[0018] Figure 1 is a bloclc diagram illustrating an exemplary distributed
computer
system according to an embodiment of the invention.
[0019] Figure 2 is a block diagram illustrating an ad placement provider
computer in
accordance with one embodiment of the present invention.
[0020] Figure 3 is a block diagram illustrating an advertiser computer in
accordance
with one embodiment of the present invention.
[0021] Figure 4 is a flowchart representing a method of generating statistics,
alerts,
and other information regarding online advertising in real time according to
one einbodiment
of the present invention.
[0022] Figure 5 is a flowchart representing a method of making computer-
generated
modifications to an online advertising program in real time according to one
embodiment of
the present invention.
[0023] Figure 6 is a block diagram illustrating the use of exeniplary types of
database
operators in queries.
DESCRIPTION OF EMBODIMENTS
[0024] Methods and systems are described that generate and automatically
respond to
statistics, alerts, and other information regarding online advertising in real
time. Reference
will be made to certain embodiments of the invention, examples of which are
illustrated in
the accompanying drawings. While the invention will be described in
conjunction with the
embodiments, it will be understood that it is not intended to limit the
invention to these
particular einbodiments alone. On the contrary, the invention is intended to
cover
alternatives, modifications and equivalents that are within the spirit and
scope of the
invention as defined by the appended claims.
[0025] Moreover, in the following description, numerous specific details are
set forth
to provide a thorough understanding of the present invention. However, it will
be apparent to
one of ordinary skill in the art that the invention may be practiced without
these particular
details. In other instances, methods, procedures, applications, components,
networks, and
database technologies that are well known to those of ordinary skill in the
art are not
described in detail to avoid obscuring aspects of the present invention.
-5-
CA 02606372 2007-10-26
WO 2006/108190 PCT/US2006/013756
[0026] Figure 1 is a block diagram illustrating an exemplary distributed
computer
system according to one embodiment of the invention. This system includes ad
placement
provider computer 102, ad server(s) 110, advertiser computers 104, and
communication
networlc(s) 106 (e.g., the Internet) for interconnecting these components. In
some
embodiments, advertiser computers 104 include networlc application 108 and
graphical user
interface (GUI) 112. As used herein, a network application is a networlc-
connected (e.g.,
Internet-connected) software application with its own user interface (e.g., a
web browser
application, an email application, a peer-to-peer transfer application, or an
instant messaging
application). Referring to Figures 1 and 2, streaming database 220 in ad
placenlent provider
computer 102 is configured to receive one or more data streams 222 containing
real-time data
concerning online advertisements for an advertiser. In some embodiments, the
real-time data
is contained in data logs sent from ad server(s) 110. Query engine 226 in ad
placement
provider computer 102 applies a continuous query to the one or more data
streams and
generates an ongoing response to the continuous query. Publish/subscribe
module 244 in ad
placement provider computer 102 sends at least a portion of the ongoing
response to the
advertiser computer 104 that subscribed to the result of the continuous query.
Advertiser
computer 104 receives at least a portion of the ongoing response to the
continuous query.
[0027] Ad placement provider computer 102 is also configured to receive a
plurality
of data streams containing real-time data concerning online advertisements in
an online
advertising program for an advertiser, e.g., using streaming database 220.
Query engine 226
in ad placement provider computer 102 applies a first continuous query to the
plurality of
data streams 222; generates a first ongoing response to the first continuous
query; applies a
second continuous query to the first ongoing response; and generates a second
response to the
second continuous query. In response to an event associated with generating
the first or
second response, advertising program manager 238 in ad placement provider
computer 102
implements a computer-generated change in the advertising program in real
time. In some
embodiments, advertiser computer 104 receives information associated with the
first
response, the second response, or both the first response and the second
response, and sends
computer-generated instructions to change the online advertising program for
the advertiser
in real time.
[0028] Figure 2 is a block diagram illustrating an ad placement provider
computer in
accordance with one embodiment of the present invention. Ad placement provider
computer
-6-
CA 02606372 2007-10-26
WO 2006/108190 PCT/US2006/013756
102 typically includes one or more processing units (CPU's) 202, one or more
network or
other comniunications interfaces 204, memory 206, and one or more
communication buses
208 for interconnecting these components. Ad placement provider computer 102
optionally
may include a user interface 208 comprising a display device 210 and a
keyboard 212.
Memory 206 may include high speed random access memory and may also include
non-
volatile memory, such as one or more magnetic disk storage devices. Memory 206
may
optionally include one or more storage devices remotely located from the
CPU(s) 202. In
some embodiments, the memory 206 stores the following prograins, modules and
data
structures, or a subset or superset thereof:
= an operating system 214 that includes procedures for handling various basic
system
services and for perforining hardware dependent tasks;
= a network communication module 216 that is used for connecting ad placement
provider computer 102 to other computers (e.g., advertiser computers 104 and
ad
server(s) 110) via one or more communication network interfaces 204 (wired or
wireless), wliich in turn connect ad placement provider computer 102 to one or
more
communications networks 106 such as the Internet, other wide area networks,
local
area networks, metropolitan area networks, and so on;
= an advertising statistics database 218 that provides advertising statistics
to one or
more advertiser computers 104 in real time and includes streaming database
220,
relational database 232 and query response data 236; and
= an advertisement program manager 238 that implements the online advertising
programs for a plurality of advertisers, which includes
o advertisers' rules 240 that specify the online advertising programs for a
plurality of advertisers, including in some embodiments how those programs
are to be modified by responses to particular continuous queries of streaming
database 220;
o advertisers' content 242 that contains the online advertising content for a
plurality of advertisers; and
o a publisli/subscribe module 244 that publishes particular query responses
generated by advertising statistics database 218 to subscribers such as
-7-
CA 02606372 2007-10-26
WO 2006/108190 PCT/US2006/013756
particular rules in the advertisers' rules 240 and/or advertiser computers 104
that subscribe to particular query responses.
[0029] Streaming database 220 includes
o data streams 222 received from multiple sources (e.g., ad servers 110) that
contain real-time data concerning online advertisements for one or more
advertisers (e.g., ad impression data, ad click through data, bid position
data,
ad spending data, and/or ad conversion data);
o adapters 224 that place the data in data streains 222 into a fornlat
suitable for
storage in the streaming database 220 and for processing by query engine 226
(e.g., by formatting and ordering the data in the data streams 222 to be
consistent with the schemas of corresponding tables in the streaming database
220); and
o query engine 226 that executes queries against streaming database 220 and/or
relational database 232, including the application of continuous queries 228
and ad hoc queries 230 to data streanls 222 and to ongoing responses produced
by other continuous queries 228 (e.g., there can be a hierarchy of continuous
queries in which a second continuous query is applied to the ongoing response
of a first continuous query, a third continuous query is applied to the
ongoing
response of a second continuous query, and so on).
[0030] Relational database 232 includes non-streanled data (e.g., advertisers'
budgets,
bids, account balances and/or cumulative expenses, keywords 234) that are used
in
combination with streaming database 220 to answer queries. In some
embodiments, relational
database 232 is also used to store advertising statistics, such as snapshots
of real-time
advertising statistics.
[0031] Query response data 236 contains the responses generated by the
execution of
queries by the query engine 226 against the streaming database 220 and/or
relational database
232. Query response data 236 are sent to the particular advertisers' rules 240
and/or
advertising computers 104 that have subscribed to the query responses
generated by
corresponding queries.
-8-
CA 02606372 2007-10-26
WO 2006/108190 PCT/US2006/013756
[0032] Figure 3 is a block diagram illustrating an advertiser computer in
accordance
with one embodiment of the present invention. Advertiser computer 104 can be
any computer
used by or on behalf of an advertiser. The aspects of Figure 3 that are
analogous to those in
Figure 2 are not described again here. The memory 306 in advertiser computer
104 stores the
following programs, modules and data structures, or a subset or superset
thereof:
= an operating system 316, analogous to operating systein 216 described above;
= a network communication module 318, analogous to network communication
module
218 described above;
= a network application 108, such as a web browser, that is used in some
embodiments
to view real-time advertising statistics, including advertiser query response
data 320;
= advertiser query response data 320 that includes the portion of the query
response data
236 that a particular advertiser subscribes to;
= advertisement management application 320 that manages the online advertising
program for a particular advertiser and includes
o advertiser content 322 that contains the advertisements used in a particular
advertiser's online advertising program;
o advertiser rules 324 that specify the online advertising program for a
particular
advertiser, including how that program is to be modified by responses to
particular continuous queries perfomled by the query engine 226; and
o advertiser data 326 that includes non-streamed data (e.g., advertisers'
budgets,
bids, account balances, etc.) that are used to manage a particular
advertiser's
online advertising program.
[0033] Referring to Figures 2 and 3, each of the above identified modules and
applications corresponds to a set of instructions for performing a function
described above.
These modules (i.e., sets of instructions) need not be implemented as separate
software
programs, procedures or modules, and thus various subsets of these modules may
be
combined or otherwise re-arranged in various embodiments. In some embodiments,
memory
206 or 306 may store a subset of the modules and data structures identified
above.
Furthermore, memory 206 or 306 may store additional modules and data
structures not
described above.
-9-
CA 02606372 2007-10-26
WO 2006/108190 PCT/US2006/013756
[0034] Although Figures 2 and 3 show computers 102 and 104 as a number of
discrete items, Figures 2 and 3 are intended more as functional descriptions
of the various
features that may be present in computers 102 and 104, rather than as a
structural schematic
of the embodiments described herein. In practice, and as recognized by those
of ordinary
skill in the art, items shown separately could be combined and some items
could be
separated. For example, some items shown separately in Figures 2 and 3 could
be
implemented on single computers and single items could be implemented by one
or more
computers. The actual number of computers used to implement ad placement
provider
computer 102 or advertiser computer 104 and how features are allocated among
those
computers will vary from one implementation to another, and may depend in part
on the
ainount of data traffic that the ad placement provider computer system or
advertiser computer
system must handle during peak usage periods as well as during average usage
periods.
[0035] In practice, the distribution of advertising program management and
data
storage functions between ad placement computer 102 and advertiser computers
104 can be
divided up in many different ways. In some embodiments, advertising management
application 320 is an applet running in network application 108 (e.g., a web
browser), with
the advertiser content 322 and rules 324 being input at advertiser computer
104, but stored in
advertisers' content 242 and advertisers' rules 240. In other embodiments,
advertising
management application 320 is a separate application. In some embodiments, an
advertiser's
rules for how their online advertising program is to be modified in response
to
alerts/notifications are stored only in advertiser rules 324 on advertiser
computer 104, rather
than in advertisers' rules 240, too. In other embodiments, all of an
advertiser's rules, including
their rules for how their online advertising program is to be modified in
response to
alerts/notifications, are stored in both advertiser rules 324 and advertisers'
rules 240.
[0036] Figure 4 is a flowchart representing a method of generating statistics,
alerts,
and other infomlation regarding online advertising in real time according to
one embodiment
of the present invention. It will be appreciated by those of ordinary skill in
the art that one or
more of the acts described may be performed by hardware, software, or a
combination
thereof, as may be embodied in one or more coinputing systems.
[0037] Streaming database 220 in ad placement provider computer 102 receives
(404)
one or more data streams 222 containing real-time data concerning online
advertisements for
-10-
CA 02606372 2007-10-26
WO 2006/108190 PCT/US2006/013756
an advertiser. In some embodiments, data streams 222 contain real-time data
concerning
online advertisements for a plurality of advertisers. In some embodiments, the
real-time data
includes one or more of ad impression data, ad click through data, bid
position data, ad
spending data, and ad conversion data. In some embodiments, the streaming
database 220
joins a plurality of data streams 222 from multiple sources.
[0038] In some embodiments, query engine 226 maintains a network of continuous
queries 228. Continuous queries 228 are applied to data streams 222 or other
continuous
queries, and can also be joined with relations (tables). In some embodiments,
each query, ad
hoc 230 or continuous 228, is compiled into a number of operators that
implement the same
operator interface. Figure 6 is a block diagram illustrating the use of
exemplary types of
database operators in queries, including stream, relation, join, group by,
filter, and cube
operators. The operators form acyclic directed graphs.
[0039] In some embodiments, data flow from one operator to another through the
shared publisli/subscribe messaging module 244. An operator finishes its task
and publishes
its data to publish/subscribe module 244. This data is then consumed by all of
its subscribers.
[0040] Most operators are non-blocking. The benefit of non-blocking query
operators
is that the workload can be partitioned and pipelined for a greater degree of
parallel
processing.
[0041] In Figure 6, the BaseStreams (e.g. BaseStreaml) are continuous data
streams
from one or more external sources, such as one or more Ad Servers 110 (Figure
1), while the
Relations (e.g., Relationl) are tables or portions of tables in a relational
database.
Continuous query Stream9 is derived from BaseStreaml through the application
of data cube
operator. Streaml0 is the result of the joins between a filtered Stream9 and
Relation5.
Stream5 is the result of a join of BaseStreaml and Relationl, followed by a
Group By
operation (in some embodiments, window aggregation is implemented using the
Group By
operator), and then followed by a filter (e.g., the HAVING clause in SQL).
Stream5 is joined
with Relation4 to form Stream6. Stream 7 is formed by a join of Stream3 and
Stream4;
Stream3 is formed by a join of BaseStreaml and Relation2, while Stream4 is
formed by a
join of BaseStream2 and Relation3.
-11-
CA 02606372 2007-10-26
WO 2006/108190 PCT/US2006/013756
[0042] In some embodiments, each operator processes data in batch, and writes
out its
state (incrementally) to disk storage in memory 206 before publishing its
results. A batch
interval can be specified by a number of events or by a time interval, or
both. Batch
processing and writing to disk increases the processing latency, but helps ad
placement
provider computer 102 to scale and to more easily recover from hardware
failures.
[0043] In some embodiments, data streams 222 are partitioned transparently,
based on
their timestamps. Within the same partition, the data may be partitioned again
based on the
semantics of a query. For example, the fields in the window partition clause
or group by
clause would serve as secondary partition keys to further parallelize
processing the data.
[0044] Query engine 226 in ad placement provider computer 102 applies (406) a
continuous query 228 to the one or more data streams 222. In some embodiments,
continuous
query 228 concerns online advertisement data for one advertiser in a plurality
of advertisers.
For example, for a particular advertiser's online advertising program,
advertiser rules 324
may include rules that correspond to continuous queries input at advertiser
computer 104,
communicated to ad placement computer 102, stored in advertisers' rules 240,
and
implemented as continuous queries 228 of data streams 222. Publish/subscribe
module 244
sends to the corresponding advertising computer 104 and/or advertisers' rules
240 the
responses to continuous queries 228. In addition, some advertiser rules 324
may include
continuous queries 228 that join data streams 222 with tables containing data
from one or
more sources (e.g., 234 and/or 326).
[0045] In some embodiments, a continuous query 228 is a sliding window query.
In
some einbodiments, the continuous queries 228 and ad hoc queries 230 are
Structured Query
Language (SQL) queries.
[0046] For example, assume that there is a data stream called
DailyCriteriaCreativeStats that provides click and impression data for each
advertisement
(also called a creative). The following continuous SQL query maintains the
seven day
moving average of the CTR for each advertisement for a given advertiser
(customer):
create continuous query ctr_monitor as
select AdGroupCreativeId,
customer_id,
Date,
sum(Clicks) over w/sum(Impressions) over w as CTR
-12-
CA 02606372 2007-10-26
WO 2006/108190 PCT/US2006/013756
from DailyCriteriaCreativeStats
window w as
partition by AgGroupCreativeId
order by Date
RANGE Interval '7' day preceding
[0047] This example is a standard SQL-2003 query. It uses the window function
feature in the ANSI-SQL OLAP feature set. To process the window clause, the
row is
partitioned based on its creative id value into multiple partitions. For each
partition, the rows
are sorted based on the Date value (although an actual sort may not be
required if the Date
value was already sorted in the input stream DailyCriteriaCreativeStats). A
row, based on the
sorting order of Date, is placed in the window if its Date value falls within
the time interval
expression. An ANSI-SQL 2003 time interval type represents the difference
between two
timestamp values. So in this case, only rows that have Date values that are
within the last
seven days are placed in the window. To compute the CTR, the sum(Clicks) and
sum(Irnpressions) are computed and then their division is computed
incrementally for the
rows in the window.
[0048] Table 1 illustrates exemplary input and output of the continuous query
ctr monitor.
Table 1
DailyCriteriaCreativeStats
Date AdGroupCreativeld Clicks Impressions CostUS$
6/24/2004 1 10 100 10.1
6/25/2004 2 20 100 20.5
6/26/2004 1 30 100 32
6/27/2005 2 40 100 40
CTR Monitor Date AdGroupCreativeId CTR
-13-
CA 02606372 2007-10-26
WO 2006/108190 PCT/US2006/013756
6/24/2004 1 0.1
6/25/2004 2 0.2
6/26/2004 1 0.3
6/27/2004 2 0.4
In this exainple, the sliding window query is incrementally maintained,
logically on a per log
record basis. However, for performance purposes, the query result may be
updated in batch
mode, such as in one-minute intervals between updates.
[0049] The query engine 226 in ad placement provider computer 102 generates
(408)
an ongoing response to a continuous query 228. In some embodiments, the
ongoing response
is generated in real time. In some embodiments, the ongoing response includes,
for a
particular ad for a predeterinined time period, one or more of the number of
ad impressions,
the number of cliclc throughs, an average or current bid position, the total
amount spent, the
number of ad conversions, the click-tlirough rate, and the cost per click
through (CPC).
[0050] As used herein and as commonly understood by those of ordinary skill in
the
art, the number of ad impressions is the number of times an ad (or a group of
ads) is
displayed by an ad placement provider or by a party that displays ads for the
ad placement
provider. The number of click throughs is the number of times that users click
on an ad (or
group of ads) or otherwise indicate that they want to view more information
related (e.g.,
hyperlinked) to the ad. The current or average bid position is the current or
average
placement position of an ad in a listing of ads. The total amount spent is the
cost to an
advertiser for a given ad or group of ads. The number of ad conversions is the
number of
times that an ad impression led to a sale (e.g., at a website hyperlinked to
the ad). For a given
ad (or group of ads), the click-through rate (CTR) is the number of click
throughs divided by
the number of impressions. For a given ad (or group of ads), the cost per
click through (CPC)
is the nunlber of click throughs divided by the total amount spent for the ad.
Each of the
preceding terms is typically determined for a particular time period.
[0051] Continuing with the description of Figure 4, the subscribe/publish
module 244
in advertisement program manager 238 sends (410) at least a portion of the
ongoing response
to a computer associated with the advertiser, e.g., advertiser computer 104.
Advertiser
-14-
CA 02606372 2007-10-26
WO 2006/108190 PCT/US2006/013756
computer 104 receives (412) at least a portion of the ongoing response to the
continuous
query 228, e.g., via networlc application 108 and advertising management
application 320.
[0052] In some embodiments, query engine 226 applies a second continuous query
to
the response (to the first continuous query) and generates a response to the
second continuous
query. In some embodiments, the response to the second continuous query is
generated in real
time. Subscribe/publish module 244 sends at least a portion of the response to
the second
continuous query to a computer associated with the advertiser, e.g.,
advertiser computer 104
and/or to an Advertiser Rule 324 associated with the advertiser. In some
embodiments, the
response to the second continuous query includes an alert. Advertiser computer
104 receives
at least a portion of an ongoing response to the second continuous query. In
some
embodiments, advertiser computer 104 receives at least a portion of an ongoing
response to
the second continuous query in real time.
[0053] Figure 5 is a flowchart representing a method of making computer-
generated
modifications to an online advertising program in real time according to one
embodiment of
the present invention. It will be appreciated by those of ordinary skill in
the art that one or
more of the acts described may be performed by hardware, software, or a
combination
tllereof, as may be embodied in one or more computing systems.
[0054] As used herein, an advertising program is one or more online
advertisements
along with a set of parameters that determine how the advertisements are used.
Exemplary
parameters include, without limitation, ad budgets, ad placement bidding rules
(e.g.,
maximum cost per click through (CPC)), keywords associated with ads, ad
distribution
preferences, groupings of ads (e.g., combining individual ads into ad groups
and combining
ad groups into ad campaigns), and rules for modifying the placement of ads, ad
groups,
and/or ad campaigns in response to alerts or other notifications.
[0055] In some embodiments, advertiser computer 104 sends (500) and ad
placement
provider computer 102 receives (502) a set of computer-implemented rules 324
for the online
advertising program for an advertiser. In some cases, rules 324 are specified
at advertiser
computer 104 by inputting user preferences via GUI 112 (e.g., via checkboxes
in GUI 112).
In other cases, rules 324 are specified at advertiser computer 104 by
inputting custom code
(e.g., SQL). Ad placement provider computer 102 stores the set of computer-
implemented
-15-
CA 02606372 2007-10-26
WO 2006/108190 PCT/US2006/013756
rules 324 for the online advertising program for the advertiser in the
advertisers' rules 240
portion of advertisement program manager 238. In some embodiments,
advertisement
program manager 238 uses the received set of rules 324 to change the
advertising program in
real time.
[0056] Streaming database 220 at ad placement provider computer 102 receives
(504)
a plurality of data streams 222 containing real-time data concenling online
advertisements in
an online advertising program for an advertiser. While the explanation of
Figure 5 will focus
on the processing of real time data for one online advertising program for one
advertiser, it
should be understood that the data streams 222 may contain real-time data
concerning online
advertisements in the online advertising programs of a plurality of
advertisers, and the
processing operations of Figure 5 may be applied to the real-time data
associated with a
plurality of online advertising programs of a plurality of advertisers.
[0057] Query engine 226 of ad placement provider computer 102 applies (506) a
first
continuous query to the plurality of data streams 222 (e.g., the continuous
query ctr_monitor
described above), and generates (508) a first ongoing response to the first
continuous query
(e.g., the rows in Table 1 produced by the continuous query ctr_monitor
described above).
[0058] The query engine 226 furthermore applies (510) a second continuous
query to
the first ongoing response, and thereby generates (512) a second response to
the second
continuous query. In some embodiments, the second response is an alert.
[0059] As noted above, rules (e.g., in advertiser rules 324 and advertisers'
rules 240)
can be specified as continuous queries 228, or may correspond to continuous
queries. For
example, a rule that identifies advertisements (creatives) that have a seven
day moving CTR
less than 1% can be defined as follows:
create continuous query low_ctr_rule as
select AdGroupCreativeld
from ctr monitor
where ctr_monitor.ctr < 0.01
An application (e.g., program manager 238 or management application 320) can
subscribe
(e.g., via publishlsubscribe module 244) to the low ctr rule. When there is a
row emitted
-16-
CA 02606372 2007-10-26
WO 2006/108190 PCT/US2006/013756
from this rule query, the application will get a callback with the information
in the select
clause. The application can then process the alert associated with the rule
accordingly.
[0060] In some embodiments, joins can be used in the rule queries to include
contextual information for an alert. For example, suppose that the text of an
advertisement is
stored in a table named Creatives. A rule that includes the text of the
advertisement when an
alert is triggered can be defined as follows:
create continuous query low ctr_rule2 as
select AdGroupCreativeId, Creatives.Line1, Creatives.Line2,
Creatives.Line3, ctr,
from ctrmonitor, Creatives
where AdGroupCreativeId = Creatives.Creativeld
and ctr < 0.01
[0061] In some cases, once a rule is fired, a customer may not want to receive
the
saine alert every time a new event occurs that would satisfy the same
condition. For example,
suppose a rule is triggered when a CTR is below 1%. Suppose further that the
customer only
wants to receive an alert if the CTR goes over 1% and then dips below 1% again
within the
last day, and furthermore wants to receive this alert no more than once per
day. To provide
for this customer preference, a first continuous query ctr_thresholdl is
defined as follows:
create continuous query ctr_thresholdl as
select AdGroupCreativeld,
ctr,
current(ctr) over w as current_ctr,
previous(ctr) over w as previous_ctr,
sum(case (when current_ctr< 0.01 and previous_ctr >=0.01)
then 1
else 0) as number_ctr_dip_below_threshold_occurence,
from ctr_monitor
window w as
(order by Date
partition by AdGroupCreativeId
range interval '1' day preceding
slide interval '1' day
[0062] The current aggregate function takes the latest value of its input
value
expression that is within the window. The previous aggregate function takes
the previous
value of its input value expression. ("Previous" really means "last", but
"last" is a keyword in
SQL that means something different, so "previous" is used as the function name
instead.)
-17-
CA 02606372 2007-10-26
WO 2006/108190 PCT/US2006/013756
Current and previous aggregate functions are extension functions that are not
defined in
ANSI SQL 2003. The slide clause tells the query engine to move the window by 1
day when
the date changes. So the number ctr dip below_threshold occurence counts the
number of
times that the CTR dips below 0.01 within the last day.
[0063] A second continuous query/rule can be defined as follows:
create continuous query low_ctr_rule3 as
select AdGroupCreativeld, ctr
from ctr_thresholdl
where ctr_dip_below_threshold_occurence = 1
With this rule, an alert will only be triggered when the CTR goes over 1% and
then dips
below 1% again within the last day.
[0064] In some cases, a customer may not want to receive an alert unless a CTR
stays
below a threshold value (e.g., 1%) for a predetermined time period (e.g., a 1
hour window).
In other words, if a CTR dips briefly below a threshold value and then
recovers, a customer
may not want to be notified. To provide for this customer preference, a first
continuous query
ctr threshold2 is defined as follows:
create continuous query ctr_threshold2 as
select AdGroupCreativeId,
ctr,
current(ctr) over w as current_ctr,
sum(case (when current_ctr >= 0.01)
then 1
else 0) as number_above_threshold
from ctr_monitor
window w as
(order by Date
partition by AdGroupCreativeId
range interval '1' hour preceding)
[0065] A second continuous query/rule can be defined as follows:
create continuous query stay_low_ctr_rule as
select AdGroupCreativeId,
ctr,
sum(number_above_threshold) as sum above_count
from ctr_threshold2
where number_above_threshold = 0
With this rule, an alert will only be triggered when the CTR is less than 1%
for the entire
preceding hour.
-18-
CA 02606372 2007-10-26
WO 2006/108190 PCT/US2006/013756
[0066] In some cases, a customer may want to look at trends and be alerted if
a CTR
dips below a threshold percentage of the CTR for an earlier time period (e.g.,
if the weelcly
CTR dips below 50% of the previous week's CTR). To provide for this customer
preference,
a first continuous query ctr weekly_trend is defined as follows:
create continuous query ctr_weekly_trend as
select AdGroupCreativeId,
customer_id,
Date,
sum(Clicks) over w/sum(Impressions) over w1 as
last_week_ctr,
sum(Clicks) over w/sum(Impressions) over w2 as
this_week ctr
from DailyCriteriaCreativeStats
window w1 as
partition by AgGroupCreativeId
order by Date
RANGE between Interval '14' day preceding and Interval
'7' day preceding
),
w2 as
partition by AgGroupCreativeId
order by Date
RANGE Interval '7' day preceding
[0067] A second continuous query/rule can be defined as follows:
create continuous query bad_weekly_ctr_trend_rule as
select AdGroupCreativeId, this_week_ctr, last_week_ctr
from ctr_weekly_trend
where this_week_ctr/last_week_ctr < 0.5
[0068] With this rule, an alert will only be triggered when this week's CTR is
less
than 50% of the previous week's CTR.
[0069] In response to an event associated with generating the first or second
response,
ad placement provider computer 102 (i.e., advertising program manager 238)
implements
(516) a computer-generated change in the advertising program in real time.
[0070] In some embodiments, ad placement provider computer 102 sends (514)
information associated with the first and/or second response to a computer
associated with
-19-
CA 02606372 2007-10-26
WO 2006/108190 PCT/US2006/013756
the advertiser (e.g., advertiser computer 104) in real time. In response to
sending the
information, ad placement provider computer 102 receives computer-generated
instructions
324 from a remote computer (e.g., advertiser computer 104) to change the
online advertising
program and implements (516) the computer-generated change in the online
advertising
program in real time.
[0071] In some embodiments, advertiser computer 104 receives (518) information
associated with a first response, a second response, or both the first
response and the second
response, and sends (520) computer-generated instructions 324 to change the
online
advertising program for the advertiser in real time.
[0072] In some embodiments, the computer-generated change modifies an
advertisement placement bid for the advertiser. hi some embodiments, the
advertisement
placement bid relates to a particular advertisement or to a group of
advertisements. For
example, the bid for an ad or for a group of ads can be raised if an alert
signals that the
display position has fallen below a particular threshold (e.g., the third
position).
Alternatively, the bid price could be raised if an alert signals that the CTR
fell below a certain
threshold, which may improve the bid position and increase the CTR. As another
example,
the bid price could be lowered if an alert signals that the remaining daily
budget for an ad or
group of ads has fallen below a predetermined threshold before a particular
time of the day.
In another example, the bid price could be lowered if an alert signals that
the display position
is higher than a specified threshold (e.g., the advertiser may not want to pay
for placement at
the highest display position).
[0073] In some embodiments, the computer-generated change modifies an
advertising
spending budget for the advertiser. The advertising spending budget could be
the budget for
a particular ad, the budget for a group of ads, or the budget for a particular
time period (e.g., a
day, week, or month). For example, the daily budget could be raised for a day
in which the ad
placement provider has unusually high traffic, to prevent the ad program from
running out of
money early in the day.
[0074] In some embodiments, the computer-generated change modifies one or more
ads being displayed for a particular advertiser. In some embodiments, the
computer-generated
change stops displaying an ad for the advertiser. For example, ads with low
CTR or low
-20-
CA 02606372 2007-10-26
WO 2006/108190 PCT/US2006/013756
conversion rates could be replaced by completely new ads or by ads with higher
CTR or
higher conversion rates.
[0075] In some embodiments, a method at a computer associated with an
advertiser
involves receiving information associated with a first response, a second
response, or both the
first response and the second response. The first response is generated at a
remote computer
by applying a first continuous query to a plurality of data streams containing
information
about online advertisements in an online advertising prograin for the
advertiser. The second
response is generated at the remote computer by applying a second continuous
query to the
first response. The method also involves sending computer-generated
instructions to change
the online advertising program for the advertiser in real time.
[0076] In some embodiments, a system includes a computer associated with an
advertiser. The computer is configured to receive information associated with
a first and/or
second respohse. The first response is generated at a remote computer by
applying a first
continuous query to a plurality of data streams containing information about
online
advertisements in an online advertising prograin for the advertiser. The
second response is
generated at the remote computer by applying a second continuous query to the
first
response. The computer is also configured to send computer-generated
instructions to change
the online advertising prograin for the advertiser in real time.
[0077] In some embodiments, a machine readable medium has stored thereon data
representing sequences of instructions, which when executed by a computer
associated with
an advertiser, cause the computer to receive information associated with a
first and/or second
response. The first response is generated at a remote computer by applying a
first continuous
query to a plurality of data streams containing information about online
advertisements in an
online advertising program for the advertiser. The second response is
generated at the remote
computer by applying a second continuous query to the first response. The
machine readable
medium also has stored thereon data representing sequences, of instructions,
which when
executed by the computer associated with the advertiser, cause the computer to
send
computer-generated instructions to change the online advertising program for
the advertiser.
[0078] In some embodiments, a system iuicludes a computer associated with an
advertiser. The computer includes means for receiving information associated
with a first
and/or second response. The first response is generated at a remote computer
by applying a
-21 -
CA 02606372 2007-10-26
WO 2006/108190 PCT/US2006/013756
first continuous query to a plurality of data streams containing information
about online
advertisements in an online advertising program for the advertiser. The second
response is
generated at the remote computer by applying a second continuous query to the
first
response. The computer also includes means for sending computer-generated
instructions to
change the online advertising program for the advertiser.
[0079] The foregoing description, for purpose of explanation, has been
described with
reference to specific embodiments. However, the illustrative discussions above
are not
intended to be exhaustive or to limit the invention to the precise forms
disclosed. Many
modifications and variations are possible in view of the above teachings. The
embodiments
were chosen and described in order to best explain the principles of the
invention and its
practical applications, to thereby enable others skilled in the art to best
utilize the invention
and various embodiments with various modifications as are suited to the
particular use
contemplated.
-22-
