Note: Descriptions are shown in the official language in which they were submitted.
CA 02973596 2017-07-11
WO 2016/118944
PCT/US2016/014650
SYSTEMS AND METHODS FOR MANAGEMENT OF AUTOMATED
DYNAMIC MESSAGING
BACKGROUND
[001] The present invention relates to systems and methods for the
generation, management of a dynamic messaging campaign. Such systems and
methods provide marketers and sales people more efficient tools for client
management and outreach. In turn, such system and methods enable more
productive
sales activity, increased profits, and more efficient allocation of sales
resources.
[002] Currently sales departments operate passively and actively. Passive
sales activity includes providing a general offer for sale of products and/or
services to
the public and waiting for customers to make the initial contact. Active sales
techniques, in contrast, involve the seller reaching out to consumers
directly. The
benefit of active sales activity is that customers can be targeted more
effectively, and
purchasing decisions may be more effectively influenced. Active sales
techniques
may include unsolicited "cold calls", or may include following up with "leads"
who
have responded to some advertisement, or who has been purchased from a
marketing
firm. While cold calling has its place, continuing a dialog with an
established lead is
by far the most targeted and effective means of sales activity.
[003] Active sale techniques have been around for as long as commerce has
been occurring. Sellers traditionally hawked their wares via in-person
solicitation or
fliers. Indeed, to this day, advertisements are routinely sent via postal mail
to
consumers. When available these mailed advertisements include degrees of
customization, such as inclusion of the receivers name printed on the
advertisement.
[004] With the advancement of technology, so too have active sales
techniques evolved. With the widespread use of telephones telemarketing became
a
staple of active sales techniques. While this initially took the form of sales
people
"cold calling" prospective customers, "robocalls" have become more popular
recently
due to the ability to reach much wider audiences with very little additional
resource
expenditure.
[005] As the intern& has become a more prominent feature of commerce, on-
line ads and email campaigns have joined the arsenal of sales departments as
ways to
CA 02973596 2017-07-11
WO 2016/118944
PCT/US2016/014650
engage a potential consumer. Email marketing in particular has become a very
effective and frequently utilized means of reaching customers. For large
customer
populations, these emails are typically minimally tailored advertisements. For
smaller
customer groups, individual emails may still be crafted by sales associates;
however
this activity (while more effective) is often very time consuming.
Additionally, a
sales person can usually only engage in a limited number of these sales
correspondences without the use of contact management software.
[006] It is therefore apparent that an urgent need exists for a dynamic
messaging system that provides the benefit of an individualized email sales
correspondence with the advantages of machine automation. Such dynamic
messaging would enable more effective sales activity and marketing campaigns.
SUMMARY
[007] To achieve the foregoing and in accordance with the present
invention,
systems and methods for management of automated dynamic messages are
providing.
Such systems and methods enable marketers and salespeople to more efficiently
follow up with leads via email (or other textual) exchanges. Using artificial
intelligence, the user is required to provide relatively minimal manual
intervention
until all objectives of the message exchange have been met.
[008] In some embodiments, a data store is populated with one or more
knowledge sets and one or more lead datasets in response to a user's input.
The lead
data includes information collected about the target consumers (or leads),
whereas the
knowledge sets provide the contextual knowledge required for the artificial
intelligence to perform categorization of any incoming responses.
[009] A campaign builder may be provided to the user for generating and
initiating campaigns. A campaign is a series of messages designed to satisfy
one or
more objectives. The information gathered through these message exchanges are
referred to as insights. The campaign builder allows the creation of a
campaign by
allowing the composition of a series of message templates with variable
fields. The
variable fields correspond to classes of data from the knowledge sets and/or
the lead
data. These templates may be generated from scratch, or may be imported from
existing campaigns (or a template library).
2
CA 02973596 2017-07-11
WO 2016/118944
PCT/US2016/014650
[0010] Once the campaign has been initiated, the system categorizes the
responses using algorithms. These categorizations have corresponding
confidence
levels. If the confidence level is too low, manual user intervention may be
required in
order to determine which subsequent action the system should perform. This may
be
done via an action interface.
[0011] Additionally, in some embodiments, the user may have access to an
insight interface (for reviewing and updating insights that have been
generated), an AT
training tool (for tuning the AT algorithms), a knowledge set interface (for
creating or
updating knowledge sets), and a statistics interface(for providing the user
information
regarding the campaign status).
[0012] Note that the various features of the present invention described
above
may be practiced alone or in combination. These and other features of the
present
invention will be described in more detail below in the detailed description
of the
invention and in conjunction with the following figures.
BRIEF DESCRIPTION OF THE DRAWINGS
[0013] In order that the present invention may be more clearly
ascertained,
some embodiments will now be described, by way of example, with reference to
the
accompanying drawings, in which:
[0014] Figure 1 is an example logical diagram of a system for generation
and
implementation of messaging campaigns, in accordance with some embodiment;
[0015] Figure 2 is an example logical diagram of a dynamic messaging
server,
in accordance with some embodiment;
[0016] Figure 3 is an example logical diagram of a user interface within
the
dynamic messaging server, in accordance with some embodiment;
[0017] Figure 4 is an example logical diagram of a message generator
within
the dynamic messaging server, in accordance with some embodiment;
[0018] Figure 5 is an example logical diagram of a message response
system
within the dynamic messaging server, in accordance with some embodiment;
[0019] Figure 6 is an example flow diagram for a dynamic message
campaign,
in accordance with some embodiment;
3
CA 02973596 2017-07-11
WO 2016/118944
PCT/US2016/014650
[0020] Figure 7 is an example flow diagram for the process of on-
boarding a
user, in accordance with some embodiment;
[0021] Figure 8 is an example flow diagram for the process of building a
campaign, in accordance with some embodiment;
[0022] Figure 9 is an example flow diagram for the process of generating
message templates, in accordance with some embodiment;
[0023] Figure 10 is an example flow diagram for the process of
implementing
the campaign, in accordance with some embodiment;
[0024] Figure 11 is an example flow diagram for the process of preparing
and
sending the outgoing message, in accordance with some embodiment;
[0025] Figure 12 is an example flow diagram for the process of
processing
received responses, in accordance with some embodiment;
[0026] Figures 13-16 are example screenshots of an interface for
building a
campaign, in accordance with some embodiment;
[0027] Figures 17-24 are example screenshots of AT training interfaces
of an
administrative dashboard, in accordance with some embodiment;
[0028] Figures 25-29 are example screenshots of context management
interfaces of an administrative dashboard, in accordance with some embodiment;
[0029] Figures 30-33 are example screenshots of insight management
interfaces of an administrative dashboard, in accordance with some embodiment;
[0030] Figures 34-37 are example screenshots of knowledge set management
interfaces of an administrative dashboard, in accordance with some embodiment;
[0031] Figures 38-41 are example screenshots of action management
interfaces of an administrative dashboard, in accordance with some embodiment;
[0032] Figures 42-45 are example screenshots of statistics interfaces of
an
administrative dashboard, in accordance with some embodiment; and
[0033] Figures 46A and 46B are example illustrations of a computer
system
capable of embodying the current invention.
4
CA 02973596 2017-07-11
WO 2016/118944
PCT/US2016/014650
DETAILED DESCRIPTION
[0034] The present invention will now be described in detail with
reference to
several embodiments thereof as illustrated in the accompanying drawings. In
the
following description, numerous specific details are set forth in order to
provide a
thorough understanding of embodiments of the present invention. It will be
apparent,
however, to one skilled in the art, that embodiments may be practiced without
some or
all of these specific details. In other instances, well known process steps
and/or
structures have not been described in detail in order to not unnecessarily
obscure the
present invention. The features and advantages of embodiments may be better
understood with reference to the drawings and discussions that follow.
[0035] Aspects, features and advantages of exemplary embodiments of the
present invention will become better understood with regard to the following
description in connection with the accompanying drawing(s). It should be
apparent,
to those skilled in the art, that the described embodiments of the present
invention
provided herein are illustrative only and not limiting, having been presented
by way
of example only. All features disclosed in this description may be replaced by
alternative features serving the same or similar purpose, unless expressly
stated
otherwise. Therefore, numerous other embodiments of the modifications thereof
are
contemplated as falling within the scope of the present invention as defined
herein
and equivalents thereto. Hence, use of absolute and/or sequential terms, such
as, for
example, "will," "will not," "shall," "shall not," "must," "must not,"
"first,"
"initially," "next," "subsequently," "before," "after," "lastly," and
"finally," are not
meant to limit the scope of the present invention as the embodiments disclosed
herein
are merely exemplary.
[0036] Note that the term "user" is utilized to describe the user of a
device
who is generating and managing a messaging campaign. It is likewise understood
that
the terms "participant", "sales associate", and "salesperson" are likewise
often utilized
interchangeably with the term "user".
[0037] Likewise, the term "recipient" is utilized to refer to the
person(s)
receiving the generated messages. Other terms such as "consumer" and "lead"
may
be interchangeably used.
CA 02973596 2017-07-11
WO 2016/118944
PCT/US2016/014650
[0038] Lastly, the following discussions and accompanying examples are
directed toward the utilization of the messaging system in the context of
sales
activities, primarily with developing sales leads. Sales activities are a
natural use case
for the presently disclosed systems and methods; however, the messaging
systems
described herein are not limited to sales activities. Indeed, the presently
disclosed
systems and methods can be employed in a variety of contexts and situations.
For
example, the disclosed messaging systems may be useful in customer support
settings,
educational campaigns, fundraising, or any other situation where a large
number of
messages within a defined context are needed.
[0039] The following disclosure includes a series of subsections. These
subsections are not intended to limit the scope of the disclosure in any way,
and are
merely for the sake of clarity and ease of reading. As such, disclosure in one
section
may be equally applied to processes or descriptions of another section if and
where
applicable.
I. DEFINITIONS
[0040] The following systems and methods for dynamic messaging a
campaign relies upon an interplay of user interaction, and sophisticates
artificial
intelligence (AI) processing of received messages. The goal of the message
campaign
it to enable a logical dialog exchange with a recipient, where the recipient
is not
necessarily aware that they are communicating with an automated machine as
opposed to a human user. This may be most efficiently performed via a written
dialog, such as email, text messaging, chat, etc. However, it is entirely
possible that
given advancement in audio and video processing, it may be entirely possible
to have
the dialog include audio or video components as well.
[0041] In order to effectuate such an exchange, an AT system is employed
within an AT platform within the messaging system to process the responses and
generate conclusions regarding the exchange. These conclusions include
calculating
the context of a document, insights, sentiment and confidence for the
conclusions.
Given that these terms are not readily familiar outside of the field of
natural language
processing, a series of definitions are provided in order to clarify the
terminology:
[0042] accuracy - the calculated probability that a classification
determined
by the AT is correct.
6
CA 02973596 2017-07-11
WO 2016/118944
PCT/US2016/014650
[0043] (Al) algorithm - a method employed to calculate the weight of a
document in a particular category.
[0044] aspect - a specific AT algorithm. Example: NaiveBayes, Sentiment.
[0045] attempt - a single message in a series for a campaign.
[0046] Al Trainer - term for the tool used to classify a document that
the
aspects were not confident scoring.
[0047] campaign - a set of possible messaging designed to be sent out to
a
lead over the course of a conversation depending on the receipt and
classification of
responses (or lack thereof).
[0048] categorization - the process in which ideas and objects are
recognized,
differentiated, and understood, generally into categories.
[0049] category - possible answers to the insight they belong to.
Example:
Insight: "Continue messaging?" has categories: "Stop" and "Continue".
[0050] classification - another word for categorization.
[0051] confidence - a calculated probability that the categorization is
correct.
[0052] context - a collection of documents that have some commonality.
Example: "all documents collected from asking 'What is a good phone
number?'.",
"messages sent from customers in a chat with Bill in customer service".
[0053] document - a set of words in a specific order used to convey a
meaning.
[0054] Hardrule - an AT algorithm that dictates a category based off a
single
token. These tokens are found to occur overwhelmingly within those specific
categories.
[0055] hardrule term - a token that is used by the Hardrule aspect.
[0056] insight - a collection of categories used to answer some question
about
a document. Example: "What does this person mean?", "How does this person
feel?",
"Should we keep emailing this person?"
[0057] knowledge set - a set of tokens with their associated category
weights
used by an aspect during classification.
7
CA 02973596 2017-07-11
WO 2016/118944
PCT/US2016/014650
[0058] lead - a person who is placed into the system at a certain time
under a
certain campaign.
[0059] lead (event) history - the notable information for a lead coming
into
the system, messages sent to that lead, responses received and alerts sent
out, in the
chronological order of their occurrences.
[0060] ngram - denotes the number of words used to make a token.
Example:
token "yes it is" is a tri-gram or an ngram of 3.
[0061] normalization - removing characters / tokens in order to reduce
the
complexity of the document without changing the accuracy of classifications.
[0062] question - an inquiry included in a message designed to limit the
response to a subset of the target language.
[0063] response - the document received after sending a message to a
lead.
[0064] (response) actions - tasks that the system can carry out for a
given
lead based on the classification of the response.
[0065] Sentiment - an AT algorithm that is used to gauge how strongly a
category expresses itself in a document.
[0066] series - a subset of a campaign designed to be sent out until a
response
is received for that subset of messages. Based on the classification of the
response, the
system may continue to another series of messaging in that same campaign.
[0067] score - a set of classifications made by the different aspects
for
different insights.
[0068] The (AI) Platform - the system that allows interaction with,
setup,
score, and modify the AT algorithms as need be. This also includes the code,
databases
and servers used for this specific purpose.
[0069] token - one or more words used as a single unit to correlate to a
category through assigning a weight.
[0070] training set - a set of classified documents used to calculate
knowledge sets.
[0071] weight - the numeric value assigned to a token or document for a
category based on the training for a particular algorithm.
8
CA 02973596 2017-07-11
WO 2016/118944
PCT/US2016/014650
[0072] word - a combination of characters used to denote meaning in a
language.
[0073] variabilization - grouping a word or set of words into a single
token.
Example: "Alex", "Sarah", and "Jill" can all be variabilized into the token
" name ".
II. DYNAMIC MESSAGING SYSTEMS
[0074] To facilitate the discussion, Figure 1 is an example logical
diagram of
a system for generating and implementing messaging campaigns, shown generally
at
100. In this example block diagram, a number of users 102a-n are illustrated
engaging a dynamic messaging system 108 via a network 106. Note that messaging
campaigns may be uniquely customized by each user 102a-n in some embodiments.
In alternate embodiments, users may be part of collaborative sales departments
(or
other collaborative group) and may all have common access to the messaging
campaigns. The users 102a-n may access the network from any number of suitable
devices, such as laptop and desktop computers, work stations, mobile devices,
media
centers, etc.
[0075] The network 106 most typically includes the internet, but may
also
include other networks such as a corporate WAN, cellular network, corporate
local
area network, or combination thereof, for example. The messaging server 108
may
distribute the generated messages to the various message delivery platforms
112 for
delivery to the individual recipients. The message delivery platforms 112 may
include any suitable messaging platform. Much of the present disclosure will
focus
on email messaging, and in such embodiments the message delivery platforms 112
may include email servers (gmail, yahoo, hotmail, etc.). However, it should be
realized that the presently disclosed systems for messaging are not
necessarily limited
to email messaging. Indeed, any messaging type is possible under some
embodiments
of the present messaging system. Thus, the message delivery platforms 112
could
easily include a social network interface, instant messaging system, text
messaging
(SMS) platforms, or even audio telecommunications systems. While audio is
possible
with the given messaging system, it is often desirable for the recipient to
have a
seamless experience where the automated messages are virtually
indistinguishable
from messages authored by a sales associate. Due to inherent difficulties in
9
CA 02973596 2017-07-11
WO 2016/118944
PCT/US2016/014650
generating realistically human sounding automated audio (much less imitating a
specific sales associate), much of the present disclosure will focus on the
generation
of written textual messages.
[0076] One or more data sources 110 may be available to the messaging
server 108 in order to provide user specific information, message template
data,
knowledge sets, insights, and lead information. These information types will
be
described in greater detail below.
[0077] Moving on, Figure 2 provides a more detailed view of the dynamic
messaging server 108, in accordance with some embodiment. The server is
comprised of three main logical subsystems: a user interface 210, a message
generator
220, and a message response system 230. The user interface 210 may be utilized
to
access the message generator 220 and the message response system 230 in order
to set
up messaging campaigns, and manage those campaigns throughout their life
cycle. At
a minimum, the user interface 210 includes APIs to allow a users device to
access
these subsystems. Alternatively, the user interface 210 may include web
accessible
messaging creation and management tools, as will be explored below in some of
the
accompanying example screenshots.
[0078] Figure 3 provides a more detailed illustration of the user
interface 210.
The user interface 210 includes a series of modules in order to enable the
previously
mentioned functions to be carried out in the message generator 220 and the
message
response system 230. These modules include a campaign builder 310, a campaign
manager 320 an AT manager 330, an insight manager 340, and a knowledge base
manager 350.
[0079] The campaign builder 310 allows the user to define a campaign,
and
input message templates for each series within the campaign. A knowledge set
and
lead data may be associated with the campaign in order to allow the system to
automatically effectuate the campaign once built. Lead data includes all the
information collected on the intended recipients, and the knowledge set
includes a
database from which the AT can infer context and perform classifications on
the
responses received from the recipients.
[0080] The campaign manager 320 provides activity information, status,
and
logs of the campaign once it has been implemented. This allows the user 102a
to
CA 02973596 2017-07-11
WO 2016/118944
PCT/US2016/014650
keep track of the campaigns progress, success and allows the user to manually
intercede if required. The campaign may likewise be edited or otherwise
altered using
the campaign manager 320.
[0081] The AT manager 330 allows the user to access the training of the
artificial intelligence which analyzes responses received from a recipient.
One
purpose of the given systems and methods is to allow very high throughput of
message exchanges with the recipient with relatively minimal user input. In
order to
perform this correctly, natural language processing by the AT is required, and
the AT
must be correctly trained in order to make the appropriate inferences and
classifications of the response message. The user may leverage the AT manager
330
to review documents the AT has processed and has made classifications for.
[0082] The insight manager 340 allows the user to manage insights. As
previously discussed, insights are a collection of categories used to answer
some
question about a document. For example, a question for the document could
include
"is the lead looking to purchase a car in the next month?" Answering this
question
can have direct and significant importance to a car dealership. Certain
categories that
the AT system generates may be relevant toward the determination of this
question.
These categories are the 'insight' to the question, and may be edited or newly
created
via the insight manager 340.
[0083] In a similar manner, the knowledge base manager 350 enables the
management of knowledge sets by the user. As discussed, a knowledge set is set
of
tokens with their associated category weights used by an aspect (AI algorithm)
during
classification. For example, a category may include "continue contact?", and
associated knowledge set tokens could include statements such as "stop", "do
no
contact", "please respond" and the like. The knowledge base manager 350
enables
the user to build new knowledge sets, or edit exiting ones.
[0084] Moving on to Figure 4, an example logical diagram of the message
generator 220 is provided. The message generator 220 utilizes context
knowledge
440 and lead data 450 in order to generate the initial message. The message
generator
220 includes a rule builder 410 which allows the user to define rules for the
messages.
A rule creation interface which allows users to define a variable to check in
a situation
and then alter the data in a specific way. For example, when receiving the
scores
11
CA 02973596 2017-07-11
WO 2016/118944
PCT/US2016/014650
from the AT, if the insight is Interpretation and the chosen category is
'good', then
have the Continue Messaging insight return 'continue'.
[0085] The rule builder 410 may provide possible phrases for the message
based upon available lead data. The message builder 420 incorporates those
possible
phrases into a message template, where variables are designated, in order to
generate
the outgoing message. This is provided to the message sender 430 which formats
the
outgoing message and provides it to the messaging platforms for delivery to
the
appropriate recipient.
[0086] Figure 5 is an example logical diagram of the message response
system
230. In this example system, the contextual knowledge base 440 is utilized in
combination with response data 550 received from the lead. The message
receiver
520 receives the response data 550 and provides it to the AT interface and
objective
modeler 530 for feedback. The AT interface 510 allows the AT platform to
process the
response for context, insights, sentiments and associated confidence scores.
Based on
the classifications generated by the AT lead objectives may be updated by the
objective modeler 530.
[0087] The message receiver 520 can then determine whether there are
further
objectives that are still pending, or whether there has been a request to
discontinue
messaging the lead. If there has been a termination request, or if all
objectives have
been fulfilled, the message receiver may deactivate the campaign for the given
lead.
If not, a scheduler 540 may be employed to assist in scheduling the next step
of the
campaign.
III. METHODS OF MESSAGING
[0088] Now that the systems for dynamic messaging campaigns have been
broadly described, attention will be turned to processes employed to generate
and
present the customized media. In Figure 6 an example flow diagram for a
dynamic
message campaign is provided, shown generally at 600. The process can be
broadly
broken down into three portions: the on-boarding of a user (at 610), campaign
generation (at 620) and campaign implementation (at 630). The following
figures and
associated disclosure will delve deeper into the specifics of these given
process steps.
[0089] Figure 7, for example, provides a more detailed look into the on-
boarding process, shown generally at 610. Initially a user is provided (or
generates) a
12
CA 02973596 2017-07-11
WO 2016/118944
PCT/US2016/014650
set of authentication credentials (at 710). This enables subsequent
authentication of
the user by any known methods of authentication. This may include username and
password combinations, biometric identification, device credentials, etc.
[0090] Next, the lead data associated with the user is imported, or
otherwise
aggregated, to provide the system with a lead database for message generation
(at
720). Likewise, context knowledge data may be populated as it pertains to the
user
(at 730). Often there are general knowledge data sets that can be
automatically
associated with a new user; however, it is sometimes desirable to have
knowledge sets
that are unique to the user's campaign that wouldn't be commonly applied.
These
more specialized knowledge sets may be imported or added by the user directly.
[0091] Lastly, the user is able to configure their preferences and
settings (at
740). This may be as simple as selecting dashboard layouts, to configuring
confidence thresholds required before alerting the user for manual
intervention.
[0092] Moving on, Figure 8 is the example flow diagram for the process
of
building a campaign, shown generally at 620. The user initiates the new
campaign by
first describing the campaign (at 810). Campaign description includes
providing a
campaign name, description, industry selection, and service type. The industry
selection and service type may be utilized to ensure the proper knowledge sets
are
relied upon for the analysis of responses.
[0093] After the campaign is described, the message templates in the
campaign are generated (at 820). If the series is populated (at 830), then the
campaign is reviewed and submitted (at 840). Otherwise, the next message in
the
template is generated (at 820). Figure 9 provides greater details of an
example of this
sub-process for generating message templates. Initially the user is queried if
an
existing campaign can be leveraged for templates, or whether a new template is
desired (at 910).
[0094] If an existing campaign is used, the new message templates are
generated by populating the templates with existing templates (at 920). The
user is
then afforded the opportunity to modify the message templates to better
reflect the
new campaign (at 930). Since the objectives of many campaigns may be similar,
the
user will tend to generate a library of campaign that may be reused, with or
without
13
CA 02973596 2017-07-11
WO 2016/118944
PCT/US2016/014650
modification, in some situations. Reusing campaigns has time saving
advantages,
when it is possible.
[0095] However, if there is no suitable campaign to be leveraged, the
user
may instead opt to write the message templates from scratch (at 940). When a
message template is generated, the bulk of the message is written by the user,
and
variables are imported for regions of the message that will vary based upon
the lead
data. Successful messages are designed to elicit responses that are readily
classified.
Higher classification accuracy enables the system to operate longer without
user
interference, which increases campaign efficiency and user workload.
[0096] Once the campaign has been built out it is ready for
implementation.
Figure 10 is an example flow diagram for the process of implementing the
campaign,
shown generally at 630. Here the lead data is uploaded (at 1010). Lead data
may
include any number of data types, but commonly includes lead names, contact
information, date of contact, item the lead was interested in, etc. Other data
can
include open comments that leads supplied to the lead provider, any items the
lead
may have to trade in, and the date the lead came into the lead provider's
system.
Often lead data is specific to the industry, and individual users may have
unique data
that may be employed.
[0097] An appropriate delay period is allowed to elapse (at 1020) before
the
message is prepared and sent out (at 1030). The waiting period is important so
that
the lead does not feel overly pressured, nor the user appears overly eager.
Additionally, this delay more accurately mimics a human correspondence (rather
than
an instantaneous automated message).
[0098] Figure 11 provides a more detailed example of the message
preparation
and output. In this example flow diagram, the message within the series is
selected
based upon which objectives are outstanding (at 1110). Typically, the messages
will
be presented in a set order; however, if the objective for a particular lead
has already
been met for a given series, then another message may be more appropriate.
Likewise, if the recipient didn't respond as expected, or not at all, it may
be desirous
to have alternate message templates in order to address the lead most
effectively.
[0099] After the message template is selected from the series, the lead
data is
parsed through, and matches for the variable fields in the message templates
are
14
CA 02973596 2017-07-11
WO 2016/118944
PCT/US2016/014650
populated (at 1120). The populated message is output to the appropriate
messaging
platform (at 1130), which as previously discussed typically includes an email
service,
but may also include SMS services, instant messages, social networks, or the
like.
[00100] Returning to Figure 10, after the message has been output, the
process
waits for a response (at 1040). If a response is not received (at 1050) the
process
determines if the wait has been timed out (at 1060). Allowing a lead to
languish too
long may result in missed opportunities; however, pestering the lead to
frequently
may have an adverse impact on the relationship. As such, this timeout period
may be
user defined. Often the timeout period varies from a few days to a week or
more. If
there has not been a timeout event, then the system continues to wait for a
response
(at 1050). However, once sufficient time has passed without a response, it may
be
desirous to return to the delay period (at 1020) and send a follow-up message
(at
1030). Often there will be available reminder templates designed for just such
a
circumstance.
[00101] However, if a response is received, the process may continue with
the
response being processed (at 1070). This processing of the response is
described in
further detail in relation to Figure 12. In this sub-process, the response is
initially
received (at 1210) and the document may be cleaned (at 1220). Document
cleaning
may include a normalization process where characters and tokens are removed in
order to reduce the complexity of the document without changing the intended
classification. Document cleaning has a number of steps to it. Upon initial
receipt of
the response, often a number of elements need to be removed, including the
original
message, HTML encoding for HTML style responses, enforce UTF-8 encoding so as
to get diacritics and other notation from other languages, and signatures so
as to not
confuse the Al. Only after all this removal process does the normalization
process
occur, which includes variabilization, removing stopwords, manual
replacements,
spelling corrections, and removal of punctuation, numbers, and any other
tokens that
are deemed unnecessary.
[00102] The normalized document is then provided to the Al platform for
classification using the knowledge sets (at 1230). As previously mentioned,
there are
a number of known algorithms that may be employed in order to categorize a
given
document, including Hardrule, NaiveBayes,Sentiment, neural nets, k-nearest
neighbor, other vector based algorithms, etc. to name a few. In some
embodiments,
CA 02973596 2017-07-11
WO 2016/118944
PCT/US2016/014650
multiple algorithms may be employed simultaneously, and then a combination of
the
algorithm results are used to make the classification. The algorithm(s)
selected may
be those with the highest confidence level in their classification, or those
who agree
most closely to one another. Responses to informational messages may be
classified
differently than responses to questions. Classification depends on the type of
responses received by each outgoing messages. The classifications may be
combined
with business logic within the objective model rule engine in order to
generate an
action set (at 1240). Campaign objectives, as they are updated, may be used to
redefine the actions collected and scheduled. For example, 'skip-to-followup'
action
may be replaced with an informational message introducing the sales rep before
proceeding to 'series 3' objectives. Additionally, 'Do Not Email' or 'Stop
Messaging' classifications should deactivate a lead and remove scheduling at
any
time during a lead's life-cycle.
[00103] After the actions are set, a determination is made whether there
is an
action conflict (at 1250). Manual review may be needed when such a conflict
exists
(at 1270). Otherwise, the actions may be executed by the system (at 1260).
[00104] Returning to Figure 10, after the response has been processed, a
determination is made whether to deactivate the lead (at 1075). Such a
deactivation
may be determined as needed when the lead requests it. If so, then the lead is
deactivated (at 1090). If not, the process continues by determining if the
campaign
for the given lead is complete (at 1080). The campaign may be completed when
all
objectives for the lead have been met, or when there are no longer messages in
the
series that are applicable to the given lead. Once the campaign is completed,
the lead
may likewise be deactivated (at 1090).
[00105] However, if the campaign is not yet complete, the process may
return
to the delay period (at 1020) before preparing and sending out the next
message in the
series (at 1030). The process iterates in this manner until the lead requests
deactivation, or until all objectives are met.
IV. EXAMPLES
[00106] The following examples include example screenshots of interfaces
for
building and managing messaging campaigns. It should be noted that while
considerable numbers of example screenshots are provided for this sales driven
16
CA 02973596 2017-07-11
WO 2016/118944
PCT/US2016/014650
example, the disclosed systems and methods for dynamic messaging are
applicable
for many purposes beyond sales and marketing. For example, educators could
benefit
greatly from such messaging capabilities. Furthermore, customer service, help-
lines,
and information services could benefit greatly from the disclosed systems and
methods of messaging campaigns.
[00107] Moreover, the following examples also focus heavily upon email
messaging. While email messaging may be particularly effective as a
communication
tool, it is entirely possible that the messages being generated may include
audio, video
and animations, text messages, Instant messages, forum postings, messaging
within a
social media platform, or any combination thereof As such, it is of paramount
importance that the following examples provide clarity of the messaging
campaign
systems and methods without unduly limiting their scope.
[00108] Figures 13-16 are example screenshots of an interface for
building a
campaign, in accordance with some embodiment. In Figure 13, shown generally at
1300, the user is presented a listing of existing campaigns, and the ability
to create a
new campaign. If selected, the user is redirected to the screen shown at
Figure 14.
Here, shown generally at 1400, the user is asked to name the campaign, have a
dashboard identifier for the campaign, and input a description for the
campaign. As
previously noted, the presently disclosed systems and methods allow a user to
have a
very large number of campaigns going on simultaneously. The description
ensures
that the user is able to keep the various campaigns organized and clear.
[00109] In addition to providing designations and descriptions for the
campaign, the user is likewise allowed to select the industry and service type
for the
given campaign. These selections enable the proper knowledge sets to be
associated
with the campaign so that the Al can more accurately classify any responses.
They
are also used to tie into Salesforce and into billing of the messaging service
so the
customer can be accurately charged for the campaigns they are running.
[00110] At Figure 15, the next step in campaign generation is message
template
formation. This can be done from scratch, or an existing campaign may be
leveraged
for the generation of these messages. This screenshot 1500 provides the
options
presented to the user.
17
CA 02973596 2017-07-11
WO 2016/118944
PCT/US2016/014650
[00111] Next, the message series is provided to the user, as shown at
Figure 16
at 1600. Here the series may be selected from a set of pull down options. The
series
typically includes a short question in order to help direct the response. Each
series is
typically directed toward meeting an objective for the lead. An example of
series and
objectives are provided in the below example table:
Series Objective
1 Verify Email Address
2 Obtain Phone Number
2 Introduce Sales Representative
3 Verify Rep Follow-Up
[00112] Delay for the series may be input, as well as message subject,
and
message body. Where appropriate, the user is able to incorporate in variables
into the
message. These variables may be defined by the user, and may be auto-populated
by
the system using the lead data. To build a message, possible phrases are
gathered for
each template component in a template iteration. A single phrase can be chosen
randomly from possible phrases for each template component. Chosen phrases are
then imported to obtain an outbound message. Logic can be universal or data
specific
as desired for individual message components.
[00113] Each series may include a number of message templates
corresponding
to multiple attempts to meet the objective. Thus, for example, it the lead
fails to
respond to the initial message, a different subsequent message may be sent
that seeks
to answer the objective.
[00114] Variable replacement can occur on a per phrase basis, or after a
message is composed. Post message-building validation may be integrated into a
message-building class. All rules interaction may be maintained with the
messaging
rule engine.
[00115] Now that the campaign building has been explored in considerable
detail, attention will be turned to the administrative tools made available to
the user.
This enables AT management, knowledge set management, insight management, and
review of campaign statistics. Figure 17 provides the top level dashboard for
the AT
management, shown generally at 1700. At this top level the user is enabled to
select
training of the AT or review of the AT document classifications. All the tools
are
located under tabs, located on the left-hand side of the screen.
18
CA 02973596 2017-07-11
WO 2016/118944
PCT/US2016/014650
[00116] Figure 18 illustrates the AT training dashboard, shown generally
at
1800. The training tool allows AT developers, who may be users or may include
messaging server administrators, to train the AT algorithms. This tool is at
the center
of how the AT is taught new things. The top left of the frame contains the
tool title,
and below that is a Context dropdown filter, which allows filtering of
documents to
train by their context. Below that is a previous message button, which
redirects back
to the previous document that was trained. On the right-side, there is the
next
message button, moves forward to the next oldest document that could use some
training. The determination of which document needs training is determined by
how
confident the classification for each insight performed by the AT is.
[00117] Figure 19, shown generally at 1900, illustrates the use of a
search bar,
which allows searching for documents with a given document id, or for a length
of
text which could exist inside a document body. Figure 20 illustrates a scores
button,
which displays the current AT classification for the document, shown generally
at
2000. This score is broken up by insight (shaded lines), and the categories
selected
for those insights. The right side of the categories container has a number
which is
the confidence value for the category selected. The confidence value is how
confident
the AT is when choosing that category for that insight. The confidence ranges
between 0 and 100 (100 being most confident) in this instant embodiment.
[00118] The center of the frame contains the unique document id, the
context
assigned to the document, and the body of the document. On the right-side of
the
center frame there is a link called "Display Normalized Document". This will
show
what the AT does to the document in terms of cleaning, parsing, and other pre-
processing techniques before it actually begins scoring.
[00119] By highlighting a portion of the document body a category
selection
box will be displayed, as seen at 2100 of Figure 21. The category selection
box
enables categorization of highlighted text manually. After selecting
categories, the
phrases and categories selected appear below the document, as seen at 2200 of
Figure
22. On the right of these new containers is a knowledge set dropdown. In order
to
train the AT the AT developer selects a knowledge set that is stored in the
new training.
This links a knowledge set not previously utilized by the AT for the
generation of
context to the document.
19
CA 02973596 2017-07-11
WO 2016/118944
PCT/US2016/014650
[00120] Moving on, Figure 23 provides a dashboard for the review of AT
training, shown generally at 2300. The review tool is used for reviewing
documents
that the AT has been confident on one or more of the classification
determinations.
Both context and insight may be filtered for when reviewing documents by
selecting
the appropriate dropdown located in the top left of the frame. For each
confident
category selected for a document, the AT developer may select whether the AI's
classification is approved by pressing either the "thumbs up" (for agreeing
with the
AI's classification) or "thumbs down" (for disagreeing with the AI's
classification)
button.
[00121] When a category is marked as either correct or incorrect, it will
highlight that category row, as shown at 2400 of Figure 24. When all confident
categories in a document are marked, the document will be removed from the
display
and the next document may be loaded. Following that there is a submit button,
which
allows the AT developer to submit any training that has been selected.
[00122] Moving on to Figure 25, the API Management tab contains tools
which
deal with the creation and management of objects, shown generally at 2500. The
contexts tool, shown at 2600 of Figure 26, helps in the management of
contexts. A
list of existing contexts may be displayed on initial load of the system to
new AT
developers/users. As with AT training, the contexts are searchable using the
search
bar located in the top right portion of the screen. By pressing the "+" button
in the top
right-hand corner or the frame, a create dialogue box is launched, as seen at
2700 of
Figure 27. This allows the creation of a new context.
[00123] Figure 28 provides the details of a context, shown generally at
2800.
The details display for any given context may be viewed by selecting any one
of the
contexts. The details display contains all the pertinent information about the
context
and an edit button (wrench icon) located in the top right of the details
display. When
the edit button is selected an edit dialogue box is launched, as seen at 2900
of Figure
29. The edit dialogue box enables editing of the information about a context.
[00124] Moving to Figure 30, an insight management screenshot is
provided,
shown generally at 3000. The insights tool helps in the management of
insights. As
with contexts, a list of existing insights may be displayed on initial load.
Likewise,
insights may be searched using the search bar located in the top right of the
frame.
CA 02973596 2017-07-11
WO 2016/118944
PCT/US2016/014650
[00125] An insight creation dialog box, as seen at 3100 of Figure 31, may
be
accessed by pressing the "+" button in the top right-hand corner or the frame.
Creating an insight will also require the creation of categories that go with
that
insight. By selecting any one of the insights, it will pull open a details
display for that
insight, as seen at 3200 of Figure 32. The details display contains all the
pertinent
information about the insight, and includes an edit button (wrench icon)
located in the
top right of the frame. In some embodiments, each insight has at least two
categories.
Clicking the edit button will bring up an edit dialogue, as seen at 3300 of
Figure 33,
which allows editing of the information for the insight.
[00126] Moving to Figure 34, an interface for knowledge set management is
provided, shown generally at 3400. As with contexts and insights, a list of
existing
knowledge sets will be displayed on initial load. Likewise, knowledge sets may
be
searched using the search bar located in the top right of the frame. A
knowledge set
creation dialog box may be launched, as seen at 3500 of Figure 35 by pressing
the "+"
button in the top right-hand corner or the frame. This will allow the creation
of a new
knowledge set. In some embodiments, a knowledge set requires that an insight
exists
such that the newly created knowledge set may be bound to that insight.
[00127] By selecting any one of the insights, it will pull open a details
display
for that knowledge set, as seen at 3600 of Figure 36. The details display
contains all
the pertinent information about the knowledge set and an edit button (wrench
icon)
located in the top right of the details display. Clicking the edit button
brings up an
edit dialogue, as seen at 3700 of Figure 37, which allows the editing of
information
about a knowledge set.
[00128] Moving to Figure 38, an actions management dashboard is
illustrated,
seen generally at 3800. One of the tools under the action dashboard is the
resolve
tool, which is seen expanded at 3900 at Figure 39. The resolve tool allows
manual
intervention into any pending responses that were considered 'not confident'
by the
AT classification. This confident/not confident determination is based not
only on the
confidence scores returned from the AT, but may also include additional
factors (for
example, the client might be new, the campaign might be new, might be for a
new
industry, etc.). Manual intervention for a non-confident determination allows
the
user, system administrator, AT developers, or other suitable individual, to
access the
document and input an action appropriate for the lead's response. Depending
upon
21
CA 02973596 2017-07-11
WO 2016/118944
PCT/US2016/014650
the embodiment, this manual intervention does not train the AT directly. The
system
keeps track of what actions were taken on each response, and this collected
data may
later be used for analytics for improvements to the AT.
[00129] Figure 40, shown generally at 4000, provides a lead history from
the
time the lead was input into the system up until the lead's campaign has been
completed. This history provides a listing of all the messages sent to the
given lead.
[00130] Moving to Figure 41, the actions taken for a given campaign may
be
reviewed, seen generally at 4100. The action review tool is laid out almost
identically
to the above action resolve tool. However, the action review tool does two
things; 1)
allows responses to be reviewed for actions that were considered confident and
did
not require manual review, and 2) review responses that were manually resolved
by
another user. This tool may be utilized to collect accuracy data on other
users by the
more senior users (such as AT developers), and allows users to check the
accuracy of
automatic actions being performed by the system.
[00131] Moving to Figure 42, the statistics dashboard is illustrated,
shown
generally at 4200. The statistics tab contains some analytics and reporting
tools. This
tools collect data from resolve actions, review actions, and review accuracy
tools.
The AT decisions tool reports on a few AT specific statistics, as seen at 4300
of Figure
43. In this specific example illustration, each graph is over a seven day
window. The
first shows how accurate the system is on a context by context basis. The next
shows
something similar based on insight. The final grouping of statistics shows how
many
responses need to be manually resolved by a user.
[00132] Lastly, Figures 44 and 45 show action statistics, shown generally
at
4400 and 4500, respectively. Each box includes statistics regarding actions
taken, and
include, for example, training by users, action accuracy, action resolution
summaries,
suggested action accuracy, and the like.
V. SYSTEM EMBODIMENTS
[00133] Figures 46A and 46B illustrate a Computer System 4600, which is
suitable for implementing embodiments of the present invention. Figure 46A
shows
one possible physical form of the Computer System 4600. Of course, the
Computer
System 4600 may have many physical forms ranging from a printed circuit board,
an
integrated circuit, or a small handheld device up to a huge super computer.
Computer
22
CA 02973596 2017-07-11
WO 2016/118944
PCT/US2016/014650
system 4600 may include a Monitor 4602, a Display 4604, a Housing 4606, a Disk
Drive 4608, a Keyboard 4610, and a Mouse 4612. Disk 4614 is a computer-
readable
medium used to transfer data to and from Computer System 4600.
[00134] Figure 46B is an example of a block diagram for Computer System
4600. Attached to System Bus 4620 are a wide variety of subsystems.
Processor(s)
4622 (also referred to as central processing units, or CPUs) are coupled to
storage
devices, including Memory 4624. Memory 4624 includes random access memory
(RAM) and read-only memory (ROM). As is well known in the art, ROM acts to
transfer data and instructions uni-directionally to the CPU and RAM is used
typically
to transfer data and instructions in a bi-directional manner. Both of these
types of
memories may include any suitable form of the computer-readable media
described
below. A Fixed Disk 4626 may also be coupled bi-directionally to the Processor
4622; it provides additional data storage capacity and may also include any of
the
computer-readable media described below. Fixed Disk 4626 may be used to store
programs, data, and the like and is typically a secondary storage medium (such
as a
hard disk) that is slower than primary storage. It will be appreciated that
the
information retained within Fixed Disk 4626 may, in appropriate cases, be
incorporated in standard fashion as virtual memory in Memory 4624. Removable
Disk 4614 may take the form of any of the computer-readable media described
below.
[00135] Processor 4622 is also coupled to a variety of input/output
devices,
such as Display 4604, Keyboard 4610, Mouse 4612 and Speakers 4630. In general,
an input/output device may be any of: video displays, track balls, mice,
keyboards,
microphones, touch-sensitive displays, transducer card readers, magnetic or
paper
tape readers, tablets, styluses, voice or handwriting recognizers, biometrics
readers,
motion sensors, brain wave readers, or other computers. Processor 4622
optionally
may be coupled to another computer or telecommunications network using Network
Interface 4640. With such a Network Interface 4640, it is contemplated that
the
Processor 4622 might receive information from the network, or might output
information to the network in the course of performing the above-described
dynamic
messaging. Furthermore, method embodiments of the present invention may
execute
solely upon Processor 4622 or may execute over a network such as the Internet
in
conjunction with a remote CPU that shares a portion of the processing.
23
CA 02973596 2017-07-11
WO 2016/118944
PCT/US2016/014650
[00136] In sum, the present invention provides a system and methods for
dynamic automated messaging driven by an artificial intelligence. The
advantages of
such a system include the ability to provide seemingly human driven email
interactions without the required manual input. Such systems may be
particularly
helpful in the context of sales and marketing, but may likewise be utilized
wherever
large distributions of email are being employed.
[00137] While this invention has been described in terms of several
embodiments, there are alterations, modifications, permutations, and
substitute
equivalents, which fall within the scope of this invention. Although sub-
section titles
have been provided to aid in the description of the invention, these titles
are merely
illustrative and are not intended to limit the scope of the present invention.
[00138] It should also be noted that there are many alternative ways of
implementing the methods and apparatuses of the present invention. It is
therefore
intended that the following appended claims be interpreted as including all
such
alterations, modifications, permutations, and substitute equivalents as fall
within the
true spirit and scope of the present invention.
24
