Note: Descriptions are shown in the official language in which they were submitted.
CA 02746734 2011-06-13
WO 2010/087879 PCT/US2009/057893
EMAIL CLIENT CAPABLE OF SUPPORTING NEAR REAL-TIME
COMMUNICATION AND METHODS FOR USING THE ADDRESSING,
PROTOCOLS AND THE INFRASTRUCTURE OF EMAIL TO SUPPORT
NEAR REAL-TIME COMMUNICATION
BACKGROUND
Field of the Invention
[0001] This invention pertains to communications, and more particularly, to an
email
client capable of supporting near real-time communication of time-based media
and
methods for using the addressing, protocols and the infrastructure of email to
support
real-time communication.
Description of Related Art
[0002] Currently there are three globally used addressing domains. The postal
system,
which is mainly used for the delivery of letters and parcels, relies on the
use of a
physical address, such as a house address, office building address or Post
Office (PO)
box. In order to assure the delivery of a letter or parcel, the physical
address of the
recipient must be provided, including a country, state or territory, a city or
town,
postal or zip code, street name and street number. The existing telephone
infrastructure defines another global addressing domain that has historically
been used
for near real-time voice communications (i.e., telephone calls). Both land-
line and
mobile telephones are addressed (i.e., called) using a telephone number, which
typically includes a country code and a variable number of additional digits
to identify
a particular phone within a given country and/or area code. When a circuit
connection
is made between the calling parties, a full duplex conversation may take
place. A third
global addressing system is email. Every email account is identified by a
unique
globally addressable email address, which defines a user name and a domain
name.
[0003] Emails are typically text messages that are sent from a sender to one
or more
recipients. The emails are created on an email client. One well-known email
client is
Microsoft Outlook, which is used to create, receive and manage email messages
on a
computer. Alternatively, free email services like Yahoo, Google or Hotmail are
available to users through a web page. Regardless of the type used, an email
client
will typically (i) list or display all the received messages, with an email
header
showing the subject of the email, the sender of the email, the date/time it
was sent and
possibly other attributes such as the size of the email; (ii) allow the user
to select
messages for review; (iii) allow the user to type and send new messages to
recipients
1
CA 02746734 2011-06-13
WO 2010/087879 PCT/US2009/057893
and reply to the received emails of others; and (iv) allow attachments, such
as still
photos, documents, or video clips, to be attached to an out-going email.
[0004] An email message must first be created in full before it can be sent. A
sender
will typically first define a recipient by entering their email address into
the
appropriate "To" field in the header of the email. The text message is then
typed into
the body of the email and files may optionally be attached. When the message
is
complete, the user sends the email. During the send sequence, the email client
initiates
a session with its email server located on a network. This session is
typically
established with the Simple Mail Transport Protocol (SMTP). During the
session, the
email client provides the SMTP server with the email address of the sender,
the email
address of the recipient, and the body of the email with any attachments. The
email
addresses of the recipient is segmented into two parts, including the
recipient's name
(e.g., "jsmith") and the domain name (e.g., "hotmail.com"). If the recipient
is in a
domain that the SMTP server controls, then the server carries out delivery
instructions
for the specific recipient, which is typically delivery of the email to an in-
box
associated with the recipient on the same SMTP server or another server
located in the
same domain. On the other hand if the recipient is in a domain that the server
does
not control, then the email server needs to communicate with a server that
controls the
recipient's domain using SMTP.
[0005] To send the email to the recipient in another domain, the SMTP server
initiates
a conversation with the Domain Name System (DNS), asking for the Mail
eXchanger
(MX) record of the recipient's domain. This MX record contains a prioritized
list of
SMTP servers for that domain. The email is then sent from the SMTP server of
the
sender to the first SMTP server in the MX list that responds. This first
responding
server then determines if the recipient is in the domain the first responding
server
controls. If so, the email is delivered to the inbox of the recipient. If not,
the above-
described process is repeated until a responding server is the one that can
deliver the
message into the recipient's inbox. Each server along the delivery route is
sometimes
referred to as a "hop". The email may then be accessed through the email
client of the
recipient, which may be located on the computer of the recipient or on the
Internet. If
an email is sent to multiple parties, the above-described process is repeated
for each
recipient.
2
CA 02746734 2011-06-13
WO 2010/087879 PCT/US2009/057893
[0006] The above-described sequence generally applies for emails sent over the
Internet. With certain proprietary systems, such as an email sent between two
Microsoft Exchange users on the same proprietary network, the SMTP protocol
may
not be used for routing the email but email addresses are still used. The
operation of
the proprietary protocol and server is essentially the same as SMTP.
[0007] The existing email infrastructure, regardless if it relies on SMTP or a
proprietary email protocol, is essentially a "store and forward" messaging
system. An
email message must first be created in its entirety before it can be sent. At
the SMTP
or proprietary mail server of the sender, as well as any intermediate email
server hops
along the path to the SMTP or proprietary mail server of the recipient, the
email
message must be received in full before it can be forwarded. Finally the email
must be
received in full at the inbox of the recipient before the recipient can review
the
mes sage.
[0008] By way of comparison, telephone conversations over the Public Switched
Telephone Network (PSTN) are progressive in nature. As words are spoken, they
are
simultaneously transmitted from the sender to the recipient, where they are
heard
effectively live or near real-time. As a result, telephone conversations can
be
conducted in a "live" or near real-time mode through a common network
connection
(i.e., a circuit). Email communication in contrast usually occurs through a
series of
separate store and forward messages, often sent back and forth between two or
more
parties at distinct times, across a network, such as the Internet.
[0009] It is well known to attach a file to an email containing time-based
media (i.e.,
media that changes with respect to time), such as a video clip. The time-based
media
attached to an email message, however, can never be reviewed by a recipient
"live",
as it is being created, due to the store and forward nature of email. Rather
the email
and the attachment containing the time-based media must first be created,
sent, stored
and forwarded at each email server hop on the network, and then received by
the
recipient in full before the time-based media of the attachment can be
reviewed. It is
therefore not possible for the recipient of an email message to review the
media in
near real-time as the media is being created.
[0010] Telephone messaging systems are also known where a voice message may be
created and sent to a recipient in the form of an email. With these systems,
the Public
Switched Telephone Network (PSTN) is used in cooperation with emails. In use,
a
3
CA 02746734 2011-06-13
WO 2010/087879 PCT/US2009/057893
recording of the message must first be made, stored, and then forwarded to the
recipient by email. Again, however, the message must first be received in full
before
the recipient can review the recorded message.
[0011] Instant messaging or IM is another example of a store and forward
system.
Similar to email as described above, messages must be completed before they
can be
forwarded to a recipient. Messages in IM systems are generally much shorter
than
those sent via email. Each line of text in IM systems is a separate message
delivered
in a store and forward manner. Existing IM systems do not provide a way for a
recipient to progressively and simultaneously review a message as the sender
creates
the message.
[0012] Live text systems are well known, although they were mostly used on
early
Unix systems with dumb terminal interfaces. In a live text system, each
individual
keystroke is sent to the recipient as soon as the sender pressed that key.
These
systems are for text only, but they do allow the recipient to progressively
review a
message as the message is being created.
[0013] Currently there is no known system or method for extending the global
addressing and routing infrastructure of email to support the live or near
real-time
communication of time-based media between a sender and a recipient using their
email addresses.
SUMMARY OF THE INVENTION
[0014] An email client capable of supporting real-time communication of time-
based
media is disclosed. The email client includes a session element configured to
establish
a session with a server when an email address addressing a recipient within a
domain
is defined. As soon as the email address is defined, a transmitting element of
the email
client is configured to progressively and simultaneously transmit time-based
media as
the time-based media is being created to the recipient via a route at least
partially
discovered by a lookup of the domain of the email address. By at least
partially
discovering the route to the recipient as soon as the email address of the
recipient is
defined, the transmitting element may progressively deliver the time-based
media to
the recipient.
4
CA 02746734 2011-06-13
WO 2010/087879 PCT/US2009/057893
BRIEF DESCRIPTION OF THE DRAWINGS
[0015] The invention may best be understood by reference to the following
description taken in conjunction with the accompanying drawings, which
illustrate
specific embodiments of the invention.
[0016] Figure 1 is a diagram of a network capable of supporting live or near
real-time
communication of time-based media between users according to the present
invention.
[0017] Figure 2 is a diagram of a communication device according to one
embodiment of the present invention.
[0018] Figure 3 is a diagram of a communication device according to another
embodiment of the present invention.
[0019] Figures 4A and 4B are flow diagrams illustrating the sequence of
creating an
email header on a communication device of the present invention.
[0020] Figures 5A through 5D are flow diagrams illustrating the sequence for
conducting communication over the network in accordance with the present
invention.
[0021] Figure 6 is a flow diagram illustrating the attachment of a media file
to an
email according to the present invention.
[0022] Figure 7 is a diagram illustrating the delivery of time-based media
over the
network according to another embodiment of the present invention.
[0023] Figure 8 is a diagram illustrating the structure of a conventional
email
according to the prior art.
[0024] Figure 9 is a diagram of the structure of a progressive email according
to the
present invention.
[0025] It should be noted that like reference numbers refer to like elements
in the
figures.
DETAILED DESCRIPTION OF SPECIFIC EMBODIMENTS
[0026] The invention will now be described in detail with reference to various
embodiments thereof as illustrated in the accompanying drawings. In the
following
description, specific details are set forth in order to provide a thorough
understanding
of the invention. It will be apparent, however, to one skilled in the art,
that the
invention may be practiced without using some of the implementation details
set forth
5
CA 02746734 2011-06-13
WO 2010/087879 PCT/US2009/057893
herein. It should also be understood that well known operations have not been
described in detail in order to not unnecessarily obscure the invention.
[0027] The present application is directed to a number of embodiments,
including (i)
the use of the email and DNS infrastructure to define the routing for the
delivery of
messages containing time-based media while using a near real-time
communication
protocol for the actual delivery of the media; (ii) various delivery options
of messages
containing time-based media using email addressing and DNS; (iii) the
modification
of SMTP or other proprietary email protocols to support the transmission of
"progressive" emails containing time-based media; (iv) the late binding of
recipient
email addresses for near real-time voice or other time-based media
communication;
and (v) conducting near real-time conversations by routing messages or
progressive
emails containing time-based media using globally addressable email addresses
and
DNS. Each of these aspects is described in detail below.
I. The Use of the Email and DNS Infrastructure to Define the Routing
for the Delivery of Messages Containing Time-based Media Using a Near
Real-time Communication Protocol for the Actual Delivery of the Media
[0028] Referring to Figure 1, a diagram of a network system capable of (i)
supporting
"live" or near real-time communication of time-based media and (ii) routing
using the
infrastructure of email and DNS according to the present invention is shown.
The
system 10 includes a network 12 with users A, B, C and D using communication
devices 14A, 14B, 14C and 14D and Servers 16A, 16B, 16C and 16D located on the
network 12. The network 12 further includes a DNS server 18. In various
embodiments, the network 12 may include the Internet, an intranet, a mobile IP
network, or any other type of network that relies on the Internet Protocol
and/or DNS,
or any combination thereof. Users A, B and C are each addressed by the servers
16A
through 16D by their respective globally addressable email addresses
"UserA @Domain A", "UserB @Domain B", and "UserC@Domain C". User D is
intentionally not identified on the network 12 by a globally addressable email
address
for reasons mentioned below.
[0029] The Servers 16A, 16B, 16C and 16D are each configured to provide one or
more services to Users A, B, C and D respectively. In this example, Server A
defines
Domain A and provides User A with the standard email delivery service using
SMTP
(or a similar proprietary service) and MX DNS records, hereafter referred to
as "MX".
6
CA 02746734 2013-10-24
Server A further provides User A with a real-time communication service,
hereafter
referred to as "RVX". Server 16B defines Domain B and provides User B with the
real-time communication service RVX, but not the email service MX. Server 16C
defines Domain C and provides User C with the email service MX, but not the
real-
time domain RVX service. Server 16D does not provide user D with either the
real-
time communication service RVX nor the email domain MX service, but other
services
that are not identified because they are not relevant.
[0030] In one embodiment, the real-time service RVX may rely on any
communication
protocol that allows users to communicate time-based media in near real-time,
but does
not require the recipient to review the time-based media in a near real-time
mode.
Known protocols with these properties include the Cooperative Transmission
Protocol
(CTP) described in detail in co-owned U.S. Patent Nos. 8,090,867 and 8,180,029
or the
near real-time synchronization protocol of voice or other time-based media as
described
in co-owned U.S. Patent No. 8,099,512 and U.S. Patent Application Publication
Nos.
2009/0103689 and 2009/0104894.
[0031] In alternate embodiments, the RVX service may rely on other
communications
protocols, individually or in combination, that provide near real-time
communication,
such as SIP, RTP, Skype, VoIP, etc.
[0032] The communication devices 14A through 14D may each be any type of
communication device, such as land-line telephones, VoIP telephones, cellular
radios,
satellite radios, military or first responder radios, mobile Internet devices,
or just about
any other type of communication device. In addition, a given user might have
multiple
communication devices 14. For example, a user may have one or more of the
following; a home computer, a work computer, a Push to Talk radio, a mobile
phone or
a personal digital assistant (PDA). Regardless of the number of communication
devices
14 each user A, B, C and D has, each will operate essentially the same and
receive the
services provided by the servers 16A, 16B, 16C and 16D as described herein
respectively.
[0033] It should be noted that the system 10 as illustrated has been greatly
simplified
compared to what would typically be implemented in actual embodiments. For the
sake
of illustration, the RVX and MX services as (or not) provided to Users A, B, C
7
CA 02746734 2011-06-13
WO 2010/087879 PCT/US2009/057893
and D as listed above have been purposely selected to highlight and describe
various
features and aspects of the present invention. In actual embodiments, however,
there
would likely be a significantly larger number of users, each with one or more
communication devices 14 and associated servers on the network 12, providing a
variety of services to each user. In addition, any combination ranging from a
single
server or a suite of servers 16 may be included on the network 12 to provide
the RVX
and/or MX for one to multiple users respectively. The communication devices
14A,
14B and 14C and the servers 16A, 16B and 16C may also communicate with one
another in a manner similar to that described above using DNS, SMTP, or other
proprietary email protocols for route discovery across one or more hops on the
network 12. The delivery route for a message to a recipient in the same domain
is
typically delivered to an inbox on the same server 16 or an associated server
in the
same domain. A message sent to a recipient in another domain will typically be
sent
to the email server of the recipient via one or more hops across the network
12. As the
routing of emails and media in near real-time across an IP network is well
known in
the art, a detailed explanation is not provided herein.
[0034] Referring to Figure 2, a diagram of a communication device 14 according
to
one embodiment of the present invention is shown. In this embodiment, the
communication device 14 is a mobile device 20 capable of wireles sly
communicating
with the network 12, such as a mobile phone or PTT radio. The mobile device 20
may optionally include one or more of the following; a keypad 22, a display
24,
speaker 26, microphone 28, volume control 30, camera 32 capable of generating
still
photos and/or video, a display control element 34, a start function element 36
and an
end function element 38. In various embodiments, the device 20 (i) is IP
based,
meaning it is designed to communicate over the network 12 using the Internet
Protocol and (ii) runs one or more RVX protocols, including any of those
listed above
or any other near real-time communication protocol. In addition, the device 20
may
optionally also locally run an email client, access an email client located on
one of the
servers 16 located on the network 12, or be capable of both running and
accessing an
email client on the network.
[0035] Referring to Figure 3, a diagram of a communication device according to
another embodiment of the present invention is shown. In this embodiment, the
communication device 14 is a computer 40 connected to the network 12, either
8
CA 02746734 2011-06-13
WO 2010/087879 PCT/US2009/057893
through a wired or wireless connection (not shown). The computer 40 optionally
includes one or more of the following; a keyboard 42, a display 44, speakers
46, a
microphone 48, a camera 50 capable of generating still photos or video, a
mouse 52, a
start function element 54 and an end function element 56. The computer 40 is
capable
of running an email client, accessing an email client located on the network
12, or
both. In various embodiments, the computer 40 (i) is IP based, meaning it is
designed
to communicate over the network 12 using the Internet Protocol and (ii) runs
one or
more RVX protocols, including any of those listed above or any other near real-
time
communication protocol. Further, the computer 40 could be a portable computer,
such
as a laptop or personal digital assistant, and is not limited to the desktop
computer as
shown. In addition, the device 40 may optionally also locally run an email
client,
access an email client located on one of the servers 16 located on the network
12, or
be capable of both running and accessing email client on the network.
[0036] The start function elements 36/54 and the end function elements 38/56
of the
mobile device 20 and computer 40 are meant to be symbolic of their respective
functions. It is not necessary for mobile device 20, computer 40, or any other
type of
communication device 14, to physically include start and end buttons per se.
Rather, it
should be understood that each of these functions might be implemented in a
variety
of ways, for example, by entering a voice command, a predefined keystroke or
command using a touch screen or other input device such as a mouse, stylus or
pointer, etc.
[0037] The network 12 uses the existing email infrastructure, including the
globally
recognizable email addresses of the recipient users and DNS for route
discovery,
while using a near real-time RVX protocol for the actual transport of messages
containing time-based media to the addressed recipient once the route is
discovered.
Like conventional emails, each message relies on a header that defines, among
other
things, a globally addressable email address of one or more recipients for
routing
purposes. Unlike conventional store and forward emails, however, the time-
based
media of the message is transmitted using a near real-time RVX protocol. As a
result,
time-based media may be simultaneously and progressively transmitted across
the
network 12, as the sender creates the media. In addition, the recipient may
optionally
simultaneously and progressively render the time-based media as it is received
over
the network. When two or more parties are conversing (e.g., generating and
reviewing
9
CA 02746734 2011-06-13
WO 2010/087879 PCT/US2009/057893
time-based media) at the same time, the network 12 is supporting near real-
time
communication using an RVX protocol for media delivery, while using the
existing
email infrastructure and DNS for routing.
[0038] Referring to Figure 4A, a flow diagram illustrating the sequence of
creating
and transmitting time-based media associated with a message on a communication
device 14 is shown. If the user of a communication device 14 wishes to
communicate
with a particular recipient, the user will either select the recipient from
their list of
contacts or reply to an already received message from the intended recipient.
If a
message from the intended recipient is not available for responding or if the
intended
recipient is not already in the contact list, the globally addressable email
address of
the recipient is manually entered into the device 14.
[0039] In response to any of the above, a message header is created (step 62),
including the globally addressable email address of the recipient in a "To"
header. As
soon as the globally addressable email address of the recipient is defined, a
DNS
lookup is performed, so that the route for delivering the media associated
with the
message to the globally addressed recipient is immediately discovered.
Thereafter, a
user may initiate the start function 36/54 and begin creating time-based media
(step
64), for example by speaking into the microphone, generating video, or both.
The
time-based media is then progressively and simultaneously encoded (step 66),
transmitted (step 68) over the network 12 using an RVX protocol using the
discovered
delivery route, and optionally persistently stored on the device 14 (step 70).
It should
be noted that although these steps 62 through 70 are illustrated in the
diagram in a
sequence, for all practical purposes they occur at substantially the same
time. The user
may select a recipient from a contacts list, initiate the start function
36/54, and then
begin speaking immediately. As the media is created, the RVX protocol
progressively
and simultaneously transmits the media across the network 12 to the recipient,
using
the DNS lookup result to discover the route without any perceptible delay to
the
sending user.
[0040] The time-based media of outgoing messages may optionally be
persistently
stored on the sending communication device 14 for a number of reasons. For
example, if time-based media of a message is created before the delivery route
is
discovered, then the time-based media may be transmitted from storage when the
delivery route is discovered. If time-based media is still being created after
the route
CA 02746734 2011-06-13
WO 2010/087879 PCT/US2009/057893
is discovered, then the time-based media is transmitted progressively and
simultaneously as the media is being created. Alternatively with the storage
of time-
based media, the sender may review stored messages at an arbitrary later time.
A
message may also be created and stored when the communication device 14 is not
connected to the network 12, where connected is defined as the ability to send
messages over the network and not connected is defined as the inability to
send
messages over the network. When the device 14 later connects, the message may
be
transmitted to the intended recipient from storage, using either an RVX
protocol or as
an attachment to an email.
[0041] Referring to Figure 4B, a flow diagram 100 illustrating the sequence
for
creating a message header (step 62 in Figure 4A) is shown. In the step 62a,
the
globally addressable email address of the sender is provided in the "From"
field of the
message header. In step 62b, the globally addressable email address of the
recipient is
entered into the "To" field of the message header. If there are multiple
recipients, the
email address of each is entered into the "To" field. In additional
embodiments, a
"CC" or "BCC" field may be used for one or all recipients. In step 62c, a
globally
unique message ID or number is assigned to the message. In step 62d, other
information, such as a conversation name, or the subject of the message, is
provided
in the header. In step 62e, the start date/time the message was created and
possibly the
end date/time of the message may be included in the header. In one embodiment,
the
steps 62a through 62e generally all occur at substantially the same time, with
the
possible exception of defining the end date/time. In other embodiments, the
steps 62a
through 62e may occur in any order.
[0042] The start and end date/times ordinarily coincide with the
implementation of
the start function 36/54 and end function 38/56 on the sending device 14
respectively.
A sender, however, might not always implement the end function 38/56 for a
given
message. When this occurs, the sender may simply stop creating and sending
time-
based media associated with the message. The message may, therefore, remain
"open-
ended" without a defined end-time/date.
[0043] In certain embodiments, the steps 62a through 62e may be performed on a
sending communication device 14. In other embodiments, the sending
communication
device may send some or all of the message header information to a server 16,
where
the steps 62a through 62e are performed. The time-based media of the message
may
11
CA 02746734 2011-06-13
WO 2010/087879 PCT/US2009/057893
also be optionally stored on a server 16 for later review by the sending user
or
transmission to the recipient.
[0044] In the embodiments described above, a message header with various
fields
including a To, From, Message ID number, Conversation Name, and message Start
and End time is provided. It should be understood that not all of these fields
are
necessary, and other fields may be included. The only required information is
at least
one recipient specified in one of the To, CC, or BCC fields, which defines the
globally addressable email address of a recipient. The other fields are all
optional.
[0045] The format of the message header is also variable. In one embodiment,
the
structure of the message header may be similar to that used with conventional
emails
or the enveloped used with emails. In other embodiments, the structure of the
message
header may take any form that is suitable for transmitting the globally
addressable
email address of the recipient(s), along with possibly other header
information, across
the network 12. While specific email header fields are discussed for
specifying
recipients, the actual header field containing the recipient address
information may
not necessarily include the globally addressable email address of the
recipient per se.
As is well known in the art, an "envelope recipient" may be used to specify
the email
address of the recipient, even though the envelope recipient may differ from
the
recipients listed in the email headers. Thus as used herein, the term message
header
should be broadly construed to include both envelope information and
conventional
message or email headers including any number of fields, such as but not
limited to
those specified in RFC 822 or 5322. In addition, the usage of the terms
"addressing"
or "globally addressable email address" is intended to be broadly construed to
include
any addressing method, including usage in conventional message or email
headers or
in a message envelope.
[0046] The network 12, under certain circumstances, may deliver messages
containing time-based media that can (i) be simultaneously and progressively
transmitted to a recipient over the network 12 and (ii) reviewed in near real-
time by
the addressed recipient as the time-based media is being created and sent by
the
sending user. Under other circumstances, the messages cannot be delivered in
real-
time. Both the near real-time and non real-time scenarios are discussed below
with
regard to Figures 5A through 5C respectively.
12
CA 02746734 2011-06-13
WO 2010/087879 PCT/US2009/057893
[0047] Referring to Figure 5A a flow diagram 80 illustrating the sequence for
potentially conducting near real-time communication with messages containing
time-
based media using a globally addressable email address over the network 12 is
shown.
The sequence is described in the context of user A sending a message to user B
using
any near real-time RVX protocol. As noted above, server 16B provides user B
with an
RVX service, but not the MX service.
[0048] In the initial step 82, server 16A receives at substantially the same
time the
message header (or the header information allowing the server to perform some
or all
of the steps 62a-62e) and the time-based media of the message to be
transmitted as it
is being progressively and simultaneously created and transmitted by
communication
device 14A. As the message header contains user B's globally addressable email
address (userB@DomainB) in the "To", "CC", or "BCC" field, server 16A requests
a
lookup for the RVX of domain B (step 84) of DNS server 18 using the DNS
protocol.
Since the RVX exists for domain B (decision 86), the lookup result is
positive. The
time-based media is then progressively and simultaneously sent using the RVX
protocol from the server 16A associated with the sender to server 16B
associated with
the recipient. The time-based media may be transmitted across one or more hops
between the two servers 16A and 16B. At each hop, a DNS lookup is performed to
discover the delivery route to the next hop, while the RVX protocol is used to
deliver
the time-based media to each next hop.
[0049] In one embodiment, the media is simultaneously and progressively
transmitted
to the communication device 14B of the recipient when the time-based media
arrives
at server 16B. The recipient is notified of the incoming message, and in
response, the
recipient may elect to simultaneously review the media in the near real-time
mode as
the media of the message is progressively received.
[0050] In an alternative embodiment, the media of the message is also
optionally
placed in an inbox and persistently stored on the recipient device 14B. With
the
persistent storage of the message, the recipient has the option of reviewing
the media
in the near real-time mode as the media is received or at an arbitrary later
time from
storage.
[0051] In yet another embodiment, the message may also be stored in an inbox
located at the server 16B associated with the user B. In this manner, the user
of device
14B may access the message at an arbitrary later time from the inbox on server
16B.
13
CA 02746734 2011-06-13
WO 2010/087879 PCT/US2009/057893
In addition, the server 16B may encapsulate the message into a file and attach
the file
to an email. As noted above, user B is not provided the MX service and
therefore
cannot receive such an email. But in situations where a user can receive
emails, the
message can be forwarded in the form of an attachment.
[0052] In yet other embodiments, the media of the message may be stored in an
out-
box of the sending user, either located on the user's sending communication
device
14, or on the server 16A associated with the sender.
[0053] Referring to Figure 5B, the flow diagram 80 is again provided to
illustrate
communication between user A and user C. As previously noted, server 16C
provides
user C with the MX service, but not a real-time RVX service. When user A
wishes to
communicate with user C, the initial sequence is essentially the same as that
described
above. Server 16A initially receives a message header (or the header
information
necessary to optionally perform steps 62a-62e) with the globally addressable
email
address of user C (userC@domainC) and the progressive and simultaneous
transmission of time-based media by user A (step 82). Since the RVX lookup
result
(decision 86) is negative, server 16A next requests an MX lookup of DNS server
18
for domain C (step 90) using the DNS protocol. With a positive result
(decision 92),
server 16A sends a conventional email with the time-based media encapsulated
as an
attachment (step 96) to server 16C. At the server 16C, the email is placed in
the
recipient's inbox. The email may also be forwarded to an inbox on
communication
device 14C. Thus when the recipient does not have the RVX service, the time-
based
media of the message is sent across the network 12 by Server 16A to server
16C, and
possibly communication device 14C, using the store and forward procedure of
SMTP
or a similar proprietary email system.
[0054] Referring to Figure 5C, the flow diagram 80 is again provided to
illustrate a
communication attempt between user A and user D. As previously noted, user D
is
not provided with either the email MX service or a near real-time RVX service.
When
user A wishes to communicate with user D, the initial sequence is essentially
the same
as that described above. Server 16A receives a message header with the
globally
addressable email address of user D (userD@domainD) (or the header information
needed to optionally perform steps 62a through 62e) and the progressive and
simultaneous transmission of time-based media by user A (step 82). Since the
RVX
lookup (decision 86) and the MX lookup for domain D (diamond 92) are both
14
CA 02746734 2011-06-13
WO 2010/087879 PCT/US2009/057893
negative, an error message is generated (step 94) and the message cannot be
delivered
(step 96). In various embodiments, the time-based media of the message may be
stored at either the sending communication device 14A, the server 16A, or
both. The
message may later be sent when the RVX and/or MX service is provided to user
D.
[0055] The scenario described with regard to Figure 5C typically occurs if an
incorrect email domain name is provided for a recipient. When the sender
attempts to
send a message using an invalid globally addressable email domain name, the
error
message (step 94) results. If the correct domain name in the email address is
provided,
the message can then be forwarded using either an RVX protocol or as an
attachment
to an email using the MX service.
[0056] In an alternative embodiment, the communication devices 14A through 14C
may be arranged in a peer-to-peer configuration. With this arrangement, at
least the
sending communication devices 14 are capable of performing the RVX and/or MX
lookups on DNS server 18 directly, without the aid of an intervening server 16
to
perform the lookup function. The communication devices 14 may also be capable
of
transmitting the media of the messages directly to other communication
devices.
Depending on whether the recipient is a member or not of the RVX and/or MX
domains, the sending communication device 14A will either (i) progressively
and
simultaneously transmit the time-based media of a message to the recipient
over the
network 12; (ii) encapsulate the time-based media of the message into a file
and
transmit an email including the file as an attachment to the recipient using
SMTP or a
similar proprietary protocol; (iii) or receive an error message if an invalid
globally
addressable user name or domain name was used in the email address and/or the
recipient is not provided the MX service.
[0057] Referring to Figure 5D, a flow diagram 100 illustrating the peer-to-
peer
embodiment is illustrated. In the initial step 101, a sending communication
device 14
indicates that it would like to communicate with a receiving communication
device
14. In decision diamond 102, the communication device 14 of the sender
performs a
DNS lookup of the recipient's globally addressable email address to determine
if the
peer recipient receives the RVX service. If the result of the look up is
positive, then
the time-based media created (step 103) using the sending communication device
14
is progressively and simultaneously transmitted (step 104) to the recipient
using the
delivery route defined by the RVX lookup. In decision diamond 105, it is
determined
CA 02746734 2011-06-13
WO 2010/087879 PCT/US2009/057893
if real-time communication is established. If yes, then the transmitted media
is
progressively and simultaneously rendered at the communication device 14 of
the
recipient as the media is received (box 106). If near real-time communication
is not
established, then the media of the message is placed in the inbox of the
recipient (box
107), either on the device 14 of the recipient, a server 16 associated with
the recipient,
or possible both. Near real-time communication may not take place with the
recipient
for a number of reasons, such as the recipient is not available, out of
network range,
or has indicated a desire to not review the message in the near real-time
mode.
[0058] On the other hand if the recipient does not receive the RVX service
(decision
102), then the media of the message is delivered in the form of an attachment
to an
email, provided the recipient receives the MX domain service. The time-based
media
is encapsulated into a file and attached to an email (step 108). When the
message is
complete, the email is transmitted using the route defined by the MX lookup
result
(step 109). In one embodiment, the email may be sent directly from the sending
peer
if the sending communication device 14 is locally running an email client. The
email
may be received either at the recipient peer device 14 if running an email
client, at a
server 16 running an email client on behalf of the recipient or possibly both
the
receiving peer 14 and server 16. In situations where both peers are running an
email
client, media may be sent in the form of an attachment to an email from the
sending
communication device 14 to the receiving communication device 14. This differs
from known telephone messaging systems, where a server, as opposed to a
sending
peer, emails a voice message to the recipient. In certain embodiments, an
attachment
may be substituted or augmented by a link to a web page containing the time-
based
media, as described in more detail below.
[0059] It should be noted that the discussion above with regard to Figures 4A,
4B and
5A through 5C has been simplified to illustrate certain aspects of the
invention. It
should be understood that actual implementations could be modified in several
ways.
For example, each time the server 16A received an email address, the server
16A
would first determine if the domain of the recipient (i.e., domain A, domain B
or
domain C), is within one or more local domains of the server 16A. If not, then
the
procedures described above with regard to Figures 5A, 5B and 5C are performed
respectively. On the other hand if the domain of the recipient is within a
local domain
of the server 16A, then the server 16A may deliver the message either (i) in
real-time
16
CA 02746734 2011-06-13
WO 2010/087879 PCT/US2009/057893
if the recipient receives a real-time communication service or (ii) as an
attachment to
an email if the recipient receives the MX service, but not a real-time
service. In
addition, it may not be necessary for the Server 16A to perform a DNS lookup
in each
instance. As is well known, previous DNS lookup results may be cached and used
upon rather than performing a new DNS lookup each time an email address of a
recipient is received.
[0060] Referring to Figure 6, a flow diagram 110 illustrating the sequence for
sending
time-based media encapsulated in an email attachment at server 16A (box 98 in
Figure 5B) or from a sending device 14A (box 107 in Figure 5D) is shown. In
either
case, time-based media generated by user A is encapsulated in a file (step
112) and is
attached to the email (step 114) when the message is complete, for example
when the
end function 38/56 is implemented. In situations where the end function 38/56
is not
implemented, the end of the message may be declared by default, after a
predetermined period of time lapses without the creation of any new time-based
media. Once the time-based media of the message is complete, either by the
implementation of the end function 38/56 or by default, the email with the
attachment
is then transmitted (step 116) by server 16A or communication device 14A to
the MX
lookup result of the recipient over the network 12 using the SMTP or a similar
proprietary protocol, in a manner similar to a conventional email.
[0061] With either the server or peer-to-peer models described above, the RVX
lookup result is initially used to deliver the time-based media. If the RVX
attempt
fails, then the MX result is used as a backup. With this arrangement, a
conventional
email with the time-based media included in an attachment and/or web link is
used to
deliver the media in circumstances where the recipient is not provided RVX
service.
The email may be created either on a server or on the sending device.
II. Delivery Options
[0062] Referring to Figure 7, a diagram illustrating the delivery of time-
based media
over the network 12 according to another embodiment of the present invention
is
shown. With this embodiment, the network 12 is essentially the same as that
described above with regard to Figure 1, with at least one exception. One or
more of
the servers 16A-16C are configured as web servers, in addition to providing
the RVX
and/or MX services as described above. With this embodiment, users receive an
email
from their respective server 16 containing a URL link when a message is sent
to them.
17
CA 02746734 2011-06-13
WO 2010/087879 PCT/US2009/057893
When the user selects the link through a web browser running on their
communication
device 14, the appropriate web server 16 serves up web pages allowing the
recipient
to access and review the message. The served web pages may also provide a
variety
of rendering options, such as review the media of the message in either the
real-time
or time-shifted modes, catch up to live, pause a live conversation, jump to
the head of
a conversation, jump to a previous point in time of the conversation, render
faster,
render slower, jump between different conversations, etc. In the figure, the
web server
functionality is provided as one of the services provided by Servers 16A, 16B
and
16C. In an alternative embodiment, the web server functionality can be
implemented
using one or more other servers (not illustrated) on the network 12 besides
16A, 16B
or 16C.
III. Email Protocol Modifications and Progressive Emails
[0063] The messages as described above are routed using globally addressable
email
address and the DNS infrastructure for defining a delivery route, while using
an RVX
protocol for the actual delivery of the time-based media in near real-time.
Although
the SMTP standard and other proprietary email protocols as currently defined
and
used are store and forward protocols, with certain modifications, SMTP and
other
proprietary email protocols could be used as an RVX messaging protocol for the
near
real-time delivery of time-based media as contemplated in the present
application.
With conventional emails, the media content must be composed in full and
packaged
before the email can be sent. On the receiving end, the email must be received
in full
before the recipient can review it. As described in detail below, SMTP,
Microsoft
Exchange or any other proprietary email protocol may be used for creating
"progressive" emails, where media may be sent in near real-time.
[0064] The existing email infrastructure can be used to support the near real-
time
transmission of time-based media by modifying the way the SMTP, Microsoft
Exchange or other proprietary email protocols (hereafter generically referred
to as an
email protocol or protocols) are used on the sending side and modifying the
way that
emails are retrieved from the server on the receiving side. Current email
protocols do
not strictly require that the entire message be available for sending before
delivery is
started, although this is typically how email protocols are used. Time-based
media can
therefore be delivered progressively, as it is being created, using standard
SMTP,
Microsoft Exchange or any other proprietary email protocol.
18
CA 02746734 2011-06-13
WO 2010/087879 PCT/US2009/057893
[0065] Email is typically delivered to user devices through an access protocol
like
POP or IMAP. These protocols do not support the progressive delivery of
messages as
they are arriving. However, by making simple modifications to these access
protocols,
a message may be progressively delivered to a recipient as the media of the
message
is arriving over the network. Such modifications include the removal of the
current
requirement that the email server know the full size of the email message
before the
message can be downloaded to the client. By removing this restriction, a
client may
begin downloading the time-based media of an email message as the time-based
media of the email message is received at the server over the network.
[0066] Referring to Figure 8, the structure of a conventional prior art email
120 using
any of the above listed email protocols is illustrated. The email 120 includes
a header
122 and a body 124. The header includes a "To" (or possibly the CC and/or BCC
fields) field, a "From" field, a unique global ID number, a subject field,
optional
Attachments, and a Date/time stamp. The body 124 of the email includes the
media to
be transmitted, which typically includes a typed message and possibly attached
files
(e.g. documents or photos). When complete, the email is sent. A DNS lookup is
performed and the email is routed to the recipient. Conventional emails are
"static",
meaning the body of the email, including attachments, is fixed once
transmission
starts. There is no way to progressively and simultaneously transmit with
conventional emails time-based media as the media is being created. Prior art
emails
120 are therefore incapable of supporting near real-time communication.
[0067] Referring to Figure 9, the structure of an email 130 according to the
present
invention is shown. Email message 130 is used for supporting near real-time
communication. The email 130 includes a header 132 including a "To" field (and
possibly the CC and/or BCC fields) and a body 134. The structure of email 130,
however, differs from a conventional prior art email 120 in at least two
regards. First,
the header 132 includes an email Start date/time and an End date/time. By
associating
a start and end time with an email 130, as opposed to just a date/time stamp
when an
email 120 is sent, the second difference may be realized. After an email 130
is created
and the sender defines a globally addressable email address of the recipient,
the DNS
lookup for routing is immediately performed. At substantially the same time,
time-
based media may be created. As the time-based media is created, it is
progressively
and simultaneously transmitted to the DNS lookup result, from hop to hop,
using the
19
CA 02746734 2011-06-13
WO 2010/087879 PCT/US2009/057893
streaming nature of SMTP, Microsoft Exchange or any other type of email
protocol.
The body 134 of email 130 is therefore "progressive". As time-based media
associated with an email message 130 is dynamically created, the time-based
media is
simultaneously and progressively transmitted to the email server of the
recipient, from
hop to hop across the network when necessary. If an email 130 is sent to
multiple
recipients, regardless if identified in the To, CC or BCC fields, the above
process is
repeated for each.
[0068] The DNS lookup is immediately performed right after the email address
of the
recipient is defined by initiating an email protocol session with the email
server
associated with the sender. This differs from conventional emails 120, where
the
email protocol session is typically initiated only after the email has been
composed in
full and the sender implements a "send" function. As a result, the delivery
route can
be discovered either before or concurrent with the progressive and
simultaneous
transmission of time-based media as it is being created. In situations where
time-
based media is created before the session is established, the time-based media
may be
either temporarily or persistently stored as the media is created. The stored
media may
then be progressively transmitted from storage once the protocol session with
the
email server is established.
[0069] The End date/time of email 130 may be either defined or open-ended.
When
the sender implements the end function 38/56 on the communication device 14,
then
the end time of the email 130 is defined. If the end function 38/56 is never
implemented, then the duration of the email 130 is "open-ended" and does not
necessarily have a defined end date/time. Open-ended emails 130 are therefore
typically terminated by default after a predetermined period of time where no
media is
created.
[0070] In summary, progressive emails 130 can be sent using SMTP, Microsoft
Exchange or any other proprietary email protocol by implementing the above-
described modifications. Similarly, recipients may simultaneously and
progressively
review the time-based media of progressive emails 130 by modifying access
protocols
such as POP, IIVIAC and the like. Together, these modifications enable the use
of
email addressing, email protocols, DNS and the existing email infrastructure
to
support real-time communication of time-based media.
CA 02746734 2011-06-13
WO 2010/087879 PCT/US2009/057893
IV. Late Binding of Recipient Addresses For Real-time Voice and Other
Time-based Media
[0071] In the context of communications, a recipient address can be described
as
"bound" when a valid delivery path through the network has been determined for
that
address. Conventional telephone calls over the PSTN are said to use "early
binding"
because the dialed phone number, the "recipient address" in this case, is used
to
establish some active path (i.e., a circuit connection) to the recipient
before any media
can be transmitted to the recipient. Only after the connection is made can the
caller
begin speaking and the media transmitted. Regardless if the call is placed to
one or
more telephone numbers, or the call is transferred a voice messaging system,
the
binding typically occurs before any words can be delivered. Since the binding
of the
recipient's address to an active destination on the network happens before any
transmission of media, it is said to be "early". In contrast, emails are said
to employ
"late" binding. A person may compose an email message and send it over a
network
without binding that message to the device on which the recipient will consume
it.
Instead, after the email is composed, the email address of the recipient is
used to route
the email to the recipient to be reviewed on a device and at a time of the
recipient's
choosing.
[0072] With the messages (as described with regard to Figures 4A, 4B and 5A-
5D) or
emails 130 described above, a user may address a recipient using their
globally
addressable email address and then immediately begin talking or generating
time-
based media. As described above, the DNS lookup to define the delivery route
is
performed immediately, as soon as the email address of the recipient is
defined. At
substantially the same time, any available time-based media is progressively
and
simultaneously transmitted across the network 12 to the recipient. Thus the
discovery
of an active delivery route and the progressive and simultaneous creation,
transmission and delivery of the time-based media occur at substantially the
same
time as the time-based media is created. In the event the actual delivery
route is
discovered after the creation of time-based media has started, then the media
may be
temporarily or persistently stored and then transmitted from storage once the
active
delivery route is defined. No network connection or circuit needs to be
established
before the user may start talking. The ability to progressively and
simultaneously
transmit the time-based media using DNS and the infrastructure of email
therefore
21
CA 02746734 2011-06-13
WO 2010/087879 PCT/US2009/057893
enables the late binding of recipient addresses for voice and other time-based
media in
a manner that previously was not possible.
V. Conversations
[0073] The messaging method and system as described (with regard to Figures 1-
3,
4A-4B and 5A-5D) is conducive for supporting conversations between sending and
receiving users. When two or more parties are conversing back and forth using
any of
the above-listed RVX protocols, such as VoIP, SIP, RTP, or Skype, then the
conversation may take place in the live near real-time mode. When the RVX
protocol
allows users to communicate time-based media in near-real-time, but does not
require
the recipient to review the time-based media in near real-time, such as with
the CTP
or synchronization protocols mentioned above, then the conversation may take
place
(i) in the near real-time mode; (ii) the time-shifted mode; or (iii)
seamlessly transition
between the two modes.
[0074] Reply messages may be routed in a number of different ways. For
example,
with the CTP and synchronization protocols, the globally addressable email
addresses
of the participants along with the DNS routing information may be embedded in
the
streaming media. When a reply is to be sent, the embedded address and routing
information is used for the reply message. Alternatively, messages may be
routed
using a conversation ID or other pointer included in the streaming media which
points
to the globally recognizable email addresses of the participants along with
the DNS
routing information. In yet another alternative, the participants may be
explicitly
addressed and a DNS lookup performed for the reply message.
[0075] The progressive email 130 embodiment described above can also be used
for
implementing conversations. When a conversation is initiated, an email 130 is
created
by the sender, at either the sending communication device 14 if running an
email
client or on a mail server on the network running an email client on behalf of
the
sender. As the media of the progressive email 130 is created, it is
progressively
transmitted to the recipient, using the routing defined by DNS. To reply, a
progressive
email 130 is created on behalf of the recipient, either on the recipient's
device 14 or
on a server running an email client on behalf of the recipient. The email
address of the
original sender is automatically inserted in the "To" field (or possibly the
CC and/or
BCC fields) of the return email 130 and the DNS lookup is performed. The media
associated with the return email may be transmitted using the streaming
feature of
22
CA 02746734 2011-06-13
WO 2010/087879 PCT/US2009/057893
SMTP, Microsoft Exchange, or another proprietary email protocol as soon as the
media is created. Recipients may simultaneously review the time-based media in
near
real-time as the media is progressively received at their email client.
[0076] Regardless of the embodiment, the "reply" function may be implemented
in a
variety ways. For example, the recipient may enter an explicit reply command
into
their communication device 14, such as by using a predefined voice or
keystroke
command, or entering a command through a touch screen. Alternatively, a reply
message or email may be generated automatically when the recipient begins
speaking
or generating other time-based media in response to an incoming message or
email
130. When a reply message is automatically created, the email address of the
original
sender is extracted from the incoming message and used for addressing the
reply
message.
[0077] In yet other embodiments, the RVX protocol used for sending and
receiving
the messages of a conversation between participants do not necessarily have to
be the
same. For example, one participant may send messages using one of the CTP,
synchronization, progressive emails, VoIP, SIP, RTP, or Skype protocols,
whereas
other participants may use a different one of the listed protocols, provided
some type
of a common conversation identifier is used. Any messages, regardless of the
protocol
used for transmission, are linked or threaded together using the unique
conversation
identifier.
[0078] In various further embodiments, conversations can be defined using a
variety
of criteria. For example, conversations may be defined by the name of a person
(e.g.,
mom, spouse, boss, etc) or common group of people (e.g., basketball team,
sales
team, poker buddies, etc). Conversations may also be defined by topic, such as
fantasy football league, ACME corporate account, or "skunk works" project.
Regardless of the contextual attribute used to define a conversation, the
ability to link
or organize the messages of a particular conversation together creates the
notion of a
persistent or ongoing conversation. With a conventional telephone call, the
conversation typically ends when the parties hang up. There is no way to
contextually
link, organize and possibly store the spoken words of multiple telephone
conversations between the same parties. On the contrary, conversations, as
defined
herein, are a set of common messages linked together by a common attribute. So
long
as messages are added to the conversation, the conversation is continuous or
ongoing.
23
CA 02746734 2011-06-13
WO 2010/087879 PCT/US2009/057893
This attribute makes it possible for a participant to contribute to a
conversation at any
arbitrary time. For example, a user may select a conversation among a list of
conversations and contribute a message to the selected conversation at
anytime. The
message is then sent to all the conversation participants. Messages are
therefore not
necessarily sent when either a conversation is first created or in reply to an
incoming
mes sage.
VI. Implementation Embodiments
[0079] The messaging methods as described with regard to Figures 1-3, 4A-4B
and
5A-5D and progressive emails 130 may be implemented in a variety of ways. For
example, cell phone and other mobile communication service providers may
provide
users with peer-to-peer mobile communication devices that operate using either
messages and/or progressive emails 130. In addition, these service providers
may also
maintain a network 12 of servers 16 for receiving messages and/or emails 130
from
non peer-to-peer communication devices, creating messages, performing DNS
lookup
operations and for routing the time-based media of messages using any one or
possibly multiple RVX protocols. In yet another embodiment, the messaging and
progressive email 130 methods may be embedded in a software application that
is
intended to be loaded into and executed on conventional telephones, mobile or
cellular telephones and radios, mobile, desktop and laptop computers. In each
of these
cases, the application enables the device to send, receive and process
messages and
progressive emails 130 as described herein. In yet other implementations, an
email
client can be modified to create, receive and process progressive emails 130.
The
email client may alternatively reside on a server on the Internet or other
network, on
sending or receiving devices, or both.
[0080] Although the above-described email methods were generally described in
the
context of a single sender and a single recipient (as discussed with regard to
Figures
4A-4B and 5A-5D) or emails 130 to a single recipient, it should be understood
the
messages and/or emails 130 might be simultaneously sent to multiple parties.
Each
recipient will either receive or not receive the message or email, depending
on their
status, as described above. As described in more detail in the above-mentioned
U.S.
applications, the media may be rendered using a number of different rendering
options, such as catch up to live, pause a live conversation, jump to the head
of a
conversation, jump to a previous point in time of the conversation, render
faster,
24
CA 02746734 2013-10-24
render slower, jump between different conversations, etc. The time-based media
exchanged by the messages and/or emails is not limited to just voice or video.
In
addition, the time-based media may be delivered to a recipient in a different
form than
it was created. For example, a voice message may be transcribed into a text
file or a
message in English may be translated into another language before being
delivered to
the recipient. Any media that varies over time, such as sensor data, GPS or
positional
information, may be transmitted. While the invention has been particularly
shown and
described with reference to specific embodiments thereof, it will be
understood by
those skilled in the art that changes in the form and details of the disclosed
embodiments may be made without departing from the scope of the invention. It
is
therefore intended that the invention be interpreted to include all variations
and
equivalents that fall within the true scope of the invention, as provided in
the attached
claims.
