Note: Descriptions are shown in the official language in which they were submitted.
CA 03026535 2018-12-04
MULTIMEDIA RESOURCE SYNCHRONOUS PUSHING METHOD BASED ON
HETEROGENEOUS NETWORK
BACKGROUND
Technical Field
The present invention relates to a heterogeneous media network transmission
method
in the field of information technologies, and in particular, to a method for
synchronously
pushing a multimedia resource based on a heterogeneous network.
Related Art
With the rapid development of network technologies, the media presentation
manner
has undergone earth-shaking changes, and new forms of media have emerged one
after
another. Traditional TV is no longer the main way for people to obtain
information and
entertainment. More terminal devices appear, such as PCs connected to the
Internet, mobile
phones belonging to almost everybody, and mobile tablet computers that are
becoming
increasingly popular. These new products have started to slowly erode the
market of
traditional TV services. The multimedia services, along with the development
of
ever-changing mobile communication and broadband wireless technologies, are
becoming
increasingly mature. Large-scale media convergence has become an inevitable
trend. In this
process, new ideas and high-end products are constantly appearing, enabling
users to
conveniently access the network, so as to enjoy richer media content and
diversified
services with ease.
At the same time, the presentation of media content is no longer only simple
audios,
audios, and subtitles, and media types will be increasingly diversified. Media
sources are no
longer only specific content providers, and a growing number of producers are
involved,
including many individual users who are also content providers and producers.
The content
from different providers has various associations. To meet the individualized
needs of
different users, the associated content usually needs to be synchronously
presented. In this
environment, heterogeneous network convergence is used as an inevitable trend
in the
development of next-generation networks, and this fully demonstrates that
future
1
CA 03026535 2018-12-04
communication is no longer a specific access technology, but instead, multiple
access
technologies coexist and work together.
In the environment of a heterogeneous media network, the media content
presented by
a terminal may be simultaneously transmitted from a plurality of transmission
channels, for
example, a broadcast channel and a broadband channel. Media resources of the
broadcast
channel have the characteristics of a short delay (basically neglected), a
large bandwidth
and high stability, and bandwidths and network delays greatly vary for
different broadband
channels (WIFI, 3G/4G, or the like).
Therefore, in a heterogeneous network media resource service formed by a
broadcast
and a broadband, due to the difference in the status of broadband channel
networks, there
are three cases of asynchronization: a server pushes media content to users
through
broadcast; if the users request for broadband resources of the same media
content, the delay
and bandwidth of the broadband network may cause the content received by the
client from
the broadcast and the broadband to be asynchronous on a timeline; the server
pushes the
same media content to users through broadcast and broadband; if the users
request to
replace a broadband resource media stream (for example, different view angles
or different
audios), when the media stream is switched, because different broadband
networks have
different delays and bandwidths, new bandwidth content and broadcast content
of the
terminal are asynchronous; the server pushes the same media content to users
through
broadcast and broadband; if the broadband network condition becomes poorer at
a
particular moment, the broadband channel play is not smooth, and when the
video is played
again smoothly, broadcast and broadband resources between different users are
asynchronous.
Therefore, how to enable the server to synchronously transmit a multimedia
resource
according to a network condition in a new-generation heterogeneous network
transmission
system has become a problem urgently needing to be resolved.
SUMMARY
The technical problem to be resolved by the present invention is to provide a
method
2
CA 03026535 2018-12-04
for synchronously pushing a multimedia resource based on a heterogeneous
network, to
implement a mechanism of synchronization between content of broadcast media
and
content of broadband media of a server in heterogeneous network transmission,
thereby
resolving the problem that user media resources of broadcast and broadband are
asynchronous caused by different network conditions in heterogeneous network
transmission.
The present invention resolves the foregoing technical problem by using the
following
technical solutions:
a method for synchronously pushing a multimedia resource based on a
heterogeneous
network, where the method is any one of the following two methods:
-method 1: including the following steps:
step 1, learning of, by a server, end-to-end broadband network delay
information of
different users;
step 2: after receiving a client request, calculating, by the server according
to the
end-to-end broadband network delay information of the different users, a
moment at which
the multimedia resource can be synchronously presented; and
step 3: sending, by the server, the multimedia resource after the moment at
which the
multimedia resource can be synchronously presented, and synchronously playing,
by a
client, content according to a start presentation time of the multimedia
resource, so that
broadband content of the different users can be synchronously presented;
-method 2: including the following steps:
step 1: learning of, by a server, broadband network delay and available
bandwidth
information of different users;
step 2: after receiving a client request, calculating, by the server according
to the
broadband network delay and available bandwidth information of the different
users, a
moment at which the multimedia resource can be synchronously presented; and
step 3: sending, by the server, the multimedia resource after the moment at
which the
3
CA 03026535 2018-12-04
multimedia resource can be synchronously presented, and notifying a client of
a play start
time of content of the multimedia resource, and synchronously presenting, by
different
clients, broadband content according to the play start time.
Further, in the step 2 of the method 1, when broadband network delays At of
the users
are different, the server sends, at a to moment, a multimedia resource at a
tin, moment to
ensure synchronization, where tin, is a start moment of a first complete
independent
decodable media unit MU after ttmp, where
0<tmi-ttmp<MU_duration
ttmp=to+At
in the formula, to is a sending moment, MU_duration is media unit duration, At
is a
downlink end-to-end broadband network delay, and ttmp is a moment that is
obtained by
calculation and at which the client receives the first independent decodable
media unit MU.
Furthermore, in the method 1,
a value of MU duration is: Os to 10s; and
a value of dt is: 0 to co.
More preferably, in the method 1,
the value of MU duration is: 0.5s; and
the value of At is: 2s.
Further, in the step 3 of the method 1, content initially sent by the server
is the
multimedia resource after the tin, moment, and to enable a broadband and a
broadcast to be
synchronous, after receiving the first complete MU by using the broadband, the
client does
not immediately play the first MU, but instead, plays, at the tm, moment, the
multimedia
resource at this moment.
Further, in the method, the server sends the multimedia resource to the
client, and the
client synchronously plays the content according to the start presentation
time of the media
resource.
4
CA 03026535 2018-12-04
Further, in the step 2 of the method 2, when the broadband network delays At
and
available bandwidths Bb of the users are different, the server sends, at a to
moment, a
multimedia resource at a tot; moment to ensure synchronization, where tim is a
start moment
of a first complete independent decodable media unit MU after ttmp, where
0<tim-tcmp<MU_duration
MU size=MU duration*Br
ttmp=to+MU_size/Bb+At
in the formula, to is a sending moment, MU_size is an average size of sent
MUs,
MU duration is media unit duration, Br is a constant bit rate at which the
multimedia
resource is played, Bb is an available bandwidth of a broadband network, At is
a downlink
broadband network delay, and ttmp is a moment that is obtained by calculation
and at which
the client receives a first independent decodable media unit MU.
Furthermore, in the method 2,
a value of MU_size is: 0 Mbits to 100 Mbits;
a value of MU duration is: Os to 10s;
a value of Br is: 0 Mbps to 50 Mbps;
a value of Bb is: 0 Tbps to 1 Tbps; and
a value of At is: 0 to co.
More preferably, in the method 2,
the value of MU_size is: 30 Mbits;
the value of MU duration is: 0.5s;
the value of Br is: 25 Mbps;
the value of Bb is: 1 Mbps; and
the value of At is: 2s.
Further, in the step 3 of the method 2, content initially sent by the server
is the
5
CA 03026535 2018-12-04
multimedia resource after the tin, moment, and to enable a broadband and a
broadcast to be
synchronous, after receiving the first complete MU by using the broadband, the
client does
not immediately play the first MU, but instead, plays, at the tm, moment, the
multimedia
resource at this moment.
Further, in the method 2, when sending the multimedia resource, the server
sends a
piece of downlink signaling to notify the client of a sequence number of a
sent first
independent decodable media resource to notify a user of a play time.
Further, the method 1 and the method 2 are applicable to a case in which at a
particular
moment, different users in a heterogeneous network media service request the
server for a
same multimedia resource or switch different multimedia resources or re-
request for a
multimedia resource due to network deterioration caused by a lot of
conditions, to make
content of the media resources synchronously presented.
The positive progress effect of the present invention is:
By using the foregoing method of the present invention, the server adopts the
method
for synchronously pushing a media resource or adaptively accessing a
synchronous media
resource for conditions of broadband networks of different users, to implement
a
mechanism of synchronously playing content of broadcast media and content of
broadband
media of the server in heterogeneous network transmission, thereby resolving
the problem
that media resources are asynchronous caused by different network conditions.
BRIEF DESCRIPTION OF THE DRAWINGS
FIG. 1 is a schematic timing diagram of Embodiment 1 of the present invention;
and
FIG. 2 is a schematic timing diagram of Embodiment 2 of the present invention.
DETAILED DESCRIPTION
The following provides preferred embodiments of the present invention with
reference
to the accompanying drawings, to describe the technical solutions of the
present invention
in detail.
Nowadays, diversified terminal presentation manners based on heterogeneous
6
CA 03026535 2018-12-04
networks have become a trend of development. When watching high-quality
broadcast
video programs, people's demands for diversified network media services are
also
increasing. Usually, media content from a broadcast channel has a small and
fixed delay,
and therefore it has little effect on the synchronization of media content.
Media content
from a broadband, for example, audios and videos, subtitles, and multimedia
applications is
susceptible to the effect of a current IP network, resulting in large and
jittery delays,
causing problems to content synchronization.
To resolve the foregoing technical problem, implementation details of the
technical
solutions are described in detail below by using two embodiments of the
present invention,
to facilitate understanding of a person skilled in the art. According to
different scenarios,
the method of any one of the following two embodiments may be adopted. If a
client can
learn of end-to-end network delay information, the method in Embodiment 1 is
adopted,
and if the client can learn of only a broadband network delay not including
the time of a
process when the media resource enters the client, the method in Embodiment 2
is adopted.
Embodiment 1
The overall technical line of this embodiment is: to learn of end-to-end
broadband
network delay information of different users by using a corresponding method
in a network,
for changing a start time at which a server starts to transmit a broadband
resource, thereby
implementing synchronization of a multimedia resource in a heterogeneous
network.
Specifically, method steps of this embodiment include:
First, a server may learn of end-to-end broadband network delay information of
different users according to some methods, for example, by using signaling
information fed
back by a client, or by means of estimation measurement.
Secondly, for a case in which at a particular moment, different users in a
heterogeneous
network media service request the server for a same multimedia resource or
switch
different multimedia resources or re-request for a multimedia resource due to
network
condition deterioration, to make content of the multimedia resources
synchronously
presented, after receiving a user request, the server calculates, according to
network delay
7
CA 03026535 2018-12-04
conditions of different users, a moment at which the multimedia resource can
be
synchronously presented.
Finally, the server sends the multimedia resource after the moment at which
the
multimedia resource can be synchronously presented, and the client
synchronously plays
content according to a play start time of the media content, so that broadband
part content
of the different users can be synchronously presented.
As shown in FIG. 1, in a specific embodiment, the specific implementation is
as
follows:
Step 1: it is known that a server receives a client request at a to moment, a
downlink
end-to-end broadband network delay of different user groups is At, and
duration of an
independent decodable media unit MU (media unit) is MU_duration.
Step 2: when end-to-end broadband network delays of the users are different,
the server
sends, at a to moment, a multimedia resource at a tin, moment to ensure
synchronization,
where tin, is a start moment of a first complete MU after ttnip, and as shown
in formula (1)
and formula (2):
0<tim4tmp<MU_duration (1)
ttmp=to+At (2)
In formula (1), MU_duration is media unit duration. In formula (2), to is a
sending
moment, At is a downlink end-to-end broadband network delay, and ttmp is a
moment that is
obtained by calculation and at which the client receives the first MU.
In a preferred implementation, the foregoing parameters can select the
following
values:
MU_duration: Os to 10s, preferably, 0.5s; and
At: 0 to co, preferably, 2s.
Step 3: because initially sent content is the multimedia source at and after
the tin,
moment, and to enable a broadband and a broadcast to be synchronous, after a
client
receives the first complete MU through the broadband, the first MU is not
immediately
8
CA 03026535 2018-12-04
played, but instead, the media content at this moment is played at and after
the tini moment.
A specific example of multimedia resource synchronization is provided below (a
manner of a client request and a manner of notifying the client by the server
are not limited,
and a signaling manner is used as an example below, and signaling is based on
the MMT
protocol).
When a user is watching an Olympic live program, because there are a large
number of
excellent performances, and people have different preferences, the program
party specially
shoots videos in a plurality of view angles for audience to select.
At this moment, when the audience watches a main program, the audience also
watches a close-up performance of a particular athlete in a small image.
However, the game
of the athlete is completed soon, and the audience wants to immediately change
an angle to
watch the posture of another athlete. Then the audience sends a request to the
server. The
request signaling should include an end-to-end network delay
fixed_end_to_end_delay.
This field can be obtained in HRBM message, as shown in the following Table
1.1:
Table 1.1
Syntax Values No. of hits
Mnemonic
HRBM ( ){
message_id 16
version 8
length 16
extension {
extension_fields_Byte
}
message_payload{
max buffer size 32
fixed _ _ end to _ end _delay 32
max transmission delay 32
}
9
CA 03026535 2018-12-04
message_id ¨ message identifier of HRBM
version ¨ message version of HRBM
length ¨ message length of HRBM
extension_fields_Byte ¨ extension field of HRBM
max buffer size ¨ maximum buffer size of HRBM
fixed_ end_ to _ end_ delay ¨ end-to-end network delay
max transmission_delay ¨ maximum transmission delay
At t0 18:00:00, the server receives the request, where
fixed_end_to_end_delay=At=2s.
Duration of an independent decodable media unit can be obtained from a video
resource,
where MU duration=0.5s.
At=2s
ttnip=t0+At=18:00:02
6c-18:00:022
The server sends, at to=18:00:00, a resource of tim=18:00:022, and a time at
which the
client receives a first media resource is 18:00:02, and according to timestamp
information
timestamp=18:00:022 of the received first media resource, a current media unit
is buffered,
and the resource is played at 18:00:022.
In this way, a user can watch the posture of another athlete in a different
view angle in
synchronization between a broadcast and a video on-demand, so as to well
resolve the
problem that user media resources of broadcast and broadband are asynchronous
caused by
different network conditions in heterogeneous network transmission.
Embodiment 2
The overall technical line used this embodiment is: to learn of information
such as
broadband network delays and available bandwidths of different users by using
a
CA 03026535 2018-12-04
corresponding method in a network, for changing a start time at which a server
starts to
transmit a broadband resource, thereby implementing synchronization of a
multimedia
resource in a heterogeneous network.
Specifically, method steps of this embodiment include:
First, a server may learn of information such as broadband network delays and
available bandwidths of different users according to some methods, for
example, by using
signaling information fed back by a client, or by means of estimation
measurement.
Secondly, for a case in which at a particular moment, different users in a
heterogeneous
network media service request the server for a same multimedia resource or
switch
different multimedia resources or re-request for a multimedia resource due to
network
condition deterioration, to make content of the multimedia resources
synchronously
presented, after receiving a user request, the server calculates, according to
network delay
conditions of different users, a moment at which the multimedia resource can
be
synchronously presented.
Finally, the server sends the resource after the moment at which the resource
can be
synchronously presented, notifies the client of a play start time of the media
content, so that
broadband part content of the different users can be synchronously presented.
As shown in FIG. 2, in a specific embodiment, the specific implementation is
as
follows:
Step 1: it is known that a server receives a client request at a to moment, a
downlink
broadband network delay of different user groups is At, an available bandwidth
is Bb, a
video bit rate is Br, and duration of an independent decodable media unit MU
is
MU_duration.
Step 2: when the broadband network delays and the available bandwidths of the
users
are different, the server sends, at a to moment, a multimedia resource at a
tin, moment to
ensure synchronization, where tin, is a start moment of a first complete MU
after tow, and as
shown in formula (1), formula (2), and formula (3);
0<tint4tmp<MU_duration (1)
11
CA 03026535 2018-12-04
MU_size=MU_duration*Br (2)
tt,w=to+MU_size/Bb+At (3)
In formula (3), to is a sending moment, MU_size is an average size of sent
MUs, and
see formula (2) for specific calculation, MU_duration is media unit duration,
Br is a
constant bit rate at which the multimedia resource is played, Bb is an
available bandwidth of
a broadband network, At is a downlink broadband network delay, and tow is a
moment that
is obtained by calculation and at which the client receives the first MU.
In a preferred implementation, the foregoing parameters can select the
following
values:
MU size: 0 Mbits to 100 Mbits, preferably, 30 Mbits;
MU duration: Os to 10s, preferably, 0.5s;
Br: 0 Mbps to 50 Mbps, preferably, 25 Mbps;
Bb: 0 Tbps to 1 Tbps, preferably, 1 Mbps; and
At: 0 to 00, preferably, 2s.
Step 3: because initially sent content is the multimedia source at and after
the tin,
moment, and to enable a broadband and a broadcast to be synchronous, after a
client
receives the first complete MU through the broadband, the first MU is not
immediately
played, but instead, the media content at this moment is played at and after
the too moment.
A specific example of multimedia resource synchronization is provided below. A
manner of a client request and a manner of notifying the client by the server
are not limited
in this embodiment. A signaling manner is used as an example below, and
signaling is
based on the MMT protocol. Certainly, in other embodiments, the signaling may
also be
applicable to other media protocols, which are not limited to the MMT
protocol.
When a user is watching an Olympic live program, because there are a large
number of
excellent performances, and people have different preferences, the program
party specially
shoots videos in a plurality of view angles for audience to select.
12
CA 03026535 2018-12-04
At this moment, when the audience watches a main program, the audience also
watches a close-up performance of a particular athlete in a small image.
However, the game
of the athlete is completed soon, and the audience wants to immediately change
an angle to
watch the posture of another athlete. Then the audience sends a request to the
server. The
request signaling should include a network delay network_delay and available
bandwidth
information network bandwidth. The request message request_message is shown in
the
following Table 2.1:
Table 2.1
Syntax Value No. of bits Mnemonic
request message() 1
message id 8 uimsbf
version 8 uimsbf
length 16 uimsbf
message_payloadO
network delay 16 uimsbf
network bandwidth 32 uimsbf
At to=18:00:00, the server receives the request, where At=2s and Bb=512
Kbit/s. The
video bit rate and duration of an independent decodable media unit can be
obtained from a
video resource, where Br-16482 Kbit/s and MU_duration=0.5s.
MU size=MU duration*Br=0.5*16482=8241 bits
AT=MU_size/Bb+At=8241/512+2=16.10s
ttrnp=to+A T=1800 :1610
61=18:00:1641
The server sends, at a to=18:00:00 moment, a resource of t1t-11=18:00:1641,
and when
sending the resource, the server needs to send a piece of downlink signaling
to notify the
client of a sequence number of a sent first independent decodable media
resource to notify
13
CA 03026535 2018-12-04
a user of a play time. A response message response_message is as follows:
Table 2.2
Syntax Value No. of bits Mnemonic
response_message() {
message id 8 uimsbf
version 8 uimsbf
length 16 uimsbf
message_payloadO
number of assets N 16 uimsbf
for(i=0; i<N; i++) 32 uimsbf
asset id 16 uimsbf
mpu sequence number 32 uimsbf
The client looks up in the table according to a first media resource sequence
known in
the signaling, to obtain time information timestamp=18:00:1641 and the size
MU_size=33
Mbits, and that a moment at which the current client receives the resource is
18:00:1610,
buffers the current media unit, and plays the resource at 18:00:1641.
In this way, a user can watch the posture of another athlete in a different
view angle in
synchronization between a broadcast and a video on-demand, so as to well
resolve the
problem that user media resources of broadcast and broadband are asynchronous
caused by
.. different network conditions in heterogeneous network transmission.
The foregoing specific embodiments further describe, in detail, the technical
problem
resolved by the present invention, and the technical solutions and beneficial
effects of the
present invention. It should be understood that the foregoing descriptions are
merely
specific embodiments of the present invention, and are not intended to limit
the present
invention. Any modification, equivalent replacement, improvement, and the like
made
within the spirit and principle of the present invention should fall within
the protection
14
CA 03026535 2018-12-04
scope of the present invention.
