Note: Descriptions are shown in the official language in which they were submitted.
MULTI-DATA-STREAM PROCESSING METHOD, APPARUTUS, COMPUTER
DEVICE AND STORAGE MEDIUM
Field
[0001] The present disclosure relates to the field of big data real-time
processing, particularly
to a multi-data-stream processing method, an apparatus, a computer device, and
a storage
medium.
Background
[0002] In the price control business system, the price is divided into two
kinds of order data:
sales amount and payment amount. For the price control, if only according to
the sales amount
control, the control effect will be low; if only according to actual amount
control, then the
control effect will be strict. To achieve the expected effect, it is necessary
to use the actual
payment amount of the order and the coupon information to calculate the actual
payment
amount for control.
[0003] Currently most general way is storing the multiple real-time data
streams to a
relational database or HDFS (Hadoop Distributed File System) first, then
providing business
with offline query function, obtaining the result set that needs to be
processed by association
query from plural tables in the relational database or HDFS, then passing the
result set to the
business system for processing.
[0004] However, the method based on offline technology for results query has
high delay, so
that the above method cannot guarantee the real-time data. Due to the
uncertainty of data entry
time of the multiple real-time data steams after processing, it is easy to
lead data loss during
association query, so the above method cannot guarantee the data reliability.
In addition, the
multiple interactive real-time data streams may not be worth saving or are not
necessarily
redundant in some types of databases. Therefore, the above method may occupy
too many
storage resources and resulting in low utilization of storage resources in the
system.
1
Date Recue/Date Received 2021-10-18
Invention Content
[0005] Based on the current technical problems, the present invention provides
a multi-data-
stream processing method, an apparatus, a computer device and a storage
medium, the
implementations of the present invention can perform multiple real-time data
stream processing
that obtaining more accurate data after processing and reducing the
occupancies of storage
resources.
[0006] The present invention provides a multi-data-stream processing method
according to
the first aspect, in an implementation, the method includes:
[0007] Continuously receiving first streaming data and second streaming data
from a
streaming processing platform, any first streaming data corresponds to zero
second streaming
data or at least one second streaming data, and any second streaming data
corresponds to one
first streaming data;
[0008] After receiving any first streaming data, wherein the first streaming
data has the
correspondingly second streaming data, if all the second streaming data
corresponding to the
first streaming data are received, performing association processing of any
first streaming data
with all the second streaming data corresponding to the first streaming data,
wherein the first
streaming data has the correspondingly second streaming data, when the
obtaining associated
data meets preset conditions after the association processing, storing any
first streaming data
into a database, wherein the first streaming data has the correspondingly
second streaming data;
[0009] After
receiving any second streaming data, if the target first streaming data
corresponding to the any second streaming data is received, and all the second
streaming data
corresponding to the target first streaming data are received, then performing
association
processing of the target first streaming data with all the second streaming
data corresponding
to the target first streaming data, when the obtaining associated data meets
preset conditions
after the association processing, storing the target first streaming data into
a database
2
Date Recue/Date Received 2021-10-18
[0010] In an implementation, before the step of performing association
processing of any
first streaming data with all the second streaming data corresponding to the
first streaming data,
wherein the first streaming data has the correspondingly second streaming
data, comprising
[0011] Obtaining a first data identification included in any first streaming
data, wherein the
first streaming data has the correspondingly second streaming data;
[0012] Determining a second data identification corresponding to the first
data identification,
the quantity of the second data identification corresponding to the first data
identification is N,
N is a positive integer greater than 0;
[0013] Generating a first data index according to the first data
identification;
[0014] Querying the second data identification corresponding to the first data
index in the
data storage middleware, the quantity of the second data identification
corresponding to the
first data index is M, M is a positive integer greater than 0;
[0015] If the second data identification corresponding to the first data
identification is the
same as the second data identification corresponding to the first data index,
determining of all
the second streaming data corresponding to any first streaming data are
received, wherein the
first streaming data has the correspondingly second streaming data; and
[0016] If the second data identification corresponding to the first data
identification is not the
same as the second data identification corresponding to the first data index,
detelliiining of all
the second streaming data corresponding to any first streaming data are not
all received,
wherein the first streaming data has the correspondingly second streaming
data.
[0017] In an implementation, after the step of determining of all the second
streaming data
corresponding to any first streaming data are not all received, wherein the
first streaming data
3
Date Recue/Date Received 2021-10-18
has the correspondingly second streaming data, comprising:
[0018] Determining a complement data between the second data identification
corresponding
to the first data identification and the second data identification
corresponding to the first data
index;
[0019] Generating a second data index according to the first data index;
[0020] Storing the second data index and the complement data into the data
storage
middleware.
[0021] The multi-data-stream processing method according to claim 1, before
the step of
performing association processing of the target first streaming data with all
the second
streaming data corresponding to the target first streaming data, comprising:
[0022] Determining the second data identification corresponding to any second
streaming
data, and a target first data identification corresponding to the second data
identification;
[0023] Generating a second data index according to the first data
identification corresponding
to the second data identification;
[0024] Querying the data storage middleware according to the second data
index;
[0025] If the second data identification corresponding to the second data
index exists in the
data storage middleware, determining of the target first streaming data
corresponding to any of
the second streaming data is received;
[0026] If the second data identification corresponding to the second data
index does not exist
in the data storage middleware, determining of the target first streaming data
corresponding to
any second streaming data is not received.
4
Date Recue/Date Received 2021-10-18
[0027] In an implementation, the method also includes:
[0028] If the second data identification corresponding to the second data
index exists in the
data storage middleware, determining whether the second data identification is
the same as the
second data identification corresponding to the second data index;
[0029] If the second data identification and the second data identification
corresponding to
the second data index are the same, determining of all the second streaming
data corresponding
to the target first streaming data are received;
[0030] If the second data identification and the second data identification
corresponding to
the second data index are not the same, determining of all the second
streaming data
corresponding to the target first streaming data are not all received;
[0031] Preferably, the method also comprises:
[0032] If the second data identification corresponding to the second data
index does not exist
in the data storage middleware, generating a first data index according to the
target first data
identification;
[0033] Storing the first data index and the second data identification
corresponding to any
second streaming data into the data storage middleware.
[0034] In an implementation, after the step of determining of all the second
streaming data
corresponding to the target first streaming data are not all received,
comprising:
[0035] According to the second data identification, updating the second data
identification
corresponding to the second data identification in the data storage
middleware.
Date Recue/Date Received 2021-10-18
[0036] In an implementation, performing association processing of any first
streaming data
with all the second streaming data corresponding to the first streaming data,
wherein the first
streaming data has the correspondingly second streaming data, comprising:
[0037] Determining a first amount of the first streaming data, wherein the
first streaming data
has the correspondingly second streaming data;
[0038] Determining a second amount corresponding to each second streaming
data;
[0039] Obtaining a third amount corresponding to any first streaming data
according to the
first amount and the second amount corresponding to each second streaming
data, wherein the
first streaming data has the correspondingly second streaming data, the third
amount is the
associated data after performing the association processing;
[0040] In an implementation, the steps of performing association processing of
the target first
streaming data with all the corresponding second streaming data, comprising:
[0041] Determining the first amount corresponding to the target first
streaming data;
[0042] Determining the second amount corresponding to each second streaming
data,
wherein the second streaming data corresponding to the target first streaming
data;
[0043] Obtaining the third amount corresponding to the target first streaming
data according
to the first amount corresponding to the target first streaming data and the
second amount
corresponding to each second streaming data, wherein the third amount is the
associated data
after performing the association processing;
[0044] The present invention provides an apparatus for the multi-data-
streaming processing,
wherein the apparatus comprises:
6
Date Recue/Date Received 2021-10-18
[0045] A streaming data receiving module configured to continuously receive
the first
streaming data and the second streaming data from a streaming processing
platform, any first
streaming data corresponds to zero or at least one second streaming data, any
second streaming
data corresponds to one first streaming data;
[0046] A first data processing module configured to after receiving any
receiving any first
streaming data, wherein the first streaming data has correspondingly second
streaming data, if
determining of all the corresponding second streaming data are received,
performing
association processing of any first streaming data with all the second
streaming data
corresponding to the first streaming data, wherein the first streaming data
has the
correspondingly second streaming data, when the obtaining associated data
meets preset
conditions after the association processing, storing any first streaming data
into a database,
wherein the first streaming data has the correspondingly second streaming
data;
[0047] A second data processing module configured to after receiving any
second streaming
data, if the target first streaming data corresponding to the any second
streaming data is
received, and all the second streaming data corresponding to the target first
streaming data are
received, then performing association processing of the target first streaming
data with all the
second streaming data corresponding to the target first streaming data, when
the obtaining
associated data meets preset conditions after the association processing,
storing the target first
streaming data into a database.
[0048] According to the third aspect, the present invention provides a
computer device
including a memory, a processor and a computer program stored in the memory
and ran on the
processor configured to achieve the steps of any above-mentioned methods when
the processor
executes the computer program.
[0049] According to the fourth aspect, the present invention provides a
computer readable
storage medium stored with a computer program configured to achieve the steps
of any above-
mentioned methods when processor executes the computer program.
7
Date Recue/Date Received 2021-10-18
[0050] In the implementation of present invention, the present server
continuously receives
first streaming data and second streaming data from a streaming processing
platform, any first
streaming data corresponds to zero second streaming data or at least one
second streaming data,
and any second streaming data corresponds to one first streaming data; after
receiving any first
streaming data, wherein the first streaming data has the correspondingly
second streaming data,
if all the second streaming data corresponding to the first streaming data are
received,
performing association processing of any first streaming data with all the
second streaming
data corresponding to the first streaming data, wherein the first streaming
data has the
correspondingly second streaming data, when the obtaining associated data
meets preset
conditions after the association processing, storing any first streaming data
into a database,
wherein the first streaming data has the correspondingly second streaming
data; and after
receiving any second streaming data, if the target first streaming data
corresponding to the any
second streaming data is received, and all the second streaming data
corresponding to the target
first streaming data are received, then performing association processing of
the target first
streaming data with all the second streaming data corresponding to the target
first streaming
data, when the obtaining associated data meets preset conditions after the
association
processing, storing the target first streaming data into a database. The
implementation of
present invention introduces a data storage middleware that achieves the
multiple real-time
data stream, which means dynamically merging the first data steam and the
second data stream,
comparing with the current technology which firstly storing the first data
steam and the second
data stream to the database and then obtaining the results by performing
processing of the first
data stream and the second data stream in the database, the present invention
method can
perform more real-time processing on multiple real-time data stream, and the
obtaining data
results can be more accurate that can reduce the storage resources.
Drawing Description
[0051] Figure 1 is an application environment diagram of a multi-data-stream
processing
method in an implementation;
8
Date Recue/Date Received 2021-10-18
[0052] Figure 2 is a process diagram of a multi-data-stream processing method
in an
implementation;
[0053] Figure 3 is a structural diagram of a multi-data-stream processing
apparatus in an
implementation;
[0054] Figure 4 is an internal structure of a computer device in an
implementation.
Specific implementation methods
[0055] In order to make application purposes, technical solutions, and
advantages clearer, the
present disclosure is further explained in detail with a particular embodiment
thereof, and with
reference to the drawings. It shall be appreciated that these descriptions are
only intended to be
illustrative, but not to limit the scope of the disclosure thereto.
[0056] An implementation of the present invention as shown in Figure 1 is the
application
environment of the multi-data-stream processing method.
[0057] Wherein, server 10 is a stream processing platform that possibly can be
implemented
by Kafka stream processing platform. Server 20 is docking server to the server
10, the server
20 will access the first data stream and the second data stream from the
server 10, the data of
the first data stream is the first streaming data, the data of the first data
steam is the second
streaming data, the first streaming data includes at least the first data
identification, the second
streaming data includes at least the second data identification. In some
application scenarios,
there is correspondence relations between the first streaming data from the
first data stream
and one or plural second streaming data from the second data stream. For
example, in an order's
price control scenario, the first data stream is order data, the second data
stream is promotional
data, in this scenario, user can purchase many kinds of products in one order,
this time the order
corresponds to a big order number, and each product in the order corresponds
to a small order
9
Date Recue/Date Received 2021-10-18
number (order line number), when user purchases the products, user can either
choose to not
use coupon or choose to use coupon (is able to use at least one coupon),
wherein, the promotion
data can include coupon number at least, which refers to the only data
identification of the
coupon used by user, if user does not use the coupon when purchasing products,
then there is
no need to perform processing on the order data corresponding to the products,
but filtering out
this order data, if user uses the coupon to purchase products, then the order
data of the products
includes order line number, coupon number, and coupon's amount corresponding
to the
coupon's number.
[0058] The server 20 performs association processing of any first streaming
data with all the
second streaming data corresponding to the first streaming data after
receiving any first
streaming data and all the second streaming data corresponding to the first
streaming data, then
obtaining the associated data, when the associated data meets preset
conditions, storing any
first streaming data into a database 40. Wherein the server 20 will firstly
store the first
streaming data and the second streaming data sent by the server 10 to data
storage middleware
30, then performing association processing of the first streaming data and all
the second
streaming data correspondingly to the first streaming data, obtaining the
associated results.
[0059] Wherein, the server 10 and the server 20 can be implemented as
independent server
or as a cluster server composed of plural servers. The server 20, the data
storage middleware
30 and the database 40 can be deployed on the same server or a cluster of
servers, they can also
respectively be deployed on an independent server or a cluster of servers. As
an example, the
Redis database can be used as data storage middleware 30, the MYSQL database
can be used
as database 40.
[0060] As shown in Figure 2, the implementation of the present invention
provides a multi-
data-stream processing method, the method includes following steps:
Date Recue/Date Received 2021-10-18
[0061] S110, continuously receiving first streaming data and second streaming
data from a
streaming processing platform, any first streaming data corresponds to zero
second streaming
data or at least one second streaming data, and any second streaming data
corresponds to one
first streaming data.
[0062] In the present implementation, the server 20 (hereinafter referred as
the server) will
access the first data stream and the second data stream of the streaming
processing platform,
then continuously receiving first streaming data and second streaming data
from a streaming
processing platform. Any first streaming data corresponds to zero second
streaming data or at
least one second streaming data, and any second streaming data corresponds to
one first
streaming data. Wherein, the first streaming data includes the first data
identification, the
second streaming data includes the second data identification, in a possible
implementation,
for the first streaming data which has correspondingly second data
identification, the first
streaming data not only has the first data identification but also has all
second data
identifications. In different application scenarios, the first data
identification and the second
identification can be different data, such as in the order price control
scenario, the first data
identification can be order number or the order line number, the second data
identification can
be coupon number.
[0063] For any first streaming data from the first data stream, if existing
the second streaming
data corresponding to the first streaming data in the second data stream, for
the any first
streaming data and the second streaming data corresponding to the first
streaming data, their
order of arriving at server is random. Which means the first streaming data
may arrive at the
server earlier than the second streaming data, and also may arrive at the
server later that all the
other second streaming data; if the first streaming data has correspondingly
two or more second
streaming data, the first streaming data may arrive at the server later than
parts of its
correspondingly second streaming data, but arriving at the server earlier that
the rest parts of
its correspondingly second streaming data, for example, the second streaming
data are B and
C corresponding to the first streaming data A, in A, B, C, B arrives at the
server earliest, then
11
Date Recue/Date Received 2021-10-18
A arrives at the server, and C arrives at the server at last. But the server
will perfoim association
processing only after any first streaming data and all its correspondingly
second streaming data
arrive at the server.
[0064] Considering the arriving order of the first streaming data and its
correspondingly
second streaming data to the server is hard to determine, and performing
association processing
only when both of the first streaming data and its correspondingly second
streaming data arrive
at the server, therefore, the processing logics are setting respectively to
the first streaming data
and the second streaming data, to achieve the association processing of both
of them, which
can improve the real-time performance of the first data stream and the second
data stream.
[0065] S120, after receiving any first streaming data, wherein the first
streaming data has the
correspondingly second streaming data, if all the second streaming data
corresponding to the
first streaming data are received, performing association processing of any
first streaming data
with all the second streaming data corresponding to the first streaming data,
wherein the first
streaming data has the correspondingly second streaming data, when the
obtaining associated
data meets preset conditions after the association processing, storing any
first streaming data
into a database, wherein the first streaming data has the correspondingly
second streaming data.
[0066] In the present implementation, after the server receives the first
streaming data,
judging whether the first streaming data has correspondingly second streaming
data. In a
possible implementation, judging method can be judging whether existing the
second data
identification except the first data identification in the first streaming
data.
[0067] The
procedure of the server judges whether all the second streaming data
corresponding to the first streaming data are received, as following:
[0068] Obtaining a first data identification included in any first streaming
data, wherein the
first streaming data has the correspondingly second streaming data;
12
Date Recue/Date Received 2021-10-18
[0069] Determining a second data identification corresponding to the first
data identification,
the quantity of the second data identification corresponding to the first data
identification is N,
N is a positive integer greater than 0;
[0070] Generating a first data index according to the first data
identification;
[0071] Querying the second data identification corresponding to the first data
index in the
data storage middleware, the quantity of the second data identification
corresponding to the
first data index is M, M is a positive integer greater than 0;
[0072] If the second data identification corresponding to the first data
identification is the
same as the second data identification corresponding to the first data index,
detellnining of all
the second streaming data corresponding to any first streaming data are
received, wherein the
first streaming data has the correspondingly second streaming data;
[0073] If the second data identification corresponding to the first data
identification is not the
same as the second data identification corresponding to the first data index,
detellnining of all
the second streaming data corresponding to any first streaming data are not
all received,
wherein the first streaming data has the correspondingly second streaming
data.
[0074] After determining of the received first streaming data has the
correspondingly second
streaming data, if the server determines all the second streaming data
corresponding to this first
streaming data are received, then performing association processing of this
first streaming data
with its all correspondingly second streaming data.
[0075] Wherein, if the second data identification corresponding to the first
data identification
and the second data identification corresponding to the first data index are
not the same, which
means only parts or none of the second streaming data corresponding to the
first streaming data
arrive at the server. Therefore, after all the second streaming data
corresponding to the first
streaming data arrive at the server, then performing association processing of
both data.
13
Date Recue/Date Received 2021-10-18
[0076] To perform the association processing in time when the unarrived second
streaming
data corresponding to the first streaming data arrive at the server, if the
server determines that
the second streaming data corresponding to the first streaming data are not
all received, wherein
the first streaming data has correspondingly second streaming data, then
performing following
steps:
[0077] Determining a complement data between the second data identification
corresponding
to the first data identification and the second data identification
corresponding to the first data
index;
[0078] Generating a second data index according to the first data index;
[0079] Storing the second data index and the complement data into the data
storage
middleware.
[0080] Wherein, the first data index and the second data index are not the
same; the
complement data is the second data identification excluded from the second
data identification
corresponding to the first data index in the second data identification
corresponding to this first
data identification. For example, the second data identifications
corresponding to the first data
identification are id 1, id 2, the second data identification corresponding to
the first data index
is id 1, then determining the complement data between the second data
identification
corresponding to the first data identification and the second data
identification corresponding
to the first data index is id 2.
[0081] Specifically, when storing the second data index and the complement
data to the data
storage middleware, adopting a form of key-value to store the second data
index and the
complement data to the data storage middleware, wherein, the second data index
is key, the
complement data is value. If the complement data includes more than two second
data
identifications, using comma to separate each second data identification.
14
Date Recue/Date Received 2021-10-18
[0082] S130, after receiving any second streaming data, if the target first
streaming data
corresponding to the any second streaming data is received, and all the second
streaming data
corresponding to the target first streaming data are received, then performing
association
processing of the target first streaming data with all the second streaming
data corresponding
to the target first streaming data, when the obtaining associated data meets
preset conditions
after the association processing, storing the target first streaming data into
a database.
[0083] In the present implementation, after the server receives the second
streaming data,
then judging whether its correspondingly first streaming data (hereinafter
referred as the target
first streaming data) has been received and whether all the second streaming
data corresponding
to the target first streaming data has been received.
[0084] In an implementation, judging whether the target first streaming data
corresponding
to the second streaming data has been received, the process is as follows:
[0085] Determining the second data identification corresponding to any second
streaming
data, and a target first data identification corresponding to the second data
identification;
[0086] Generating a second data index according to the first data
identification corresponding
to the second data identification;
[0087] Querying the data storage middleware according to the second data
index;
[0088] If the second data identification corresponding to the second data
index exists in the
data storage middleware, determining of the target first streaming data
corresponding to any of
the second streaming data is received;
[0089] If the second data identification corresponding to the second data
index does not exist
in the data storage middleware, determining of the target first streaming data
corresponding to
Date Recue/Date Received 2021-10-18
any second streaming data is not received.
[0090] In the present implementation, a second streaming data includes a
second data
identification and the first data identification corresponding to this second
identification (the
target first data identification), therefore, after the server receives the
second streaming data,
obtaining the second data identification and the target first data
identification from the second
streaming data. Possibly, the second streaming data does not include the
target first data
identification, but the server or the data storage middleware maintain each
first data
identification in advance and the mapping relations between each first data
identification and
its correspondingly second identification, therefore, after the server obtains
the second
identification from the second streaming data, then obtaining the target first
data identification
corresponding to the second data identification through the mapping relations.
In the other
implementations, a second streaming data can conclude plural second data
identifications.
[0091] After obtaining the second data identification and its target first
data identification,
the server generates the second data index according to the target first data
identification, then
querying the data storage middleware through the generated second data index.
[0092] According to the above-mentioned, if the target first streaming data
arrives at server
earlier than the second streaming data, then the server will generate the
second data index based
on the first data identification included in the first streaming data, and
storing this second data
index and all the second data identification included in the first streaming
data (such as storing
in a key-value form) to the data storage middleware, therefore, if the first
streaming data
corresponding to the second streaming data has arrived at the server in
advance, then the second
data index generated from the present implementation can query relative
results from the data
storage middleware; Otherwise, if the first streaming data corresponding to
the second
streaming data has not arrived at the server, then querying data storage
middleware through the
second data index will not get any results, because this data storage
middleware does not use
the second data index (such as key) to store the data's records at this time.
Therefore, judging
whether the target first streaming data is received or not through querying
the data storage
16
Date Recue/Date Received 2021-10-18
middleware by the second data index.
[0093] In an implementation, if the second data identification corresponding
to the second
data index does not exist in the data storage middleware, the server performs
following steps:
[0094] Generating the first data index according to the target first data
identification;
[0095] Storing the first data index and the second data identification
corresponding to any
second streaming data to the data storage middleware.
[0096] In the present implementation, according to the above-mentioned, if the
data storage
middleware does not exist the second data identification corresponding to the
second data index,
which means the first streaming data (the target first streaming data)
corresponding to the
second streaming data received by the server has not arrived, so the server
can firstly determine
the first data identification (the target first data identification)
corresponding to the second data
identification included in the second streaming data, then generating the
first data index
according to the target first data identification, then storing this first
data index and the second
data identification included in the second streaming data (such as storing in
a key-value form)
to the data storage middleware, and when the target first streaming data
arrives, the server can
perform relative processing.
[0097] In another hand, because the server can perform association processing
after all the
second streaming data corresponding to the target first streaming data arrive
at the server,
considering that the target first streaming data may correspond to plural
second streaming data,
therefore, the second streaming data received by the present server may not be
the last one
which arrives at the server in this plural second streaming data, or still
existing parts of second
streaming data corresponding to the target first streaming data will arrive at
server after the
current second streaming data arrives, therefore, if the present server
queries data storage
middleware by the current second streaming data to obtain the relative results
(relative second
data identification), then further judging if all the second streaming data
corresponding to the
17
Date Recue/Date Received 2021-10-18
target first streaming data have been received.
[0098] In an implementation, the process of judging whether all the second
streaming data
corresponding to the target first streaming data are all received, as
following:
[0099] If the second data identification corresponding to the second data
index exists in the
data storage middleware, determining whether the second data identification
and the second
data identification corresponding to the second data index are the same;
[0100] If the second data identification and the second data identification
corresponding to
the second data index are the same, determining of all the second streaming
data corresponding
to the target first streaming data are received;
[0101] If the second data identification and the second data identification
corresponding to
the second data index are not the same, determining of all the second
streaming data
corresponding to the target first streaming data are not all received;
[0102] In the present implementation, according to the above-mentioned, in the
data storage
middleware, the second data index generated by the first data identification
of the target first
streaming data corresponds to all the second data identification included in
the target first
streaming data, to quickly determine if this second streaming data are the
second streaming
data corresponding to the target first streaming data or the last received
second streaming data
when receiving the second streaming data, in the present implementation, the
server queries
the relative second data identification from the data storage middleware
according to the second
data index, comparing all the queried second data identifications with the
second data
identification included in the received second streaming data, if the all the
queried second data
identifications and the second data identification included in the received
second streaming
data are not the same, then determining of all the second streaming data
corresponding to the
target first streaming data are not all received, at this time, then updating
the results (the second
data identification corresponding to the second data index) corresponding to
the second data
18
Date Recue/Date Received 2021-10-18
index in the data storage middleware, specifically, deleting the results
corresponding to the
second data index from the second data identification included in the received
second streaming
data in the data storage middleware. For example, the second data
identifications corresponding
to the second data index in the data storage middleware are id 1, id 2, but
the second data
identification included in the received second streaming data is id 1, and
these data
identifications are different after comparison, then updating the result
corresponding to the
second index from (id 1, id 2) to (id 2). However, if all the queried second
data identifications
and the second identification included in the received second streaming data
are the same,
determining of all the queried second data identification corresponding to the
target first
streaming data are all received, and then the server can perform association
processing of the
target first streaming data with all its correspondingly second streaming
data.
[0103] In an implementation, in step S120, performing association processing
of any first
streaming data with all the second streaming data corresponding to the first
streaming data,
wherein the first streaming data has the correspondingly second streaming
data, comprising:
[0104] Determining a first amount of the first streaming data, wherein the
first streaming data
has the correspondingly second streaming data;
[0105] Determining a second amount corresponding to each second streaming
data;
[0106] Obtaining a third amount corresponding to any first streaming data
according to the
first amount and the second amount corresponding to each second streaming
data, wherein the
first streaming data has the correspondingly second streaming data, the third
amount is the
associated data after performing the association processing.
[0107] In an implementation, in step S130, performing association processing
of the target
first streaming data with all the corresponding second streaming data,
comprising:
[0108] Determining the first amount corresponding to the target first
streaming data;
19
Date Recue/Date Received 2021-10-18
[0109] Determining the second amount corresponding to each second streaming
data,
wherein the second streaming data corresponding to the target first streaming
data;
[0110] Obtaining the third amount corresponding to the target first streaming
data according
to the first amount corresponding to the target first streaming data and the
second amount
corresponding to each second streaming data, wherein the third amount is the
associated data
after performing the association processing;
[0111] In the above-mentioned two implementations, the server performs similar
association
processing on the target first streaming data (has correspondingly second
streaming data) and
its corresponding second streaming data.
[0112] For
example, after the server receives a first streaming data which has
correspondingly second streaming data, if determining of all the second
streaming data
corresponding to this first streaming data are all received, or after
receiving a second streaming
data, if determining of the first streaming data corresponding to this second
streaming data is
received and all the second streaming data corresponding to this first
streaming data are all
received, then the server determines the first amount corresponding to the
first streaming data
and the second amount of each second streaming data corresponding to this
first streaming data,
obtaining the third amount corresponding to the target first streaming data
according to
calculation of the first amount corresponding to the target first streaming
data and the second
amount corresponding to each second streaming data.
[0113] Besides, after association processing, and/or when all the second
streaming data
corresponding to the first streaming data have arrived, deleting the first
streaming data and all
its correspondingly second streaming data in the data storage middleware to
avoid the waste of
storage space. Furthermore, after association processing, and/or when all the
second streaming
data corresponding to the first streaming data have arrived, deleting the
first data index and
second data index.
Date Recue/Date Received 2021-10-18
[0114] The following is a supportive illustration of the present invention's
implementation
through an order price control scenario.
[0115] In this scenario, the first data stream is order data stream, the first
streaming data is
order data that at least includes order number or order line number, if user
uses coupon to
purchase this order, then the order data also includes the coupon number and
the coupon
amount corresponding to his coupon number; the second data stream is promotion
data stream,
the second streaming data is promotion data that at least includes coupon
number. The server
access order data stream and promotion data stream through the stream
processing platform
Kafka, if any i orders of data and their correspondingly promotion data have
arrived, the server
performs association processing on both, but any orders of data and their
correspondingly
promotion data arrive at server randomly, therefore, there may be several
kinds of situations:
1, the orders data arrive at server earlier than the promotion data; 2, the
orders data arrive at
server later than the promotion data; 3, when the orders data correspond to
plural promotion
data, the orders data arrive at server earlier than parts of the
correspondingly promotion data.
[0116] Assuming that the order number (or order line number) included in
orders data is order
A, coupon numbers are coupon 1, coupon 2; the promotion data corresponding to
this orders
data is the first promotion data including the coupon number coupon 1 and the
second
promotion data including the coupon number coupon 2,
[0117] For the above-mentioned cases, the execution process can be as follows:
[0118] Assuming that the server receives the orders data first, the server
will store the orders
data to the data storage middleware Redis and obtain the orders data's first
data identification
(order number) orderA, because this orders data include coupon number coupon
1, coupon 2,
the promotion data corresponding to this orders data can be determined,
furthermore, the first
data index prepare operate coupon orderA can be generated based on order A,
then querying
data storage middleware Redis through the first data index, because the orders
data arrive at
21
Date Recue/Date Received 2021-10-18
first, the query will not have any results at this time, so the server
generates the second data
index based on the first data identification, such as operating coupon orderA,
and storing the
second data index and coupon number in form of key-value to the Redis, wherein
separating
each coupon number by comma, such as (operating coupon orderA couponl,
coupon2).
[0119] When the promotion data corresponding to the orders data arrived, the
server obtains
the second data identification from the received promotion data, if the first
promotion data
arrives, then storing the first promotion data to the Redis and obtaining its
second data
identification coupon 1 and its correspondingly first data identification
orderA, the server will
generate the second data index operating coupon orderA according to the first
data
identification orderA, and querying the second data identifications
corresponding to the second
data index by querying Redis which are couponl and coupon2, due to the
difference between
this promotion data's second data identification couponl and the second data
identifications
coupon 1 and coupon 2 corresponding to the second data index, which means this
promotion
data is not the last arriving promotion data of this orders data, therefore,
updating the value
corresponding to the second data index from couponl, coupon 2 to coupon 2 in
Redis.
Furthermore, after second promotion data arrives, and after the above-
mentioned process which
can determine of all the promotion data corresponding to this orders data have
all arrived at the
server, then performing association processing of this orders data with their
correspondingly
promotion data which are the first promotion data and the second promotion
data.
[0120] Except the coupon number included in each promotion data, activity
information is
also included, the server can judge (specific judge method is not limited in
the present
implementation) whether this coupon number is valid by this active information
(can include
activity status, collection method, activity times, etc.). Determining the
order amount (or
payment amount, which is the first amount) corresponding to the order number
and the coupon
amount (the second amount) corresponding to the coupon number by the pre-
stored orders data,
then summing up the order amount and the valid coupon amount of the coupon
number to
obtain the final amount (the third amount), comparing the final amount with
the pre-set
threshold, if the comparing result meets the pre-set condition, then
determining of the order
22
Date Recue/Date Received 2021-10-18
corresponding to the present order number is abnormal order, storing the order
data
corresponding to this order number to the database for the follow-up report
displaying and/or
alarm processing later (for example, sending alarm information to responsible
personnel, etc.)
[0121] For the above-mentioned two cases, the server's execution process can
be as follows:
[0122] Assuming that the server receives the first promotion data (can be the
second
promotion data) at first, the server will store the first promotion data to
data storage middleware
Redis and obtain the first promotion data's second data identification coupon
1 and its
correspondingly first data identification orderA, then the second data index
operating coupon orderA can be generated based on order A, then querying data
storage
middleware Redis through the second data index, because the orders data have
not arrived yet,
the query from Redis will not have any results at this time, so the server
generates the first data
index prepare operate coupon orderA based on the first data identification,
and storing the
first data index and coupon number couponl in form of key-value to the Redis,
such as
(prepare operate coupon orderA-coupon1).
[0123] Later when the second promotion data arrived, the server executes the
above-
mentioned process which can determine that the orders data is not arrived yet,
this can
incrementally update the value corresponding to the first data index, for
example the result is
(prepare operate coupon orderA couponl, coupon2).
[0124] At last when the orders data arrived, the server obtains the first data
identification
orderA from the orders data, and generates the first data index
prepare operate coupon orderA based on orderA, and querying the data storage
middleware
Redis by the first data index, obtaining coupon number couponl and coupon2 at
this time, the
queried coupon number and the coupon number included in the orders data are
the same, which
means all the promotion data corresponding to this orders data have arrived,
therefore,
performing association processing, the association processing is as above-
mentioned and will
not be repeated here.
23
Date Recue/Date Received 2021-10-18
[0125] For the third case in the above-mentioned, the server's execution
process can be as
follows:
[0126] Assuming the server receives the first promotion data at first, the
server will store the
first promotion data to data storage middleware Redis and obtain the first
promotion data's
second data identification couponl and its correspondingly first data
identification orderA, then
the second data index operating coupon orderA can be generated based on order
A, then
querying data storage middleware Redis through the second data index, because
the orders data
have not arrived yet, the query from Redis will not have any results at this
time, so the server
generates the first data index prepare operate coupon orderA based on the
first data
identification, and storing the first data index and coupon number couponl in
form of key-
value to the Redis, such as (prepare operate coupon orderA-coupon1).
[0127] Later when server receives the orders data, storing the orders data to
Redis and
obtaining their first data identification orderA and coupon numbers couponl
and coupon2, then
generating the first data index prepare operate coupon orderA based on the
first data
identification orderA, obtaining the coupon number couponl by querying the
Redis through
the first data index, due to the difference between the second data
identification included in the
orders data and the second data identification corresponding to the first data
index, which
means the other promotion data have not arrived yet, therefore, the sever
obtains the
complement data between the second data identifications (coupon number couponl
and
coupon2) included in the orders data and the second data identification
(coupon number
coupon 1) corresponding to the first data index, which can obtain the coupon
number coupon2,
then generating the second data index operating coupon orderA based on the
first data
identification orderA, and storing the second data index and coupon number
coupon2 in form
of key-value to the Redis, such as (operating coupon orderA-coupon2).
[0128] When the second promotion data arrived, the server obtains the second
data
identifications from the second promotion data which are coupon number coupon2
and its
24
Date Recue/Date Received 2021-10-18
correspondingly first data identification orderA, generating the second data
index
operating coupon orderA based on the first data identification, obtaining the
second data
identification corresponding to the second data index which is coupon number
coupon2 by
querying the data storage middleware Redis, at this time coupon number coupon2
included in
the second promotion data and the coupon number of the second data
identification
corresponding to the second data are the same, which means all the promotion
data
corresponding to the orders data have arrived, therefore, performing
association processing,
the association processing is as above-mentioned and will not be repeated
here.
[0129] In an implementation, as shown in Figure 4, a multi-data-stream
processing apparatus
is provided, including following modules:
[0130] A streaming data receiving module 110 configured to continuously
receive the first
streaming data and the second streaming data from a streaming processing
platform, any first
streaming data corresponds to zero or at least one second streaming data, any
second streaming
data corresponds to one first streaming data;
[0131] A first data processing module 120 configured to after receiving any
receiving any
first streaming data, wherein the first streaming data has correspondingly
second streaming
data, if determining of all the corresponding second streaming data are
received, performing
association processing of any first streaming data with all the second
streaming data
corresponding to the first streaming data, wherein the first streaming data
has the
correspondingly second streaming data, when the obtaining associated data
meets preset
conditions after the association processing, storing any first streaming data
into a database,
wherein the first streaming data has the correspondingly second streaming
data;
[0132] A second data processing module 130 configured to after receiving any
second
streaming data, if the target first streaming data corresponding to the any
second streaming data
is received, and all the second streaming data corresponding to the target
first streaming data
are received, then performing association processing of the target first
streaming data with all
Date Recue/Date Received 2021-10-18
the second streaming data corresponding to the target first streaming data,
when the obtaining
associated data meets preset conditions after the association processing,
storing the target first
streaming data into a database.
[0133] In an implementation, a first data processing module includes following
sub modules:
[0134] A
first identification obtaining sub module configured to obtain a first data
identification included in any first streaming data, wherein the first
streaming data has the
correspondingly second streaming data;
[0135] A second identification determination sub module configured to
determine a second
data identification corresponding to the first data identification, the
quantity of the second data
identification corresponding to the first data identification is N, N is a
positive integer greater
than 0;
[0136] A first index generation sub module configured to generate a first data
index
according to the first data identification;
[0137] A query sub module configured to query the second data identification
corresponding
to the first data index in the data storage middleware, the quantity of the
second data
identification corresponding to the first data index is M, M is a positive
integer greater than 0;
[0138] A determination sub module configured to when the second data
identification
corresponding to the first data identification is the same as the second data
identification
corresponding to the first data index, determining of all the second streaming
data
corresponding to any first streaming data are received, wherein the first
streaming data has the
correspondingly second streaming data; when the second data identification
corresponding to
the first data identification is not the same as the second data
identification corresponding to
the first data index, determining of all the second streaming data
corresponding to any first
streaming data are not all received, wherein the first streaming data has the
correspondingly
26
Date Recue/Date Received 2021-10-18
second streaming data.
[0139] In an implementation, the first data processing module also includes
following sub
modules:
[0140] A complement data determination sub module configured to determine a
complement
data between the second data identification corresponding to the first data
identification and
the second data identification corresponding to the first data index;
[0141] A second index generation sub module configured to generate a second
data index
according to the first data index;
[0142] A storage sub module configured to store the second data index and the
complement
data into the data storage middleware.
[0143] In an implementation, the second data processing module includes
following sub
modules:
[0144] A data identification determination sub module configured to determine
the second
data identification corresponding to any second streaming data, and a target
first data
identification corresponding to the second data identification;
[0145] A second index generation sub module configured to generate a second
data index
according to the first data identification corresponding to the second data
identification;
[0146] A query sub module configured to query the data storage middleware
according to
the second data index;
[0147] A first determination sub module configured to when the second data
identification
27
Date Recue/Date Received 2021-10-18
corresponding to the second data index exists in the data storage middleware,
determining of
the target first streaming data corresponding to any of the second streaming
data is received;
and when the second data identification corresponding to the second data index
does not exist
in the data storage middleware, determining of the target first streaming data
corresponding to
any second streaming data is not received.
[0148] In an implementation, the second data processing module also includes:
[0149] A
judging sub module configured to when the second data identification
corresponding to the second data index exists in the data storage middleware,
determining
whether the second data identification is the same as the second data
identification
corresponding to the second data index;
[0150] A second determination sub module configured to when the second data
identification
and the second data identification corresponding to the second data index are
the same,
determining of all the second streaming data corresponding to the target first
streaming data
are received; and when the second data identification and the second data
identification
corresponding to the second data index are not the same, determining of all
the second
streaming data corresponding to the target first streaming data are not all
received;
[0151] In an implementation, the second data processing module also includes:
[0152] A first index generation sub module configured to when the second data
identification
corresponding to the second data index does not exist in the data storage
middleware,
generating a first data index according to the target first data
identification;
[0153] A storage sub module configured to store the first data index and the
second data
identification corresponding to any second streaming data into the data
storage middleware.
[0154] The second data processing module also includes:
28
Date Recue/Date Received 2021-10-18
[0155] A data identification updating sub module configured to update the
second data
identification corresponding to the second data identification in the data
storage middleware
according to the second data identification.
[0156] In an implementation, the first data processing module also includes:
[0157] A first amount determination sub module configured to determine a first
amount of
the first streaming data, wherein the first streaming data has the
correspondingly second
streaming data;
[0158] A second amount determination sub module configured to determine a
second amount
corresponding to each second streaming data;
[0159] A
association processing sub module configured to obtain a third amount
corresponding to any first streaming data according to the first amount and
the second amount
corresponding to each second streaming data, wherein the first streaming data
has the
correspondingly second streaming data, the third amount is the associated data
after performing
the association processing;
[0160] In an implementation, the second data processing module also includes:
[0161] A first amount determination sub module configured to determine the
first amount
corresponding to the target first streaming data;
[0162] A second amount determination sub module configured to determine the
second
amount corresponding to each second streaming data, wherein the second
streaming data
corresponding to the target first streaming data;
[0163] A association processing sub module configured to obtain the third
amount
29
Date Recue/Date Received 2021-10-18
corresponding to the target first streaming data according to the first amount
corresponding to
the target first streaming data and the second amount corresponding to each
second streaming
data, wherein the third amount is the associated data after performing the
association
processing;
[0164] For the specific limitation of the multi-data-stream processing
apparatus can refer to
the above-mentioned the multi-data-stream processing method, which will not be
repeated here.
Each module of the above data cache apparatus can be achieved fully or partly
by software,
hardware, and their combinations. The above modules can be embedded in the
processor or
independent of the processor in computer device and can store in the memory of
computer
device in form of software, so that the processor can call and execute the
operations
corresponding to the above modules.
[0165] In an implementation, a computer device is provided to be a server and
whose internal
structure diagram is shown in Figure 4. The computer device includes a
processor, a memory,
a network interface, and a database connected through a system bus. The
processor of the
computer device is configured to provide calculation and control capabilities.
The memory of
computer device includes non-volatile storage medium and internal memory. The
memory of
non-volatile storage medium has operation system, computer programs and
database. The
internal memory provides an environment for the operation system and computer
program
running in a non-volatile storage medium. The network interface of the
computer device is
used to communicate with an external terminal through a network connection.
The computer
program is executed by the processor to implement a multi-data-stream
processing method.
[0166] The skilled in the art can understand that the structure shown in
Figure 4 is only partial
structural diagram related this application solution and not constitute
limitation to the computer
device applied on the current application solution, the specific computer
device can include
more or less components than what is shown in the figure, or combinations of
some
components or different components to what is shown in the figure.
Date Recue/Date Received 2021-10-18
[0167] In an implementation, a computer device is provided which includes a
memory, a
processor, and a computer program stored on the memory and running on the
processor. The
processor performs the following steps when executing the computer program:
[0168] Continuously receiving first streaming data and second streaming data
from a
streaming processing platform, any first streaming data corresponds to zero
second streaming
data or at least one second streaming data, and any second streaming data
corresponds to one
first streaming data; after receiving any first streaming data, wherein the
first streaming data
has the correspondingly second streaming data, if all the second streaming
data corresponding
to the first streaming data are received, performing association processing of
any first streaming
data with all the second streaming data corresponding to the first streaming
data, wherein the
first streaming data has the correspondingly second streaming data, when the
obtaining
associated data meets preset conditions after the association processing,
storing any first
streaming data into a database, wherein the first streaming data has the
correspondingly second
streaming data; and after receiving any second streaming data, if the target
first streaming data
corresponding to the any second streaming data is received, and all the second
streaming data
corresponding to the target first streaming data are received, then performing
association
processing of the target first streaming data with all the second streaming
data corresponding
to the target first streaming data, when the obtaining associated data meets
preset conditions
after the association processing, storing the target first streaming data into
a database.
[0169] In an implementation, the processor executes the computer program,
before achieving
the step of performing association processing of any first streaming data with
all the second
streaming data corresponding to the first streaming data, wherein the first
streaming data has
the correspondingly second streaming data, also achieving following steps:
[0170] Obtaining a first data identification included in any first streaming
data, wherein the
first streaming data has the correspondingly second streaming data;
determining a second data
identification corresponding to the first data identification, the quantity of
the second data
identification corresponding to the first data identification is N, N is a
positive integer greater
31
Date Recue/Date Received 2021-10-18
than 0; generating a first data index according to the first data
identification; querying the
second data identification corresponding to the first data index in the data
storage middleware,
the quantity of the second data identification corresponding to the first data
index is M, M is a
positive integer greater than 0; if the second data identification
corresponding to the first data
identification is the same as the second data identification corresponding to
the first data index,
determining of all the second streaming data corresponding to any first
streaming data are
received, wherein the first streaming data has the correspondingly second
streaming data; and
if the second data identification corresponding to the first data
identification is not the same as
the second data identification corresponding to the first data index,
determining of all the
second streaming data corresponding to any first streaming data are not all
received, wherein
the first streaming data has the correspondingly second streaming data.
[0171] In an implementation, the processor executes the computer program,
after achieving
the step of determining of all the second streaming data corresponding to any
first streaming
data are not all received, wherein the first streaming data has the
correspondingly second
streaming data, also achieving following steps:
[0172] Determining a complement data between the second data identification
corresponding
to the first data identification and the second data identification
corresponding to the first data
index; Generating a second data index according to the first data index; and
storing the second
data index and the complement data into the data storage middleware.
[0173] In an implementation, the processor executes the computer program,
before achieving
the step of performing association processing of the target first streaming
data with all the
second streaming data corresponding to the target first streaming data, also
achieving following
steps:
[0174] Determining the second data identification corresponding to any second
streaming
data, and a target first data identification corresponding to the second data
identification;
generating a second data index according to the first data identification
corresponding to the
32
Date Recue/Date Received 2021-10-18
second data identification; querying the data storage middleware according to
the second data
index; if the second data identification corresponding to the second data
index exists in the data
storage middleware, determining of the target first streaming data
corresponding to any of the
second streaming data is received; and if the second data identification
corresponding to the
second data index does not exist in the data storage middleware, determining
of the target first
streaming data corresponding to any second streaming data is not received.
[0175] In an implementation, the processor executes the computer program to
also achieve
following steps:
[0176] If the second data identification corresponding to the second data
index exists in the
data storage middleware, determining whether the second data identification is
the same as the
second data identification corresponding to the second data index; if the
second data
identification and the second data identification corresponding to the second
data index are the
same, determining of all the second streaming data corresponding to the target
first streaming
data are received; and if the second data identification and the second data
identification
corresponding to the second data index are not the same, determining of all
the second
streaming data corresponding to the target first streaming data are not all
received;
[0177] In an implementation, the processor executes the computer program to
also achieve
following steps:
[0178] If the second data identification corresponding to the second data
index does not exist
in the data storage middleware, generating a first data index according to the
target first data
identification; and storing the first data index and the second data
identification corresponding
to any second streaming data into the data storage middleware.
[0179] In an implementation, the processor executes the computer program,
after achieving
the step of determining of all the second streaming data corresponding to the
target first
streaming data are not all received, also achieving following steps:
33
Date Recue/Date Received 2021-10-18
[0180] According to the second data identification, updating the second data
identification
corresponding to the second data identification in the data storage
middleware.
[0181] In an implementation, the processor executes the computer program to
perform
association processing of any first streaming data with all the second
streaming data
corresponding to the first streaming data, wherein the first streaming data
has the
correspondingly second streaming data, also achieving following steps:
[0182] Determining a first amount of the first streaming data, wherein the
first streaming data
has the correspondingly second streaming data; determining a second amount
corresponding
to each second streaming data; and obtaining a third amount corresponding to
any first
streaming data according to the first amount and the second amount
corresponding to each
second streaming data, wherein the first streaming data has the
correspondingly second
streaming data, the third amount is the associated data after performing the
association
processing.
[0183] In an implementation, the processor executes the computer program
perform
association processing of the target first streaming data with all the
corresponding second
streaming data, also achieving following steps:
[0184] Determining the first amount corresponding to the target first
streaming data;
determining the second amount corresponding to each second streaming data,
wherein the
second streaming data corresponding to the target first streaming data; and
obtaining the third
amount corresponding to the target first streaming data according to the first
amount
corresponding to the target first streaming data and the second amount
corresponding to each
second streaming data, wherein the third amount is the associated data after
performing the
association processing.
[0185] In an implementation, a computer readable storage medium stored with a
computer
34
Date Recue/Date Received 2021-10-18
program is provided to achieve the following steps when processor executes the
computer
program.
[0186] Continuously receiving first streaming data and second streaming data
from a
streaming processing platform, any first streaming data corresponds to zero
second streaming
data or at least one second streaming data, and any second streaming data
corresponds to one
first streaming data; after receiving any first streaming data, wherein the
first streaming data
has the correspondingly second streaming data, if all the second streaming
data corresponding
to the first streaming data are received, performing association processing of
any first streaming
data with all the second streaming data corresponding to the first streaming
data, wherein the
first streaming data has the correspondingly second streaming data, when the
obtaining
associated data meets preset conditions after the association processing,
storing any first
streaming data into a database, wherein the first streaming data has the
correspondingly second
streaming data; and after receiving any second streaming data, if the target
first streaming data
corresponding to the any second streaming data is received, and all the second
streaming data
corresponding to the target first streaming data are received, then performing
association
processing of the target first streaming data with all the second streaming
data corresponding
to the target first streaming data, when the obtaining associated data meets
preset conditions
after the association processing, storing the target first streaming data into
a database.
[0187] In an implementation, the processor executes the computer program,
before achieving
the step of performing association processing of any first streaming data with
all the second
streaming data corresponding to the first streaming data, wherein the first
streaming data has
the correspondingly second streaming data, also achieving following steps:
[0188] Obtaining a first data identification included in any first streaming
data, wherein the
first streaming data has the correspondingly second streaming data;
determining a second data
identification corresponding to the first data identification, the quantity of
the second data
identification corresponding to the first data identification is N, N is a
positive integer greater
than 0; generating a first data index according to the first data
identification; querying the
Date Recue/Date Received 2021-10-18
second data identification corresponding to the first data index in the data
storage middleware,
the quantity of the second data identification corresponding to the first data
index is M, M is a
positive integer greater than 0; if the second data identification
corresponding to the first data
identification is the same as the second data identification corresponding to
the first data index,
determining of all the second streaming data corresponding to any first
streaming data are
received, wherein the first streaming data has the correspondingly second
streaming data; and
if the second data identification corresponding to the first data
identification is not the same as
the second data identification corresponding to the first data index,
determining of all the
second streaming data corresponding to any first streaming data are not all
received, wherein
the first streaming data has the correspondingly second streaming data.
[0189] In an implementation, the processor executes the computer program,
after achieving
the step of determining of all the second streaming data corresponding to any
first streaming
data are not all received, wherein the first streaming data has the
correspondingly second
streaming data, also achieving following steps:
[0190] Determining a complement data between the second data identification
corresponding
to the first data identification and the second data identification
corresponding to the first data
index; Generating a second data index according to the first data index; and
storing the second
data index and the complement data into the data storage middleware.
[0191] In an implementation, the processor executes the computer program,
before achieving
the step of performing association processing of the target first streaming
data with all the
second streaming data corresponding to the target first streaming data, also
achieving following
steps:
[0192] Determining the second data identification corresponding to any second
streaming
data, and a target first data identification corresponding to the second data
identification;
generating a second data index according to the first data identification
corresponding to the
second data identification; querying the data storage middleware according to
the second data
36
Date Recue/Date Received 2021-10-18
index; if the second data identification corresponding to the second data
index exists in the data
storage middleware, determining of the target first streaming data
corresponding to any of the
second streaming data is received; and if the second data identification
corresponding to the
second data index does not exist in the data storage middleware, determining
of the target first
streaming data corresponding to any second streaming data is not received.
[0193] In an implementation, the processor executes the computer program to
also achieve
following steps:
[0194] If the second data identification corresponding to the second data
index exists in the
data storage middleware, determining whether the second data identification is
the same as the
second data identification corresponding to the second data index; if the
second data
identification and the second data identification corresponding to the second
data index are the
same, determining of all the second streaming data corresponding to the target
first streaming
data are received; and if the second data identification and the second data
identification
corresponding to the second data index are not the same, determining of all
the second
streaming data corresponding to the target first streaming data are not all
received;
[0195] In an implementation, the processor executes the computer program to
also achieve
following steps:
[0196] If the second data identification corresponding to the second data
index does not exist
in the data storage middleware, generating a first data index according to the
target first data
identification; and storing the first data index and the second data
identification corresponding
to any second streaming data into the data storage middleware.
[0197] In an implementation, the processor executes the computer program,
after achieving
the step of determining of all the second streaming data corresponding to the
target first
streaming data are not all received, also achieving following steps:
37
Date Recue/Date Received 2021-10-18
[0198] According to the second data identification, updating the second data
identification
corresponding to the second data identification in the data storage
middleware.
[0199] In an implementation, the processor executes the computer program to
perform
association processing of any first streaming data with all the second
streaming data
corresponding to the first streaming data, wherein the first streaming data
has the
correspondingly second streaming data, also achieving following steps:
[0200] Determining a first amount of the first streaming data, wherein the
first streaming data
has the correspondingly second streaming data; determining a second amount
corresponding
to each second streaming data; and obtaining a third amount corresponding to
any first
streaming data according to the first amount and the second amount
corresponding to each
second streaming data, wherein the first streaming data has the
correspondingly second
streaming data, the third amount is the associated data after performing the
association
processing.
[0201] In an implementation, the processor executes the computer program
perform
association processing of the target first streaming data with all the
corresponding second
streaming data, also achieving following steps:
[0202] Determining the first amount corresponding to the target first
streaming data;
determining the second amount corresponding to each second streaming data,
wherein the
second streaming data corresponding to the target first streaming data; and
obtaining the third
amount corresponding to the target first streaming data according to the first
amount
corresponding to the target first streaming data and the second amount
corresponding to each
second streaming data, wherein the third amount is the associated data after
performing the
association processing.
[0203] The skilled in the art can understand that all or partial of procedures
from the above-
mentioned methods can be performed by computer program instructions through
related
38
Date Recue/Date Received 2021-10-18
hardware, the mentioned computer program can be stored in a non-volatile
material computer
readable storage medium, this computer can include various implementation
procedures from
the abovementioned methods when execution. Any reference to the memory, the
storage, the
database, or the other media used in each implementation provided in current
application can
include non-volatile and/or volatile memory. Non-volatile memory can include
read-only
memory (ROM), programable RON (PROM), electrically programmable ROM (EPRPMD),
electrically erasable programmable ROM(EEPROM) or flash memory. Volatile
memory can
include random access memory (RAM) or external cache memory. As an instruction
but not
limited to, RAM is available in many forms such as static RAM (SRAM), dynamic
RAM
(DRAMD), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM),
enhanced SRAM (ESDRAM), synchronal link (Synchlink) DRAM (SLDRAM), memory bus
(Rambus), direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and
memory bus dynamic RAM (RDRAM), etc.
[0204] The technical features of the above-mentioned implementations can be
randomly
combined, for concisely statement, not all possible combinations of technical
features in the
abovementioned implementations are described. However, if there are no
conflicts in the
combinations of these technical features, it shall be within the scope of this
descriptions.
[0205] The above-mentioned implementations are only several implementations in
this
disclosure and the description is more specific and detailed but cannot be
understood as the
limitation of the scope of the invention patent. Evidently those ordinary
skilled in the art can
make various modifications and variations to the disclosure without departing
from the spirit
and scope of the disclosure. Therefore, the appended claims are intended to be
construed as
encompassing the described embodiment and all the modifications and variations
coming into
the scope of the disclosure.
39
Date Recue/Date Received 2021-10-18
