Note: Descriptions are shown in the official language in which they were submitted.
CA 02302469 2000-02-23
WO 99/11014 PGT/F'I98/00652
1
DEFINING QUALITY OF TRANSMISSION IN A TRANSMISSION SYSTEM
FIELD OF THE INVENTION
The invention relates to a method of defining the quality of trans-
mission in a transmission system, in which method test information that can be
recognized is transmitted over a transmission channel.
BACKGROUND OF THE INVENTION
In a radio system, the quality of the connection between a base sta-
tion and a subscriber terminal varies constantly. This variation is caused by
interfering factors occurring on the radio path and radio wave attenuation as
a
function of distance. For example; when the terminal moves farther away from
the base station, the attenuation on the transmission path between the termi-
nal and the base station increases. Efforts are usually made to compensate for
this attenuation by power regulation.
A commonly used variable for denoting the quality of a connection
in digital cellular radio systems is the bit error rate (BER) which denotes
the
number of erroneously received bits compared with all received bits. In the
GSM cellular radio system, for example, a base station and a terminal con
tinuously measure the bit error rate in the connection between them.
Another way to evaluate connection quality is based on the evalua
tion of the quality of the information transmitted on the connection; in view
of
speech, for example, how the receiver finds the quality of the speech. The
above mentioned interference on the signal path also affect the quality of re
ceived voice. A so-called threshold phenomenon, which means that the voice
quality stays good for a long period of time when the distance between the
base station and the terminal increases and then suddenly radically degrades,
is typical for digital systems which allow error correction methods to be
used.
In analogue systems the quality degrades steadily. Quality evaluation is a
useful method when communication networks implemented by different tech-
niques are compared with one another. The bit error rates in different radio
systems, for example, are not necessarily comparable, or even accessible (as
in analogue systems).
In the quality evaluation of information, for example in the quality
evaluation of speech, a signal transmitted to the signal path is usually com-
pared with a signal received over the signal path. A discontinuous signal, for
example speech, is commonly used as an audio signal, since the codecs in
CA 02302469 2000-02-23
WO 99/11014 PGT/FI98/00652
2
digital systems are not optimized for a continuous signal, for example a sine
wave.
Methods of evaluating quality have been developed. In general,
quality is denoted on a scale of one to five (bad, poor, satisfactory, good
and
excellent) in which the best value is denoted by the value five. A known
method is a so-called subjective measurement. In this method a group of test
persons evaluates the quality of a received signal within a given procedure.
The best known procedure is the so-called mean opinion score (MOS). The
recorded speech samples to be evaluated are played to a heterogeneous
90 group of persons who give their opinion about the quality, usually on a
scale of
one to five, and a mean value is formed from the opinions.
Another known method is based on an objective measurement in
which speech quality is defined computationally by means of mathematical
models. At its simplest, the signal-to-noise ratio or the above bit error rate
is
used here to measure speech quality, and in more complex solutions, psy-
choacoustically weighted computational methods are used.
Methods have also been disclosed in which the above evaluation
manners are combined by using neuronetwork applications which are taught
to give a similar response as a heterogeneous group of persons. In general,
the aim in all known methods is to evaluate how closely a received test infor-
mation signal resembles a transmitted one, and the aim is to define the
quality
of the transmission on the basis of a detected difference.
The prior art solutions have several problems, however. The sub
jective methods are not practical; it is expensive and time-consuming to em
ploy a group of persons for the evaluation. Furthermore, irrespective of aver
aging, the results obtained are not necessarily repeatedly similar.
The objective methods, on the other hand, do not correlate closely
with the speech quality experienced by persons. The bit en-or rate, for exam-
ple, denotes poorly how well speech is recognized. The combinations of the
subjective and objective methods, in tum, require heavy computational power.
The neuronetwork applications are complex and demanding to implement.
BRIEF DESCRIPT10N OF THE INVENTION
It is thus an object of the invention to provide a method and an ar-
rangement implementing the method in such a manner that the above prob-
lems can be solved. An object of the invention is to implement a method by
CA 02302469 2000-02-23
WO 99/11014 PCT/FI98/00652
3
which the evaluation of the quality of information can be cost-effectively and
reliably implemented. This is achieved by a method of the type described in
the introduction, the method being characterized in that the same predeter-
mined symbol is repeatedly transmitted in the transmission of the test informa-
tion, and that a measurement of information revealing whether the information
is recognized or not is performed to the received test information, and that
the
measurements are performed over a predetermined period of time, and that
the quality of transmission at the time is defined on the basis of the measure-
ments performed during the time.
The invention further relates to an arrangement for defining the
quality of transmission in a transmission system, which system comprises
means for transmitting test information that can be recognized over a trans-
mission channel. The arrangement of the invention is characterized in that the
arrangement comprises means for repeatedly transmitting the same predeter-
mined test information symbol, means for performing a measurement of infor-
mation revealing whether the information is recognized or not to the received
test information, and means for defining the quality of transmission at the
time
on the basis of the measurements performed over a predetermined period of
time.
The preferred embodiments of the invention are disclosed in the
dependent claims.
Several advantages can be achieved by the method and arrange-
ment of the invention. The invention enables speech quality, for example, to
be evaluated more reliably without expensive equipment and demanding com-
putation. The solution of the invention utilizes speech recognition methods
and
equipment to evaluate speech quality. Similarly, pattern recognition methods
are utilized in the quality definition of image information. !n the solution
of the
invention, the aim is not to define how closely a received signal resembles a
transmitted one, but only to find out whether the received test information is
recognized or not, and the quality of the connection is estimated on the basis
of several successive recognition decisions.
The solution of the invention is also suitable to be used in any
transmission system in which information is transferred via a transmission
path
subject to interference. A typical example is a cellular radio system, but the
invention can also be applied to transmission systems which employ fixed ca-
bling.
CA 02302469 2000-02-23
WO 99/11014 PCT/FI98/00652
4
BRIEF DESCRIPTION OF THE DRAWINGS
The invention is now described in closer detail in connection with
the preferred embodiments with reference to the accompanying drawings, in
which
Figure 1 is a flow diagram showing an example of the method of the
invention,
Figures 2a and 2b are flow diagrams of a definition of a quality
grade,
Figure 3 is a flow diagram of changing test information,
Figures 4a and 4b illustrate transmission and reception of test in-
formation, and
Figure 5 illustrates a structure of the arrangement in accordance
with the invention.
DETAILED DESCRIPTION OF THE INVENTION
In the following, the invention will be described when it is applied
particularly to the evaluation of speech quality. Similarly, the invention
can,
however, be applied in connection with transmission of other kind of informa-
tion, image transmission, for example, as it is obvious to those skilled in
the
art. In such a case the speech recognition algorithms and equipment are re-
placed by pattern recognition systems within the basic idea of the invention.
The aim is thus to estimate the quality of speech transmitted over a
transmission path. In the solution of the invention, a transmitter repeatedly
transmits desired test information to the transmission path. The same prede-
termined symbol is repeatedly transmitted in the test information
transmission.
A receiver receives the information and performs a measurement of informa-
tion revealing whether the information is recognized or not. The measure-
ments can be performed over a predetermined period of time, and the quality
of transmission is defined on the basis of the measurements performed during
the time.
Figure 1 illustrates the operation of the method in accordance with
the invention. In a first step 100 a transmitter transmits the desired
information
which is preferably a particular predetermined word. In a second step 102 a
receiver receives the word proceeded over the transmission path. In a third
step 104 a recognition is performed which reveals whether the transmitted
word is recognized or not. In a subsequent step 106 the information on the
CA 02302469 2000-02-23
WO 99/11014 PCT/FI98/00652
transmission quality is updated on the basis of the recognition. Next, a
return
to the first step 100 follows.
In a preferred embodiment of the invention the desired test informa
tion symbol is thus a particular predetermined word which is recorded and
5 stored in advance, and which can thus be repeatedly transmitted unchanged
over the transmission path. This particular word is continuously transmitted.
The interval between the words can be set as desired in such a manner that
successive words can be distinguished from one another. The words are re-
ceived at the other end of the transmission path. On the transmission path,
the
signal has met with interference typical of the transmission path, which
effect
the quality of the signal. A speech recognition which, on the basis of the
word,
gives a decision value indicating whether the word is recognized or not is per-
formed to the received word. If desired, the received words can be stored in a
memory for a later examination or check. The storage can be performed by a
predetermined criterion, in such a manner that only the unrecognized words
are stored, for example. The words are received over a predetermined period
of time, and the quality estimate of transmission at the time is defined on
the
basis of the decisions made during the time. A typical measurement time can
be five seconds, for example. The quality estimate is given on the known scale
of one to five.
Speech recognition can be performed by known speech recognition
methods and algorithms. A received word is evaluated in speech recognition,
and as a result of the evaluation a binary decision whether the word was rec-
ognized or not is thus obtained. Since the solution of the invention only aims
to
recognize a word and not to estimate how near or far the received word is in
view of the transmitted word, the solution of the invention is essentially
easier
to implement in practice than the previous methods.
Figure 2a illustrates a definition of the estimate of the transmission
quality, in other words steps 104 and 106 of Figure 1. A recognition test is
thus
performed to each word in step 200 by using a known speech recognition
method. If the word was recognized, the quality estimate is upgraded in step
202. Next, the evaluation of the following word can be introduced in step 204.
If the word was not recognized in step 200, the quality estimate is degraded
in
step 206. Next, the evaluation of the following word can also be introduced in
step 204. The update of quality information in steps 202 and 206 can be per-
formed in such a manner that word recognitions performed over a particular
CA 02302469 2000-02-23
WO 99/11014 PCT/FI98/00652
6
determined period of time are taken into account when defining the grade. The
transmission quality can be defined on the basis of the words received during
the last five seconds, for example, and when a new word arrives the recogni-
tion information of the oldest word is removed from the computation.
Figure 2b illustrates another embodiment of the definition of the es-
timate of the transmission quality. It is possible to predict the changes in
the
quality of the signal more accurately in this embodiment. A recognition test
is
thus performed to each word in step 200. If the word was recognized, it is
next
checked in step 208 whether successive recognitions have occurred several
times. If not, the quality estimate is upgraded by an increment A given in
step
210. If several successive recognitions occur, the quality estimate is
upgraded
by an increment n*A given in step 212, which is thus greater than the incre-
ment used in step 210. Next, the evaluation of the following word can be intro-
duced in step 204.
If the word was not recognized in step 200, it is next checked in
step 214 whether successive failures in recognition have occurred several
times. If not, the quality estimate is degraded by a decrement A given in step
216. If several successive failures in recognition occur, the quality estimate
is
degraded by a decrement n*A given in step 218, which is thus greater than the
decrement used in step 216. Next, the evaluation of the following word can
also be introduced in step 204. It is an aim of this method to make the
quality
estimation react quicker to great and abrupt changes. In a cellular radio sys-
tem, for example, great changes can take place when a terminal enters or ex-
its a tunnel. The quality of a signal can then suddenly change from good to
bad, or vice versa. If several parallel decisions occur, they are detected in
the
above method, and thus the change unit of the quality information can be in-
creased. The upgradation of the change unit and the number of the required
successive similar decisions are system-dependent parameters.
In accordance with a preferred embodiment of the invention, sev
eral words that belong to different levels with respect to recognition can
alter
natively be used as a test information symbol. The test word used at a par
ticular time can be selected on the basis of the quality information. Let us
as
sume, for example, that three words of different degrees of difficulty with re
spect to recognition are used. The test can be started with a word of medium
degree of difficulty. If it is found that the quality information is
continuously
good, in other words the percentage of recognition of the word is high, the
CA 02302469 2000-02-23
WO 99/11014 PCT/FI98/00652
7
transmission of a more difficult word can be introduced. Similarly, if a word
of a
medium degree of difficulty is rarely recognized, in other words the
percentage
of recognition is low, an easier word can be introduced for testing the connec-
tion. The recognition percentages for changing the test word are system-
s specific parameters and they can vary. This embodiment is illustrated in the
flow diagram of Figure 3. A recognition test is performed to a test word in
step
300 and at once, the recognition percentage at the time is computed, in other
words how many words from the transmitted ones are recognized. If the word
was recognized, it is checked in step 302 whether the recognition percentage
has reached the word upgradation threshold. If not, a transfer to the next
word
takes place in step 304. If the threshold is reached, the next most difficult
word
is selected as the test word in step 306. Next, a transfer to the following
word
takes place in step 304.
If the word was not recognized in step 300, it is checked in step 308
whether the recognition percentage has reached the word degradation thresh
old. If not, a transfer to the following word takes place in step 304. If the
threshold is reached, the next easiest word is selected as the test word in
step
310. Next, a transfer to the following word takes place in step 304. The flow
diagram only shows the word changing algorithm, not an update of the quality
information, which is performed as shown in Figure 2a or 2b. The word can
also be changed directly on the basis of the quality estimate, or on the basis
of
a parameter computed on the basis of another recognition test.
Let us next take a closer look at an arrangement of the invention for
defining the quality of transmission. The transmission and reception of test
information in accordance with the invention can be performed in various
ways. Examples of different alternatives are shown in Figures 4a and 4b. Fig-
ure 4a shows a transmitter 400 and a receiver 404. The transmitter 400
transmits test information 404 to the receiver 402 in which the recognition
and
quality evaluation in accordance with the invention are performed. Figure 4b
shows a so-called loop-back solution, in which a first transceiver 406
transmits
test information 404 to a second transceiver 408. The second transceiver 408
transmits received test information 410 back to the first transceiver 406
which
receives the signal and performs the recognition and quality evaluation in ac-
cordance with the invention. An advantage of the latter solution is that trans-
mission and reception can be performed physically in the same place. A dis-
advantage is that the test information travels twice over the transmission
path,
CA 02302469 2000-02-23
WO 99/11014 PCT/F198/00652
8
which makes it impossible to define which transmission direction includes
more interference.
Figure 5 is a flow diagram illustrating an example of the structure of
the arrangement in accordance with the invention. The arrangement of the
invention for defining the quality of transmission comprises memory means
500 in which previously known test information to be transmitted can be
stored, in other words a number of words of different levels with respect to
recognition in a preferred embodiment of the invention. The arrangement fur-
ther comprises control means 502 which read the test information from the
memory means 500. The arrangement comprises means 504 for continuously
transmitting test information over a transmission channel 506, controlled by
the control means 502. The control means 502 are typically implemented by a
processor and the transmission means 504, in turn, by means of a modem or
a radio transmitter. The arrangement of the invention further comprises means
508 for receiving test information from the transmission channel 506. The re-
ception means 508 typically comprise a radio receiver and a modem. From the
modem, the test information is conveyed to processing means 510 which are
arranged to perform an information measurement revealing whether the infor-
mation is recognized or not to the received information by means of speech
recognition software. The arrangement further comprises processing means
510 for defining the transmission quality at the time on the basis of the meas-
urements performed during a predetermined period of time. The measurement
results are stored in memory means 512. If desired, the received test informa-
tion, in other words the received words, can be stored in the memory for a
later examination or check in the memory means 512. The storage can be
performed by a predetermined criterion, for example in such a manner that
only unrecognized words are stored.
The arrangement of the invention can be preferably implemented by
a microcomputer apparatus which comprises a voice card and software capa
ble of producing voice information stored in a memory, and a modem via which
the apparatus is connected to a transceiver in a cellular radio system, for ex
ample. The recognition of information can preferably be performed by known
speech recognition software. The transmission and reception functions can
both be preferably implemented by the same apparatus, or alternatively, two
separate apparatuses can be used.
CA 02302469 2000-02-23
WO 99/11014 PCT/FI98/00652
9
In accordance with a preferred embodiment of the invention, in
connection with the reception of test information other measurements are also
performed, such as field strength or bit error ratio measurements. When the
test information results are stored for a later examination, the results of
the
measurements performed at the same time are also stored. The reception and
measurement times are attached to the stored material, whereby results ob-
tained by different methods can be compared with one another.
Although the invention is described above with reference to the ex
ample in accordance with the accompanying drawings, it is to be understood
that the invention is not restricted thereto but it can be modified in various
ways within the scope of the inventive idea disclosed in the appended claims.
