CA 02392849 2004-11-16
SPEECH INTERVAL DETECTING METHOD AND DEVICE
This is a divisional of application S.N.
2,258,908 filed April 3.0, 1998.
Technical Field
Parent application S.N. 2,258,908 relates to a
speech speed converting method and a device for
embodying the same which are able to achieve
easiness of hearing expected in speech speed
conversion without extension of playback time in
various video devices, audio devices, medical
-cl.evices, etc. such as a television set, a radio, a
tape recorder, a video tape recorder, a video disk
player, a hearing aid, etc.
The present divisional case relates to a speech
interval detectinggmethod and a device for embodying
the same which are able to discriminate between speech
intervals and non-speech intervals of an input signal
in the event that the.speech which is delivered together
with noises or background sounds in a broadcast program,
a recording tape, or a daily life is processed to change
height of the voice or speech speed, the meaning of the
speech is mechanically recognized, the speech is coded
to transfer or record, or the like.
Parent application 2,258,908 relates to a
speech speed converting method and a device
"for embodying the same which converts a speech
speed in real- time by processing the speech
made by the human being, and carries. out a
CA 02392849 2002-07-30
2
series of processes without omission of information,
while monitoring always a data length of the input
speech, an output data length calculated previously
according to a conversion function, which is concerned
with a previously giVen scaling factor, and a data
length of the speech being output actually in constant
process unit when a delivered speed (speech speed) of
listening speech is made slow.
Furthermore, in the speech speed converting method
lo and the device for embodying the same, for example, the
non-speech interval which has a length in excess of a
variable threshold value being set according to a delay
degree (conversion factor) expected in speech speed
conversion can be reduced appropriately while aiming
at minimizing the time difference between the image and
the speech caused by extension of the speech in watching
the television receiver, and maximum slowness
impression which can be accomplished within a
decided time range can be created automatically by
changing adaptively a conversion factor according to
a degree of time difference between the input data
length and the output data length, while keeping
substantially a speaking time of the converted
speech within a speaking time of an original speech.
Moreover, the present system calculates the
power of input signal data at a predetermined time
interval in frame unit having a predetermined time width,
and then discriminates between the speech interval and
the non-speech interval every frame by using the
threshold value for the power which is changed according
CA 02392849 2002-07-30
-3_
to the maximum value and the difference between the
maximum value and the minimum value, while holding the
maximum value and the minimum value of the power within
the past predetermined time period, so as to respond
sequentially to change in respective powers of the input
speech and the background sound. As a result,
improvement in quality of processed sound, improvement
.in the speech recognition rate, increase in the coding
ef f iciency, and improvement in quality of the decoded
lo speech can be achieved by detecting precisely the speech
interval of the input signal in the case that changed
in height of the voice or speech speed, mechanical
recognition of the meaning of the speech, and coding
of the speech to transfer or record, and the like are
effected by processing the speech which is delivered
together with noises or background sounds in a broadcast
program, a recording tape, or a daily life.
In addition, the speech processing can be executed
in real time while shortening a calculation time and
2o also reducing a cost, by employing only the power which
can be derived relatively simply as a f eature parameter.
Background Art
In case the speech speed converting method is
applied to the actual broadcast, there are some cases
where delay from the original speech such as emergency
news becomes an issue. Particularly, it is possible
that this delay has a bad effect on the visual media
in contrast with the effect expected in the speech speed
conversion.
CA 02392849 2002-07-30
-4-
Therefore, as approaches for achieving the speech
speed converting effect (slowness impression) without delay
from the original speech, there have been reported the
method of suppressing extension in time by changing the
speech speed from slowly to quickly as a function of
a lapse time from a start point of one breath speech
to an end point instead of uniformly slow conversion,
and then reducing appropriately the non-speech interval
between sentences (R. Ikezawa et al., "An Approach for
io Absorbing Extension in Time Caused in Speech Speed
Conversion", Spring Conference,. Japanese Acoustic
Society, 2-.6-2, pp.331-332, 1992), the method of
achieving this approach in real time (A. Imai et al.,
"Real Time Absorption Method for Extension in Time
3 5. Gau-&ed in Speach Speed Conversion", in International
Conference, IEICE, D-694, pp 300, 1995), etc.
The former sets an appropriate function manually
under that assumption that all speech styles have been
known. The latter also sets a function defining a
20 factor manually, and fixes this function after the
function has been set once.
In addition, only the constant remaining time is
set manually to reduce the non-speech interval. If a
deal-of "inconsistency" is integrated, the extended
25 speech being accumulated in a buffer is cleared
manually.
Therefore, in the speech speed converting device
in the prior art, there has been such a problem that,
since various speaking styles (speech speed, "timing"
30 in speech, etc..) are present in the broadcast speech
CA 02392849 2002-07-30
-5-
according to the speaker and also appropriate
parameters must be set manually respectively, the
device has many operation points, setting per se is
difficult, and it is difficult for the common user to
handle the device.
Besides, in the above speech speed converting
device, the speech interval and the non-speech interval
must be recognized separately. There are various
systems as the speech interval detecting system in the
io prior art.
As one of the speech interval detecting system in
the prior art, such a system has been known that a noise
level and a speech level are calculated based on the
power of the speech signal, etc., then a level threshold
value 3-s sat based on the.calculation result, then this
level threshold value and the input signal are compared
with each other, the.n the interval is decided as the
speech interval if the level of the input signal is
higher than the level threshold value and the interval
is decided as the non-speech interval if the level of
the input signal is lower than the level threshold
value.
As methods of setting the level threshold value
employed in this system, there are first to third
representative systems. According to the first system,
a value which is. obtained by adding a preselected
constant to a noise level value of the input speech is
employed as the level threshold value. According to
the second system which is an improved first system,
the level threshold value is set to a relatively large
CA 02392849 2002-07-30
_6_
value when a value obtained by subtracting the noise
level value from a maximum level value of the input
speech signal is large, whereas the level threshold
value is set to a relatively, small value when the value
obtained by subtracting the noise level value from a
maximum level value of the input speech signal is small
(for example, Patent Application Publication (KOKAI)
Sho 58-130395, Patent Application Publicati.on (KOKAI)
Sho 61-272796, etc.).
According to the third system, in addition to these
level threshold value setting methods, the input signal
is monitored continuously, then the input signal is
regarded as the noise level when the level of the input
signal is steady over a constant time period, and then
athr-eshold val-ue, employed for the speach interval
detection is set while updating. the noise level
sequentially (Proceeding in International Conference,
IEICE, D-695, pp 301, 1995).
However, in the above speech interval detecting
system in the prior art, there have been problems
described in the following.
To begin with, the first system has an advantage
that it is simple, and can operate well when the average
level of the speech is a middle level. However, the
first system is easy to detect the noise, etc. "errously
as speech when the average level of the speech is
too large, and it is easy to detect the speech with
omission of a part of the speech when the average level
of the speech is too small.
CA 02392849 2002-07-30
-7-
Then, the second system can overcome the problem
arisen in the first system. However, there has been
such a problem that, since the event that levels of the
noises and the background sounds in the input signal
are kept substantial.ly constant is employed as a premise,
the second system can follow the variation in level of
the speech, but the precise speech interval detection
cannot be assured when levels of the noises and the
background sounds are changed at every moment.
Then, since the variation in such noise level is
considered into the third system, erroneous detection
is not caused even when the noise level is changed
sequentially.
However, not only the noise but also the background
sound s-uch as music, ymitation s.munsi, etc. as sound
effects are included in the broadcast program, etc.,
and commonly these levels are changed at every moment
and at the same time the speech is always continued to
deliver, so that the input signal level seldom becomes
steady over a predetermined time period. In such case,
there has been such a problem that, since the noise level
cannot be set correctly even by the third system, it
is difficult to detect precisely the speech interval.
The present invention has been made in view of the
above circumstances, and it is an object of the present
invention to provide a speech interval detecting
method and a device for embodying the same which is
capable of discriminating the speech interval and the
non-speech interval by executing the speech
processing in real time so as to respond sequentially
CA 02392849 2005-11-23
-8-
to change in the respective levels of the input
speech and the background sound, while shortening the
calculation time and also reducing the cost, since
only the power which can be derived relatively simply
as a feature parameter is employed.
Summary of the Invention
In order to achieve the above object,
there is provided a speech interval detecting
method comprising the steps of: calculating a
frame power of a frame having a predetermined
width of an input signal data, and holding and
updating a maximum value and a minimum value of
the frame power of the frame each time a
predetermined time period passes; setting and
updating a threshold value for power so as to be
varied according to the maximum value and
difference between the maximum value and the
minimum value at every predetermined time period;
and comparing the threshold value most recently
updated with power of a current frame to decide
whether the current frame belongs to a speech
interval or a non-speech interval.
CA 02392849 2004-11-16
_9'
According to the above configuration, in the
speech interval detecting method set forth above,
a frame power of an input signal data is calculated in
unit of predetermined frame width at a predetermined
time interval, then a maXimum value and a minimum value
of the frame power within a past predetermined time
period are held, then a threshold value for power is
decided according to the maximum value being held and
difference between the maximum value and the; minimum
value, and then the threshold value and power of a
current frame are compared with each other to decide
whether or not the current frame belongs to a speech.
interval or a non-speech. interval. Therefore, the
speech interval and the non-speech interval can be
diseri-mi-nated by -ex-eeuting the- speech proc.e.ssing in
real time while responding sequentially to change in
respective levels of the input speech and the background
sound.
According to the speech interval detecting method set
forth above the threshold value is set to a first value
when the difference between the maximum value and the
minimum value is less than a predetermined value and the
threshold value is set to a.second value that is smaller
than the first value when the difference is not less than
the predetermined value..
In order to further achieve the above object,
there is provided a speech interval detecting
device comprising: a power calculator for
calculating a frame power of a frame having a
CA 02392849 2005-11-23
-10-
predetermined width of an input signal data each time a
predetermined time period passes; an instantaneous power
maximum value latch for holding and updating a maximum
value of the frame power of the frame at every
predetermined time period; an instantaneous power minimum
value latch for holding and updating a minimum value of
the frame power of the frame at every predetermined time
period; a power threshold value decision portion for
setting and updating a threshold value for power so as to
be varied according to the maximum value being held in
the instantaneous power maximum value latch and
difference between the minimum value being held in the
instantaneous power minimum value latch and the maximum
value at every predetermined time period; and a
discriminator for comparing the threshold value most
recently obtained by the power threshold value decision
portion with power of a current frame to decide whether
the current frame belongs to a speech interval or a non-
speech interval.
According to the above configuration, a power
calculator calculates a frame power of an input
signal data in unit of predetermined frame width at a
predetermined time interval, an instantaneous power
maximum value latch holds a maximum value of the frame
power within a past predetermined time period, an
instantaneous power minimum.value latch holds a
minimum value of the frame power within the past
predetermined time period, a power threshold value
decision portion decides a threshold value for power
changed according to the maximum value being held in
the instantaneous power maximum value latch and
CA 02392849 2004-11-16
-11-
difference between the maximum value and the minimum
value being held in the instantaneous power minimum
value latch, and a discriminator compares the threshold
value obtained by the power threshold value decision
portion with, power of a durrent frame to. decide whether
or not the current frame belongs to a speech interval
or a non-speech interval. Therefore, while shortening
a calculation time and also reducing a cost by employing
only the power which can be derived relatively simply
as a feature parameter, the speech interval and the
non-speech interval can be discriminated by executing
the speech processing in real time so as to respond
sequentially to change in the respective levels of the
input speech and the background sound.
According to the speech interval detecting
device set forth above, the power threshold value
decision portion sets the threshold value to a
first value when the difference between the
maximum value and the minimum value is less than a
predetermined value and the power threshold value
decision portion sets the threshold value to a
second value that is smaller than the first value
when the difference is not less than the
predetermined value.
CA 02392849 2002-07-30
-12-
Brief Description of the Drawings
FIG.1 is a block diagram showing a speech speed
converting device according to an embodiment of the
present invention;
FIG. 2 is a block diagram showing a speech interval
detecting device according to an embodiment of the
present invention;
FIG.3 is a schematic view showing an example of
an operation of the speech interval detecting device
shown in FIG.2;
FIG . 4 is a schematic view showing a method of
generating connection data, which is employed to
connect th-e same block repeatedly in a connection data
generator shown in FIG.1;
FIG. 5 is a block diagram showing an example of a
detailed configuration of an I/O data length
monitor/comparator in a connection order generator
shown in FIG.1; -and
FIG.6 is a schematic view showing an example of
connection order which is generated by the connection
order generator shown in FIG.1.
Best Mode for Carrying out the Invention
The present invention will be explained in detail
with . reference to the accompanying drawings
hereinafter.
FIG.1 is a block diagram showing a speech speed
converting device according to an embodiment of the
CA 02392849 2002-07-30
-13-
present invention.
The speech speed converting device shown in FIG.1
comprises a terminal 1, an A/D converter 2, an analysis
processor 3, a block data splitter 4, a block data memory
S. a connection data generator 6, a connection data.
memory 7, a connection order generator 8, a speech data
connector 9, a D/A converter 10, and a terminal 11. When
the speech-speed converted speech data are synthesized
by applying an analyzing process to input speech data
io from a speaker based on attributes of the speech data
and then using a desired function according to the
analyzed information, the speech speed converting
device can eliminate omission of the speech information
against change in scaling factor by executing these
processes without inconsistency while comparing a data
length (input data length) of input speech data, a
target data length calculated by multiplying such data
length by any scaling factor, and a data length (output
data length) of actual output speech data, and can
monitor time difference between the original speech
being changed at every moment and the converted speech .
And, the speech speed converting device can eliminate
adaptively the time difference from the original speech
because of the speech speed conversion by changing the
scaling factor adaptively, e.g., by increasing the
speech speed conversion factor temporarily when the
time difference is small and conversely decreasing the
speech speed conversion factor temporarily when the
time difference is large, and further changing a
3o remaining rate of the non-speech interval adaptively
CA 02392849 2002-07-30
-14-
based on the speech speed conversion factor, an amount
of expansion, etc.
The A/D converter 2 executes an A/D conversion of
the speech signal being input into the terminal 1, e. g.,
the speech signal being output from an analog speech
output terminal of the video device, the audio device,
etc. such as the microphone, the television set, the
radio,'and others, at a predetermined sampling rate
( e. g., 32 kHz), and supplies the resultant speech data
Zo to the analysis processor 3 and the block data splitter
4 neither too much nor too less while buffering such
speech data-into a FIFO memory.
The analysis processor 3 extracts the speech
intervals and the non-speech intervals by analyzing the
speeeeh data being output from the A/D converter 2, then
generates split information to determine respective
time lengths necessary for the split process of the
speech data being executed in the block data splitter
4 based on these intervals, and then supplies such split
information to the block data splitter 4.
Now, embodiments of the speech interval detecting
method and the device f or embodying the same according
to the present invention will be explained hereunder.
In the speech interval detecting method and the
device for 6mbodying the same according, to the present
invention, in view of the fact that level variation in
the speech in the input signal is reflected on a maximum
value of the power being input immediately before and
level variation in the background sound is reflected
on a. minimum value of the power being input immediately
CA 02392849 2002-07-30
-15-
before if power of. the input signal is etnployed as an
index, a threshold value can be decided by such a process
that a value obtained by subtracting a predetermined
value from the maximum value of power being input
immediately before is get to a basic threshold value
and then correction is applied to increase the basic
threshold value as a value obtained by subtracting the
minimum value from the maximum value of power being
input immediately before is decreased (as an S/N is
io reduced), when noises are seldom present to determine
a threshold value for speech/non-speech
discrimination.
Then, the speech interval detecting method and the
device for embodying the same calculates the power of
the input speech data at a pre"termine.d time interval
in unit of frame having a predetermined time width, and
then discriminates between the speech interval and the
non-speech interval every frame by using the threshold
value for the power which is changed according to the
maximum value and difference between the maximum value
and the minimum value, while responding sequentially
to change in respective powers of the input speech and
the background sound to hold the maximum value and the
minimum value of the power in the past predetermined
time interval.
The explanation will be.made concretely with
reference to the drawings hereinafter.
FIG.2 is a block diagram showing the speech
interval detecting device.
CA 02392849 2002-07-30
-16-
An speech interval detector 31 shown in FIG.2
comprises a power calculator 32 for calculating the
power of the digitized input signal data at a
predetermined time interval by a predetermined frame
width, an instantaneous power maximum value latch 33
for holding the maximum value of the frame power within
the past predetermined time period, an instantaneous
power minimum value latch 34 for holding the minimum
value of the frame power within the past pre,determined
lo time period, a power threshold value decision portion
35 for deciding a threshold value for power which is
changed according to both the maximum value and the
difference between the maximum value held in the
instantaneous power maximum value latch 33 and the
minimum value held in the instantaneous power minimum
value latch 34, and a discriminator 36 for
discriminating whether or not the speech belongs to the
speech interval or the non-speech interval, by
comparing the threshold value decided by the power
threshold value decision portion 35 with the power at
the current frame.
The speech interval detector 31 calculates the
power with respect to the input signal data at a
predetermined time interval in frame unit having a
predetermined time width, and then discriminates
between the speech interval.and the non-speech interval
every frame by using the threshold value for power which
is changed according to the maximum value and the
difference between the maximum value and the minimum
value, while responding sequentially to change in
CA 02392849 2002-07-30
-17_
respective powers of the input speech and the background
sound to hold the maximum value and the minimum value
of the power within the past predetermined time period.
The power calculator. 32 calculates a sum of squares
or square mean value of' the signal at a time interval
of 5 ms over a frame width of 20 msec, for example, then
sets the frame power at that time to -"P" by representing
this value logarithmically, i.e., in decibel, and then
supplies this frame power " P " to the instantaneous power
lo maximum value latch 33, the instantaneous power minimum
value latch 34, and the discriminator 36.
The instantaneous power maximum value latch 33 is
designed to hold the maximum value of the frame power
"P" within the past predetermined time period (e.g.,
6 seconds), and always supplies the held value "P
to the power threshold value decision portion 35.
However, when the frame power "P" to satisfy "P>Pupper"
is supplied from the power calculator 32 , the maximum
value "PuBper"-iS immediately updated.
The instantaneous power minimum value latch 34 is
designed to hold the minimum value of the frame power
"P" within the past predetermined time period (e.g.,
4 seconds), and always supplies the held value "Piower
to the power threshold value decision portion 35.
However, when the frame power "P" to satisfy "P<Plower
is supplied from the power calculator 32, the minimum
value "Plower . is immediately updated-.
The power threshold value decision portion 35
decides a threshold value "Pthr" of the power by
CA 02392849 2002-07-30
-1$-
executing calculations given in following equations,
for example ; with the use of the maximum value "Pupper
held in the instantaneous power maximum value latch 33
and the minimum value "Plower held in the instantaneous
power minimum value latch 34, and then supplies the
threshold value "Pthr" to the discriminator 36.
FOr Pupper - Plower ~ 60 [ dB
Pthr = Pupper - 3 5 . . . (1)
For Pupper - Piower < 60 [ uB ] r
Pthr = Pupper - 35 + 3 5 X{ 1 - (Pupper - Plower )/ 6 0}.,.( 2)
In this case, it is desired that an upper limit
Of Pthr should be set to Pthr = Pupper-13 in order to prevent
the malfunction of the device of the present invention
when a level of the background sound becomes close to
a level of the speech. Also, a constant 35 in above
Eqs. corresponds to a basic threshold value when the
above mentioned noises are seldom present.
The discriminator 36 compares the power "P"
supplied from the power calculator 32 every frame with
the threshold value "Pthr" supplied from the power
threshold value decision portion 35, then decides every
frame that the frame belongs to the speech interval when
"P>Pthr" is satisfied and that the frame belongs to the
non-speech interval when "PSPt,,r" is satisfied, and then
outputs a speech/non-speech discriminating signal
based on these decision results.
Accordingly, as shown in FIG.3, under the
situation that the value of the input signal data is
CA 02392849 2002-07-30
-19-
being changed, the maximum value "Pupper" and the minimum
value "Ploa1er" can be latched from the power "P" being
output from the power calculator 32 by the instantaneous
power maximum value latch 33 and the instantaneous power
minimum value latch 34 respectively, then the threshold
value "Pthr" is decided based on the maximum value "Pupper's
and the minimum value "Plower". and then it is decided
based on this threshold value "Pthr" whether or not the
frames belong to the speech interval, or the non-speech
io interval respectively.
In this manner, in this embodiment, the power of
the input signal data is calculated at a predetermined
time interval in unit of frame having a predetermined
time width and then, with responding sequentially to
the change in the powers of the input speech and the
background sound to keep the maximum value and the
minimum value of the power within the past predetermined
time period, the speech interval and the non-speech
interval are discriminated by using the threshold value
for power which changes according to the maximum value
and the difference between the maximum value and the
minimum value. Therefore, with regard to the speech
.which is delivered together with noises or background
sounds in a broadcast program, a recording tape, or a
daily life,_the speech interval and the non-speech
interval can be precisely discriminated frame by frame.
In this embodiment, since a level of the background
sound is estimated based on the minimum value of the
instantaneous power within the past predetermined time
period, the speech interval and the non-speech interval
CA 02392849 2002-07-30
-20-
of the input signal can be discriminated even if the
level of the background sound is varied at every moment
in the broadcast program, etc. and simultaneously the
speech is continued to deliver.
, As a result, in the case that
(a) height of the voice and speed of the speech
in the input signal are changed by processing the
speech,
(b) the meaning of the speech in the input signal
lo is mechanically recognized,
(c) the speech in the input signal is coded to
transfer or record, etc., improvement in quality of
processed sound, improvement in the speech recognition
rate, increase in the coding efficiency, and
15 improvemeint in quality of the decoded speech can be
achieved.
Since only the power which can be derived
relatively simply as a feature parameter is employed,
a calculation time can be shortened and also a
20 configuration of the.overall device can be simplified
to reduce a cost. In addition, speech processing can
be executed in real time.
Next, in the speech speed converting method of the
present invention, processes will be continued further
25 as follows.
That is, the decision whether or not the speech
is voiced sound with vibration of the vocal cords or
voiceless sound without vibration of the vocal cords
is applied to the interval in which the power exceeds
30 the predetermined threshold value Pthr, i. e., the speech
CA 02392849 2002-07-30
-21-
interval. Not only the magnitude of the power but also
zero crossing analysis, autocorrelation analysis, etc.
can be applied to this decision.
When a time length of the block is decided to
analyze the speech data, periodicity is detected by
applying the predetermined autocorrelation analysis to
the speech interval (voiced sound interval, voiceless
sound interval) and the non-speech interval, and then
the block lengths are decided based on this periodicity.
lo Then, pitch periods which are vibration periods of the
vocal cords are detected from the voiced sound interval,
and then the voiced sound interval is split such that
respective pitch periods correspond to respective block
lengths. At that time, since the pitch periods of the
voiced $ound interval is distributed over the wide range
of about 1.25 ms to 28.0 ms, as precise pitch periods
as possible are detected by executing the
autocorrelation analysis using different window widths,
or the like. The reason why the pitch period is used
2o as the block length of the voiced sound interval is to
prevent change in height of the voice due to repetition
in block unit. As with the voiceless sound interval
and non-speech interval, the block lengthis detected
by detecting the periodicity within 5 ms.
Then, 'the block data splitter 4 splits the speech
data output from the A/D converter 2 in accordance with
the block length decided by the analysis processor 3,
and then supplies the speech data which are obtained
by this split process in unit of block and the block
length to the block data memory 5. The block data
CA 02392849 2002-07-30
-22-
splitter 4 also supplies both end portions of the speech
data obtained by the split process in unit of block,
i.e., a predetermined time length (e.g., 2 ms) after
a start portion and a predetermined time length ( e. g.,
2 ms) before an end poiction, to the connection data
generator 6.
The'block data memory 5 stores the speech data
supplied in unit of block from the block data splitter
4 and the block length temporarily by virtue of ring
lo buf fer . The block data memory 5, as the case may be,
supplies the speech data being stored temporarily in
unit of block to the speech data connector 9 and supplies
the block lengths being stored temporarily to the
connection order generator 8.
The connection data generator 6 applies windows
to the speech data in the end portion of the preceding
block, the start portion of the concerned block, and
the start portion of the succeeding block every block,
as shown in FIG.4, then executes overlapping addition
of the end portion of the preceding block and the end
portion of the concerned block and overlapping addition
of the start portion of the concerned block and the start
portion of the succeeding block, then generates
connection data for every block by connecting them, and
then supplies the connection data to the connection data
memory 7.
The connection data memory 7 stores the connection
data of respective blocks supplied from the connection
data generator 6 temporarily by virtue of ring buffer,
3o and then supplies the connection data being stored
CA 02392849 2002-07-30
-23-
temporarily to the speech data connector 9 if necessary.
The connection order generator 8 generates the
connection order of the speech data in unit of block
and corinection data in order to attain the desired
speech speed which is set by a listener. In this case,
the listener can set an extension factor in time for
respective attributes (voiced sound interval,
voiceless sound interval, and non-speech interval) by
using a digital volume as an inter~ace. This value is
io stored in a writable memory. Also, this value can be
provided by selecting one of the method (uniform
extension mode) in which such value is processed as a
fixed extension factor and the method (time extension
absorption mode) in which a speech speed converting
effect can be achieved within a limited time range by
controlling respective speech attributes totally and
adaptively while aiming at such set factor, not to
integrate the inconsistency for a predetermined time.
According to the, connection order generator.8,
when the speech synthesis is performed actually by using
the extension factor being set in the memory, time
difference between a delivered time of the original
speech and an output time of the converted speech- can
be always monitored by grasping, in real time, time
relationships among the input speech data length and
.the output speech data length at the same time and the
speech data length to be synthesized, so that the time
difference can be suppressed automatically within.a
constant length by feeding back this information. At
the same time, it can be checked whether or not
CA 02392849 2002-07-30
-24-
inconsistency in time (e.g., request such that the
output speech data length must be set shorter than the
input speech data length) is caused by using a scaling
factor being changed into any value at any timing, and
therefore omission of speech information in synthesis
can be prevented.
Next, the process in the connection order
generator 8 will be explained in detail hereunder. When
the scaling factor of the speech is set by any function,
lo the speech data length (= input data length) in
processing unit specified by the block data splitter
4 is sequentially calculated based on respective block
lengths supplied from the block data memory 5, and then
a length which is derived by multiplying the input data
length by the scaling factor being set by the listener
is set as a target data length. The speech data
connector 9 connects the speech data to coincide with
this target data length, and also feeds back the speech
data length (=output data length), which is a length
of the output speech data being output actually,
sequentially to the connection order generator 8.
Then, as shown in FIG. 5, a target length which is
generated by an I/O data length monitor/comparator 20
provided in the connection order generator 8 is sent
to the speech data connector 9 as connection order
information. The I/0 data length monitor/comparator'
20 comprises an input data length monitor 21 for
monitoring the input data length; an output target
length calculator 22 for calculating a target length
(target data length) of the output data generated by
CA 02392849 2002-07-30
-25-
the speech speed factor conversion, which is effected
based on the input data length obtained by the input
data length monitor 21 and the value given by the
listener (or a function memory built in the device),
for example, and also cotrecting this target data length
automatically; a comparator 23 for comparing the target
data length obtained by the output target length
calculator 22 with the input data length obtained by.
the input data length monitor 21, and then setting the
lo target data length to coincide with the input data
length if the target data length is shorter than the
input data length.,but outputting the target data length
as it is if the target data length is longer than the
input data length; an output data length monitor 24 for
receiving ready-connected information concerning the
output data supplied from the speech data connector 9
to monitor the output data length; and a comparator 25
for comparing the output data length obtained by the
output data length monitor 24 with the target data
length obtained by the comparator 23, and then setting
the target data length to coincide with the output data
length if the target data length is shorter than the
output data length, but outputting the target data
length as it is if the target data length is longer than
the output data length. Then, as described later, the
I/0 data length monitor/comparator 20 reads out values
being set in the memory for every attribute of the speech
at a predetermined time interval, then calculates the
target data length in order to attain extension factors
for every read attribute, then generates the connection
CA 02392849 2002-07-30
-26-
information, into which the scaling information of the
speech are added, at every moment based on the target
data length and the output data length obtained by the
output data length monitor 24, and then connects the
speech data and the corinection data for every block,
as shown in FIG.6.
First, the input data length and the target data
length are compared sequentially with each other, and
then the target data length is corrected to coincide
1.o with the input data length if it has been decided that
the input data length is longer than the target data
length, but change of the target data length is
suspended if it has been.decided that the input data
length is less than the target data length.
Then, the target data length and the-actual output
data length are compared sequentially with each other,
and then the target data length is corrected to coincide
with the output data length if it has been decided that
the output data length is longer than the target data
length, but change of the target data length is
suspended if it has been decided that the output.data
length is less than the target data length.
Connection instructions indicating the extension
information, connection information, etc. are
generated to coincide with the target data lengths
obtained by these comparing processes, and then
supplied to the speech data connector 9.
Then, controlling conditions for the speech speed
conversion factor in the connection order generator 8
will be explained hereunder. For example, in case the
CA 02392849 2002-07-30
-27-
speech speed conversion is desired in the limited time
range such as the time frame in the broadcast, the input
data length and the output data length are monitored
sequentially so as to. measure time difference between
both data at a time interval being previously set
arbitrarily, and then such a. function for changing the
scaling factor adaptively may be set that the speech
speed conversion factor is increased temporarily if an
amount of delay is small but the speech speed conversion
1o factor is decreased temporarily if an amount of delay
is large."
For example, in this embodiment, assume that a
start time of the first voiced sound appearing after
a time when the non-speech interval of more than 200ms
appears is set to "t=0" , and then a cosine function given
by a following Eq. 3 may be employed as a function which
can provide a factor corresponding to the start time
of the voiced sounds appearing in the range of "OS t
T~~.
f(t) = rs + 0.5(rs - re)(cos7'Ct/T + 1.0)...(3)
Where t: 0 t T
rs: an external input value by the
listener (1.OSrsS1.6)
re: a value given as an initial value
(e.g., re = 1.0)
Then, the time difference between the input data
length and the output data length is calculated at a
certain constant time interval, e.g., every one second,
CA 02392849 2002-07-30
-28-
and then the process is executed such that the initial
value re is increased from "1.0" by "0.05" and
conversely is decreased to about."0.95" according to
the time difference at that time. However, in case the
non-speech interval of more than 200 ms has not appeared
yet at a point of time in excess of the time period T,
a factor of 1.0, for example, is applied to the
succeeding voiced sound interval. In this case, a new
factor may be given by using a variable such as the pitch,
lo the power, etc. as an index.
Further, a remaining rate of the non-speech
interval may be changed adaptively in view of the speech
speed conversion factor, the extension amount, etc.
This may be set arbitrarily as a'function.
Then, a compression allowable limit (a value
indicating how long at least interval must be saved
without reduction) of the non- speech interval is set
to correspond to the external input value rs. This
limit may be expressed by the above function, but it
may be set discretely, for example, as described in the
following.
At rs = 1.0, this limit is reducible up to 300 ms
At rs = 1.1, this limit is reducible up to 250 ms
At rs = 1.2, this limit is reducible up to 230 ms
At rs = 1.3, this limit is reducible up to 200 ms
At rs = 1.4, this limit is reducible up to 200 ms
At rs = 1.5, this limit is reducible up to 150 ms
At rs = 1.6, this limit is reducible up to 100 ms
In addition, a reduction system of the non-speech
CA 02392849 2002-07-30
-29-
interval can be implemented by shifting a pointer to
any address on the ring buffer. In this embodiment,
omission of the speech information can be prevented by
shifting the pointer to the start portion of the voiced
sound immediately after the concerned non-speech
interval.
Furthermore, the speech data connector 9 reads the.
speech data from the block data memory 5 in unit of block
in compliance with the connection order decided by the
io connection order generator 8, then extends the speech
data of the designated block, then connects the speech
data and the connection data while reading out the
connection data from the connection data memory 7 and
suppressing the connection process not to cause excess
and deficiency in capacity of the FIFO memory provided
in the D/A converter 10, and then generates the output
speech data to supply them to the D/A converter 10.
The D/A converter 10 D/A-converts the output
speech data at a predetermined sampling rate ( e. g., 32
2o kHz) while buffering the output speech data supplied
from the speech data connector 9 by virtue of the FIFO
memory, then generates the output speech signal, and
then outputs it from the terminal 11.
In this manner, in this embodiment, when the
speech-speed converted speech data are synthesized by
applying an analyzing process to input speech data from
a speaker based on attributes of the speech data and
then using a desired function according to the analyzed
information, the speech speed converting device can
3o eliminate omission of the speech information against
CA 02392849 2002-07-30
-30-
change in extension/scaling factors since these
processes can be executed without inconsistency while
comparing the input data length, the target data length
calculated by multiplying the input data length by any
scaling factor, and the actual output speech data length.
And, the speech speed converting device can. eliminate
adaptively the time difference between the original
speech and the converted speech because of the speech
speed conversion by monitoring the time difference.
io which varies at every moment and changing the scaling
factor adaptively, e.g., by increasing the speech speed
conversion factor temporarily when the time difference
is small and conversely decreasing the speech speed
conversion factor temporarily when the time difference
is large, and further changing a remaining rate of the
non-speech interval adaptively based on the speech
speed conversion factor., an amount of expansion, etc.
Therefore, the speech speed conversion factor. and the
non-speech interval can be controlled adaptively
according to set conditions only by setting the
conversion factor employed as the several-stage aims
once by the user, and thus an expected effect for the
speech speed conversion can be achieved stably within
the time range being delivered actually.
As a result, the most suitable speech speed
converting effect .for respective speakers can be
provided automatically to the broadcast program in
which the speakers are changed frequently, etc. In
addition, the present invention makes it possible for
the aged person and the visually or acoustically
CA 02392849 2002-07-30
-31-
handicapped person, who are difficult to listen the
rapid talking, to listen the emergency news, which
needs real time property, and the speech in the
visual media such as the television stably and slowly
without delay in time by an extremely simple
operation.
lo Industrial Applicability
According to the speech interval detecting
method and the device for embodying the same of the
present invention, the speech interval and the non-
speech interval can be discriminated by executing the
speech processing in real time so as to respond
sequentially to change in the respective levels of
the input speech and the background sound, while
shortening the calculation time and also reducing the
cost, since only the power which can be derived
relatively simply as a feature parameter is employed.
