Note: Descriptions are shown in the official language in which they were submitted.
CA 02596411 2007-08-08
Method and System for Providing an Acoustic Signal with Extended Bandwidth
The invention is directed to a method and a system for providing an acoustic
signal, in
particular a speech signal, with extended bandwidth.
Acoustic signals transmitted via an analog or digital signal path usually
suffer from the
drawback that the signal path only has a restricted bandwidth such that the
transmitted
acoustic signal differs considerably from the original signal. For example, in
the case of
conventional telephone connections, a sampling rate of 8 kHz is used resulting
in a
maximal signal bandwidth of 4 kHz. Compared to the case of audio CD's, the
speech
and audio quality is significantly reduced.
Furthermore, many kinds of transmissions show additional bandwidth
restrictions. In
the case of an analog telephone connection, only frequencies between 300 Hz
and 3.4
kHz are transmitted. As a result, only 3.1 kHz bandwidth are available.
In principle, the bandwidth of telephone connections could be increased by
using
broadband or wideband digital coding and decoding methods (so-called broadband
codecs). In such a case, however, both the transmitter and the receiver have
to sup-
port corresponding coding and decoding methods which would require the
implementa-
tion of a new standard.
As an alternative, systems for bandwidth extension can be used as described,
for ex-
ample, in P. Jax, Enhancement of Bandlimited Speech Signals: Algorithms and
Theo-
retical Bounds, Dissertation, Aachen, Germany, 2002 or E. Larsen, R. M. Aarts,
Audio
Bandwidth Extension, Wiley, Hoboken, NJ, USA, 2004. These systems are to be im-
plemented on the receiver's side only such that existing telephone connections
do not
have to be changed. In these systems, the missing frequency components of an
input
signal with small bandwidth are estimated and added to the input signal.
An example of the structure and the corresponding signal flow in such a state
of the art
bandwidth extension system is illustrated in Fig. 6. In general, both the
lower and the
upper frequency ranges are re-synthesized.
1
CA 02596411 2007-08-08
At block 601, an incoming or received acoustic signal x(n) in digitized form
is proc-
essed by sub-sampling and block extraction so as to obtain signal vectors
x(n). Here,
the variable ndenotes the time. In this Figure, it is assumed that the
incoming signal
x(n) has already been converted to the desired bandwidth by increasing the
sampling
rate. In this conversion step, no additional frequency components are to be
generated
which can be achieved, for example, by using appropriate anti-aliasing or anti-
imaging
filter elements. In order to not amend the transmitted signal, the bandwidth
extension is
performed only within the missing frequency ranges. Depending on the
transmission
method, the extension concerns low frequency (for example from 0 to 300 Hz)
and/or
high frequency (for example 3400 Hz to half of the desired sampling rate)
ranges.
In block 602, a narrowband spectral envelope is extracted from the narrowband
signal,
the narrowband signal being restricted by the bandwidth restrictions of the
telephone
channel. Via a non-linear mapping, a corresponding broadband envelope signal
is es-
timated from the narrowband envelope. The mappings are based, for example, on
codebook pairs (see J. Epps, W. H. Holmes, A New Technique for Wideband En-
hancement of Coded Narrowband Speech, IEEE Workshop on Speech Coding, Con-
ference proceedings, pages 174 to 176 June 1999) or on Neural Networks (see J.-
M.
Valin R. Lefebvre, Bandwidth Extension of Narrowband Speech for Low Bit-Rate
Wideband Coding, IEEE Workshop on Speech Coding, Conference Proceedings,
pages 130 to 132, September 2000). In these methods, the entries of the
codebooks or
the weights of the neural networks are generated using training methods
requiring
large processor and memory resources.
Furthermore, in block 603, a broadband or wideband excitation signal having a
spec-
trally flat envelope is generated from the narrowband signal. This excitation
signal cor-
responds to the signal which would be recorded directly behind the vocal
cords, i.e. the
excitation signal contains information about voicing and pitch, but not about
form and
structures or the spectral shaping in general. Thus, to retrieve a complete
signal, such
as a speech signal, the excitation signal has to be weighted with the spectral
envelope.
For the generation of excitation signals, non-linear characteristics (see U.
Kornagel,
Spectral Widening of the Excitation Signal for Telephone-Band Speech
Enhancement,
IWAENC 01, Conference Proceedings, pages 215 to 218, September 2001) such as
two-ray rectifying or squaring, for example, may be used.
2
CA 02596411 2007-08-08
For bandwidth extension, the excitation signal xeSC (n) is spectrally colored
using the
envelope in block 604. After that, the spectral ranges used for the extension
are ex-
tracted using a band stop filter in block 606 resulting in signal vectors
yex1(n) . The
band stop filter can be effective, for example, in the range from 200 to 3700
Hz.
The signal vectors x(n) of the received signal are passed through a
complementary
band pass filter in block 605. Then, the signal components yeS, (n) and y,,(n)
are
added to obtain a signal vector y(n) with extended bandwidth. In block 607,
the differ-
ent signal vectors are assembled again and an over-sampling is performed
resulting in
a signal y(n).
In these prior art systems, the elements and their parameters are implemented
once
and, then, remain unchanged. Thus, all incoming acoustic signals are treated
the same
way. In view of this, it is an object underlying the present invention to
provide a more
flexible method and apparatus for providing an acoustic signal with extended
band-
width.
This problem is solved by the method according to claim 1 and the apparatus
accord-
ing to claim 16.
In accordance with the invention, a method for providing an acoustic signal
with ex-
tended bandwidth is provided, comprising:
(a) automatically determining a current upper and a current lower bandwidth
limit of
received acoustic signal,
(b) automatically determining at least one complementary signal to complement
the
received acoustic signal between a predefined lower broadband bandwidth limit
and the current lower bandwidth limit and/or between the current upper band-
width limit and a predefined upper broadband bandwidth limit, wherein the pre-
defined lower broadband bandwidth limit is smaller than the current bandwidth
3
CA 02596411 2007-08-08
limit and the predefined upper broadband bandwidth limit is larger than the
cur-
rent upper bandwidth limit,
(c) automatically assembling the at least one complementary signal and the re-
ceived acoustic signal to obtain an acoustic signal with extended bandwidth.
By determining current upper and lower bandwidth limits of a received acoustic
signal
and determining a complementary signal between the current bandwidth limits
and the
respective predefined broadband (or wideband) bandwidth limits, the method
according
to the invention allows an adaptation of the bandwidth extension to the
acoustic signal
actually received. For example, when the transmitter uses an ISDN telephone, a
broader frequency range is used compared to the case of a mobile phone with a
hands-free system. Therefore, the bandwidth of a received acoustic signal will
be ex-
tended only in those ranges where it is necessary so that the quality of the
resulting
signal is very high.
In this way, on the one hand, no spectral gaps will occur even if the received
signal
covers only a very narrow frequency range. On the other hand, when receiving
signals
covering a relatively broad frequency range, no frequencies are cut-off when
determin-
ing the complementary signal.
The received acoustic signal may be a digital signal or may be digitized. In
the above
method, steps (a) to (c) may be preceded by the step of converting the
received acous-
tic signal to a predetermined sampling rate. Furthermore, steps (a) to (c) may
be pre-
ceded by the step of extracting a signal vector from the acoustic signal, in
particular,
the converted acoustic signal. The signal vector may be obtained by sub-
sampling the
acoustic signal and may comprise a predefined number of entries. Then,
subsequent
(in time) signal vectors may overlap. The use of signal vectors simplifies
further proc-
essing of the signals.
Steps (a) to (c) may be preceded by the step of determining a spectral vector
of the
received acoustic signal. In particular, a window function may be applied to
signal vec-
tors of the received acoustic signal. For example, a Hann or a Hamming window
may
be used (see K. D. Kammeyer, K. Kroschel, Digitale Signalverarbeitung, 4th
Edition,
Teubner, Stuttgart, Germany 1997). Signal vectors, in particular the signal
vectors
4
CA 02596411 2007-08-08
weighted in this way, may be transformed into the Fourier domain using a
discrete Fou-
rier transform. The resulting vector is a short-term spectral vector. This
allows for fur-
ther processing in the Fourier domain.
In the above methods, step (b) may comprise determining a broadband spectral
enve-
lope signal and a broadband excitation signal between the lower and upper
broadband
bandwidth limits such that the product of spectral envelope signal and
excitation signal
corresponds to the received acoustic signal according to a predetermined
criterion.
Such a decomposition into an envelope signal and an excitation signal
simplifies de-
termining the current bandwidth limits and increases the accuracy when
determining a
complementary signal.
Step (a) may comprise comparing a determined broadband spectral envelope
signal
and a long-term power spectrum of the received acoustic signal. It turned out
that the
long-term power spectrum is a suitable basis for determining current bandwidth
limits
of the acoustic signal.
Thus, if current bandwidth limits have been determined in step (a) in this way
using a
broadband spectral envelope signal of the received acoustic signal,
determining a
complementary signal in step (b) based on these current bandwidth limits and
compris-
ing determination of an envelope signal enables to iteratively adapt the
current band-
width limits by comparing again the (newly) determined envelope signal and a
long-
term power spectrum. In other words, determining current bandwidth limits in
step (a)
may use a spectral envelope signal determined according to step (b),
particularly in a
preceding step or in a preceding iteration of the method.
In particular, if the received acoustic signal has been transformed into the
Fourier do-
main, determining a long-term power spectrum may comprise performing a first
order
recursive smoothing of the absolute values squared of the sub-band signals
corre-
sponding to the acoustic signal. This can be done, in particular, only if a
wanted signal,
such as a speech signal, has been detected in the received acoustic signal.
In addition, the long-term power spectrum may be normalized, particularly with
respect
to a long-term power spectrum within predetermined frequency limits.
5
CA 02596411 2007-08-08
Alternatively, the long-term power spectrum may be determined in the time
domain.
This can be done by determining the auto-correlation and performing an LPC
analysis
to obtain corresponding prediction coefficients.
The comparing step may comprise selecting the minimal and maximal frequency
for
which the long-term power spectrum is larger than or equal to the power
spectrum of
the determined broadband spectral envelope signal plus a predetermined
constant.
This is a particularly simple and reliable way to determine the bandwidth
limits. The
predetermined constant can be chosen based on empirical or theoretical data.
The
predetermined constant may be negative.
In the above methods, determining a broadband spectral envelope signal may com-
prise selecting an envelope signal from a codebook according to a
predetermined crite-
rion.
By using codebooks, the required computing power can be reduced for
determining an
envelope signal. In principle, different kinds of criteria can be used when
selecting an
envelope signal from a codebook. In particular, using a predetermined distance
crite-
rion such as a cepstral distance can be used, particularly if the codebook
entries have
the form of cepstral vectors.
In particular, selecting an envelope signal may comprise equalizing the
received
acoustic signal and selecting an envelope signal from the codebook having
minimal
distance to the equalized acoustic signal according to a predetermined
distance crite-
rion, in particular, having a minimal cepstral distance.
Equalizing the acoustic signal allows to modify it such that a comparison with
envelope
signals from the codebook can be simplified. In particular, the received
acoustic signal
can be equalized in such a way that the resulting signal shows a long-term
power
spectrum corresponding to the long-term power spectrum of the signal used for
training
the codebook. Equalizing can be restricted to frequencies between the current
upper
and lower bandwidth limits of the received acoustic signal; outside these
limits, the sig-
nal may remain unchanged. In particular, equalizing the received acoustic
signal can
6
CA 02596411 2007-08-08
be performed using a normalized long-term power spectrum of the signal used
for
training the codebooks, particularly using the normalized long-term power
spectrum
divided by the normalized long-term power spectrum of the received acoustic
signal
itself.
The codebook may comprise pairs of corresponding envelope signals, each pair
com-
prising a broadband envelope signal between the lower and upper broadband band-
width limits and a corresponding narrowband envelope signal between a lower
narrow-
band bandwidth limit being larger than the lower broadband bandwidth limit and
an
upper narrowband bandwidth limit being smaller than the upper broadband
bandwidth
limit, and selecting an envelope signal may comprise determining a narrowband
enve-
lope signal having minimal distance to the equalized acoustic signal according
to the
predetermined distance criterion and selecting the corresponding broadband
envelope
signal of this pair.
In this way, a simple comparison between the received acoustic signal and the
ele-
ments of the codebook can be performed as narrowband signals usually match a
re-
ceived acoustic signal with a narrow bandwidth more closely.
When using a cepstral distance to select an envelope signal, the received
acoustic sig-
nal, particularly in its equalized form, has to be transformed into the
cepstral domain.
Thus, the step of selecting an envelope signal can further comprise the steps
of deter-
mining the absolute value squared of the sub-band signals of the received
acoustic
signal, determining an auto-correlation in the time domain, particularly by
performing
an inverse discrete Fourier transform on the vector of the absolute value
squared, de-
termining prediction coefficients, particularly using the Levinson-Durbin
algorithm, per-
forming a recursion to obtain the cepstral coefficients.
In order to determine a spectral envelope from the cepstral vectors, the
method may
further comprise the steps of recursively transforming a cepstral vector into
prediction
error coefficients, augmenting the prediction error filter vector by adding a
predeter-
mined number of zeros and subsequently performing a discrete Fourier transform
to
obtain an inverse spectrum, determining the reciprocal of each sub-band
component to
obtain a spectral envelope vector.
7
CA 02596411 2007-08-08
In the above methods, the step of selecting an envelope signal may be preceded
by
providing adapted narrowband codebook envelope signals being adapted to the
cur-
rent lower and upper bandwidth limits.
Such an adaptation of the codebook entries allows for an improved selection of
a cor-
responding envelope signal from the codebook. In particular, if the received
acoustic
signal shows a broader bandwidth than the original narrowband envelope signals
of the
codebook, the adaptation would result in envelope signals in the codebook
having an
extended bandwidth. In this way, particularly fricatives can be more reliably
detected.
The providing step may comprise processing broadband codebook envelope signals
using a long-term power spectrum of the received acoustic signal.
Due to the use of the power spectrum of the received acoustic signal, a
suitable adap-
tation to the acoustic signal can be obtained. The long-term power spectrum
may be
normalized; furthermore, the long-term power spectrum of the received acoustic
signal
may be divided by a normalized long-term power spectrum of a broadband signal
used
for training of the codebook. The processing of the broadband codebook
envelope sig-
nals may be performed only for frequencies outside the current bandwidth
limits; within
the bandwidth limits, the envelope signals may remain unchanged. Processing
using
the long-term power spectrum may comprise weighting broadband codebook
envelope
signal vectors using the long-term power spectrum of the received acoustic
signal.
In the above methods, determining a broadband excitation signal may be based
on
prediction error filtering and/or a non-linear characteristic. In this way,
suitable excita-
tion signals can be generated. Possible non-linear characteristics are
disclosed, for
example, in U. Kornagel, Spectral Widening of the Excitation Signal for
Telephone-
Band Speech Enhancement.
In the above methods, the at least one complementary signal may be based on a
product of the determined broadband spectral envelope and the determined
broadband
excitation signal, and step (c) may comprise summing the received acoustic
signal be-
tween the current lower and upper bandwidth limits and the at least one
complemen-
tary signal being restricted to the band between the lower broadband bandwidth
limit
8
CA 02596411 2007-08-08
and a current lower bandwidth limit and/or to the band between the current
upper
bandwidth limit and the upper broadband bandwidth limit.
Thus, the complementary signal is based on spectrally coloring the excitation
signal
using the envelope signal. By adding a complementary signal only outside the
current
bandwidth limits of the received acoustic signal, artifacts are avoided in the
resulting
signal with extended bandwidth.
Step (c) may also comprise adapting the power of the complementary signal
and/or the
received acoustic signal. With this step, the power of the received acoustic
signal can
be maintained.
In the above-described methods, at least one of the steps may be performed in
the
cepstral domain. Particularly if the entries of the codebook are cepstral
vectors, this
allows for performing the method in a simpler way.
Steps (a) to (c) of the above methods may be repeated at predetermined time
intervals.
Then, the repeated adaptation to the currently received acoustic signal leads
to a per-
manent high quality of the resulting broadband signal.
Steps (a) to (c) of the above methods may be repeated only if a wanted signal
compo-
nent, such as speech activity, is detected in the received acoustic signal.
Particularly in
the case of speech signals, an extension of the bandwidth of the received
acoustic sig-
nal is advantageous. Thus, restricting the method to the case of detected
speech activ-
ity reduces the required computing power and avoids the presence of artifacts
due to
mal-adaptation.
The invention also provides a computer program product comprising one or more
com-
puter-readable media having computer-executable instructions for performing
the steps
of the above-described methods when run on a computer.
Furthermore, an apparatus for providing an acoustic signal with extended
bandwidth is
provided, comprising:
9
CA 02596411 2007-08-08
bandwidth determining means for automatically determining a current upper and
a current lower bandwidth limit of a received acoustic signal,
complementary signal means for automatically determining at least one com-
plementary signal to complement the received acoustic signal between a prede-
fined lower broadband bandwidth limit and the current lower bandwidth limit
and/or between the current upper bandwidth limit and a predefined upper
broadband bandwidth limit, wherein the predefined lower broadband bandwidth
limit is smaller than the current bandwidth limit and the predefined upper
broad-
band bandwidth limit is larger than the current upper bandwidth limit, and
assembling means for automatically assembling the at least one complementary
signal and the received acoustic signal to obtain an acoustic signal with ex-
tended bandwidth.
Analogous to the above-described method, such an apparatus provides an advanta-
geous way to extend the bandwidth of a received acoustic signal. In
particular, due to
the determination of current upper and lower bandwidth limits of the received
acoustic
signal and a corresponding determination of a complementary signal, the
quality of the
resulting output signal is increased compared to the case of bandwidth
extension sys-
tems with fixed parameters.
The complementary signal means may comprise a means for determining a
broadband
spectral envelope signal and a broadband excitation signal between the lower
and up-
per broadband bandwidth limits such that the product of spectral envelope
signal and
excitation signal corresponds to the received acoustic signal according to a
predeter-
mined criterion.
The bandwidth determining means may be configured to compare a determined
broadband spectral envelope signal and a long-term power spectrum of the
received
acoustic signal.
The bandwidth determining means may be configured to select the minimal and
maxi-
mal frequency for which the long-term power spectrum is larger than or equal
to the
CA 02596411 2007-08-08
power spectrum of the determined broadband spectral envelope signal plus a
prede-
termined constant.
In the above-described apparatus, the means for determining a broadband
spectral
envelope signal may comprise a means for selecting an envelope signal from a
code-
book according to a predetermined criterion.
The means for selecting an envelope signal may be configured to equalize the
re-
ceived acoustic signal and select an envelope signal from the codebook having
mini-
mal distance to the equalized acoustic signal according to a predetermined
distance
criterion, in particular, having a minimal cepstral distance.
In the above-described apparatus, the codebook may comprise pairs of
corresponding
envelope signals, each pair comprising a broadband envelope signal between the
lower and upper broadband bandwidth limits and a corresponding narrowband enve-
lope signal between a lower narrowband bandwidth limit being larger than the
lower
broadband bandwidth limit and an upper narrowband bandwidth limit being
smaller
than the upper broadband bandwidth limit, and the means for selecting an
envelope
signal may be configured to determine a narrowband envelope signal having
minimal
distance to the equalized acoustic signal according to the predetermined
distance crite-
rion and to select the corresponding broadband envelope signal of this pair.
The means for determining a broadband spectral envelope signal may comprise a
means for providing adapted narrowband codebook envelope signals being adapted
to
the current lower and upper bandwidth limits.
The means for providing may be configured to process the broadband codebook
enve-
lope signal using a long-term power spectrum of the received acoustic signal.
In the above-described apparatus, the means for determining a broadband
excitation
signal may be configured to determine the broadband excitation signal based on
pre-
diction error filtering and/or a non-linear characteristic.
The at least one complementary signal may be based on a product of the
determined
broadband spectral envelope and the determined broadband excitation signal,
and the
11
CA 02596411 2007-08-08
assembling means may be configured to sum the received acoustic signal between
the
current lower and upper bandwidth limits and the at least one complementary
signal
being restricted to the band between the lower broadband bandwidth limit and a
cur-
rent lower bandwidth limit and/or to the band between the current upper
bandwidth limit
and the upper broadband bandwidth limit.
In the above-described apparatus, at least one of the means may be configured
to per-
form at least part of its function in the cepstral domain.
The means of the above-described apparatus may be configured to perform their
re-
spective function repeatedly at predetermined time intervals.
The apparatus may further comprise a wanted signal detector, in particular, a
speech
detector, and the means may be configured to perform their respective function
only if
a wanted signal component is detected in the received acoustic signal.
Further features and advantages of the invention will be described in the
following with
reference to the figures.
Fig. 1 illustrates the structure of an example of an apparatus for providing
an acoustic
signal with extended bandwidth;
Fig. 2 is a flow diagram of an example of a method for providing an acoustic
signal
with extended bandwidth;
Fig. 3 illustrates an example of a normalized long-term power spectrum for
training a
codebook;
Fig 4 illustrates examples of codebook entries;
Fig. 5 illustrates the determination of current bandwidth limits;
Fig. 6 illustrates the structure of a prior art system.
12
CA 02596411 2007-08-08
Fig. 1 shows the structure of the signal flow in an apparatus for providing an
acoustic
signal with extended bandwidth. Fig. 2 is a flow diagram illustrating an
example of a
method for providing an acoustic signal with extended bandwidth which could be
per-
formed by the apparatus corresponding to Fig. 1. In view of this, Fig.'s 1 and
2 will be
described in the following simultaneously.
According to step 201, an acoustic signal, such as a speech signal, is
received via a
telephone line. Because of the restricted bandwidth of the telephone line, an
extension
of the bandwidth is desired to improve the signal quality. Thus, the signal is
to be aug-
mented so as to obtain a predetermined broader bandwidth. It is to be
understood that
the method described in the following can be used for bandwidth extension
independ-
ent of the type of incoming signal and independent of the type of transmission
line, i.e.,
it need not be a telephone line.
The acoustic signal x(n) received by block 101 has already been pre-processed
by
increasing the sampling rate up to the predetermined broadband or wideband
band-
width. In this way, however, no additional frequency components are generated.
This
can be achieved, for example, by using suitable anti-aliasing or anti-imaging
filters.
This kind of bandwidth extension, preferably, is performed only for the
"missing" fre-
quency ranges; in the case of an analog telephone line, these ranges may be
between
0 and 300 Hz and 3400 Hz up to half of the desired sampling rate, for example,
up to
3700 Hz.
From the resulting signal x(n), n denoting the time variable, signal vectors
x(n) are
generated (step 202). This can be achieved by taking every r sampling values
up to a
certain length. Thus, a signal vector with N. elements has the form:
x(n) = [x(nr), x(nr -1),..., x(nr - NQõo + 1)]T.
It is to be noted that an overlap may exist between neighboring signal
vectors. For a
desired or final sampling rate of 11.025 kHz, one may take the following
values:
r = 64,
NanQ = 256.
13
CA 02596411 2007-08-08
After that (step 203), a windowing procedure is performed on the signal vector
so as to
obtain a windowed signal vector xjn) :
x W (n) = Fx(n).
The window matrix F is a diagonal matrix of the form
ho 0 0 ... 0
0 h, 0 ... 0
F= 0 0 h2 ... 0
0 0 0 ... h
The elements of this matrix can be chosen corresponding to different kinds of
windows.
Typical windows are the Hann or Hamming window. The weighted signal vectors
are
transformed into the Fourier domain using a discrete Fourier transform:
XW(n) = DFT{xw(n)}.
The resulting short-term spectral vector has the form:
X w(n) = f X(e i~ , n), X(e'n', n),..., X(e'n" , n),..., X(e'O"D~-1 , n)~ ,
'
wherein S2, denotes the frequency variable.
Based on the spectral vectors, a long-term power spectrum of the received
acoustic
signal is determined in block 102 (step 204). There are different
possibilities to esti-
mate such a long-term power spectrum. According to one alternative, a first
order re-
cursive smoothing is performed on the absolute value squared of the sub-band
signals
X (e'"" , n) :
14
CA 02596411 2007-08-08
2
~3freS~ (S2F,, n -1) + (1- /3fre ) X. (e'n", n)I , during speech activity
S. (SZ'õ n) _
S~ (52u , n -1), else.
Preferably, the time constant 8fre is chosen to be close to 1 (0 /3fre < 1)
so as to
obtain a sufficiently large averaging time.
In principle, the recursive smoothing according to the first line of the above
equation
may be performed continuously. However, in order to avoid any artefacts, it
may be
performed only if a wanted signal component is present in the received
acoustic signal,
for example, if speech activity is detected. For this purpose, a speech
detector may be
provided as described, for example, in E. Hansler, G. Schmidt, Acoustic Echo
and
Noise Control - A Practical Approach, Wiley, Hoboken, NJ, USA, 2004.
In order to simplify the further processing, the long-term power spectrum may
be nor-
malized to the long-term power within a predefined frequency band:
S~ (52P, n)
Sxc,norm (Q/t 5 n) _ P.
E(f2,,, n)
/l=fl,
The band limits S2. and S2Pu denote the lower and upper limits of a predefined
fre-
quency band. For example, this frequency band may correspond to a telephone
band
with minimal bandwidth for which the present method is to be used, for
example, the
limits may be 400 Hz and 3300 Hz. Preferably, the limits correspond to a band
which is
smaller or at most equal to the frequency band of the narrow frequency band
within
which the codebook described below has been trained; these limits being
denoted by
S2, and S2n.
Alternatively, to determine the long-term power spectrum in the frequency
domain, an
estimation can be performed in the time domain as well. For this purpose, an
auto-
correlation is estimated for about 10 to 20 sampling cycles of offset.
Afterwards, predic-
tion coefficients can be determined using an LPC (linear predictive coding)
analysis.
CA 02596411 2007-08-08
The long-term power spectrum is obtained via a discrete Fourier transform and
a divi-
sion.
In block 103 (step 205), the acoustic signal is equalized. The equalizing is
performed
on the spectral vector determined above:
Xeg (n) = Heg (n)X,, (n).
The equalizing matrix Heq (n) is a diagonal matrix of the form
He9(en) 0 ... 0
_ 0 Heq(e'"',n) ... 0
He9(n)
0 0 ... He9(e'Q" -1,n)
with the entries
1 if ((52p < S2, (n - 1)) or (0, > SZõ (n -1)))
He9 (e , n) = SxX,norm (52,, n), else
Sxx,norm (o{( I n)
and
Heq,max , if Heq (e j"v, n) > Hee.m.
He9 (e'~" , n) = Hevmin I lf (He9 (e,fl, I n) < He9,min
Heq (e'n" , n), else,
I n the equations above, S2, (n -1) and S2u (n -1) denote the current lower
and upper
bandwidth limits of the received acoustic signal. Thus, for obtaining an
updated equal-
ized signal, the bandwidth limits at time (n -1) are taken as the current
bandwidth lim-
its. Furthermore, Sxs,norm (RU In) denotes the normalized long-term power
spectrum of
the broadband signal which has been used for training the codebook.
Normalizing of
16
CA 02596411 2007-08-08
such a power spectrum is performed analogously to the case of the long-term
power
spectrum of the received acoustic signal described above. An example for such
a nor-
malized long-term power spectrum used for training a codebook is shown in
Figure 3.
The equalizing is restricted to minimal and maximum values, for example, to
He9,m;,, _ -12 dB,
Heg,max =12 dB.
As can be seen from the above, the acoustic signal is equalized only within
the current
bandwidth limits one time step before. Outside these bandwidth limits, no
equalizing
takes place.
In the following, determining a broadband spectrum envelope will be described
in more
detail. An envelope signal corresponding to the received acoustic signal will
be deter-
mined using a codebook. The used codebook comprises a number of pairs of corre-
sponding narrowband and broadband envelope signals. The codebook has been ob-
tained by training with a large database on the basis of a starting long-term
power
spectrum (see Y. Linde, A. Buzo, R. M. Gray, An Algorithm for Vector Quantizer
De-
sign, IEEE Trans. Comm., vol. COM-28, no. 1, pages 84 - 95, Jan. 1980).
As indicated in Figure 2, the codebook entries are adapted in step 206 (block
104). In
particular, the narrowband codebook entries c; .S (n) are adapted.
This is achieved by starting with the broadband entries of the codebook. If
the broad-
band envelope signals are provided as cepstral vectors c; b(n) , the
corresponding
spectra C; h(n) are determined. Based on these broadband spectral envelopes,
the
adapted or optimized narrowband spectra are determined by a multiplication
with a
weighting matrix:
C;,s (n) = Amoa (n)C;,b (n)=
The weighting matrix is a diagonal matrix of the form:
17
CA 02596411 2007-08-08
Hmod (e'" n) 0 . . . 0
0 Hmod (e'o' I n) . . . 0
Hmod (n)
-
0 0 ... Hmod (e,QNDFT-, e n)
with the entries
1, if~SZ,(n-1)<S2IU <S2õ(n-1)~,
joa
Hmod (e n) - Sxx,norm Ru . n)
!~ else.
Sz,norm \~LU , n)
Afterwards, cepstral vectors are determined from the resulting spectral
narrowband
envelopes.
The conversion from spectral vectors to cepstral vectors and vice versa will
be de-
scribed in the following with respect to step 207 in which broadband spectral
envelopes
are determined (block 105).
A broadband spectral envelope from the codebook matching the acoustic signal
best is
determined by comparing the narrowband codebook entries with the spectral
envelope
of the spectrum of the acoustic signal (after equalizing). The narrowband
codebook
entry is selected that has the smallness distance to the acoustic signal
spectrum. In
principle, different distance criteria can be used. The cepstral distance is
particularly
useful as the codebook entries are provided in the form of cepstral vectors.
When an optimal narrowband codebook entry has been selected, the corresponding
broadband codebook entry is determined as the optimal broadband spectral
envelope
for the received acoustic signal. Due to the adaptation of the narrowband
codebook
entries as described above, an optimal narrowband envelope can be selected in
a very
reliable way.
Converting a spectral vector, particularly of the received acoustic signal, to
a cepstral
vector can be achieved by:
18
CA 02596411 2007-08-08
1. Determining the absolute value squared of each sub-band signal Xeq (e'n, ,
n).
2. Applying an inverse discrete Fourier transform on this vector results in an
esti-
mation of the auto-correlation in the time domain.
3. Using the Levinson-Durbin algorithm, prediction coefficients (with an order
of
about 10 to 20) can be determined from the auto-correlation.
4. By performing a recursion with respect to the order, the prediction
coefficients
are used to determine the cepstral coefficients. Usually, the order
corresponds
to one and a half of the prediction order.
The optimal cepstral vector of the broadband codebook is designated by Cpt
b(n). The
resulting broadband spectral envelope has the form:
~ T
Copt,b (n) = lcopt,b(e j", , n), Copt,b (e'~' , n),..., Copt b(eJ vDF7 n)]
Conversion of cepstral vectors into spectral vectors is achieved by:
1. Converting the cepstral vectors using a recursion with respect to the order
(as
above) to obtain prediction error filter coefficients.
2. By augmenting the prediction error filter vector by a predetermined number
of
zeros and subsequent performing of a discrete Fourier transform, an inverse
spectrum is obtained.
3. By determining the reciprocal of each sub-band component, the vector Copt,b
(n)
is generated. Divisions by zero have to be treated separately, for example by
adding a suitable constant.
Fig. 4 illustrates an example of a codebook with four pairs of entries. In
each diagram,
a corresponding original narrowband envelope, and a corresponding adapted
narrow-
19
CA 02596411 2007-08-08
band envelope are shown. The original broadband and narrowband codebook
entries
have been obtained on the basis of a large database for an ISDN telephone
connec-
tion. As can be seen in this figure, after the adaptation, the resulting
optimized entries
have a higher upper limit frequency. This allows for an improved detection of
fricatives.
In step 208 (block 103), an excitation signal corresponding to the received
acoustic
signal is generated. This broadband excitation signal shows a spectrally flat
envelope.
It corresponds to a signal which would be recorded directly behind the vocal
cords.
For determining a broadband excitation signal, first of all, the spectral
envelope of the
equalized short-term spectrum Xeq (n) is estimated in the form of prediction
error filter
coefficients. Applying an inverse discrete Fourier transform on this spectral
vector al-
lows to determine the corresponding time signal. After that, the vector in the
time do-
main is filtered by a prediction error filter. The corresponding filter
coefficients are those
that have been determined previously.
Then, a non-linear characteristic, such as a two-way rectification or
squaring, is applied
to the filtered time domain vector. This generates the missing low frequency
and high
frequency signal components. A transformation in the Fourier domain provides,
then,
the spectrum of the extended excitation signal XeSC (n).
Alternatively, determining an excitation signal can be performed in the time
sub-band
or Fourier domain as well. Examples for this alternative can be found in B.
Iser, G.
Schmidt, Bandwidth Extension of Telephony Speech, Eurasip Newsletter, Volume
16,
Number 2, pages 2 to 24, June 2005.
In the following step 209 (block 107), the broadband spectral envelope and the
excita-
tion signal are used for spectrally coloring the excitation signal. This can
be achieved
by multiplication in the sub-band or Fourier domain:
Yex: (n) = diag{Copi,b (n)}XeSC (n)=
The diagonal matrix diag{CoP, b(n)} has the form:
CA 02596411 2007-08-08
CoP, b(ei1, n) 0 ... 0
0 Copt h(e'"' , n) ... 0
diag{Cop, b (n)} _
JQNDFT '
0 0 ... Cop,b (e ,n)
Because of the non-linearity or the prediction error filtering when generating
the excita-
tion signal, the power of the acoustic signal need not be maintained.
Therefore, a
power adaptation may be performed:
Ye,f (n) = K(n)Ye, (n).
The correction factor K can be chosen to be
At.
JIXw\e'Q'njz
K(n) = u=N,
z
jna
~YerWe n
wherein S2U and S2Pu denote the same bandwidth limits as in the estimation of
the
long-term power spectrum above.
The current bandwidth limits are adapted in step 210 (block 108). According to
one
possibility, the bandwidth limits are determined starting with a comparison of
the spec-
trum of the received acoustic signal and the broadband spectral envelope being
re-
duced by a predefined constant:
z z l
S2, (n) = min S2u E f X (e'~ "~ > CopI b (e'll" ") + K,
l
S2" (n) = max S2~ E ~IXW (e'~"'" I z > C~,p, b (e'n '" + K, ~ }.
JJ
21
CA 02596411 2007-08-08
The parameter Kc can have the value:
Kc = -12 dB.
In Fig. 5, an example for determining the bandwidth limits is illustrated. The
above, in-
termediate limit values are given by the points of intersection between the
lowered
broadband spectral envelope and the spectrum of the received acoustic signal.
These intermediate limit values may be recursively smoothed to eliminate
temporary
mal-estimations. In this case, preferably, smoothing is performed only if
speech activity
is detected in the current signal frame.
~1(n) = Qbandl ~! (n -1) + (1- ~36aõd, )S2, (n), during speech activity,
S2, (n -1), else,
Tiu (n) _ j8bandl ?iu (n -1) + (1- j8b,,,,d, )S2v (n), during speech activity,
S2õ (n -1), else.
Then, the received acoustic signal is passed through an adaptive band pass
filter to
retain only components within the current bandwidth limits (block 109) to
obtain a spec-
tral vector Y,e, (n). Similarly, the spectrally colored excitation signal is
passed through a
complementary adaptive band stop filter (block 110) so as to obtain a vector
Y,,, (n).
An output signal with a standard bandwidth is generated (step 211) by starting
with
summing these two spectral vectors:
Y(n) = I'lel (n) + Yex, (n).
The components of these vectors are generated as:
Yrel (n) = <T re! (n)Aw (n)f
22
CA 02596411 2007-08-08
Yex, (n) = G eX, (n)X ex, (n),
wherein the weighting matrices G1e, (n) and Gex1(n) are diagonal matrices:
Gtel (e'f2 , n) 0 . . . 0
0 Gter (e'n' , n) . . . 0
G tel (n)
ioN
0 0 ... Grer (e p~, _, ~ n)
Ge, (e'c , n) 0 . . . 0
G (n) = 0 Gex, (e'K2' , n) ... 0
eX~
0 0 . . . Gex, (e j'N-' , n)
The elements of the matrix G1e, (n) are determined as:
jQ' 1, if S2, (n) <- S2AI <- S2õ (n),
Giel (e , n)
0, else.
The weights of the complementary weighting matrix are determined so as to
yield the
unity matrix when summed:
inõ inõ
Ge, (e , n) =1- Gtei (e ~ n).
Alternatively, the transitions at the bandwidth limits can be realized in a
smoother way.
The resulting output spectrum Y(n), then, is transformed into the time domain
via an
inverse Fourier transform:
y(n) = IDFT{Y(n)},
23
CA 02596411 2007-08-08
followed by windowing the resulting vector. Particularly when using the above-
indicated
values for Nana and r and a Hann window, this window function can be used
again to
obtain windowed time domain vectors:
y w (n) = I+y (n)=
The resulting time domain vectors are, then, assembled using an overlap add
method
(as described in K. D. Kammeyer, K. Kroschel, Digitale Signalverarbeitung) to
obtain
the final output signal y(n).
In the above-described steps of the method, more complex filter bank systems
may be
used instead of the conventional discrete Fourier transform and inverse
discrete Fou-
rier transform (see, for example, P. P. Vaidyanathan, Multirate Systems and
Filter
Banks, Prentice Hall, Englewood Cliffs, NJ, USA, 1992).
Further alternatives to the above-described variants are possible as well. For
example,
the steps performed in the Fourier domain may also be performed in the time
domain.
Furthermore, equalizing the acoustic signal may be performed when adapting the
nar-
rowband codebook entries. Also, the above-described equalizing step may be aug-
mented. For example, if an amplification or an attenuation is detected at
particular fre-
quencies, it may be adjusted within the bandwidth limits as well. In this
case, the output
vector Yfer (n) is modified with the weighting matrix Hm.d (n).
In addition to the above-described codebook analysis for estimating the
broadband
spectral envelopes, a so-called linear mapping (see B. Iser, G. Schmidt,
Bandwidth
Extension of Telephony Speech) may be used additionally.
Further modifications and variations of the present invention will be apparent
to those
skilled in the art in view of this description. Accordingly, the description
is to be con-
strued as illustrated only and is for the purpose of teaching those skilled in
the art the
general manner of carrying out the present invention. It is to be understood
that the
forms of the invention shown and described herein are to be taken as the
presently
preferred embodiments.
24
