Note: Descriptions are shown in the official language in which they were submitted.
CA 02306095 2000-04-12
WO 99/27668 PCT/US98/12034
VOICE RECOGNITION UNIT
FOR AUDIENCE MEASUREMENT SYSTEM
technical Field of the Invention
The present invention relates generally to an
audience measurement system that identifies members of an
audience of a program to which a receiver, such as a
television, radio, or computer, is tuned, and/or that
identifies a program to which a receiver, such as a
television, radio, or computer, is tuned.
background of the Invention
Measuring audiences of programs has long been
of interest to program suppliers and advertisers because
audience measurements provide the data from which the
effectiveness of programs and advertisements may be eval-
IS uated. A variety of well known methods have been em-
ployed in order to provide an estimate of the total audi-
ence to a program, to a portion of a program, and/or to a
commercial. In addition to the total audience, these
methods provide additional estimates of demographically
significant audience segments (e.g., the number of men
and/or women between certain ages who watched a selected
- 1 -
CA 02306095 2000-04-12
WO 99/27668 PCTIUS98/12034
portion of a given program). These methods involve manu-
ally and/or automatically measuring the viewing and/or
listening habits of audience members of statistically
selected households and then estimating the total audi-
ence based on these measurements. Audience members of
statistically selected households are usually referred to
as panelists.
The measurement of the viewing and/or listening
habits of an audience generally requires three separate
determinations: (1) a determination of the channels or
stations to which the monitored receivers are tuned with-
in the statistically selected households; (2) a determi-
nation of the programs which were available on the tuned
channels at the times during which the receivers were
tuned to these channels or stations; and, (3) a determi-
nation of the household members and visitors who were
present in the area of the receivers during the time that
the receivers were tuned to selected programs.
All three determinations can affect the accu-
racy of audience measurements. Of the three determina-
tions discussed above, determination (3) perhaps most
- 2 -
CA 02306095 2000-04-12
WO 99127668 PCT/US98/12034
affects the accuracy of audience measurements because the
members of an audience of a receiver are usually required
to cooperate in the identification process. When members
are required to participate in the identification pro-
cess, they frequently make errors, or they simply forget
to identify either themselves or the other members of the
audience. For example, one known system which allows
audience members to identify themselves in order to indi-
cate their presence is a "People Meter." The "People
Meter" may be a remote control and/or a plurality of push
buttons activated by the audience members when they enter
or exit the monitored area. However, audience members
frequently forget to press any button at all, or they
sometimes make errors in pressing the right button.
In an attempt to overcome this problem, several
measurement systems have been developed. For example, in
U.S. Patent No. 3,056,135, Currey et al. describe a sys-
tem which utilizes strategically placed switches for
counting the number of persons entering, leaving, and re-
maining within a monitored area, and a photographic re-
corder for periodically recording the composition of the
- 3 -
CA 02306095 2000-04-12
WO 99/27668 PCT/US98/12034
audience. This approach requires that the photographic
record be viewed by an operator, which both invades the
privacy of the audience members, and imposes an unaccept-
able cost on the measurement operation.
These problems led to a variety of suggestions
for passive, non-obtrusive methods of counting (but not
identifying) viewers, and of tracking their movement
about the monitored area. For example, Kiewit, et al.,
in U.S. Patent No. 4,644,509, describe an ultrasonic
sonar system which counts and tracks members of the audi-
ence. However, the described system does not identify
the members of the audience and, therefore, requires the
members of the audience to manually identify themselves.
Other systems, such as that described by Kiewit
in U.S. Patent 4,930,011, rely upon electronically active
tags which are worn or carried by members of an audience
and which uniquely identify corresponding ones of these
audience members. These systems respond to the tags
whenever the audience members wearing the tags are within
the monitored area of a receiver. However, audience
- 4 -
CA 02306095 2000-04-12
WO 99/27668 PCTIIJS98/12034
members often forget to wear their tags, in which case
such audience members are not identified.
Lu, in U.S. Patent No. 4,858,000, and in U.S.
Patent No. 5,031,228, discloses an image recognition
system for identifying the members of an audience of a
receiver. The system uses a video camera to acquire
images of the faces of the audience members, and a com-
puter subsystem to recognize the faces in these images by
comparing these faces to reference faces stored in a data
base. The system also locates and tracks viewers, and
includes infrared illuminators that provide a control-
lable level of imperceptible illumination of the moni-
toyed area for the video camera. However, the system is
generally expensive and difficult to install.
The system disclosed by Lu et al., in U.S.
Patent No. 5,550,928 also uses image recognition to iden-
tify audience members. This system incorporates algo-
rithms for finding bodies and then heads of members of an
audience. Once the head is found, a face finding algo-
rithm extracts facial image representations that are
provided to two face recognition routines, an Eigenface
- 5 -
CA 02306095 2000-04-12
WO 99/27668 PCTNS98/IZ034
recognition routine and a template matching routine.
This system can identify audience members who are moving
about the monitored area and/or who are not facing the
cameras. However, it is still complex and expensive.
Determinations (1) and (2) discussed above also
can affect the accuracy of audience measurements because
one of these determinations alone is normally not suffi-
cient to identify the programs to which receivers are
tuned. That is, a determination of the channels to which
a receiver in a statistically selected household is tuned
does not lead to a determination of the tuned programs
without also knowing what programs were available on the
tuned channels. By the same token, a determination of
the programs that were available on the tuned channels
does not lead to a determination of the tuned programs
without knowing the channels to which the receiver in a
statistically selected household is tuned. Thus, both
determinations are typically required in order to deter-
mine the programs tuned by the audience members.
Acquisition of information about available programs and
entering this information into a database for later
- 6 -
CA 02306095 2000-04-12
WO 99!17668 pCTIUS98/I2034
matching with the tuned channels leads to acquisition and
data entry errors that adversely affect the accuracy of
audience measurements.
Thomas et al., in U.S. Patent No. 5,481,294,
addresses these problems with an audience measurement
system that detects and records ancillary codes which are
embedded in, and identify, the programs to which receiv-
ers are tuned. Thus, only one determination needs to be
made if an ancillary code is present in the viewed pro-
grams. If ancillary codes are not available in the tuned
programs, program signatures are extracted from the pro-
grams and are compared to reference signatures previously
extracted.
The metering apparatus disclosed in this patent
may include non-intrusive sensors, such as microphones
near the receiver, that detect the ancillary codes in the
audio output of the monitored receivers. If microphones
are used to detect the ancillary codes, second micro-
phones may be provided to pick up background noise. The
outputs of the ancillary code sensing microphones and the
noise sensing microphones are processed in order to sub-
CA 02306095 2000-04-12
WO 99/27668 PCT/US98/12034
tract the background noise from the signals provided by
the ancillary code sensing microphones.
This audience measurement system may also in-
clude a "People Meter" as described above for additional
backup.
Turner et al., in U.S. Patent No. 4,907,079,
disclose a system for monitoring audiences of receivers.
The members of the audience provide control commands
(such as channel selections) to a microprocessor through
an infrared unit which includes a keyboard. Information
collected by the microprocessor may be stored and trans-
mitted to a central computer over conventional phone
lines. A motion detector coupled to the microprocessor
detects when an audience member has entered or exited the
monitored area. A character generator coupled to the
microprocessor allows communication with the audience
members via alphanumeric messages displayed, for example,
on a television screen. Also, a speech output unit cou-
pled to the microprocessor generates speech signals to
communicate to the audience members.
_ g _
CA 02306095 2000-04-12
WO 99/27668 PCT/US98/12034
The Turner et al. system also includes a speech
input unit coupled to the microprocessor to allow audi-
ence members to audibly identify themselves. Audible
speech from the audience members is detectable by a mi-
crophone, is amplified, is digitized, and is provided to
the microprocessor. The microprocessor performs a power
density spectrum analysis on the signal to derive voice
signatures of the particular audience members. These
signatures are compared to reference signatures, which
were previously stored during initialization of the sys-
tem, in order to identify viewers. The microprocessor
may also normalize the digital voice data and remove
small amplitude samples in order to minimize the effects
of background noise. When the system disclosed in this
Turner et al. patent is activated, an initial display
message is placed on the viewing screen which requests
the audience to speak their first and last names in order
to provide the reference signatures. Voice recognition
by itself does not, however, permit a viable audience
measurement system. Moreover, the disclosed signal pro-
_ g _
CA 02306095 2000-04-12
AttOrIley ~?OCKet
28Q49j33953
cessing that minimises the effects of background noise
may not be adequate under many conditions.
Kimura, in U.S. Patent No. 5,267,323, disc~.ases
a voice operated remote control having a tranamit~ex 10A.
The transmitter l0A includes first and second microphones
M3. and M2. The (first micrvphQne M9. serves to input a
voice oommaz~d and the second microphone M2 serves to pick
up ambient sound or noise. A noise xemover 104 removes
the ambiez~.t noise by producing a different~.al signal
indicative of the difference between the a~.gnaZs from the
t~,rst a,~id eecor~.d microphones M1 and M2.
The present invent~.on is directed to a system
which identifies members of an audience 4f a receiver and
which salves one or more t~f the probxems rioted above.
Summary of he Tnve~ '-n ~ ion
Tn accordance urith a first aspeot o~ the pres-
ent invention, a system for identifying a metrtber of an
audience of a reoexver comprises first and second micxo-
phones, a noise cancellation unit, and a voice recogni-
_ ~.0 _ C~'~
~f~~c~0
P
CA 02306095 2000-04-12
Attorney Docket
~B~Q9~33953
t~.on unit. The first and second microphones axe arxan~ed
so that ore of the First anal second Microphones picks up
rplati~rely rnvre of a voice szgr~al from the member of the
GudienCe and the other of the first and sacdnd micro-
phones picks up relatively more of a sound Bigna~. from
the receiver. The noise cancellation unit i~3 Gaug7.ed to
tr~e first and seco:~d microphones, arid the noise cancel.la-
Lion u,n.~.t is arranged to substantially elim~.nate the
sound sigrs.a~. ~rom the voice signal. The 'mice xecogr~i-
tion .nit is coup3.ed to the anise canceilativh unit, and
- x0a -
C~O~,O~~
CA 02306095 2000-04-12
WO 99/27668 PCT/US98/12034
the voice recognition unit is arranged to identify the
voice signal from the member of the audience.
In accordance with another aspect of the pres-
ent invention, a system comprises a microphone, a voice
recognition unit, and a program sensor. The system iden-
tifies a member of an audience of a program to which a
receiver is tuned. The microphone is located to receive
a voice signal from the member. The voice recognition
unit is coupled to the microphone and is arranged to
identify the member from the voice signal. The program
sensor is arranged to identify the program.
In accordance with yet another aspect of the
present invention, a system comprises a passive people
meter, a microphone, a voice recognition unit, and a con-
troller. The system identifies a member of an audience
of a program to which a receiver is tuned. The passive
people meter is arranged to provide a first identifica-
tion of the member by passively identifying the member.
The microphone is located to receive a voice signal from
the member. The voice recognition unit is coupled to the
microphone and is arranged to provide a second identifi-
- 11 -
CA 02306095 2000-04-12
WO 99/27668 PCT/US98/12034
cation of the member by identifying the member from the
voice signal. The controller is arranged to select at
least one of the first and second identifications in
order to identify the member.
In accordance with still another aspect of the
present invention, an audience recognition unit comprises
a housing, a microphone supported by the housing, and a
voice recognition unit supported by the housing and cou-
pled to the microphone. The voice recognition unit is
arranged to recognize a member of an audience from an
identifier spoken by the member. Accordingly, the audi-
ence recognition unit identifies the member of the audi-
ence of a program to which a receiver is tuned.
- 12 -
CA 02306095 2000-04-12
WO 99/Z7668 PCT/US98/12034
These and other features and advantages of the
present invention will become more apparent from a de-
tailed consideration of the invention when taken in con-
junction with the drawings in which:
Figure 1 illustrates the system of the present
invention for use in identifying members of an audience
of a program to which a television receiver is tuned;
Figure 2 is a block diagram of an embodiment of
the system which is in accordance with the present inven-
tion and which includes an audience recognition unit
(ARU?;
Figure 3 is a more detailed diagram of the
audience recognition unit shown in Figure 2;
Figure 4 is a diagram of a noise cancellation
arrangement for use in the audience recognition unit
shown in Figure 2;
Figure 5 illustrates a hand held version of the
audience recognition unit shown in Figure 3;
Figure 6 illustrates a table top version of the
audience recognition unit shown in Figure 3;
- 13 -
CA 02306095 2000-04-12
WO 99/27668 PCT/US98/I2034
Figure 7 is a flow chart depicting a program
which may be implemented by the audience recognition unit
shown in Figure 3 during a training period; and,
Figure 8 is a flow chart depicting a program
which may be implemented by the audience recognition unit
shown in Figure 3 during voice recognition of audience
members.
An audience measurement system 10, which deter-
mines the members of an audience to a program to which a
monitored receiver is tuned, is shown in Figure 1. The
audience measurement system 10 includes an audience
recognition unit 12, which is shown in Figure 1 as a hand
held unit but which alternatively may be any other type
of unit, such as a table top unit. The audience recog-
nition unit 12 permits the members of the audience to be
identified either passively or actively, and may or may
not interact with a receiver 14 during the identification
process.
- 14 -
CA 02306095 2000-04-12
WO 99/27668 PCT/US98112034
A program sensor 16 cooperates with the re-
ceiver 14 in order to determine the programs (and/or
channels) to which the receiver 14 is tuned. The program
sensor 16 may be any known sensor which determines chan-
nets (and/or programs). For example, the program sensor
16 may sense ancillary codes which are embedded in the
program signals to which the receiver 14 is tuned and
which specifically identify the tuned programs. In addi-
tion, or alternatively, the program sensor 16 may include
a signature extractor which extracts characterizing sig-
natures from the program signals to which the receiver 14
is tuned so that the signatures may be compared to refer-
ence signatures in order to identify the programs (and/or
channels) to which the receiver 14 is tuned. In addi-
tion, or alternatively, the program sensor 16 may be an
apparatus which determines the channels to which the
receiver 14 is tuned by monitoring the tuner of the re-
ceiver 14.
The audience measurement unit 12 and the pro-
gram sensor 16 communicate with a base unit 18 which may
be either a site unit or a home unit. As is known in the
- 15 -
CA 02306095 2000-04-12
WO 99!27668 PCTIUS98112034
art, a site unit may be provided at each audience area
within a household, and all site units within the house-
hold may communicate to a home unit over the electrical
power lines in the walls of the household. The home unit
may then communicate by way of the public telephone sys-
tem with a central unit which gathers tuning and audience
information from other households so that the information
can be combined into reports useful to program suppliers,
advertisers, and the like. In households having only one
viewing area, the base unit 18 may be arranged to both
receive data from the audience measurement unit 12 and
the program sensor 16, and may communicate this data by
way of the public telephone system to the central unit.
Thus, as shown in Figure 2, the audience
recognition unit 12 provides its recognition data to the
base unit 18 by way of a communication link 20 which, for
example, may be an infrared or RF communication link.
The program sensor 16 provides its program identification
data to the base unit 18 by way of a communication link
21 which, for example, may be a hard wire, but which may
alternatively be any other communication link, such as an
- 16 -
CA 02306095 2000-04-12
WO 99127668 PCT/US98/12034
infrared or RF communication link. The base unit 18
collects the audience recognition data from the audience
recognition unit 12 and the program identification data
from the program sensor 16, and may communicate this data
over a public telephone system 22 to a central site 24
which accumulates similar data from other households.
As indicated in Figure 2, the audience measure-
ment system 10 may also include a passive people meter
26. For example, the passive people meter 26 may be of
the type which passively identifies audience members in a
monitored area in accordance with the teachings of U.S.
Patent No. 5,550,928 described above. The passive people
meter 26 provides its people identification data to the
base unit 18 by way of a communication link 28 which, for
example, may be a hard wire, but which may alternatively
be any other communication link, such as an infrared or
RF communication link.
Accordingly, if the audience measurement system
10 includes both the audience recognition unit 12 and the
passive people meter 26, the passive people meter 26 may
be used to passively identify audience members in the
- 17 -
CA 02306095 2000-04-12
WO 99/27668 PCT/US98/12034
event that the audience recognition unit 12 is unable to
do so. Alternatively, the audience recognition unit 12
may be used to actively or passively identify audience
members in the event that the passive people meter 26 is
unable to passively identify audience members. As a
still further alternative, the outputs of both the audi-
ence recognition unit 12 and the passive people meter 26
may be applied simultaneously to a soft decision logic
that identifies audience members based upon both of these
outputs. In any of these alternatives, confidence is in-
creased that audience members can be accurately recog-
nized.
The audience recognition unit 12 is shown in
more detail in Figure 3. The audience recognition unit
12 includes first and second microphones 30 and 32. The
first and second microphones 30 and 32 are positioned in
the audience recognition unit 12 so that one of the first
and second microphones 30 and 32 tends to pick up rela-
tively more of the voice signals from the audience mem-
bers, and the other of the first and second microphones
and 32 tends to pick up relatively more of the audio
- 18 -
CA 02306095 2000-04-12
WO 99/27668 PCT/US98112034
output from the monitored receiver. For example, the
first and second microphones 30 and 32 may be separated
so that, in most orientations of the audience recognition
unit 12, one of the first and second microphones 30 and
32 is closer to the audience members and the other of the
first and second microphones 30 and 32 is closer to the
monitored receiver. A preferred separation between the
first and second microphones 30 and 32 is 6.72 inches,
although any other suitable separation may be used. In
this embodiment of the present invention, the first and
second microphones 30 and 32 may be pointed in opposite
directions. Alternatively, one or both of the first and
second microphones 30 and 32 may be located remotely from
the audience recognition unit 12 so that one of the first
and second microphones 30 and 32 is closer to the audi-
ence members and the other of the first and second micro-
phones 30 and 32 is closer to the monitored receiver.
The analog output of the first microphone 30 is
amplified by an amplifier 34, and this amplified analog
output is converted to a digital signal by an analog to
digital converter 36. Similarly, the analog output of
- 19 -
CA 02306095 2000-04-12
WO 99/27668 pCT/US98/12034
the second microphone 32 is amplified by an amplifier 38,
and this amplified analog output is converted to a digi-
tal signal by an analog to digital converter 40.
Even though the first and second microphones 30
and 32 can be positioned or directed so that one of the
first and second microphones 30 and 32 tends to pick up
relatively more of the voice signals from the audience
members and the other of the first and second microphones
30 and 32 tends to pick up relatively more of the audio
output from the monitored receiver, the first and second
microphones 30 and 32 pick up both the voice signals from
the audience members and the audio output from the re-
ceiver 14. Also, both the first and second microphones
30 and 32 pick up background noise other than the audio
output from the receiver 14. In order to reduce or elim-
mate the effects of background noise and the audio out-
put from the receiver 14, the outputs from the analog to
digital converters 36 and 40 are provided to a noise
cancellation arrangement 42. The noise cancellation
arrangement 42 processes the outputs from the analog to
digital converters 36 and 40 so as to cancel the audio
- 20 -
CA 02306095 2000-04-12
WO 99/27668 PCT/US98/12034
output of the receiver 14 as well as background noise
from the voice signals provided by the audience members,
so that these voice signals may be more easily identi-
fied.
The noise cancellation arrangement 42 may use
signal subtraction in order to subtract background noise
and the audio output of the receiver 14 from the voice
signals provided by the audience members. For example,
if the first microphone 30 is directed toward the audi-
ence members, and if the second microphone 32 is directed
toward the receiver 14, the signal from the first micro-
phone 30 may be designated SP (p meaning people), and the
signal from the second microphone 32 may be designated as
Sr (r meaning reference). In general, the signal Sp may
be given according to the following equation:
Sp = aVp + bVr ( 1 )
where SP is the signal from the first microphone 30, Vp is
the contribution to the signal from the first microphone
30 due to the voice signals of the members of the audi-
- 21 -
CA 02306095 2000-04-12
WO 99/27668 PCTIUS98/12034
ence, Vr is the contribution to the signal from the first
microphone 30 due to the audio output of the receiver 14
and other background noise, a and b are parameters dis-
cussed below, and a is greater than b.
Similarly, the signal Sr may be given according
to the following equation:
Sr = cVp + dVr ( 2 )
where Sr is the signal from the second microphone 32, c
and d are parameters discussed below, and c is less than
d. Equations (1) and (2) may be subtracted with a
weighting parameter a according to the following equa-
tion:
D = eSP - Sr = (ae - c) ~ p + (be - d} ~V (3)
where D is the weighted difference between equations (1)
and (2), and a is a weighting parameter discussed below.
If a, b, c, and d are fixed parameters deter-
mined by the environment and the features of the first
- 22 -
CA 02306095 2000-04-12
WO 99/27668 PCT/US98112034
and second microphones 30 and 32, the weighting parameter
a can be selected so that (be - d) - 0, or so that a =
d/b > 1. Accordingly, if (be - d) - 0, the difference D
will consist only of the signal Vp. In this case, the
signal VP from equation (3) is given by the following
equation:
D
p ae - c'
Because a, e, and c are constants in a given household,
these constants may be treated as a single constant k ac-
cording to the following equation:
k ael c (5)
- 23 -
CA 02306095 2000-04-12
WO 99/27668 PCT/US98/12034
so that the signal Vp is given by following equation:
VP = kD. (()
The weighting parameter a may be determined
adaptively during periods when the receiver 14 is on but
no member of the audience is talking. During such a
period, the audience recognition unit 12 can automati-
cally find the weighting parameter a so that D = 0. This
weighting parameter a is then inserted into equations (5)
and (6) in order to determine VP, and VP is processed by a
voice recognition unit 44 as discussed below.
This noise cancellation approach assumes that
the first microphone 30 is directed to, or is closer to,
the audience than is the second microphone 32, and that
the second microphone 32 is directed to, or is closer to,
the receiver 14 than is the first microphone 30. For
example, if the audience recognition unit 12 is a hand
held unit, the first microphone 30 and the second micro-
phone 32 may be mounted on the audience recognition unit
12 so that they are separated and so that the user is
- 24 -
CA 02306095 2000-04-12
WO 99127668 PC'T/US98112034
directed to orient the audience recognition unit 12 with
the first microphone 30 directed toward the audience and
the second microphone 32 directed toward the receiver 14.
Similarly, if the audience recognition unit 12 is a table
top unit, the first microphone 30 and the second micro-
phone 32 again may be mounted on the audience recognition
unit 12 so that they are separated and so that the user
is directed to orient the audience recognition unit 12
with the first microphone 30 directed toward the audience
and the second microphone 32 directed toward the receiver
14. Alternatively, whether the audience recognition unit
12 is a hand held unit or a table top unit, the first and
second microphones 30 and 32 may be connected to the
audience recognition unit 12 by lines or other communica-
tion media which permit the first microphone 30 to be
mounted in close proximity to the audience and the second
microphone 32 to be mounted in close proximity to the
receiver 14.
Instead of canceling noise as discussed above,
the noise cancellation arrangement 42 may use phase
cancellation. It is noted that sound travels at approxi-
- 25 -
CA 02306095 2000-04-12
WO 99/Z7668 PCT/US98/12034
mately 1,120 ft per second in air. Sounds not originat-
ing equidistant from both of the first and second micro-
phones 30 and 32, which are preferably separated by 6.72
inches, reach these microphones at different times and
are canceled or are at least partially canceled. Based
on the actual frequencies of the sounds, and the spacing
between the first and second microphones 30 and 32, dif-
ferent phase angles result. The noise cancellation ar-
rangement 42 implementing phase cancellation uses these
different phase angles in order to cancel the audio out-
put from the receiver 14 and other background noise from
voice signals provided by the members of the audience.
In this embodiment of the present invention, the first
and second microphones 30 and 32 are preferably direc-
tional microphones mounted on the same side of the audi-
ence recognition unit 12. The audience recognition unit
12 may have any orientation with respect to the audience
and the receiver 14 as long as the first and second mi-
crophones 30 and 32 and not pointed directly at the re-
ceiver 14.
- 26 -
CA 02306095 2000-04-12
WO 99/27668 PCT/US98/I2034
In this embodiment of the present invention,
the noise cancellation arrangement 42 may be of the form
shown in Figure 4. According to this embodiment, the
noise cancellation arrangement 42 includes an amplifier
60, which amplifies the signal from the first microphone
30, and an amplifier 62, which amplifies the signal from
the second microphone 32. Because the amplifiers 60 and
62 may be essentially the same, only one of the amplifi-
ers is shown in detail in Figure 4. The outputs of the
amplifiers 60 and 62 are summed by a summer 64 so that
background noise is substantially eliminated. The output
from the summer 64 is detected and converted to a DC
level by a detector 66 and is supplied to a timer 68.
If the output of the summer 64 is above a
predetermined threshold, the timer 68 is triggered and
provides an output to a switch 70 in order to connect the
output of the summer 64 to the voice recognition unit 44.
The voice recognition unit 44 provides a hold signal
through a terminal 72 in order to hold the timer 68 in
its triggered state until a predetermined time after the
voice recognition unit 44 last recognizes a voice.
- 27 -
CA 02306095 2000-04-12
WO 99/27668 PCTNS98/12034
The voice recognition unit 44 may be an RSC164
voice recognition processor provided by Sensory Inc. The
voice recognition unit 12 compares the output from the
noise cancellation arrangement 42 to prestored reference
voice signals (identifiers) stored in a memory. These
prestored reference voice signals (identifiers) may be
spoken identifier words or voice features from a speaker-
's frequency spectrum. When matches are found, members
of the audience are identified, and the voice recognition
unit 44 provides a suitable identification record to a
controller 46 for each match. The controller 46 stores
the identification record in a memory 48 for later
transmission to the base unit 18 by way of a communica-
tion arrangement 50. The communication arrangement 50
may use the communication link 20 as previously de-
scribed. The controller 46 may also be coupled to a
clock 52 so that the times that the members of the audi-
ence are identified may also be stored in the memory 48.
A prompt unit 54 may also be coupled to the
controller 46. The controller 46 controls the prompt
unit 54 in order to prompt audience members to either
- 28 -
CA 02306095 2000-04-12
WO 99127668 PCT/US98/12034
provide their corresponding voice signals to the first
and second microphones 30 and 32, or use a keypad 56 in
order to manually identify themselves. The prompt unit
54 may be a voice synthesizer and/or a graphic or video
display (such as an LCD or LED) of the audience recog-
nition unit 12 arranged to provide audible and/or visible
prompts to the members of the audience. Alternatively,
the prompt unit 54 may be the speakers and/or the screen
of the receiver 14 arranged to provide audible and/or
visible prompts to the members of the audience in re-
sponse to interaction with the audience recognition unit
12.
The audience recognition unit 12 may be
implemented as a hand held unit 90 which is illustrated
in Figure 5. The hand held unit 90 includes a housing 92
that houses the elements of the audience recognition unit
12 which are shown in Figure 3. The housing 92 supports
an LED display 94 which may perform the functions of the
prompt unit 54. As noted above, however, the prompt unit
54 alternatively may be a voice synthesizer, in which
case the housing 92 would incorporate an output speaker
- 29 -
CA 02306095 2000-04-12
WO 99127668 PCTIUS98/12034
to project audible voice prompts from the voice synthe-
sizer to the members of the audience. The LED display 94
(and/or the voice synthesizer) may perform functions
other than prompting. The housing 92 also houses an IR
transmitter 96 for establishing the communication link 20
shown in Figure 2. Alternatively, the communication link
20 may be established by an ultrasonic or RF link, a hard
wire, or the like. The housing 92 further supports a
keypad 98, which may be the keypad 56 shown in Figure 3,
for allowing audience members to enter data. For exam-
ple, predetermined ones of the keys in the keypad 98 may
be assigned to members and visitors of the audience so
that they may identify themselves manually if they cannot
be recognized through their voice signals. The housing
92 may contain a rechargeable battery which is arranged
to provide power to the audience recognition unit 12 and
which may be charged overnight or when the audience rec-
ognition unit 12 is not in use. The hand held unit 90
also has a power on/off button 95 in order to turn the
hand held unit 90 on and off, and an LED 97 in order to
- 30 -
CA 02306095 2000-04-12
WO 99/27668 PCT/US98/I2034
indicate that the hand held unit 90 is ready to identify
audience members and visitors.
Alternatively, the audience recognition unit 12
may be implemented as a table top unit 100 which is
illustrated in Figure 6. The table top unit 100 includes
a housing 102 that houses the elements of the audience
recognition unit 12 which are shown in Figure 3. The
housing 102 may support an LED display (not shown) in
order to implement the functions of the prompt unit 54.
As noted above, however, the prompt unit 54 alternatively
may be a voice synthesizer, in which case the housing 102
would incorporate an output speaker to project audible
voice prompts from the voice synthesizer to the members
of the audience. The LED display or the voice synthe-
sizer may perform functions other than prompting. The
housing 102 may also house an IR or RF transmitter (not
shown) for establishing the communication link 20 shown
in Figure 2. Alternatively, the communication link 20
may be established by an ultrasonic link, a hard wire, or
the like. The housing 102 supports a keypad 104, which
may be the keypad 56 shown in Figure 3, for allowing
- 31 -
CA 02306095 2000-04-12
WO 99127668 PCT/IJS98/12034
audience members to enter data. For example, predeter-
mined ones of the keys in the keypad 104 may be assigned
to members of the audience so that they may identify
themselves manually if they cannot be recognized through
their voice signals. The table top unit 100 may contain
a rechargeable battery which is arranged to provide power
to the audience recognition unit 12 and which may be
charged overnight or when not in use. Alternatively, the
table top unit 100 may have a power cord for providing
power to the audience recognition unit 12 from an elec-
trical outlet. Also, the table top unit 100 may have a
power on indicator light 106 and a voice detected indica-
for light 108.
During a training mode, the controller 46 of
the audience recognition unit 12 may implement a program
120 depicted in Figure 7 by way of a flow chart. The
program 120 is executed during the training mode in order
to store identifier words or voice features associated
with the members of the audience. During training, an
audience member should assume a position at a specified
distance from the audience recognition unit 12. This
- 32 -
CA 02306095 2000-04-12
WO 99/Z7668 PCT/US98/12034
audience member initiates training by pressing an
appropriate key or button on the keypad 56 of the audi-
ence recognition unit 12.
When the training mode is initiated so that the
program 120 is executed, the audience recognition unit
12, at a block 122, determines whether a predetermined
amount of time has passed since the last audience member
activity. If the audience recognition unit 12 determines
that there has been no audience member activity for the
predetermined amount of time, the program 120 assumes
that the training mode has ended and the program 120
exits. If the predetermined amount of time has not pass-
ed since the last audience member activity, the audience
recognition unit 12, at a block 124, continues to wait
for a wakeup command to be spoken. The wakeup command
should be a unique word or word combination not normally
present in the audio of the receiver I4. The audience
member should speak the wakeup command in a normal voice
to the first and second microphones 30 and 32. When the
audience recognition unit 12 detects the wakeup command,
the audience recognition unit 12, at a block 126, prompts
- 33 -
CA 02306095 2000-04-12
WO 99/27668 PCTNS98/12034
the audience member to say the identifier ward which the
viewer has chosen to identify himself or herself. Such
an identifier word may be the name of the viewer.
The audience recognition unit 12 then deter-
mines at a block 128 whether the audience member has said
the identifier. If not, the audience recognition unit
12, at the block 126, again prompts the audience member
to say the identifier word. If the audience recognition
unit 12 determines at the block 128 that the audience
member has said the identifier word, the audience recog-
nition unit 12, at a block 130, suitably processes the
identifier word and stores the identifier word in the
memory 48. Flow returns to the block 122 where the
training mode is terminated or another audience member
enters a corresponding identifier word.
During an identification mode, the controller
46 of the audience recognition unit 12 may implement a
program 140 depicted in Figure 8 by way of a flow chart.
The program 140 is executed during the identification
mode in order to identify members of the audience. The
identification mode may be initiated by pressing an
- 34 -
CA 02306095 2000-04-12
WO 99127668 PCT/US98112034
appropriate key or button on the keypad 56 of the audi-
ence recognition unit 12.
When the identification mode is initiated so
that the program 140 is executed, the audience recogni-
tion unit 12, at a block 142, determines whether an audi-
ence member desires to end the identification mode by
pressing an appropriate key or button on the keypad 56 of
the audience recognition unit 12. If the audience recog-
nition unit 12 determines that the audience member de-
sires to end the identification mode, the program 140
exits. If the audience recognition unit 12 determines
that the audience member does not desire to end the iden-
tification mode, the audience recognition unit 12, at a
block 144, waits for a wakeup command to be spoken. The
audience member should speak the wakeup command in a
normal voice to the first and second microphones 30 and
32. When the audience recognition unit 12 detects the
wakeup command, the audience recognition unit 12, at a
block 146, prompts the audience member to say his or her
identifier word.
- 35 -
CA 02306095 2000-04-12
WO 99/2?668 PCT/US98/12034
The audience recognition unit 12 then deter-
mines at a block 148 whether the audience member has said
the identifier word. If not, the audience recognition
unit 12, at the block 146, continues to prompt the audi-
ence member to say the identifier word. If the audience
recognition unit 12 determines at the block 148 that the
audience member has said the identifier word, the audi-
ence recognition unit 12, at a block 150, instructs the
voice recognition unit 44 to compare the identifier word
to the vocabulary of identifier words stored in the mem-
ory 48.
If the audience recognition unit 12 determines
at a block 152 that the voice recognition unit 44 detects
a match between the spoken identifier word and one of the
identifier words stored in the memory 48, the audience
recognition unit 12, at a block 154, stores the spoken
identifier word and the time of the match as an
identification record in the memory 48 for immediate or
later communication to the base unit 18. (Alternatively,
the audience recognition unit 12, at the block 154, may
communicate the spoken identifier word and time directly
- 36 -
CA 02306095 2000-04-12
WO 99/27668 PCT/I1S98/12034
to the base unit 18.) If the audience recognition unit
12 determines at the block 152 that the voice recognition
unit 44 cannot match the spoken identifier word to one of
the identifier words stored in the memory 48, the audi-
ence recognition unit 12, at a block 156, prompts the
audience member to try again or to use another farm of
identification. For example, the prompt 140 may prompt
the audience member to speak his or her identifier word a
predetermined number of times before prompting the audi-
ence member to use another form of identification (such
as entering the identifier word through use of the keypad
56). Alternatively, or additionally, if the controller
46 determines that the voice recognition unit 44 cannot
detect a match at the block 152, the controller 46 at the
block 156 may send a message to the base unit 18 indi-
Gating that identification should be made from the pas-
sive people meter 26. After the audience recognition
unit 12, at the block 154, stores the spoken identifier
word and the time of the match as an identification re-
cord in the memory 48, or after the audience recognition
unit 12, at the block 156, prompts the audience member to
- 37 -
CA 02306095 2000-04-12
WO 99/27668 PCT/US98/12034
try again, or to use another form of identification, or
to send a suitable message to the base unit 18 to seek an
identification from the passive people meter 26, or the
like, flow returns to the block 142.
The advantage of using a wakeup command during
normal voice recognition is that the comparison process
is simplified. That is, during normal audience recogni-
tion, each spoken identifier word is compared to every
reference identifier word stored in the memory 48. How-
ever, the wakeup command only needs to be compared to one
reference in the memory 48.
The block 126 of Figure 7 and the block 146 of
Figure 8 may be arranged to also prompt each audience
member to speak a cue in addition to the member's identi-
fier word. The advantage of coupling a cue to an identi-
fier word is to minimize the possibility that the audi-
ence recognition unit 12 may incorrectly identify a per-
son as an audience member when that person's identifier
word is contained in the audio output of the receiver 14,
but that person is not in the audience. For example, the
identifier word may be a name, and the cue may be a word
- 38 -
CA 02306095 2000-04-12
WO 99/27668 PCT/US98/12034
common to all audience members, such as the word "enter,"
or the word "exit," or the like. Thus, a member of the
audience, when entering (or leaving) the monitored area,
speaks his or her name, as the identifier word, coupled
with the cue. For example, the member may say "John
enter" or "Mary enter." Alternatively, the cue may be
unique to each audience member. Either arrangement
reduces the likelihood that a word audibly emitted by the
receiver 14 will be mistaken as an identifier word spoken
by a member of the audience.
Certain modifications of the present invention
have been discussed above. Other modifications will
occur to those practicing in the art of the present
invention. For example, the invention has been described
above in connection with measuring the audiences of tele-
vision programs. However, the present invention is use-
ful also in measuring the audiences of radio programs and
programs of other media.
Also, the program sensor 16 is described above
as sensing the program and/or channel to which the re-
ceiver 14 is tuned. Instead, the audience recognition
- 39 -
CA 02306095 2000-04-12
WO 99127668 PCT/US98/12034
unit 12 may be arranged to accept manual entry of program
identifications by the audience members through use of
the keypad 56.
Moreover, the functions of a remote control may
be incorporated into the audience recognition unit 12.
Furthermore, instead of relying upon two micro-
phones, such as the first and second microphones 30 and
32, the audience recognition unit 12 may rely upon a
single, close range directional microphone. If so, noise
cancellation may not be necessary.
In addition, the use of a wakeup command may be
eliminated.
Also, as described above, the passive people
meter 26 may be of the type which passively identifies
audience members in a monitored area in accordance with
the teachings of U.S. Patent No. 5,550,928 described
above. However, the passive people meter 26 may be any
known type of passive device that is capable of passively
identifying members of an audience. For example, the
passive people meter 26 may use an ultrasonic and/or
infrared transducer or an array of such transducers in
- 40 -
CA 02306095 2000-04-12
WO 99/27668 PCT/US98/12034
order to capture an image of the audience area and may
use processing algorithms in order to recognize members
in the audience.
Moreover, the passive people meter 26, as des-
cribed above, passively identifies audience members in a
monitored area. If the passive people meter 26 cannot
passively identify the members in the audience, it can be
arranged to count the members of the audience and to
prompt manual identification of the members until the
IO number of manually identified members equals the count.
Alternatively, if the passive people meter 26 cannot
passively identify every member in the audience, it can
be arranged to count the members of the audience and to
prompt manual identification of those members which it
cannot passively identify. As a still further alterna-
tine, a counter may be used instead of the passive people
meter 26. The counter can comprise an ultrasonic and/or
infrared transducer or an array of such transducers or
other device or devices which can detect people in the
audience and can count the number of detected people.
Then, the audience measurement system 10 could be ar-
- 41 -
CA 02306095 2000-04-12
WO 99/27668 pCT/US98/12034
ranged to prompt manual identification until the number
of people manually identified equals the count.
Furthermore, the audience measurement unit 12,
the program sensor 16, and the passive people meter 26
are shown in Figure 2 as being separate elements. In-
stead, the audience measurement unit 12 may include the
program sensor 16 and/or the passive people meter 26.
Accordingly, the description of the present
invention is to be construed as illustrative only and is
for the purpose of teaching those skilled in the art the
best mode of carrying out the invention. The details may
be varied substantially without departing from the spirit
of the invention, and the exclusive use of all modifica-
tions which are within the scope of the appended claims
is reserved.
- 42 -
