Note : Les descriptions sont présentées dans la langue officielle dans laquelle elles ont été soumises.
CA 02354114 2001-07-16
, !r? t ~L d. i'-'.: 3 i y;, ~'F'tf7~~~
g
:~j'_ i
' ~ '~~31
FIELD OF THE INVENTION PRUP~~E~TE ~i~i~:zI.EG~IJEf;W '
The present invention pertains generally to a vision system for use by the
visually
impaired, for those with a visual memory impairment, or for improvement of
seeing,
remembering, visual communication, or for information display, capture,
processing,
or the like.
BACKGROUND OF THE INVENTION
SUMMARY OF THE INVENTION
A "getting" is a region of a space, such as a polarization space, time--
polarization
space, time-frequency space, time-frequency-polarization space, or th.e like,
or a
region of time such as a time interval or periodic train of time intervals or
random or
pseudorandom tune variations, or a region of frequency such as a frequency
spectrum,
frequency band, frequency region, or the like.
The concept of "getting" generalizes the concept of "setting" (time and place,
more commonly known as "time-space" ) and emphasizes the capture, obtaining,
ma-
nipulating, display, or the like, of visual information.
A diverter is a reflective or partially reflective surface or object or
material or
boundary that diverts at least a portion of rays of light to or from an
instrument.
Preferably the diverter either is, or has properties similar to, a mirror,
beamsplitter,
or selector. The diverter may also be a holographic optical element, or other
material
that diverts rays of light to or from an optical path of a user of the
apparatus of the
invention. An example of a diverter is an automobile windshield that a driver
can see
through, but where the diverter (windshield) diverts some rays of light that
would
otherwise reach the driver's eyes into a camera or similar sensing element. A
diverter
may also be comprised of a boundary between different materials, such a,s a
difference
in materials within eyeglasses, for diverting rays of light; into an eye of a
wearer
of the eyeglasses. Such a boundary may be a step boundary such as an interface
between two materials having different properties, or it may be a delta
boundary (an
approximation to a Dirac delta measure in space) such as a thin film,
deposited, for
example, at a 45 degree angle, within the lens material of eyeglass lenses.
A key inventive concept is that of a diverter transmitting light in a getting
of
greater biological sensitivity for allowing the diverter to appear to a
biological in-
strument such as the human eye to be transparent, while having a getting of
lesser
2
CA 02354114 2001-07-16
biological sensitivity for allowing the diverter to appear reflective to a,
camera or
display system, aremac, or other kind of instrument or indicator without so
much
appearing so strongly reflective to the biological vision system. Within the
getting of
lesser biological sensitivity there is a getting of preferably not lesser
machine sensitiv-
ity in which the diverter is reflective. Thus the reflectivity, from the
machine's point
of view, preferably improves, or at the very least, remains the same within
the getting
of lesser biological sensitivity, or even if the reflectivity from the
machine's point of
view decreases, it does not decrease as much as the reflectivity from the
biological
point of view.
For example, in one embodiment; the getting of lesser biological sensitivity
is a
short time interval during which the diverter is reflective, owing to an
electrochromic
characteristic. Embodiments of the invention that use an electrochromic
diverter are
called electrochromic embodiments. An electrochromic diverter is a diverter
having
either a reflectivity that is responsive to an electrical input to the
diverter or to the
surrounding structure of the apparatus, or a transmissivity that is responsive
to an
electrical input to the diverter or to the surrounding structure of the
apparatus. Thus
an electrochromic diverter is a diverter that can be changed in reflectivity
character-
istics or transmissivity characteristics in response to an electrical input.
It might,
for example, become darker under control of an electrial signal. Thus a camera
or
other sensor seeing by reflection in the diverter will see only the reflection
and not
the transmission, or will at least see less of the transmission. Even if the
amount of
reflection the machine sees is no greater in the getting of lesser biological
sensitivity, it.
will still see roughly the same amount of reflection and less of the
transmission. Thus
a nearly equal reflectivity seen by the machine, along with a lesser
transrnissivity, may
still give rise to better signal to noise ratio if we regard the transmission
(transmitted
light) as noise (unwanted signal) and the reflected light as signal (desired
signal).
Therefore a camera synchronized with a periodic train of pulses delivered to a
control
system will allow the camera to see during brief intervals of lesser
biological sensi-
tivity. Interspersed with these intervals of lesser biological sensitivity
during which
the camera sees, there are intervals of greater biological sensitivity during
which the
biological system sees through the diverter. The intervals of greater
biological sensi-
tivity are preferably longer in time, such as by having the periodic train of
pulses of
the control system be such that the intervals of greater biological
sensitivity are of
greater than fifty percent duty cycle, and the intervals of lesser biological
sensitivity
3
CA 02354114 2001-07-16
be less than or equal to fifty percent duty cycle. Preferably the duty cycle
of the
intervals of lesser biological sensitivity are much shorter than fifty
percent, so that
the biological system perceives the diverter as being transparent. In some
embod-
iments, the diverter is embodied as a discontinuity, such as by being embedded
in
lens material of eyeglasses, so that this transparency makes it completely
invisible to
biological systems. Preferably; the diverter is also invisible to persons
other than a
person using the apparatus of the invention.
The term "biological" refers to a response of a biological vision system such
as a
human biological vision system, or the like, or to an instrumented biological
vision
system such as an adversarial security guard or the like looking through a
video
surveillance system at a time in the present (e.g. as by way of closed circuit
television)
or in the future (e.g. by way of playing back recordings of closed circuit
television
depicting the user of the apparatus of the invention).
It is desired that the diverter appear transparent to the user of the personal
imag-
ing system, such that the user can see through it while at the same time light
can
be diverted to a camera, or diverted from a display, aremac, or the Like for
display
into or viewing by at least one eye of the user, or for projection onto
subject matter
visible to the user. Moreover, it is sometimes further desired that the
diverter appear
transparent to others. For example, in an eyeglass embodiment of the
invention, it
is preferable that both of the user's eyes be visible to others, so that the
eyeglasses
have a normal appearance. It may also be desirable that this normal appearance
be preserved even though others may be looking through instruments such as
video
surveillance cameras. Thus, for example, in some embodiments of the invention
it
is preferable that the apparatus have a normal looking appearance to
adversarial
security guards looking presently or in the future (ie at surveillance
recordings) by
wav of video surveillance cameras. Accordingly, the apparatus of the invention
us-
ing a synchronized electrochromic diverter is preferably not synchronized, or
easily
synchronizable by adversaries, to the video surveillance cameras, and will
therefore
appear more transmissive to the video surveillance cameras than to the
instruments
such as a camera, or the like, within the apparatus of the invention.
A getting of lesser biological sensitivity may also be a spectral band. An
embod-
iment of the invention using a spectrally concealed diverter uses a getting of
greater
biological sensitivity in a spectral band of greater sensitivity to the
biological vision
system. It uses a spectral band of lesser sensitivity for a getting of lesser
biological
4
CA 02354114 2001-07-16
sensitivity, in which the diverter appears reflective. Thus concealment of the
diverter
is by way of spectral response. In some embodiments, there is a combination of
spec-
tral getting and temporal getting, so that the diverter is transparent most of
the time
and in most of the visible spectrum, but is reflective in a lesser visible
portion of the
spectrum and for only brief time pulses.
However, in the preferred embodiment of the spectrally concealed diverter,
there
is no significant electrically controlled temporal variation in the optical
properties of
the diverter. This results in a pure spectral getting, which makes the
diverter easier
to manufacture. The lower cost of the purely spectral concealment is important
when
the diverter is large, as, for example, when the diverter is the entire
windshield (or a
large portion thereof ) of a vehicle.
In the vehicle embodiment, a windshield that transmits more visible light and
transmits less infrared lght may be constructed. Thus an optical instrument
such as
an infrared camera or other sensor seeing by reflection in the diverter
(windshield)
will see only the reflection and not the transmission, or might at least see
at least
approximately the the same amount of the reflection and less of the
transmission.
Therefore an optical instrument such as an infrared seeing camera mounted
outside
the vehicle, above the windshield, looking down, will see what the driver sees
(e.g.
it will see down the road or the like, in the direction the driver is facing)
by way of
a reflected image. Even if the amount of reflection the camera sees is no
greater in
the getting of lesser biological sensitivity, it will still see roughly the
same amount of
reflection and less of the transmission. Thus a nearly equal reflectivity seen
by the
machine, along with a lesser transmissivity, may still give rise to better
signal to noise
ratio if we regard the transmission (transmitted light) as noise (unwanted
signal) and
the reflected light as signal (desired signal).
Two preferred embodiments of spectral getting include embodiments having' an
infrared camera in which the camera optics are lowtap spectrally concealed,
and a
bandpass camera, in which the camera optics are bandtap spectrally concealed.
In a spectrally concealed camera system in whiclu the spectral concealment is
lowtap concealment, an infrared camera is spectrally concealed. Lowtap
spectral
concealment is concealment in which the diverter has a. normal appearance in
the
normal visible spectrum as a typical adversarial person would see it, or as
seen in
normal biological vision. In the preferred embodiment, the diverter is a
dichroic
element. The dichroic element passes visible light and reflects infrared
light, so it is a
CA 02354114 2001-07-16
diverter that is like clear glass in the visible, and like a mirror in the
infrared regions
of the spectrum. Lowtap concealment taps into the low end of the spectrum,
e.g. the
instrumentation such as a camera or other optical instrument taps into the
lower end
of the frequency spectrum.
In a spectrally concealed embodiment the spectral concealment is bandtap con-
cealment, using a bandpass instrument. For example, a bandtap system allows an
optical instrument such as a camera to sample rays of light in a narrow
spectral band
in which the diverter is reflective. The diverter is transmissive in other
regions of the
spectrum, and therefore the apparatus allows the user to see through it,
naturally,
and the apparatus also allows others to see the user. natnxrally.
The diverter may also be polarizationally concealed. A polarizationally
concealed
diverter may also be temporally concealed, transmitting light in a getting
time interval
of greater biological sensitivity and reflecting light in a getting time
interval of lesser
biological sensitivity, or it may also be spectrally concealed, by having a
getting
spectrum of greater biological sensitivity where light is transmitted, and a
getting
spectrum of lesser biological sensitivity where light is reflected. A
polarizationally
concealed diverter may also be purely polarizationally concealed.
The polarizationally concealed diverter reflects, to a greater degree; light
with
certain polarization properties, and transmits, to a greater degree, light not
having
these properties. In a preferred embodiment, the diverter is a dichroic
optical element
reflecting polarized light into a camera or similar optical instrument, and
transmitting
light of the opposite polarization, so that to the biological system the
diverter appears
transparent but to the camera the diverter appears, at least partially,
reflective.
BRIEF DESCRIPTION OF THE DRAWINGS
The invention will now be described in more detail, by way of examples which
in no way are meant to limit the scope of the invention, but, rather, these
examples
will serve to illustrate the invention with reference to the accompanying
drawings, in
which:
FIG. 1 is a diagram showing an eyetap system with concealed getting.
FIG. 2A shows a concealed getting eyetap system in eyeglasses with comfort
bands.
FIG. 2B shows a concealed getting eyetap system in eyeglasses with infrared
ghost
image Mocker.
FIG. 3 shows an eyeglass based aimer.
6
CA 02354114 2001-07-16
FIG. 4 shows an aimer concealed in an eyeglass lens designed to look like an
eyeglass lens of bifocal eyeglasses.
FIG. 5 shows an eyetap embodiment of the invention.
FIG. 6 shows a vehicular embodiment of the invention for use in airplanes,
boats,
trucks; buses, cars, and other vehicles.
FIG. 7 shows a cargo carrier roof box embodiment.
FIG. 8 shows an eyetap system using auxiliary noneyet;ap cameras to
interpolate
an eyetap viewpoint with a getting of biological sensitivity.
FIG. 9 shows a block diagram of the system using auxiliary noneyetap cameras
to interpolate an eyetap viewpoint with a getting of biological sensitivity.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
While the invention shall now be described with reference to the preferred em-
bodiments shown in the drawings, it should be understood that the intention is
not
to limit the invention only to the particular embodiments shown but rather to
cover
all alterations, modifications and equivalent arrangements possible within the
scope
of appended claims.
FIG. 1 depicts an eyetap system with concealed getting. The apparatus is meant
for being viewed by eye 100, but of course the apparatus does not actually
include the
eye 100 as a part of the apparatus of the invention. A diverter 110 diverts a
portion of
the rays of light that would have otherwise entered through the lens of eye
100. This
light is diverted into a sensor 130. A satisfactory sensor 130 is a video
camera. Sensor
130 is preferably sensitive to a different getting than eye 100, or, more
importantly,
sensor 130 is preferably sensitive to a different getting than other sensors
such as eyes
of other biological organisms such as other people.
A common problem with EyeTap devices is that the images can become milky or
otherwise reduced in contrast or quality because of light shining into the
sensor 130
or the diverter 110 other than light from intended subject matter. Thus a
housing
for the apparatus preferably includes at least a portion 150 that preferably
functions
as a light shield to the getting of sensor 130, more than it functions as a
light shield
to the eye 100 or to other sensors.
For example, a portion 150 of the housing for the apparatus is electrochromic,
so
that it becomes more opaque during a time interval when sensor 130 is more
sensitive
to light, and more transparent during a time interval when sensor 130 is less
sensitive
7
CA 02354114 2001-07-16
to light. The result can be a light shield that is more transparent to the
user, and
less transparent to the sensor 130, so that there can be a light shield around
the
camera that does not appreciably block the user's ability to see around and
through
the apparatus.
Alternatively, sensor 130 can be an infrared camera; and all or part of the
housing
for the apparatus can be made of material that is opaque or at least dark in
the
infrared, but more clear or transparent in the visible spectrum. In this way,
a very
good and quite complete lens hood, shield, or light Mocker can be constrlacted
around
the camera and diverter that does not obstruct the vision of the user.
For purposes of illustration, portion 150 is the top of the housing. since
light
coming from above is a major problem. It should be understood; however; that
portion 150 could be any portion of the housing, or it could be another
element or
accessory. For example, in an eyeglass based version of the invention, portion
150
could be a separate element such as a baseball cap with a plastic brim that is
clear
in the visible and dark or opaque in the infrared, so that the wearer's vision
is not
obstructed (and the wearer does not have the dark and evil appearance when
seen
by others, as might happen with a baseball cap) but the sensor's effective
vision is
hooded, vignetted, or shielded, or stray light is kept out of the sensor or
diverter or
associated portions of the system.
In a preferred eyeglass based embodiment of the invention, the camera body is
borne by the nose 140 of the wearer. This means that tl a camera body is the
nose-
bridge, or is very closely integrated with the nosebridge of the eyeglasses,
and a
portion of the camera body may extend into the cavity to the left of the nose
140
(here a right eyetap is illustrated but the left eye or both eyes could be
tapped within
the scope of the invention).
A back portion 120C of the housing is shown, and preferably blocks more of the
getting to which sensor 130 is sensitive, compared to how much it blocks the
getting
to which eye 100 or other sensors are sensitive, so that the user can see in a
less
obstructed way, and so that others can see the user's eyes in a less
obstructed way.
Portion 120 of the housing is the portion around sensor 130, such as the
portion
in front of and behind sensor 130.
Wont portions 120A and 120B must allow incoming light to reach diverter 110
so front portions 120A and 120B are transparent in the getting to which sensor
130
is sensitive. However, the portion 120A nearest sensor 130 is preferably dark
in the
8
CA 02354114 2001-07-16
human getting (getting of greater biological sensitivity) to help conceal a
lens of sensor
130 from being seen by persons other than the user. Conversely, portion 120B
further
from a lens of sensor 130 is preferably transparent in the human getting near
sensor
130, so that others can easily see the eye 100 of the user, a,nd so that less
of the field
of view of eye 100 is obstructed.
In one embodiment, using an infrared camera for sensor 130, the front portion
120
is entirely transparent in the infrared, whereas the transparency in visible
light varies
such that the left side is opaque in the visible, and the right side is
transparent.
In some embodiments the diverter 110 allows getting selectivity to separate
human
getting from machine getting. In one embodiment the diverter 110 is a 45
degree ''hot
mirror" that reflects infrared (heat) into sensor 130, and transmits visible
light into
eye 100.
The illustration of Fig. 1 simply serves as an example of how variable
character-
istics of the getting work to conceal or reveal views across the various
gettings. The
invention is not necessarily always directed at making the user's eyes visible
to others.
For example, the invention may take the form of dark sunglasses that are
transparent
in the infrared, to allow infrared cameras to see out without losing light,
but very
dark to hide the wearer's eyes in the visible, so that others cannot see the
strange
eye movements that are typical of a person looking at the world through a
computer
screen.
FIG. 2A shows a an eyeglass based embodiment of the invention. The portion
150 of the eyeglass frames may be the entire eyeglass frames, entirely made
out
of, for example, visibly clear plastic that is opaque in the infrared.
Preferably the
eyeglass frames are cttt on a numerical control machine, responsive to live or
previously
recorded input from a machine vision system, Cyberwarc~ (TM) scanner, or the
like;
aimed at the head of the intended wearer, so that the eyeglasses will fit
perfectly
onto the head of the wearer, to ensure that the apparatus has a carefully
positioned
EyeTap.
One or more comfort bands 210 are for going behind the head of the wearer, to
keep the glasses on. Preferably there is at least one comfort band that goes
below the
bump at the back of the wearer's head, and one that goes above the bump.
Preferably
there is means for keeping the two comfort bands from separating from each
other
more than by a, certain amount of distance.
Comfort bands 210 may be released or connected to a back portion 220 of the
9
CA 02354114 2001-07-16
eyeglass frames. A release capability allows the eyeglasses to be removed
under certain
conditions. A sleeve 230 receives an attachment pin 230A located at the ends
of
comfort bands 210. This sleeve is mounted within a block of material 240 and
held
in place with security set screw 250.
Security set screw 250 may be a thumb screw that can be operated without tools
or other equipment. Alternatively, security set screw 250 can be a special
security
screw that requires a special wrench or key.
The key can be used to install the eyeglasses and then the wearer of the
eyeglasses
can conveniently leave behind the key, so that he or she has an articulable
inability
to remove the eyeglasses. Alternatively, the wearer can carry the key, on his
or
her person. but conveniently forget where it was placed on his or her body
(e.g.
forget what pocket it was in) and therefore have an articulable inability to
remove
the eyeglasses. The wearer can also leave the key in a locker in a secure
locker
morn, such as the locker room of a secure facility. The secure facility may be
a
decontamination facility, so that assailants or terrorists washing the wearer
to remove
the eyeglasses would need to either wait outside the locker room, or accompany
him
or her therein. Such a method of doing business in security eyeglasses would
deter
assailants, terrorists, or corrupt officials from abusing the wearer of the
eyeglasses;
because such assailants, terrorists, or corrupt officials would need to either
allow the
wearer to escape to the safety of the locker room, or the assailants,
terrorists. or
corrupt officials would need to comply with conditions of the locker room in
order
to enter with the wearer. These conditions can include various security
screening
procedures as might be found in a secure decontamination facility.
In the event of attempted breakage of the eyeglasses by assailants,
terrorists,
corrupt officials, or other criminals. an automatic alarm may sound and may
include
audible deterrence, in addition to chemical deterrence such as automatic
release of
a marker chemical such as that sold under the trade name Dye Witness (TM).
This
alarm may be engaged to the continuity of comfort bands 210, so that if, for
example,
wire cutters or bolt cutters were used to cut through comfort bands 210, the
alarm
would be activated, marking the assailants for later identification and
prosecution.
Comfort bands 210 are preferably made of copper, so that they are somewhat
soft and flexible, and also so that they tarnish nicely to match typical hair
color of a
typical wearer. Copper, being a conductive material, is also suitable for the
continuity
based safety alarm described above.
CA 02354114 2001-07-16
Additionally, comfort bands 210 may sense, by conduction, the scalp of the
wearer;
to determine if the eyeglasses are worn (to thwart short-circuit forced
removal of the
apparatus), as well as to receive brainwave EEG signals. For this purpose a
lattice
of comfort bands that form a skull cap may be preferable, and may also
function as
a shield for other electrodes.
Additionally, the comfort bands may form part of a ground plane antenna for
wireless communications, especially when they help ground the antenna to the
head
of the user, to eliminate or reduce electromagnetic radiation wasted on the
user, and
therefore improve transmission to the airwaves while reducing exposure of the
user
to electromagnetic radiation.
FIG. 2B depicts an eyeglass based embodiment, showing an extra infrared Mocker
299 more extensive than portion 150 of the eyeglass frames. In this situation,
a wide
angle lens on camera sensor 130 makes it responsive to a wide field of view.
This
field of view is too great to be re-synthesized by an aremac, so a small field
aremac
260A resynthesizes the central 1/3 of the camera's field of view in linear
dimension
(e.g. 1/9 of the image area) whereas a framing aimer 260F allows the wearer to
see
a rectangular viewfinder to aim the camera sensor 130.
One aspect of the wide angle embodiment of the invention is the Mocker 299
that
typically appears to the wearer (and to other people looking at the wearer) as
a
clear plastic sheet, almost invisible, whereas in the infrared region of the
spectrum
to which camera sensor 130 is sensitive, the Mocker 299 is opaque. Therefore
Mocker
299 prevents ghost images from passing through diverter 110 into camera sensor
130.
Rays 200L from subject matter in view of the apparatus of the invention are
diverted, at least in part, as rays 200D for pickup by sensor 130. Sensor 130
may be
sensitive to some rays 200T that could be transmitted through diverter 110.
Despite
the fact that a hot mirror may be used for diverter 110, it may still ad.min a
small
portion of visible light due to incomplete stoppage. Many hot mirrors, for
example,
still transmit a small portion of infrared light. Alternatively, sensor 130
may be
sensitive to Borne visible light despite inserting a visible blocking infrared
filter in
front of sensor 130. A satisfactory visible blocking infrared filter is a
"Covert Infrared"
filter. or a Kodak (TM) wratten infrared filter.
However, despite attempts to keep out contamination, the diverter 110 is often
of
dichroic nature (a typical diverter might be a dichroic filter) which admit
some out of
band light. especially the more we deviate from 45 degrees (assuming the hot
mirror
11
. ,. CA 02354114 2001-07-16
is a 45 degree hot mirror). Thus the wide angle nature of the apparatus means
that
some infrared rays 200T from stray subject matter to the right of the wearer
would
leak through the diverter 110 as transmitted rays that are not entirely
reflected by
diverter 110.
Thus rays 200B of light from subject matter to the right of the wearer of the
appa-
ratus are blocked by Mocker 299, since Mocker 299 is an IR blocking sheet
(typically
of plastic material that is transparent in visible light). Therefore rays 200B
do not
contribute to rays 200T that would otherwise adversely affect the sensor 130
with a
ghost image of subject matter to the right of the wearer superimposed with
images
of subject matter in front of the wearer.
FIG. 3 depicts an embodiment of the invention for use as a camera ,jammer for
providing privacy to the wearer of the device. Diverter 110 is preferably a
hot mirror
to reflect infrared rays from an infrared laser 330. IR, laser 330 directs a
beam of
IR light as ray 331, to be reflected by diverter 110 and emerge as ray 332;
which
the wearer of the eyeglasses can aim into an offending camera. The light,
source this
reduces the contrast of the camera. A much weaker laser 335 produces ray 336
of
visible (typically red) light that reflects partially in diverter 110 and
emerges as ray
337 for being viewed by a wearer of the apparatus.
Diverter 110 typically only reflects a small portion of of ray 336. Therefore
ray 337
will be much weaker than ray 336, because the hot mirror is for reflecting
infrared
light and transmitting visible light. Thus the wearer of the apparatrus can
see a
relatively unobstructed view through diverter 110, together with a moderately
bright
aimer in ray 337. The aimer serves to allow the wearer to aim the apparatus at
the
intended target, namely the offending camera.
Multiple aimers can also be incorporated into the wearer's apparatus, and
image
tracking with a wearable camera can also be used to track the offending
cameras,
and keep a large number of cameras saturated with bright infrared light. Thus
if
the tracking algorithm continues to work after the wearer has sighted the
offending
cameras in a viewfinder, the wearer could continue to do other things while a
wearable
computer maintained the locations of the offending cameras and kept them
saturated
in infrared light.
Thus, for example, a person discussing this patent application, and showing a
patent agent the drawings while meeting in a restaurant, such as Second Cup,
where
cameras are concealed in objects that look like smoke detectors, could keep
all the
12
CA 02354114 2001-07-16
offending cameras saturated, in order to maintain privacy and confidentiality
of the
patent drawings; despite the use of the surveillance cameras in the
restaurant.
A similar invention may also be used as a light meter, for photographers to
have
a wearable spot meter. Instead of infrared laser 330, a lightmeter can be
aimed by
the wearer of the apparatus to measure light from a given location that is
visible by
way of laser 335.
Alternatively, instead of laser 330, a retinal scanner could be used, so that
the
device could be used to positively identify persons encountered by the wearer,
in
which laser 335 is used to aim the retinal scanner into an eye of a subject to
be
positively identified.
Such a device, trademark of applicant, for eyedentification (T1VI) produces
light at
a wavelength in the near infrared, with laser 330. A satisfactory wavelength
for laser
330 is 890 nanometers. A camera and laser 330 combination are aimed by the
wearer
of the apparatus, into an eye of a subject to be identifeyed (TM). To
identifeye (TM)
the subject, the wearer positions the aimer of laser 335 to indicate to the
wearer that
the laser 330 is positioned so that the laser 330 shines into at least one of
the subjects
eyes, and a headworn camera of the apparatus captures a recording of blood
vessels
in the subjects eyes. The pattern of blood vessels in the subject's eyes
provides a
unique identification of the subject. Such a device is particularly suited to
unwilling
subjects, such as clerks or functionaries who refuse to show identification
cards when
asked for identification.
The EyeTap technology may be combined with the EyeDentification (TM) tech-
nology illustrated here, so that face recognition and retinal scans can be
provided
from unwilling or uncooperative subjects in a dynamic realtime environment. By
combining facial thermographs and retinal scans of unwilling or uncooperative
sub-
jects, the apparatus can covertly and quickly identify persons in a dynamic
real world
environment.
Such an apparatus is particularly useful for war crimes, or for investigation
of
atrocities perpetrated by governments and large organizations, comprised of
officials
who might otherwise refuse to show identifcation. Unlike ordinary citizens who
can be
required to show their identification cards to police, the officials who have
previously
escaped accountability can now be brought under some degree of scrutiny.
FIG. 4 depicts an aimer in an eyeglass lens 400 made to have the same
appearance
as lenses of typical bifocal eyeglasses. Laser 330 is concealed in the
eyeglass frame
13
. CA 02354114 2001-07-16
at the edge of (or beyond the edge of ) the eyeglass lens 400. Preferably
laser 330
operates vitrionically (e.g. preferably from being embedded in the lens
material),
with ray 331 being preferably inside the lens material. Optics 431 reflects
beam 331 to
generate beam 3318 which is directed toward diverter 41CI. Diverter 410 is
preferably
disguised as the cut line typical of flat top bifocal eyeglasses. Therefore,
although
diverter 410 is preferably transparent in the visible (or other getting of
biological
sensitivity) and reflective in the infrared (or other getting of machine
sensitivity),
any residual incompleteness of this transparency in the getting of biological
sensitivity
(e.g. residual incompleteness of transparency in the spectrum of visible
light) will be
more forgiven by virtue of placement in the location common to the cut line of
bifocal
eyeglasses.
Laser 335 concealed also in the eyeglass frames produces ray 336 reflected by
optics
436 to produce ray 337 of reflected light.
Preferably ray 331 is infrared and ray 336 is visible red light. Ray 3318 is
diverted
away from an eye of the wearer of the eyeglasses by diverter 410 so that it
emerges
as ray 332. Ray 3378 is diverted by diverter 410 so that it emerges toward an
eye of
the wearer of the eyeglasses, as ray 337.
FIG. 5 depicts an embodiment of the invention used in an eyetap device. A
diverter 110 is used to divert eyeward bound light (light coming toward and
collinear
with rays passing through eye 100) into sensor 130. Portion 120C of a housing
is
opaque to infrared light (or light of another machine getting), whereas
portion 120B
is transparent to both the machine and biological gettings.
Aremac 160 with associated optics is for resynthesizing light rays for being
viewed
by eye 100. In the infrared getting, for example, the wearer can see in total
darkness,
by way of seeing through infrared sensor 130.
Preferably diverter 110 is invisible to others, but in some embodiments,
diverter
110 is still visible to others. Therefore, preferably there is a dummy side of
the device,
even if it is only a one eyetap device. Thus in Fig. 5 there is shown a, right
eyetap,
but the left eye. position has at least a dummy diverter 110D; so that the
eyeglasses
have a normal appearance.
FIG. 6 depicts an embodiment of the invention used in a vehicle such as an
aircraft,
boat, truck, or car. Rays 600L of headward bound light are rays of light that:
. shine toward; and
14
CA 02354114 2001-07-16
~ are collinear with a line passing through a point approximately at,
a head 600. Typically the head 600 will be the head of a driver. Preferably
the point
will be a point close to the eyes of the driver. At least a portion of the
rays 600L are
diverted into a camera or similar kind of sensor 130 by a c:liverter 110.
Preferably the
diverter is the windshied of the vehicle. Alternatively the diverter may be
affixed to
the windshield or be near the windshield.
Preferably the windshield reflects infrared light and passes visible light, so
that the
getting of machine sensitivity is infrared light, and the getting of
biological sensitivity
is visible light.
Preferably the sensor 130 is responsive to infrared light, and is located at a
point
such that it responds to rays 600L of headward bound light. If the windshield
or
diverter 110 were flat, the distance from sensor 130 to the point 130P
(principal
point) where the central optical axis of sensor 130 intersects diverter 110,
would be
equal to the distance from the head 600 to that same point. This distance is
known
as the EyeTap distance.
However, generally the windshield or diverter 110 is curved, so that the
distance
may be adjusted. Since the curve is often anisotropic, (e.g. giving
astigmatism in the
reflection), the sensor 130 may also be compensated optically and
computationally.
l~Toreover, aremac 160 will preferably be responsive to an output of sensor
130.
Preferably a processor is responsive to an output from sensor 130 and aremac
160
is responsive to an output from the processor. Thus the processor can
compensate
further for distortions in the windshield or diverter 110 and thus cause
aremac 160 to
synthesize rays 160L of synthetic light. Preferably each ray 160L of synthetic
light is:
~ responsive to; and
~ collinear with,
a corresponding ray 600L of headward bound light. The above condition is
called the
collinearity criterion.
Thus the apparatus preferably meets the eyetap criterion. An autofocus camera
of sensor 130 preferably drives the focus of the aremac 160. Alternatively the
camera
sensor 130 may be fixed focus, of sufficient depth of field, and a separate
distance
sensor adjusts a focus distance of aremac 160 depending on a sense of where a
driver
of the vehicle is looking.
CA 02354114 2001-07-16
An eye tracker 601 may determine where the driver is looking, and focus the
aremac 160 to match the depth plane of subject matter that the driver is
looking at.
Tracker 601 may also track the position of the head, a,nd it may also be
connected
to the processor so that aremac 160 can resynthesize the output to maintain
the
collinearity criterion.
Preferably sensor 130 is concealed in a car antenna or the like, or some
similar
appendage of the vehicle that does not have an unusual appearance such as
might
attract vandals to vandalize the sensor when the vehicle is parked.
FIG. 7 shows a roof box cargo carrier embodiment of the invention in which a
roof
box cargo carrier 700 is used to conceal a much larger sensor 130, as well as
other
navigational apparatus.
FIG. 8 shows an eyetap system with one or more additional auxiliary cameras
that are responsive to other points away from the eye point of the wearer.
Sensor 130 is an infrared eyetap sensor responsive to eyeward bound rays 200L
of
infrared light, whereas an even wider angle camera sensor 830 in the
nosebridge of
the eyeglasses is responsive to rays 800L of incoming light.
Auxiliary camera sensors 830A in the left and right temple side pieces are
respon-
sive to rays 800A.
Preferably the two camera sensors 830A and sensor 830 are responsive to a
getting
of biological sensitivity, such as visible light whereas sensor 130 is
responsive in a
getting of lesser biological sensitivity such as the infrared portion of the
spectrum.
Typically the image from sensor 130 is a greyscale image whereas the images
from
the other three cameras are color images. The epipola,r geometry between
sensor
130 and the other three sensors is fixed and known, so that sensor data from
the
three color cameras can be used to interpolate the viewpoint of sensor 130.
The
interpolated viewpoint might ordinarily be slightly lacking, as viewpoint.
interpolation
is somewhat imprecise. However, the luminance signal of sensor 130 is combined
with
an interpolated chrominance signal, so that the result is a natural color
image from
the viewpoint of the right eye of the wearer.
Alternatively, a statistical analysis of the color images is used to colorize
the
image from sensor 130. In this way. the apparatus can be completely covert,
since
the diverter 110 operates in a getting of lesser biological sensitivity, and
is therefore
concealable in eyeglass lens material, or the like.
Fig. 9 shows a block diagram of this embodiment, in which left camera sensor
16
CA 02354114 2001-07-16
930L and right camera sensor 9308 are both connected to a viewpoint
interpolator
900 to generate an interpolated signal 901. Signal 901 is a,n approximation to
a video
signal that would have been seen from the viewpoint of the wearer's right eye
100.
Eyetap camera 130 by way of diverter 110 provides an actual signal from this
same
viewpoint, but at a getting of lesser biological sensitivity.
In applications where it is desired to have a video signal that has a natural
ap-
pearance (such as natural color or natural visible light greyscale) signal 901
may be
used.
Alternatively, an even better Point of Eye (PoE) signal 911 may be generated
by
colorizing eyetap signal 931 with statistics derived from interpolated PoE
signal 901.
The final eyetap signal 911 rnay be fed to a processor 920; to provide a
computer
mediated reality environment (or at least a viewfinder function) by way of
aremac
160.
In all aspect's of the present invention, references to "camera" mean any
device or
collection of devices capable of simultaneously determining a quantity of
light arriving
from a plurality of directions and or at a plurality of locations, or
determining some
other attribute of light arriving from a plurality of directions and or at a
plurality of
locations.
References to "processor" . or "computer" shall include sequential
instruction, par-
allel instruction, and special purpose architectures such as digital signal
processing
hardware, Field Programmable Gate Arrays (FPGAs), programmable logic devices,
as well as analog signal processing devices.
References to "eyeward bound" light refer to rays of light that are collinear
with
rays passing through a point near the center of projection of an eye of a user
of the
apparatus of the invention, regardless of whether or not the rays are blocked
before
they reach this point.
References to ''eyesward bound" light refer to rays of light that are
collinear with
rays passing through a point near the eyes of a user of the apparatus of the
invention,
regardless of whether or not the rays are blocked before they reach this
point.
References to "headward bound" light refer to rays of light that are collinear
with
rays passing through a point near the head of a user of the apparatus of the
invention,
regardless of whether or not the rays are blocked before they reach this
point.
From the foregoing description, it will thus be evident that the present
invention
provides a design for differently sensitive optics for an EyeTap device. As
various
17
CA 02354114 2001-07-16
changes can be made in the above embodiments and operating methods without
departing from the spirit or scope of the invention, it is intended that all
matter
contained in the above description or shown in the accompanying drawings
should be
interpreted as illustrative and not in a limiting sense.
Variations or modifications to the design and construction of this invention,
within
the scope of the invention, may occur to those skilled in the art upon
reviewing
the disclosure herein. Such variations or modifications, if within the spirit
of this
invention, are intended to be encompassed within the scope of any claims to
patent
protection issuing upon this invention.
18
