Note: Descriptions are shown in the official language in which they were submitted.
CA 02247649 2000-02-21
Patent Application
of
Steve Mann
for
COVERT CAMERA VIEWFINDER OR DISPLAY HAVING
APPEARANCE OF ORDINARY EYEGLASSES
of which the following is a specification:
FIELD OF THE INVENTION
The present invention pertains generally to a wearable apparatus that provides
the
wearer with an electronic display device, graticule, reticule, or crosshairs,
similar to
or suitable for a camera viewfinder screen, while at the same time appearing
to others
as ordinary bifocal eyeglasses.
BACKGROUND OF THE INVENTION
In photography (and in movie and video production), it is desirable to capture
events in a natural manner with minimal intervention and disturbance. Current
state-of-the-art photographic or video apparatus, even in its most simple
"point and
click" form, creates a visual disturbance to others and attracts considerable
attention
on account of the gesture of bringing the camera up to the eye. Even if the
size of
the camera could be reduced to the point of being negligible (e.g. no bigger
than
the eyecup of a typical camera viewfinder, for example), the very gesture of
holding a
device up to, or bringing a device up to the eye is unnatural and attracts
considerable
attention, especially in establishments such as gambling casinos or department
stores
where photography is often prohibited. Although there exist a variety of
covert
cameras such as a camera concealed beneath the jewel of a necktie clip,
cameras
concealed in baseball caps, and cameras concealed in eyeglasses, these cameras
tend to
produce inferior images, not just because of the technical limitations imposed
by their
2
CA 02247649 2000-02-21
small size, but, more importantly because they lack a means of accurately
determining
which objects in the scene are within the field of view of the camera to aim
the
camera for obtaining a picture having good photographic or videographic
composition.
Because of the lack of a viewfinder, investigative video and photojournalism
made
with such cameras suffers from poor composition. Accordingly, such covert
cameras
are often fitted with very wide angle lenses so that the poor aim will not
result in
missing important subject matter. As a result of these wide angle lenses,
details in
the scene are typically much more poorly rendered than they would be if a
normal or
tele lens were used.
It is desired that a camera viewfinder provide a very clearly defined
rectangular or
oval shaped outline showing clearly what is included in the field of view of
the camera
and what is not. It is desired that the rectangular outline be in sharp focus
along
with the subject matter, even though the rectangular outline marking is
typically
much closer to the eye (since it is located inside the camera) than the
subject matter
in the scene.
Alternatively, or in addition to this rectangular outline, it may be desired
to have
crosshairs, or a small circular outline, brackets, or the like, to indicate
where the
center of the camera's field of view is located. Brackets, or a small circular
outline,
may also indicate field of view of a special portion of the image, such as the
portion
over which automatic gam control (AGC) or automatic focus (AF) is determined.
Traditional camera viewfinders often include the ability to overlay virtual
objects,
such as camera shutter speed, or the like, on top of reality, as described in
U.S.
Pat. No. 5664244 which describes a viewfinder with additional information
display
capability. In this case, :it is desired that both the virtual objects that
exist within
the viewfinder and the real objects that exist beyond the viewfinder appear in
sharp
focus.
Open-air viewfinders are often used on extremely low cost cameras, as well as
on
some professional cameras for use at night when the light levels would be too
low to
3
CA 02247649 2000-02-21
tolerate any optical loss in the viewfinder. Examples of open-air viewfinders
used on
professional cameras, in addition to regular viewfinders, include those used
on the
Grafflex press cameras of the 1940s (which had three different kinds of
viewfinders),
as well as those used on some twin-lens reflex cameras. However, such open air
viewfinders do not generally provide a sharply defined boundary. The
rectangular
boundary defined by such viewfinders is in poor focus, making it difficult for
the user
of such a camera to determine the exact extent of coverage of the camera.
SL~~MMARY OF THE INVENTION
It is an object of this invention to provide a covert wearable viewfinder that
may
be used without a camera of any kind, so that the wearer can simply imagine
what it
is like to be shooting, and can hone his or her compositional skills by having
a reticle,
graticule, crosshairs, or the like, over in front of his or her vision at all
times while
wearing the glasses.
It is an object of this invention to provide a wearable eyeglass based device
allowing
the wearer to covertly view data, such as from the screen of a wearable
computer
(wearcomp) system but where the device appears to others to be simply a pair
of
ordinary bifocal eyeglasses.
It is an object of this invention to provide a wearable eyeglass based device
allowing
the wearer to covertly view data, such as from the screen of a wearable
computer
(wearcomp) system but where the device appears to others to be simply a pair
of
ordinary reading glasses.
It is an object of this invention to provide a wearable eyeglass based device
allowing
the wearer to covertly view electronically stored pictures but where the
device appears
to others to be simply a :pair of ordinary bifocal eyeglasses.
It is an object of this invention to provide a wearable eyeglass based device
allowing
the wearer to covertly view electronically stored pictures but where the
device appears
to others to be simply a :pair of ordinary reading glasses.
4
CA 02247649 2000-02-21
It is an object of this invention to provide a method of covertly positioning
a
camera in which both hands may be left free, and in which the direction in
which the
camera is facing is clearly indicated by means of some marking that appears as
if it
were superimposed on the real objects in the scene.
It is a further object of this invention to provide a means of covertly
exposing a
film or acquiring a picture electronically where the spatial extent (field of
view) of
the image may be ascertained without having to hold any device up to the eye.
What is described i~~ a wearable camera and viewfinder for capturing video of
exceptionally high compositional and artistic calibre. In addition to the fact
that
covert versions of the apparatus can be used to create investigative
documentary
videos having very good composition, for everyday usage the device need not
neces-
sarily be covert. In fact, it may be manufactured as a fashionable device that
serves
as both a visible crime deterrent, as well as a self-explanatory (through its
overt
obviousness) tool for documentary videomakers and photojournalists.
Another feature of the invention is that the wearable camera has a viewfinder
such
that the image may be presented in a natural manner suitable for long-term
usage
patterns.
There are several re~~sons why it might be desired to wear the camera over a
sustained period of time:
1. There is the notion of a personal visual diary of sorts.
2. There is the idea of being always ready. By constantly recording into a
circular
buffer, a retroactive record function, such as a button that instructs the
device
to "begin recording from five minutes ago" may be useful in personal safety
(crime reduction) s~s well as in ordinary everyday usage, such as capturing a
baby's first steps o:n video. With the prior art in photography and video, we
spend so much time preparing the camera and searching for film, batteries,
etc.,
or at the very least, just getting the camera out of its carrying case, that
we
CA 02247649 2000-02-21
often miss important moments like a baby's first steps, or a spontaneous
facial
expression during the opening of a gift.
3. There is the fact that the wearable camera system, after being worn for a
long period of time, begins to behave as a true extension of the wearer's mind
and body. As a result, the composition of video shot with the device is often
impeccable without even the need for conscious thought or effort on the part
of
the user. Also, one can engage in other activities, and one is able to record
the
experience without the need to be encumbered by a camera, or even the need to
remain aware, at a conscious level, of the camera's existence. This lack of
the
need for conscious thought or effort suggests a new genre of documentary video
characterized by long-term psychophysical adaptation to the device. The result
is a very natural first-person perspective documentary, whose artistic style
is
very much as if a recording could be made from a video tap of the optic nerve
of
the eye itself. Events that may be so recorded include involvement in
activities
such as horseback riding, climbing up a rope, or the like, that cannot
normally
be well recorded from a first-person perspective using cameras of the prior
art.
Moreover, a very natural first-person perspective genre of video results. For
example, while wearing an embodiment of the invention, it is possible to look
through the eyepiece of a telescope or microscope and record this experience,
including the approach toward the eyepiece. The experience is recorded, from
the perspective of the participant.
4. A computational system, either built into the wearable camera, or worn on
the
body elsewhere and connected to the camera system, may be used to enhance
images. This may be of value to the visually impaired. The computer may
also perform other l;asks such as object recognition. Because the device is
worn
constantly, it may also function as a photographic/videographic memory aid,
e.g. to help in way-:finding through the recall and display of previously
captured
6
CA 02247649 2000-02-21
imagery. For example, the wearable camera system may recognize the face of
someone within the camera's field of view and superimpose the name of the
person it recognized into the viewfinder. This is presented to the wearer in
the
form of a virtual name tag.
An important aspect of the proposed invention is the capability of the
apparatus
to mediate (augment, anal to a limited extent diminish, or otherwise alter)
the visual
perception of reality.
The proposed camera, viewfinder is related to the displays that are used in
the field
of Virtual Reality (VR) in the sense that both are wearable. However, an
important
difference is that the proposed invention allows the wearer to continue to see
the real
world, while VR displays block out the ability to see the real world.
It is possible with this invention to provide a method of exposing a film or
ac-
quiring a picture electronically where the tonal characteristics of the
picture may be
ascertained without having to hold any device up to the eye.
It is possible with this invention to provide a method of exposing a film or
acquiring
a picture electronically where no apparent difference in body movement or
gesture
between when a picture is being taken and when no picture is being taken is
detectable
by others.
It is possible with this invention to provide the user with a means of
determining
the composition of the picture from a display device that is located such that
only the
user can see the display device, and so that the user can ascertain the
composition
of a picture or take a picaure or video and transmit images) to one or more
remote
locations without the knowledge of others in the immediate environment.
It is possible with this invention to provide the user with a means of
determining
the composition of the picture from a display device that is located such that
only
the user can see the display device, as well as an optional additional display
device
that the user can show to others if and when the user desires to do so.
It is possible with this invention to provide the user with a means of
determining
7
CA 02247649 2000-02-21
the composition of the picture from a display device that is located such that
both
the user as well as others can see it, if the user should so desire.
It is possible with this invention to provide a means for a user to experience
additional information overlaid on top of his or her visual field of view such
that the
information is relevant to the imagery being viewed.
It is possible with this invention to provide a means and apparatus for a user
to
capture a plurality of images of the same scene or objects, in a natural
process of
simply looking around, and then have these images combined together into a
single
image of increased spatial extent, spatial resolution, dynamic range, or tonal
fidelity.
It is possible with this invention to provide a stereo viewfinder means in
which
there are two viewfinders, one concealed in each eyeglass lens of what appears
to
others to be bifocal glasses, or above each lens of what appears to others to
be
ordinary reading glasses.
The proposed invention facilitates a new form of visual art, in which the
artist
may capture, with relatively little efFort, a visual experience as viewed from
his or
her own perspective. Wirh some practice, it is possible to develop a very
steady body
posture and mode of movement that best produces video of the genre pertaining
to this invention. Because the apparatus may be lightweight and close to the
head,
there is not the protrusion associated with carrying a hand-held camera. Also
because
components of the apparatus of the invention are preferably mounted very close
to
the head, in a manner that balances the weight distribution, the apparatus
need not
restrict the wearer's head moverrlent or encumber the wearer appreciably.
Mounting
close to the head minimizes the moment of inertia about the rotational axis of
the
neck, so that the head can be turned quickly while wearing the apparatus. This
arrangement allows one to record the experiences of ordinary day-to-day
activities
from a first-person perspective. Moreover, because both hands are free, much
better
balance and posture is possible while using the apparatus. Anyone skilled in
the arts
of body movement control as is learned in the martial arts such as karate, as
well as
8
CA 02247649 2000-02-21
in dance, most notably ballet, will have little difficulty capturing
exceptionally high
quality video using the apparatus of this invention.
With known video or movie cameras, the best operators tend to be very large
peo-
ple who have trained for many years in the art of smooth control of the
cumbersome
video or motion picture film carrieras used. In addition to requiring a very
large per-
son to optimally operate such cameras, various stabilization devices are often
used,
which make the apparatus even more cumbersome. The apparatus of the invention
may be optimally operated by people of any size. Even young children can
become
quite proficient in the use of the wearable camera system.
A typical embodiment of the invention comprises one or two spatial light modu-
lators or other display means built into a pair of bifocal eyeglasses or
reading glasses,
together with one or more sensor arrays. Typically one or more CCD (charge
coupled
device) image sensor arrays and appropriate optical elements comprise the
camera
portion of the invention. In the bifocal embodiment of the invention,
typically a
beamsplitter or a mirror silvered on both sides is used to combine the image
of the
viewfinder with the apparent position of the camera, or to display the extent
of cov-
erage of the camera. The beamsplitter is typically imbedded into the eyeglass
lens,
and is made much wider than it needs to be, so that it appears to be a cut
line
in the eyeglass lens typical of bifocal eyeglasses. The viewfinder is simply a
means
of determining the exter.~t of coverage of the camera in a natural manner, and
may
comprise either of:
~ A reticle, graticule., rectangle, or other marking that appears to float
within a
portion of the field of view.
~ A display device that shows a video image, or some other dynamic information
perhaps related to the video image coming from the camera.
One aspect of the invention allows a photographer or videographer to wear the
apparatus continuously and therefore always end up with the ability to produce
a
9
CA 02247649 2000-02-21
picture from something that was seen a couple of minutes ago. This may be
useful to
everyone in the sense that we may not want to miss a great photo opportunity,
and
often great photo opportunities only become known to us after we have had time
to
think about something we previously saw.
Such an apparatus might also be of use in personal safety. Although there are
a
growing number of video surveillance cameras installed in the environment
allegedly
for "public safety", there have been recent questions as to the true benefit
of such
centralized surveillance infrastructures. Most notably there have been several
exam-
ples in which such centralized infrastructure has been abused by the owners of
it (as
in roundups and detainment of peaceful demonstrators). Moreover, "public
safety"
systems may fail to protect individuals against crimes committed by the
organiza-
tions that installed the systems. The apparatus of this invention allows the
storage
and retrieval of images b~y transmitting and recording images at one or more
remote
locations. Images may be transmitted and recorded in different countries, so
that
they would be difficult to destroy, in the event that the perpetrator of a
crime might
wish to do so.
The apparatus of the invention allows images to be captured in a natural
manner,
without giving an unusual appearance to others (such as a potential
assailant).
Accordingly, the present invention in one aspect comprises eyeglasses with an
LED
imbedded inside the glass or plastic material from which the eyeglasses are
made, and
wires to connect the LED to the outside world. The LED is so close to the eye
that
one cannot focus on it, but it does provide a blurry circular disk with which
the
wearer can orient the glasses in a particular direction with respect to a
particular
object. Preferably there is also a small crosshairs scratched onto the inside
surface
of the eyeglass lens, so that these will appear sharp within the blurry
circular disk,
and will appear at all depths of focus, e.g. no matter what the wearer is
looking at,
the crosshairs will appear sharp and in focus, as if they formed an image with
infinite
field of view. The crosshairs will appear in the image of the wearer's eye
lens, so
CA 02247649 2000-02-21
that particles of dust or dirt in the eye will also be visible within the
blurry circle of
confusion of the LED. Such eyeglasses may be useful for their amusement value,
e.g.
to give the wearer the feeling that he or she is shooting at something or
aiming at
something. Even without any form of camera in the eyeglasses, such eyeglasses
could
be marketed as a toy or personal amusement device, because of the unique
appearance
the wearer has, and the imagined sense of personal empowerment involved in
sighting
an "enemy" in the crosshairs etched onto the inside glass.
According to another aspect of the invention, there is provided a plurality of
LEDs,
with wires running inside the lens material. Preferably. either two LEDs are
used to
mark the left and right edges of a rectangular boundary, or four are used, one
for
each corner of the rectangular boundary.
According to another aspect of the invention, there is provided a wearable
camera
concealed in the eyeglas~~es, in which the above mentioned crosshairs are used
to aim
the camera.
According to another aspect of the invention, there is provided a similar
wearable
camera system in which the above rectangular boundary is used to aim the
camera.
According to another aspect of the invention, there is provided a wearable cam
era system in which four L-shaped corners are used to define the four corners
of a
rectangular boundary which is used to aim the camera.
According to another aspect of the invention, there is provided an elongated
beam-
splitter imbedded within an eyeglass lens. The beamsplitter is made much
longer than
necessary, in order that it have the appearance of a normal cut line in a
normal pair
of flat top bifocal eyeglasses. Preferably this beamsplitter is oriented at a
45 degree
angle, so that it can show the output of an electronic display that might, for
example,
be connected to the output of a WearComp system that is processing images from
a
camera also concealed inside the eyeglass frames.
According to another aspect of the invention, there is provided eyeglasses in
which
a viewscreen in the eyeglasses provides an electronic viewfinder connected to
the
11
CA 02247649 2000-02-21
output of the camera also concealed in the eyeglass frames. Preferably this
viewfinder
allows the wearer to examine not just composition, but also exposure,
contrast, and
details of the image, as well as meta information such as that determined by
an object
recognition program that might, for example, recognize faces of people in the
picture
and remind the wearer who it is that the wearer wanted to photograph, or
remind
the wearer of people in the picture who are not satisfied with their picture
portraits,
or who need to be re-photographed in better lighting or the like.
BRIEF .DESCRIPTION OF THE DRAWINGS
The invention will now be described in more detail, by way of examples which
in no way are meant to :limit the scope of the invention, but, rather, these
examples
will serve to illustrate the invention with reference to the accompanying
drawings, in
which:
FIG. 1 illustrates the principle and components of typical flat-top bifocal
eye-
glasses.
FIG. 2 illustrates an older style of bifocal eyeglasses in which the cut line
runs
along the entire width of the lens.
FIG. 3 illustrates the concealment of an LED inside an eyeglass lens.
FIG. 4 illustrates how the concealed LED can be used to center the subject
matter
in the field of view of a camera also concealed within the eyeglasses.
FIG. 5 shows an embodiment of the invention in which there are two LEDs con-
cealed within an eyeglass lens, so that the left and right edges of a
rectangular bound-
ary may be defined.
FIG. 5a shows a top view, while FIG. 5b shows an inside view and FIG. 5c shows
the view seen by the wearer when the apparatus is in operation.
FIG. 6 shows an embodiment; of the wearable viewfinder invention in which the
viewfinder contains four LEDs concealed in what appears to others like an
ordinary
eyeglass lens of ordinary trifocal construction.
12
CA 02247649 2000-02-21
FIG. 6a shows the inside view, and FIG. 6b shows the view as seen by the
wearer,
when the device is in operation.
FIG. ? shows an embodiment of the wearable camera invention where there is a
beamsplitter concealed in an eyeglass lens such that it looks like the cut
line across
the top of a flat top bifocal eyeglass lens.
FIG. 8 shows an improvement of the wearable camera invention where there is
a bifocal inset lens that magnifies the screen but not the objects seen
through the
glasses, and where there is also ambient light amplification so that the
screen is backlit
in proportion to the amount of light in the environment, but in such a way
that it is
more brightly lit than with passive ambient backlighting.
FIG. 9 shows an alternate embodiment of the wearable camera system where the
display magnification is done through reflection rather than refraction, and
where
the camera and display do not function along collinear rays directly, but,
instead, a
WearComp is used to correct for this noncollinearity.
DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
While the invention shall now be described with reference to the preferred em-
bodiments shown in the drawings, it should be understood that the intention is
not
to limit the invention only to the particular embodiments shown but rather to
cover
all alterations, modifications and equivalent arrangements possible within the
scope
of appended claims.
In all aspects of the present invention, references to "camera" mean any
device or
collection of devices capable of simultaneously determining a quantity of
light arriving
from a plurality of directions and or at a plurality of locations, or
determining some
other attribute of light arriving from a plurality of directions and or at a
plurality
of locations. Similarly references to "viewscreen" shall not be limited to
just minia-
turized television monitors or computer monitors, but shall also include
computer
data display means, as well as fixed display means, where such fixed display
means
13
CA 02247649 2000-02-21
include crosshairs, graticules, reticles, brackets, etc., and other video
display devices,
still picture display devices, ASCII text display devices, terminals, and
systems that
directly scan light onto the retina of the eye to form the perception of an
image.
References to "processor", or "computer" shall include sequential instruction,
par-
allel instruction, and special purpose architectures such as digital signal
processing
hardware, Field Programmable Gate Arrays (FPGAs), programmable logic devices,
as well as analog signal processing devices.
When it is said that object ''A" is "borne" by object "B", this shall include
the
possibilities that A is attached to B, that A is part of B, that A is built
into B, or
that A is B.
FIG. 1 is a diagram depicting a pair of bifocal eyeglasses, and will serve to
define
the terminology pertaining to various parts of the bifocal eyeglasses, as this
terminol-
ogy will be used in this disclosure. Eyeglasses are normally held upon the
head of the
wearer by way of temple side pieces 110. These temple side pieces 110 support,
by
way of a hinge held on with rivets 120, a frame 130. Within frame 130 are two
lenses
140. Lenses 140 contain inset lenses 150 which may have different
prescriptions than
lenses 140 in which they are set. There is typically a cut line 160 between
lens 140
and lens 150.
Typically lens 140 will provide a prescription for distant objects while inset
lens
150 will provide a prescription for nearby objects. Lens 150 is commonly used
for
reading. Accordingly, lens 140 rnay, in some cases, have infinite focal length
(zero
power) and simply serve as a support for lens 150, which is typically a lens
of positive
focal length (e.g. a magnifying glass).
In some cases, lens 140 may be non existent, as in typical reading glasses, in
which
lens 150 is mounted directly to frame 130, and the wearer looks over frame 130
to see
distant objects and looks through lens 150 to see nearby objects.
In many modern multifocal eyeglasses, there is no visible cut line 160, and
instead
there is a gradual transition from the prescription of lens 140 to that of
lens 150. Such
14
CA 02247649 2000-02-21
multifocal lenses are known as progressive. The purpose of such gradual
transition
is to accomodate a variety of distances, in situations where the wearer
normally has
an inability to focus over any appreciable range of distances, as well as for
improved
appearance. Since the need for multifocal lenses shows a deficiency on the
part of the
user, there has been a trend toward hiding this deficiency, just as there has
been a
trend toward contact lenses and laser eye treatments to eliminate the need for
eyewear
altogether. However, amid the desire among some to hide the fact that they
need
bifocals or even just ordinary glasses, there are others who like to wear
eyeglasses.
Even some people who do not need eyeglasses often wear so-called pseudo-
intellectual
glasses, which are glasses in which lens 140 has infinite focal length.
Moreover, bifocal
eyeglasses and reading glasses are often associated with intellectuals, and
thus there
is a portion of society that would readily wear glasses having the general
appearance
of those depicted in FIG. 1, even if they did not require a prescription of
any kind.
The eyeglass lenses 14.0 may also contain markings 170 made directly on the
glass.
Such markings, for example, may be the manufacturer's name or an abbreviation
(such as the letters "GA" engraved on the left lens of Giorgio Armani (trade-
mark)
glasses). For illustration in this disclosure, the markings "L" and "R" denote
left and
right lenses, as labeled from the perspective of the wearer. Such markings
will help
make it clearer which lens and which side of the lens is being shOWll.
FIG. 2 depicts the left lens of an older style of bifocal eyeglasses. The cut
line
extends all the way across. Older styles of bifocal eyeglasses such as that
depicted
in FIG. 2 are becoming popular among some individuals, so that a eyeglasses
with
lenses like those depicted in FIG. 2 would not look particularly out of place.
Lens
240 is suitable for looking at distant objects, while lens 250 is suitable for
looking at
near objects. Lens 240 may have infinite focal length (zero power) if the
glasses are
intended to be worn by a person who has normal vision (e.g. does not need
corrective
eyewear). Such a person may wear these glasses simply to facilitate in doing
fine
soldering, needlework, or the like, by virtue of lenses 250 which may simply
serve the
CA 02247649 2000-02-21
same purpose as ordinary magnifying glasses.
Alternatively, lenses 250 and 240 may both be of infinite focal length, and
the
cut line 260 therebetween may be simply for cosmetic purposes, e.g. to provide
the
wearer with the appearance of a traditional intellectual who might wear old-
style
bifocal eyeglasses.
Thus eyewear as depicted in FIG. 2 may be constructed to meet the prescription
of one who requires bifocal glasses, or one who requires only ordinary
unifocal glasses,
or one who requires no glasses at all.
Accordingly, the eyewear as depicted in FIG. 2 may be constructed for nearly
anyone, and may also be used as a basis in which to conceal additional
apparatus.
FIG. 3 shows how an LED (light emitting diode) may be concealed within the
lens material (such as glass or plastic) of an eyeglass lens 340. Lens 340 may
comprise
two separate prescriptions, one prescription, or no prescription at all,
depending on
the wearer's needs or lack thereof. A thin wire 310, preferably made of
stainless steel
or other material that is silver in. colour, is imbedded on or within the lens
material
340. Wire 310 carries electrical current to the anode terminal of an LED 320
also
imbedded in the lens material. Wire 330 carries electrical current from the
cathode
terminal of the LED. (Fc>r purposes of this disclosure conventional current,
in which
electricity flows from plus to minus, is used rather than electron current in
which
electrons flow from minus to plus. )
LEDs normally come in clear plastic housings, which are quite large (e.g. on
the order of three millimeters in diameter). However, the internals of an LED
may
be imbedded into the lens material 340, so that the lens material itself forms
the
protective housing around these internals. In this way, the LED 320 is too
small to
be easily seen by the unaided eye of someone who might be looking at the
wearer's
eyeglasses. Thus, so long as the LED 320 is not illuminated, it will remain
essentially
invisible to others.
A miniature shroud 321 is ty pically placed over LED 320 so that people other
16
CA 02247649 2000-02-21
than the wearer of the glasses cannot easily see the light from LED 320. This
shroud
is typically made in an irregular shape, so that it has the appearance of a
spec of dust
or small particle of dirt. The side facing the wearer (and behind the LED from
the
wearer's perspective) is preferably black, while the side facing away from the
wearer
is preferably dust or dirt-coloured, and may comprise dirt or dust particles
or other
imperfections imbedded into the glass.
As an alternative to an LED, a scintillating fiber optic or other light source
imbed-
ded in the glass may be used.
Moreover, a cut line on the inside of the glass, such as crosshairs scratched
into
the glass, may be used to project this image directly onto the retina of the
wearer,
so that it is in focus for ;x,11 focal distances of the wearer's own eye lens.
FIG. 4 depicts subject matter being captured using an embodiment of the in-
vention. In this example, subject matter may, for example, comprise a
department
store manager or clerk who has illegally chained his fire exits shut, but also
(despite
the use of video surveillance in his department store) has a store policy
prohibiting
photography by customers in the department store in order that his criminal
activity
is not documented.
An arbitrary point 9:10 on the subject matter of interest radiates light in
all
directions, and some of this light may be collected by a customer wearing
eyeglasses
in which a video camera has been concealed.
Light from 410 passes through the customer's eyeglasses, in particular,
through
a lens 340 of the customer's eyeglasses, and then through lens 420 of
customer's eye
430. This light converges to a paint 440 on the customer's retina. To the left
of eye
430 is shown the image of the department store manager upon the customer's
retina,
and point 450 of this image corresponds to point 410 of the subject matter.
LED 320 is located in eyeglass lens 340 which is very close to the wearer's
eye.
Humans with normal healthy vision can focus on objects that are between about
4
inches (approximately lOcm) and infinity, away from the lens 420 in the eye
430.
17
CA 02247649 2000-02-21
Thus objects such as LED 320 'which are closer than 4 inches must appear out
of
focus. LED 320 is so close to the eye, in fact, that it will appear extremely
out of
focus. The customer will not be able to see the LED in his eyeglasses, and in
fact the
customer will see a very large circular-shaped blob which is the out-of-focus
image
of the point source LED 320.
The circular disk that one sees from a point source of light that is out of
focus is
known as the circle of confusion.
Rays of light from LED 320 are denoted by dashed lines which eye lens 420 is
too
weak to focus, so that they spread out and strike the retina at 460, defining
a circular
disk of light 470. LED 320 is typically red or green, so that disk 470 appears
as a
large circle of red or green light.
The exact shape of this disk 9:70 is determined by the shape of the opening in
the
eye, and disk 470 will also show imperfections in the eye lens 420, such as
dust on
eye lens 420, or any irregularities in the eye iris of lens 420.
However, despite these irregularities, the circular blob 470 will indicate to
the
wearer the direction in which a video camera that has been hidden inside the
eyeglass
frames is pointing.
Thus the customer m.ay make use of LED 320 imbedded in eyeglass lens 340 to
orient his wearable camera in the direction of the subject matter, and to know
that
the subject matter is centered in the camera's field of view.
Moreover, the colour and state of the LED (e.g. whether the LED is flashing,
and at what rate, and in the case of a multicolour LED whether it is red or
green
or whichever other colours it may assume) may convey additional information to
the
wearer of the apparatus.
Typically LED 320 turns red to indicate that a recording device is active, or
a
transmitter is active. Typically the LED begins flashing when hard disk space
on the
recording device is almo~~t full. The rate of flashing is typically used to
indicate to
the wearer how much disk space remains.
18
CA 02247649 2000-02-21
In order that light from the LED that might be reflected off the inside
surface of
the glass 340, or off the wearer's eye 430, is not seen by others (such as the
depart-
ment store manager or t:he like), LED 320 is automatically adjusted in
brightness in
accordance with ambient light levels. Typically the camera is capable of
measuring
the quantity of light received, and also estimating the scene contrast, and
from this
information, provides a control voltage to LED 320 so that it becomes bright
when
necessary (such as outdoors on a sunny day) and darker when it does not need
to be
so bright (such as in a dimly lit <;orridor or stairwell of a department
store).
One drawback of the invention described in FIG. 4 is that the circular disk
470
varies in size depending on the opening and closing of the iris of eye lens
420. Thus
although it may be used to center the subject matter in the viewfinder, it
cannot
easily be used as an indicator of the extent of coverage of the camera, as
with a
traditional viewfinder in which the user can see whether or not the subject
matter is
wholly contained within the field of view of the camera.
FIG. 5 therefore depicts an unproved viewfinder in which two LEDs are
concealed
with wiring along the cut line of (or where the cut line would be in) a
bifocal eyeglass
lens 340. LEDs 520 and 521 may be of similar construction to LED 320 and may
be
similarly shrouded so that others facing the wearer do not readily see the
light from
LEDs 520 and 521.
Wire 510 carries electric current to LED 520, which is connected in series
with
LED 521 by wire 511, followed by wire 512 which completes the circuit. It is
preferable
that the LEDs be wired in series, so that a single current limiting resistor
or drive
circuit can drive both of them at equal brightness. (Wiring LEDs in parallel
is known
to provide unreliable and sometimes unpredictable results.)
FIG. 5a is a top view of FIG. 5, looking at the eyeglass lens 340 on edge. The
surface of the glass 340 that faces away from the wearer is designated 341,
while that
facing toward the wearer is designated as 342. LEDs 520 and 521 are imbedded
inside
the glass but located near surface 341. On the other surface 342 are scratch
marks
19
CA 02247649 2000-02-21
550 and 551 which are constructed to look like part of the optical cut lines
around
normal bifocal insets.
These cut lines are made in an inside-out bracket shape.
FIG. 5b is an inside view of F'IG. 5, looking at the eyeglass lens from the
wearer's
side.
FIG. 5c is an inside view of FIG. 5, looking at the eyeglass lens from the
wearer's
side, but showing how it appears when the LEDs 520 and 521 are turned on, and
the
glass is too close to a wearer's eye to focus on. Instead, light from LED 520
projects
an image of scratch marls 550 directly onto the retina of the wearer's eye.
Since the
image of scratch mark 550 is not inverted (e.g. since it is projected directly
onto
the retina), it will appear to the wearer as if it is inverted. This is
because upright
objects are normally presented inverted (upside down) on the retina, and this
is what
we are used to. (See for example, George M. Stratton's 1896 article in the
journal
"Psychological Review" , entitled "Some Preliminary Experiments on Vision" . )
It is
for this reason that the two halves of the brackets are each backwards.
What the wearer sees is inward-facing brackets as shown in FIG 5c. These are
seen as dark lines within the circles of confusion 570 and 571. Circles of
confusion
570 and 571 arise from I~EDs 520 and 521 respectively, since each is a point
source
that is too close to the e;ye for the eye lens to focus on.
Brackets 550 and 551 are sufficient to indicate to the wearer what subject
matter
will be within the camera's field of view and what will not. Most notably,
brackets
550 and 551 are made to match exactly the horizontal field of view of the
camera, and
some fraction of the vertical field of coverage of the camera. Typically this
fraction of
vertical coverage is 1/3, so that the wearer can easily imagine a grid 580
where there
are 12 squares of size equal to that defined by each bracket. Grid 580 is
shown as a
dashed line because the wearer does not in fact see grid 580 but merely
imagines it,
given the visual cues 550 and 557..
Moroever, since brackets 550 and 551 define the central 1/3 horizontal strip
along
CA 02247649 2000-02-21
what will be the final picture, the wearer can concentrate on this portion of
the frame,
and compose a picture using the well-known "rule of thirds" in which it is
suggested
that the most artistically pleasing images usually result by placing important
subject
matter on a 1/3 boundary.
Thus the viewfinder of FIG 5 provides the wearer with a practical wearable
camera
system in which there is an awareness of picture composition.
FIG. 6 shows an embodiment of the wearable camera system in which the
viewfinder
means is concealed in eyeglass lens 340 configured to appear as if it were an
ordinary
trifocal eyeglass lens.
The same series configuration of LEDs as that depicted in FIG 5 is used, but a
second row higher up, in. which wire 610 carries electricity to the anode
terminal of
LED 620 which is connected in series with LED 621 by way of wire 611, and in
which
wire 612 completes the circuit.
Each pair of LEDs has its own current limiting resistor or the like which is
typically
mounted in the eyeglass frames so that a single set of wires concealed within
the frames
can power the LEDs. These wires are typically connected to a waist-worn power
supply and the wiring fi°om the glasses to the power supply is
typically concealed
within an eyeglass safety strap. A satisfactory eyeglass safety strap for
concealment
of wiring is one sold under the trade name "Croakies" .
FIG. 6a shows the inside surface 342 of lens 340 after it has been marked for
use
with the four LEDs depicted in FIG 6. Four "L"-shaped scratches or similar
marks
are made on the inside surface 342 of eyeglass lens 340. L 650 will be seen in
the
upper left hand corner, :L 651 will be seen in the upper right corner, L 652
will be
seen in the lower left corner, and L 653 will be seen in the lower right
corner of the
camera's field of view.
FIG. 6b shows the inside surface 342 of lens 340 after it has been marked for
use
with the four LEDs depicted in I' IG 6, and when it is placed too close to the
eye to
focus on, and when further, all four LEDs are turned on.
21
CA 02247649 2000-02-21
Although each L appears in its proper place (e.g. the upper left L appears to
the
wearer to be situated at the upper left corner of the frame), each of them is
inverted
within its corresponding circle of confusion. LED 620 defines a circle of
confusion
670. LED 621 defines a circle of confusion 671. LED 520 defines a circle of
confusion
672. LED 521 defines a circle of confusion 673.
Note that it is acceptable if these circles of confusion overlap. For example
670
may overlap with 672, as may circle of confusion 671 overlap with 673.
However, so
long as the overlap does not extend into the "L"-shaped marking the apparatus
will
work fine. For example, as long a,s circle of confusion 670 does not extend
into L 652,
then L 652 will continue to be clearly defined. (Otherwise, L 652 will be seen
as a
double image. )
With the embodiments depicted so far, the camera is typically concealed in the
nosebridge of the eyeglasses, or behind one or both of the fasteners 120. In
stereo
systems (e.g. where there are two cameras) one camera is concealed behind
fastener
120 on each side, and the camera extends back into the temple side piece, with
the
wire extending back down a hollow eyeglass safety strap.
In stereo embodiments of the invention, the lenses are typically made to look
like decorative jewels typically found on eyeglasses, where fastener 120 is
located.
Alternatively, a thin metal plated plastic where the metal coating is thin
enough to
see through, is placed over each lens, and is made to have the appearance of a
fastener
120.
In some embodiments, a narrow angle camera (also called tele camera) is
mounted
behind each of the fasteners 120, and a wide angle camera (also called simply
wide
camera) is also concealed. within the nose bridge of the eyeglass frames 130.
However, the difference in location between these cameras and the eye location
where the viewfinders are located, introduces some error in the viewfinder and
camera
alignment at close distance. For example, if the wearer wishes to look into an
eyepiece
of a telescope or microscope, the camera will not likely see into the
eyepiece, and the
22
CA 02247649 2000-02-21
glasses will therefore have failed to record this experience as the wearer had
actually
experienced it (e.g. video recorded from the apparatus will fail to show the
view
through the telescope or microscope).
Accordingly, an improvement to the invention may be made by making the camera
collect rays of light that are collinear with the rays of light entering the
eye from the
viewfinder.
FIG. ? shows both front and side views of an eyeglass lens 140 in which such
an
improved design has been implerriented. The inset bifocal lens 750 is
constructed such
that it extends all the way through the glass 140 and so that the cut line 760
on the
outside of the glass is at a much different height than the cut line 761 on
the inside
of the glass. Cut lines 760 and 761 in fact define a flat surface that extends
from
inside to outside surfaces of the glass, and is angled at a 45 degree angle.
This surface
is typically coated with a thin coating of a reflective material such as
metal, or the
like, so that it is partial)'.y transparent and partially reflective. This
surface acts as
a beamsplitter to reflect some light that enters the glasses along the cut
line, and to
reflect some of this light clown through lens 720 into CCD sensor array 710.
At the top
of the eyeglass lens 140 there is a, graticule, reticle, crosshairs, cropping
"L" pattern,
brackets, or spatial light modulator 770. Whether 770 is a fixed pattern such
as a
graticle, reticle, crosshairs, cropping "L" pattern, or brackets, or a moving
pattern
defined by an electronic cLisplay medium such as a spatial light modulator,
770 will be
hereafter referred to as a "viewscreen" . Viewscreen 770 may display a
combination of
fixed and variable information, such as an electronic image from camera sensor
array
710 together with a grid and other markings and annotations such as shutter
speed,
light levels, or the like.
Viewscreen 770 may also display meta information derived from the image formed
on sensor array 710. This meta information may, for example, include a virtual
name
tag superimposed on a person's body, and derived from a wearable face
recognizer
system attached to the output of sensor array 710. In this manner, the
wearable
23
CA 02247649 2000-02-21
camera system may assist the wearer in recognizing people who the wearer might
wish to photograph.
When sensor array 710 is connected to viewscreen 770, sensor array 710 may
incor-
porate a polarizer and so may viewscreen 770, so as to prevent video feedback.
Video
feedback may be prevented by making the surface between cut lines 760 and 761
into a
beamsplitter with appropriate polarization characteristics, or by placing
polarizers on
one or both of array 710 and viewscreen 770 or a combination of both.
Beamsplitters
that are imbedded in glass and have polarization characteristics are well
known, and
are referred to as dichroic beamsplitters, and as polarizing cube
beamsplitters, when
the polarization characteristics are deliberately optimized.
Viewscreen 770 is typically mounted on the top of the eyeglass lens, so that a
small opening in the frames 130 allows light from natural sources like the sun
780, or
artificial electric lighting typically found in indoor settings, to enter.
Typical bifocal eyegl;~ss inset lenses 150 are mounted on the outside surface
of
eyeglass lens 140, but there is no reason why they could not be mounted
instead
on the inside surface. Accordingly, 790 depicts a bifocal inset lens mounted
on the
inside surface of lens 140. This lens is like a very powerful reading glass,
that allows
the wearer to see viewsc:reen 770 in sharp focus, and to read any text that
might
be displayed on viewscreen 770 in addition to picture compositional aids
displayed
thereon.
Three major drawbacks of the embodiment depicted in FIG. 7 are that objects
beyond lens 790 appear out of focus, it relies on a bright light source 780
directly
over the glasses, and the path between the camera lens 720 and the sensor
array 710
is open so that the image is washed out by light leaking in.
FIG. 8 shows a side view of an alternative embodiment of the invention in
which
lens 790 is replaced with simply having an inward curving portion of the
eyeglasses
defined between 810 and 790. In the same way that sunglass lenses may be
curved
without providing any magnification or other prescription, this curvature
between
24
CA 02247649 2000-02-21
790 and 810 provides no magnification of objects seen through the glasses.
However, light from viewscreen 770 passes within the glass 140 and therefore
is
not affected by curvature 810. Thus a magnified well focused view of
viewscreen 770
is seen.
Camera lens 720 is :now located at the edge of eyeglass lens 140, so that the
path between camera lens 720 and sensor array 710 is now wholly contained
inside
eyeglasses frame 130. Eyeglass frame 130 is typically made of black plastic so
that it
forms a good housing for the one or more cameras contained therein.
The output of sensor array 71.0 goes into WearComp ( "wearable computer" )
830,
where it is digitized and analyzed in terms of brightness, contrast, etc..
The WearComp has a display output, typically in the form of a VGA signal 840,
which is connected to viewscreen 770. Viewscreen 770 is typically a spatial
light
modulator. Preferably viewscreen 770 is a full color high resolution VGA
screen, but
the spatial light modulator from a display known by the trade name
"Cyberdisplay"
or "Smart Slide" , manufactured by Kopin Corporation, is a satisfactory
viewscreen
770. This item produces polarized light, owing to the fact that it is a liquid
crystal
between crossed polarizers. The desired polarization is selected by choosing
which
side of viewscreen 770 will face downwards. While viewing through lens 790,
one can
try both orientations, and select t;he one with greatest brightness. If the
beamsplitter
defined by cut lines 760 and 761 does not provide sufficient polarization to
prevent
video feedback, and additional polarizer is used on sensor array 710.
The WearComp 830 will typically display the digitized video from sensor 710
along with annotation, such as a menu of commands for the wearable camera
system.
WearComp 830 may also do additional processing, such as face recognition, and
other
forms of photographic/videographic memory aids, so that the unit will function
as a
visual memory prosthetic: (VMP) as defined in http://wearcam.org/vmp.htm
In the embodiment pictured here in FIG 8, an external backlight 870 is used to
backlight viewscreen 770, so that it can be seen clearly and brightly.
However, this
CA 02247649 2000-02-21
backlight is controlled by an amplifier drive circuit 880 which amplifies the
ambient
light sensor 890.
Light sensor 890 determines how bright the environment is, and drives the back-
light 870 appropriately. In this way, the display appears bright in bright
sunny
conditions, and dark when indoors in low light. Thus, for example, when
walking
down a dark street late a,t night, a potential assailant is unlikely to see
the backscat-
tered light from the apparatus because light sensor 890 has controlled
backlight 870
in such a manner that there is just sufficient light to see the viewscreen 770
in this
dark setting.
Additionally, the estimated ambient light quantity reported by light sensor
890
may be overridden or replaced by a light control signal 850 from WearComp 830,
which is calculated by analyzing t;he input 820 from the sensor array 710. In
this way,
the quantity of light entering the eye from viewscreen 770 can be precisely
controlled
in nearly exact correspondence with the actual quantity of light entering the
eye from
the scene itself. Thus not only are rays of light from the scene collinear
with rays of
light from corresponding points in viewscreen 770, but they are also in tonal
register.
Recording is typically retroactive, in the sense that the wearable camera
system,
by default, typically records into a 5-minute circular buffer, so that
pressing "record"
begins recording starting from 5 minutes ago, e.g. starting from 5 minutes
before
"record" is pressed. This means that if the wearer presses "record" within a
couple
of minutes of, for example, witnessing a police brutality or human rights
violation,
the event will have been sucessfu:lly recorded.
Typically the recording is also transmitted by way of a wireless Internet
connection
to wearcomp 830, so that corrupt officials such as a corrupt department store
security
guards who might be a close personal friends of the manager who might, for
example
have the fire exits illegally chained shut, cannot seize and destroy the
storage medium
upon which the recording was made.
FIG. 9 shows an alternate embodiment of the wearable camera system in which
the
26
CA 02247649 2000-02-21
collinearity error is corrected computationally. Such a wearable camera system
is suit-
able for augmented reality and mediated reality (as described
http://wearcam.org/mr.htm),
as well as for more traditional forms of photography and videography. A camera
910,
concealed within the nosebridge of the eyeglass frames 130, or a pair of
cameras 910
concealed within the temple side pieces 110 of eyeglass frames 130, is
connected to
a vision processor 931 within a WearComp 830. The WearComp is typically worn
around the waist, in a shirt pocket, or comprises components spread throughout
an
undershirt, as described in http:/ /wearcam.org/procieee.html
The output of vision processor 931 is a scene description, which is supplied
to a
coordinate transformer 932. The coordinate transformer maps from camera coordi-
nates of camera 910, or camera pair 910 to eye coordinates. These eye
coordinates
are passed on to a graphics rendering system 933, and rendered in eye
coordinates,
onto viewscreen 770.
A complete machine vision analysis of the scene, followed by a complete
graphics
rendering of the entire scene is typically too formidable a task for a small
battery
powered wearcomp 830.
Accordingly, in typical use of the invention, only a small fraction of the
scene
details are rendered. For example, in the wearable face recognizer, only a
virtual
name tag is rendered. Because of the coordinate transformer, this name tag
appears
to the wearer as if it were attached to the subject, even though there is a
discrepancy
between camera location 910 and the eyeball location of the wearer. In the
stereo
version, two views are rendered, one for each eye, so that the name tag
appears to
hover in the same depth plane as the subject matter.
Light from viewscreen. 770 is projected down through the glass to beamsplitter
960.
Some will pass through beamsplitter 960, and unfortunately some will be
reflected
outward where people might be able to see it. Because of the desire that the
apparatus
be covert, beamsplitter 9n0 must be a polarizing beamsplitter and the
viewscreen 770
must be polarizing, and oriented appropriately, to minimize light reflecting
off the
27
CA 02247649 2000-02-21
beamsplitter 960.
A curved concave mirrorlike surface 961 reflects some light back up to the
beam-
splitter 960 and into the eye, while providing magnification as concave
mirrorlike
surfaces do. This mirrorlike surface is disguised as part of a bulge 950 in
the eyeglass
lens. This bulge may be the actual prescription of the wearer, or may be
simply a
magnifying region useful to anyone who does not normally wear glasses but
wishes
to be able to do fine work. Alternatively, the bulge may be made so that it
does
not provide magnification, but simply looks like the inset lens of bifocal
eyeglasses.
Mirrors 960 and 961 are partially silvered, and imbedded in the glass in such
a way
as to appear as cut lines for inset bifocal lens 950. Although mirrors 960 and
961
only need to be about as wide as they are deep, they may be extended across
much
further than necessary, so that they will look like normal bifocal cut lines.
The apparatus of this invention allows the wearer to experience the camera
over
a long period of time. F'or example, after wearing the apparatus sixteen hours
per
day for several weeks, it begins to function as a true extension of the mind
and
body. In this way, photographic composition is much more optimal, because the
act
of taking pictures or shooting video no longer requires conscious thought or
effort.
Moreover, the intentiona,lity of the picture-taking process is not evident to
others,
because picture-taking is not preceeded by a gesture such as holding a
viewfinder
object up to the eye. The wearable viewfinder is an important element of the
wearable
camera invention allowing the wearer to experience everyday life through a
screen, and
therefore be always ready to capture anything that might happen, or even
anything
that might have happened previously by virtue of a retroactive record
capability of the
invention. Moreover, additional information beyond just exposure and shutter
speed
may be displayed in the camera viewfinder. For example, the camera allows the
wearer
to augment, partially diminish, or otherwise partially alter his or her
perception of
visual reality. This mediated-reality experience may be shared. The wearer may
allow
others to partially alter his or her perception of reality. In this way the
invention is
28
CA 02247649 2000-02-21
useful as a new communications medium, in the context of collaborative
photography,
collaborative videography, and telepresence. Moreover, the invention may
perform
other useful tasks such as functioning as a personal safety device and crime
deterrent
by virtue of its ability to maintain a video diary transmitted and recorded at
multiple
remote locations. As a tool for photojournalists and reporters, the invention
has clear
advantages over other competing technologies.
From the foregoing description, it will thus be evident that the present
invention
provides a design for a wearable camera with a viewfinder. As various changes
can
be made in the above embodiments and operating methods without departing from
the spirit or scope of the invention, it is intended that all matter contained
in the
above description or shown in trae accompanying drawings should be interpreted
as
illustrative and not in a :Limiting sense.
Variations or modifications to the design and construction of this invention,
within
the scope of the invention, may occur to those skilled in the art upon
reviewing
the disclosure herein. Such variations or modifications, if within the spirit
of this
invention, are intended to be encompassed within the scope of any claims to
patent
protection issuing upon this invention.
29
