Note: Descriptions are shown in the official language in which they were submitted.
CA 02220826 1997-11-12
Doc.~o.10706
System and Method for Capturing a Range Image of a Reflective Surface
Field of the Invention
The invention relates to three dimensional (3-D) computer vision and more
particularly to three dimensional im:~gin~ of objects having reflective surfaces.
5 Background of the Invention
Human vision relies on experience and stereoscopic vision to determine ranges ofobjects. Each of two eyes capture an image from locations offset by a predetermined
distance, and the brain determines a range to at least an object within a field of view. When
peering into a mirror, a person sees images within the mirror of objects that are outside the
10 mirror. Stereoscopic vision establishes a distance to the images that is substantially the
distance light travels from the objects to the person's eyes. As such, the Fnp~ h language has
an expression "tricks with mirrors" wherein mirrors are used to alter a physical space. For
example, placing mirrored tile on a wall in a small room, makes the room appear much
larger.
Stereoscopic systems similar to human vision have been proposed for computer
vision with applications in measurement, testing, robotics, modeling, m~nllf~turing,
navigation, and so forth. It is the very breadth of applications for computer vision that has
made them popular both in terms of research and adoption.
Two popular techniques currently in use for optical ranging of a target surface are
20 known, respectively, as the standard optical triangulation system and the Biris (bi-iris)
system, the latter employing an apertured mask in a converging lens system of an im~ging
device having a position sensitive detector, e.g. a CCD camera.
These systems are described and compared in a paper entitled "Practical
Considerations for a Design of Industrial Systems, SPIE ~ol. 959, 1988, pp 225-246 by F.
25 Blais et. al., and also in. "Optical Range Image Acquisition for the Navigation of a Mobil
CA 02220826 1997-11-12
Doc.No. 10706
Robot, " published in the proceedings of the 1991 IEEE International Conference on
Robotics and Automation, Sacramento, California, April 9-11, 1991 by F. Blais et. al. A
further article of note is "Active, Optical Range Image Sensors," by Paul J. Besl, Machine
Vision and Applications (1988) 1:127-152. All these documents are hereby incorporated
5 herein by reference.
A sample machine vision system is disclosed in U.S. Patent number 5, 270, 795
entitled Validation of Optical Ranging of a Target Surface in a Cluttered Environment, in the
name of F. Blais. The system comprises a collim~tecl laser light source and an image capture
means for capturing light diffused by opaque surfaces. It merges the features of standard
10 optical triangulation systems and those of Biris systems. The system uses a laser light source
and a detector having two irises distanced apart. Light from the laser diffused from opaque
surfaces is detected at the detector through each of the two irises. The resulting images are
correlated to determine range. Unfortunately, when surfaces are reflective or transparent,
collim~te-1 light sources are often reflected as collim~te~l beams mi.~in~ the image capture
15 means. As such, many range sensing means relying on optical range sensing are limited to
substantially opaque surfaces. The content of U.S. Patent number 5, 270, 795 to F. Blais is
hereby incorporated herein by reference.
In U.S. Patent 4,645,347 titled Two Dimensional Tm~gin3~ Device in the name of Rioux, a
mask aperture having a plurality of apertures therein is described for use in im~gin~;. Such a
20 mask is cost effective in reducing a number of required detectors in an im~ging system and
for reducing effects caused by vibration.
Object of the Invention
In an attempt to overcome these and other limitations of the prior art, it is an object of
the present invention to provide a system and a method for ranging reflective surfaces and for
25 determinin~ a geometry for said surfaces.
_
CA 02220826 1997-11-12
Doc.No.10706
Summary of the Invention
In accordance with the invention there is provided a method of measuring a distance from a
first location to a reflective surface. The method comprises the steps of:
directing diffused light toward the reflective surface forming an image therein;
5 using a range im~gin~ means having a detector, measuring a distance from the detector to the
image, and
determining the distance to the reflective surface based on a known relationship between the
location of a diffused light source, the location of the detector and the distance from the
detector to the image.
10 In accordance with the invention there is provided a method of measuring a distance to a
reflective surface comprising the steps of:
projecting toward the reflective surface a pattern of diffused light forming an object;
using an im~gin~ means comprising a detector, said detector having a known spatial relation
to the object, receiving some of the projected light reflected off of the reflective surface and
15 forming an image of the object;
using a processing means, determining a range to an image of the object; and
using the processing means, determinin~ a distance to the reflective surface based on the
known spatial relation and the image range.
In accordance with another aspect of the invention there is provided an im~gin~ system
20 comprising
a diffused light source for directing diffused light toward a surface and for forming at least an
image therein;
. CA 02220826 l997-ll-l2
Doc.No.10706
an im~ginp means comprising at least a detector for receiving diffused light reflected from
the surface and having a predetermined spatial relation with the diffused light source; and
image processing means for det~rmining a distance from the system to an image ofthe at least an image and for determining a distance from the system to at least a location on
S the surface from which diffused light is reflected.
An advantage of the present invention is the ability to model reflective surfaces using
an optical range measurement device.
Brief Description of the D,l a~
Exemplary embodiments of the invention will now be described in conjunction with10 the following drawings, in which:
Fig. 1 is a simplified diagram of a prior art im~ging device im~gin~ a reflective surface,
Fig. 2 is a simplified diagram of a prior art im~in~ device failing to image a reflective
surface,
Fig. 3 is a simplified diagram of a device according to the present invention im~gin~ a
15 reflective surface;
Fig. 4 is a simplified diagram of a device according to the present invention im~gin~ a planar
reflective surface and indicating the effect of a field of vision upon a detector;
Fig. 5 is a simplified diagram of a device according to the present invention incorporating
two detectors and a single diffused light source;
20 Fig. 6 is a simplified diagram of a device according to the present invention incorporating
two detectors and two diffused light sources,
Fig. 7 is a simplified diagram of a device according to the present invention incorporating
three detectors and a diffused light source;
Fig. 8 is a simplified diagram of a device according to the present invention incorporating
25 two detectors and two diffused light sources;
CA 02220826 1997-11-12
Doc.~o. 10706
Fig. 9 is a simplified diagram of a device according to the present invention incorporating a
plurality of detectors and a single diffused light source and im~gin~ a transparent substrate
having a predetermined non-zero thickness;
Fig. 10 is a simplified diagram of a device according to the present invention im~gin~ a
5 reflective surface that is non-planar;
Fig. 11 is a diagram showing a triangulation method of determining image range;
Fig. 12 is a simplified diagrarn of a Biris range system employing a single Biris detector and
four diffused light sources; and
Fig. 13 is a flow diagram of a method of determininp distance and reflective surface
10 geometry according to the present invention.
Detailed Description of the Invention
Reflective surfaces reflect incident light at an angle equal to the angle of incidence. In Fig. 1
is shown a reflective surface 5 with an object in the form of a collimated laser light source 2
having an iris with a predetermined shape 3, and a detector 9, on a first side thereof and
15 according to the prior art. On a second side thereof is an image 7 of the object. Once a range
for a reflective surface 5 has been determined, a detector 9 and a collimated laser light source
can be aligned (as shown in Fig. 1) to allow for the detector to capture a reflected collim~ted
beam 4c. Unfortunately, absent knowledge of range, orientation, and characteristics such as
shape, ~ nment is not possible and detecting the collim~te-1 reflected beam is a matter of
20 chance.
Referring to Fig. 2, a reflective surface 5 in the form of a polished mirror is shown
having an object, a collim~tc~l laser light source 2 having an iris with a predetermined shape
3, and detector 9 according to the prior art on a first side thereof and having an image on a
second side thereof. Paths of light are shown for light from the collim~ted laser light source
25 2. As can be seen, light reflected by the reflective surface 5 misses the detector 9 and
therefore is not used in determinin~ a range for the surface 5. Alternatively, when light is
diffused at the object, it is imaged by the detector 9 and is used for determinin~ a range.
CA 02220826 1997-11-12
Ooc. No. 10706
Resulting range images treat reflective surfaces and transparent surfaces as holes or worse
yet, inconsistently treat them as either holes or, when alignment happens to be appropriate, as
surfaces. Further, the number of diffused and reflected signals reaching a detector leads to
inconsistent range detection.
S Referring to Fig. 3, an embodiment of the present invention is shown opposite a
reflective surface 5 in the form of a highly polished mirror. A laser light source 2 is diffused
by diffusion means 30 producing a recognizable pattern of light and dark (two diffusing
circles and an opaque frame). Diffusion means 30 acts as diffusing irises and results in a
plurality of objects. A detector 9 in the form of a CCD is positioned for receiving light from
the objects once the light is reflected off a reflective surface. The detector 9 is provided with
a frequency selective filter means 56 for filtering frequencies other than a predetermined
range of frequencies. This allows use of the invention in lighted conditions. Alternatively, the
filter means filters ambient light of predetermined frequencies. Further alternatively, no filter
is used. Preferably, a diffused light source provides illumination of a large percentage of a
l S field of view for the detector. Preferably, diffused light source thickness is small to result in
sharp images at the detector 9.
In operation, light emitted by the laser 2 is diffused by the diffusion means 30. The
diffused light reflects off the reflective surface 5 and reaches the detector 9. It will be clear to
those skilled in the art that diffused light is likely to reach the detector 9 whereas collim~tecl
light would be less likely to reach the detector 9 in sufficient quantities. A range detection
algorithm executed within a processing means 60, det~rmines a distance between the detector
9 and images 7 of the objects (diffusion means) 30. The processing means 60 then determines
a range between the detector 9 and the reflective surface 5 in dependence upon the known
relative locations of the objects 30 and the detector 9. Further, when sufficient images 7 are
imaged by the detector 9 and in dependence upon a recognizable pattern of the objects 30, a
surface shape of the reflective surface 9 is determined.
CA 02220826 1997-11-12
Doc.~o. 10706
In order to determine a range to an image at least two data points are required. These
points can be acquired by projecting at least two light sources, by projecting an object with at
least two discernible features, by capturing an image with at least two detectors or by
capturing images with a detector provided with two irises (Biris). Each of these systems is
5 discussed in detail below.
Referring to Fig. 4, using a collim~ted light source and a diffuser, allows light within
a narrow band of frequencies to be projected and to form several objects 31-36. Several
objects result in several images 71-76 in reflection. Unfortunately, many detectors have small
fields of view 10 of for example 30 degrees. 6 objects 31-36 form 6 images 71-76 in a mirror
5 but only objects 33, 34, 35, and 36 are detected by the detector 9. The images are all visible
to a human eye in a location chosen for a detector 9. Unfortunately, due to a field of view 10
associated with the detector 9, the images 71 and 72 are not detected by the detector 9. Since
a range is calculated to each image and there is a known distance between each object and the
detector 9, a detçrrnin~tion of which image is associated with each object is possible. This
15 det~rmin~tion is then used to accurately determine a distance to the reflective surface 5.
To distinguish between associated objects and images, some calculations are
performed. A distance D; is measured between the detector 9 and each image. Each distance
corresponds to a series of potential locations and orientations of the reflective surface 5. The
resulting equation for each distance Dj is a simple triangulation with an unknown angle at a
20 vertex thereof. Thus, the distance D; is equal to the distance of 4a added to the distance of 4b.
The distances of 4a and 4b are interrelated by the distance from the object to the detector K
(shown in Fig. 10). A discussion of the calculations required and the definitions of K, K 1,. . is
presented later.
Associating an image for which a distance is to be calculated with an object
25 determines a value for K. When the association is correct, a triangle is formed with a known
base K and an unknown height. Relying on the rules of geometry, a unique triangle has been
defined. In dependence upon the orientation of the base and an angle defined by a vertex of
CA 02220826 1997-11-12
Doc.No. 10706
the triangle, surface orientation is determined. Therefore, by determinin~ a distance to an
image and using the known distance from the detector to an associated object, a distance to
an imaged point on a surface is determined. The orientation of the surface can be determined
using two measurements.
Unfortunately, as shown in Fig. 4, some images are not detected. When this occurs,
images and objects must be associated correctly in order to determine an accurate
measurement of distance. A method of associating images and objects is to project objects
that are distinguishable one from the other. Another method is to project objects spaced apart
such that two objects associated incorrectly with images will result in the determination of
different surfaces, correlating determined surface location and orientation allows for
verification of results.
Applying this latter method to im~gin~ a reflective surface is now described. Animage is captured by the detector 9 (shown in Fig. 4) of a plurality of images. The number of
images (4) is determined and a preliminary association is made. A distance to each image and
a corresponding surface is determined. Each surface is then compared to determine whether
or not the determined surfaces are consistent. Associating the images with different objects is
performed in order to determine a consistency of other associations. Once sufficient
associations have been made, a most consistent surface is selected as the surface measured.
When the surface or the detector is moving, the consistency of the surface is measured
against other swrface shape and distance determinations.
When the reflective surface 5 has an irregular shape or is curved, further data points
are required to resolve surface geometry. A method of doing so is shown in Fig. 10 and
described later.
Using a single diffused light source in the form of an LED, a single object exists. By
capturing an image with each of at least two detectors, A distance to the image can be
ascertained. Referring to Fig. 5, an embodiment of the present invention is shown opposite a
flat reflective surface. A laser light source 2 is diffused by diffusion means 30 producing a
CA 02220826 1997-11-12
Doc.No. ~0706
recognizable pattern of light and dark. The diffusion means 30 acts as an object. A plurality
of detectors 9a and 9b in the form of CCDs are positioned proximate the object. The
detectors are provided with a frequency selective filter means 56 for filtering frequencies
other than the laser frequency. Alternatively, the filter means filters ambient light at
S predetermined frequencies. Further alternatively, no filter is used. The use of two detectors
results in the capture of two data points for the image (as is shown in Fig. 5) and allows for
triangulation to be employed.
Referring to Fig. 6, a system provided with 2 detectors and 2 diffused light sources is
shown. The system results in the acquisition of 4 data points. Such a system is useful for
10 range determination and for surface modeling of the reflective surface 5. For each image, a
distance and surface orientation is calculated. Due to the increased number of data points, a
solution is now possible (and unique) for surface distance, surface orientation, and surface
curvature (over a section of the surface). Moving the detectors or the surface allows image
capture of the entire surface and allows for surface modeling. Preferably the relative motion
15 is accurately measured to ensure accurate modeling.
Referring to Fig. 7, an embodiment wherein a single diffused light source projects
images to each of 3 image capture means is shown. In principal, the device operates similarly
to that of Fig. 6. Triangulation is achieved between each pair of points (three different pairs)
to determine information relating to surface distance, surface geometry, and surface
20 orientation.
In Fig. 8, a similar system to that of Fig. 6 is shown wherein the light sources are
diffused light sources in the form of LEDs 21. Alternatively, the light sources 21 can be
regular diffused light sources (bulbs) or other diffused lighting means. The elimin~ion of a
laser light source reduces the overall cost of the system. When using limited frequency LEDs,
25 a frequency dependent optical filter may be employed to filter out ambient light from light
reaching the detector.
CA 02220826 1997-11-12
Doc.No. 10706
Transparent surfaces tend to be reflective in part. In Fig. 9, a transparent surface 50 in
the form of a glass plate is shown. Using the present invention, both a surface geometry and a
thickness can be determined for the glass plate. As is noted in dotted line at 4d, a further
image is detected having a different distance (in this case Dj plus twice the thickness of the
5 glass plate). A processing means 60 resolves the images 7 and 7d and calculates image
distances in order to determine plate thickness and surface geometry.
Referring to Fig. 10, an embodiment of the present invention shown opposite a
reflective surface with regular curved geometry is shown. A laser light source 2 is diffused by
diffusion means 30 producing a recognizable pattern of light and dark. The diffusion means
10 30 acts as a plurality of objects. Detectors 9a and 9b in the form of CCDs are positioned
proximate the objects 30. Alternatively, the detectors 9a and 9b are positioned in a location
having a known spatial relation to a location of the objects 30. The detectors 9a and 9b are
provided with a frequency selective filter means 56 for filtering frequencies other than the
laser frequency. Alternatively, the filter means filters ambient light of predetermined
15 frequencies. Further alternatively, no filter is used.
Returning to Fig. 9, In operation, light emitted by the laser 2 is diffused by the
diffusion means 30. The diffused light is reflected from the reflective surface S and reaches
the detectors 9a and 9b. It will be clear to those skilled in the art that diffused light will reach
the detectors 9a and 9b whereas collimated light would be less likely to reach the detectors
20 9a and 9b in sufficient quantities. A range detection algorithm executed within a processing
means 60, determines a distance between the detectors 9a and 9b and an image 7 of the
objects (diffusion means) 30. Where, as in this instance, the reflective surface is non-linear,
the image 7 is distorted relative to the objects 30 and/or m~gnified relative to the curvature of
the surface. The processing means 60 correlates objects 30 and images 7. Alternatively,
25 points on an object (known) are correlated with points on a captured image (reflected). Using
the known spatial relation between the detectors 9a and 9b and the objects 30, the processing
means determines a distance to at least some of the points on the reflective surface 5.
Locations of these points are then used to determine a surface geometry for the reflective
CA 02220826 1997-11-12
Doc.No. 10706
surface 5 and therefore orientation and m~gnification of the curved surface can be
determined. At least 3 captured images are required to determine surface geometry.
Preferably, more images are captured. Further images at different angles or from different
known locations allow for the construction of a geometric model of the reflective surface 5.
5 Alternatively, a large object with a discernible pattern is used to model the entire surface.
Further alternatively, the diffused light source is stationary and only the detectors 9a and 9b
move. In this last embodiment, the diffused light source's pattern is used to determine
detector angle or location; image overlap is used to construct a model of the reflective surface
5.
Referring to Fig. 1 l, calculations used in clet(~rminin~ a distance to a detected image
are shown. A detector is spaced a distance Kl from a first object and a distance K2 from a
second object. The objects are spaced apart by a distance of K and the two objects and the
detector are co-linear. Therefore, K= I K2 - Kl 1. The detector detects images of each of the
first and second objects. The images are at angles of a and ~, respectively, Between the
15 images of the first and second objects is a distance K similar to that distance between the
objects themselves. Therefore, a triangle is formed with a known base and a known angle
opposite the base. This results in a determined triangle. Once determined the values of D l
and D2 are calculated using trigonometric functions or other known methods.
A second series of triangles is now formed in which the bases are Kl and K2
20 respectively. Angles of a corner of each triangle adjacent the base are known. The distance
around the perimeters of the two triangles are D l + Kl and D2 + K2, respectively. This is
known because the optical paths from the detector to the images D 1 and D2 are equal in
length to the optical path from each of the first object and the second object, respectively. As
the light travels in a substantially straight line from the object to the reflective surface and
25 from the reflective surface to the detector, the distance Dl is equivalent to the distance in
traversing two sides of a triangle; the third side of the triangle has a length Kl. Similarly, D2
is equivalent to the distance in traversing two sides of a triangle; the third side of the triangle
has a length K2.With this information the two triangles can be evaluated to determine all
11
CA 02220826 1997-11-12
~>c. N~>. 10706
distances and thereby measure a distance to a point on a substantially flat reflective surface.
This method of determinin~ range is known in the art as photogrammetry.
Referring to Fig. 12 an embodiment of the invention using a Biris detector 9 having
two apertures 91 and 92, a lens 96, and a CCD 97 and 4 diffusing light sources 3 is shown.
5 Existing implementations of Biris systems weight standard plane of light range measurement
contributions with Biris contributions to increase accuracy of a range measurement. A plane
of light measurement forms a large part of a range measurement since it relies on a large
triangulation base.
When surface curvature is small, because of a small separation between apertures 91
10 and 92 in the detector 9 mask a range is calculated by the system. The distance between the
apertures 91 and 92 is referred to as b. Further, an angle of incidence of light is determined in
dependence upon a point on a CCD within the detector 9 where the light is incident. The
point is referred to as p. From these two values (p and b) range can be determined (as
explained with reference to Fig. 11 and U.S. Patent 5,270, Validation of Optical Ranging of a
15 Target Surface in a Cluttered Environment to F. Blais).
When curvature of a surface is monotonic, range measurements from 2 different
profiles are used to evaluate the curvature of the surface. Alternatively, p is used as with
other curved surfaces. Once a curvature of a surface is known, a relation between b and p
becomes known and is used to validate range measurements.
One possible algorithm that includes signal validation and range refinement based on
low curvature property of a reflective surface is as follows:
1. for each detected image, compute a range;
2. to each image associate an object;
3. for each image range and associated object, compute a surface range and
orientation;
CA 02220826 1997-11-12
Doc.No. 10706
4. evaluate the surface orientation/curvature and range resulting from each
co~ u~ion in step 3 and correlate them;
5. when the computed values do not correlate, associate the images with different
objects and return to step 3, and
6. when the computed values correlate, minimi7~ any range surface curvature errors
computed in step 3.
In measuring range for curved surfaces compensation for curvature of the surface is
applied. The compensation is used because curvature of a reflective surface alters image
locations. When curvature is sufficiently small, the surface is treated as a planar surface for
10 each measurement and a curved surface is reconstructed in dependence upon a plurality of
range measurements. Alternatively, when the curvature of the surface is not sufficiently
small, compensation is required.
For a lens of radius r and index of refraction n, the focal length is
r=(n- 1 )f (for a mirror n= -1).
The distance of the position of the image to the surface is modified by the curvature
of the surface; however, the technique to compute the location of the reflective surface
assumes a substantially flat surface (r=oo) and therefore, any change in image location results
in a corresponding change to a measurement of range. In order to determine curvature for the
surface using the above equations, we require its range. In order to determine the surface
range using the above equations, we require a known surface curvature. Because both
curvature and range are unknown, an iterative approach to converge measured range and
curvature to corresponding values. A detailed mathematical explanation of a method
employed is presented in Appendix A attached.
CA 02220826 1997-11-12
Doc.No. 10706
Using a system as defined in the present invention, effects of laser speckle arereduced. Smooth reflective surfaces are known to reduce laser speckle. Further,
polychromatic light sources may be used, thereby avoiding laser speckle.
Anamorphic lenses may increase camera accuracy while retaining field of view. It5 will be apparent to those of skill in the art that as laser field of view increases, further
constraints are imposed upon dimensions of light sources arrays.
For im~ginp reflective surfaces with a high degree of curvature, light source and
detector location is important. As such, specific light source-detector geometries for high
curvature surfaces are employed. This improves a chance that sufficient images will fall
10 within a field of view of a detector, and that those images are correctly associated with
objects. Distinguishing between objects can be achieved in many ways. For example, each
object (diffused light source) can have a different wavelength or a different shape. Allowing
sufficient images to fall within a field of view is a geometrical problem to be solved in
dependence upon known criteria regarding a surface.
Referring to Fig. 13, a method of determininp distance and reflective surface
geometry is shown in a flow diagram. Diffused light is projected at a reflective surface a
range for which is to be determined. The diffused light is in the form of at least an object.
The diffused light reflects off the reflective surface and is detected by at least a detector. At
least two and preferably at least three instances of reflected light are detected by the at least a
20 detector.
For each instance a range is calculated to an image (as viewed by the at least adetector) of the at least an object. Each image is associated with an object in the form of a
diffused light source. The association is estimated and may be incorrect. In dependence upon
each association, a range and orientation are calculated for the reflective surface. The
25 calculated ranges and orientations for each image are compared to determine a surface. When
a surface results, it is refined by reducing errors in surface range and curvature. When an
inadequate surface results, at least an image is associated with a different object and the range
14
CA 02220826 1997-11-12
~c.~.~0706
and orientation is recalculated. The method continues from the calculation of range and
orientation as set out above.
A diffused light source is a primary light source providing diffused light.
Alternatively, a diffused light source is a secondary light source tr~n~mit~in~ or reflecting
S light incident thereon in a diffused fashion. A discernible object reflecting some diffused
light acts a diffused light source. Further, as described above, the primary source of light
need not provide diffused light and, in an embodiment, is a collimated laser light source in
conjunction with a diffuser acting as a secondary light source.
Numerous other embodiments of the invention may be envisioned without departing
10 from the scope of the invention.
