Note: Descriptions are shown in the official language in which they were submitted.
3B~
Three di ensional ima~inq method and device
BACRGROUND OF THE INVENTION
The present invention relates to a three dimensional
imaging method and device, that is to say a method and
S device for obtaining three dimensional data of a target
surface or other object, whether such data is displayed in
three dimensional form or not. Indeed, the data may never
be displayed as such, but may merely be used to control
other equipment. Such a device is useful for supplying
three dimensional data to other instruments. The data can
be valuable in the science of robotics, where objects are
required to be identified on the basis of their three
dimensional shape and manipulated accordingly. Such data
is also useful in monitoring the accuracy o the shape of
a series of articles intended to be identical with each
other.
One of the objectives of the present invention is to
develop a low cost, compact, high-speed three dimensional
imaging device to be used in automatic assembly. The
ability to analyze the three dimensional data of a scene
very quickly, i.e. in a fraction of a second, has many
practical advantages, especially in an industrial
environment. It reduces interference problems, as well as
the results of mechanical, acoustic and electromagnetic
vibrations. Moreover, moving objects can readily be
surveyed and measured.
The output from such a three dimensional imaging device
is similar to a television signal except that the amplitude
of the signal is related to the geometric characteristics
of the object. In contrast, ordinary two dimensional
television cameras provide an output signal having an
amplitude that is not geometrically related to the object
but represents the surface reflectance properties of the
object, combined with the ambient light conditions, its
orientation and the intensity and spectral characteristics
of such ambient light. The result is thus usually affected
by the orientation of the object and the proximity of other
objects. It is primarily for these reasons that the
extraction of three dimensional features from a two dimen-
sional image of a scene is difficult to realize~
There are many applications, such as in robotics, where
~0 a relatively low resolution, three dimensional image is
sufficient to enable the equipment to discriminate between
various known objects, as opposed to a detailed inspection
task where it is necessary to make fine comparisons between
the ideal shape and the actual manufactured part.
An objective of the present invention is to provide a
system that is versatile in that it enables selection
either of high resolution, i.e. sensitive discrimination,
with some sacrifice in speed, or alternatively, extremely
fast operation, e.g. down to 0.01 second for a typical
image size of 128 x 128 pixels, for those situations where
less fine resolution can be tolerated. Speeds of this
order of magnitude are normally quite fast enough for most
assembly tasks. Indeed they represent a substanti(al
improvement over speeds hitherto available in three
dimensional imaging devices.
Among the various techniques for obtaining three
dimensional data that have been proposed in the past, the
use of an active triangulation system employing a beam of
radiation, e.g. laser light, that is projected onto an
area of the surface to be examined, and a position sensi-
tive detector for measuring deviations in the reflected
beam, is the approach that is believed most likely to lead
ultimately to achievement of the desirable criteria
mentioned above. Systems employing such triangulation
have, for example, been described by K.~. Morander in "THE
OPTOCATOR. A HIGH PRECISION, NON-CONTACTING SYSTEM FOR
DIMENSION AND SURFACE MEASUREMENT AND CONTROL," 5th Inter-
national Conference on automated inspection and product
control, 1980, pp. 393-6; and by T. Kanade and H. Asada in
"NONCONTACT VISUAL THREE-DIMENSIONAL RANGING DEVICES,"
SPIE Proc. Vol. 283, 1981, pp. 48-53. The principles
employed in these systems are explained in more de~ail
below in relation to Figure 1 of the accompanying drawings.
These systems have certain disadvantages, however,
particularly in relation to requiring comparatively bulky
apparatus and limiting the size of the field of view that
can be obtained. They also exhibit shadow effect prGblems
when used to obtain the profile of objects of certain
shapes.
~5 SUMMARY OF T~E INVENTION
.
An objective of the present invention is to provide
improvements in respect of these disadvantages. This
result is achieved by so synchronising scanning of the
projected and detected beams that the detected position in
the detector remains unchanged when the beams scans a flat
reference surface. The distance of the reference surface
from the detector can be set arbitrarily. The detected
position changes only when the beam is reflected from a
surface point that is either nearer to, or further from
t.he reference surface.
?J~15~
More specifically, the invention provides for project-
ing a beam of radiation onto an area of the surface to be
examined and detecting a beam reflected from the projected
beam by such surface area. The projected and detected
beams define a plane which for convenience can be referred
to as the "beam plane." This plane intersects the plane of
the reference surface (reference plane) along a line that
extends in the direction that is considered to be the X
direction. So long as the object surface is flat and
extends in the X direction, the detected beam strikes the
position sensitive detector at the same position
regardless of the scanned orientation of the beams, any
deviation in such position representing a deviation from
the reference plane of the intersection of the beams, i.e.
a deviation of the surface area under examination in a
direction perpendicular to the reference plane. This
latter direction is considered the Z direction and
represents the important information about the examined
surface that is required to compile the three dimensional
data.
An arrangement functioning in this manner has been
disclosed in U.S. patent 4,171,917 issued October ~3, 1979
to R.A. Pirlet. Pirlet achieves synchronism of his beams
by means of a pair of octagonal mirrors that are caused to
rotate in synchronism and, together with other mirrors, to
cause the beams to intersect at the surface of the object
being examined, which, in his example, is the surface of
strip material that is travelling past the device, e.g. a
strip or beam coming from a rolling mill or the like.
Synchronised rotation of the mirrors in the Pirlet system
causes the point of intersection of the beams, i.e. the
area of the surface under examination at any given moment,
to scan across the strip in a direction perpendicular to
the direction of travel of the strip. For example, in
Pirlet's Figure 3a, the intersection point 16 scans in the
-- 5
vertical direction (along the line 3-4 in Figure l or 2)
while the strip l travels from left to right, direction 5
in Figu~e l, which latter direction is equivalent to the X
direction as defined above. Consequently, the Pirlet
system is dependent on relative movement of the device and
the object under examination in order to achieve scanning
of the surface in the second dimension. There is no
provision in Pirlet's system, nor would it be practicable
for there to be any such provision, having regard to the
geometry of his system, for scanning the point 16 from
let to right in Figure 3a, i.e. in the X direction
defined by the intersection o~ ~he beam plane with the
reference plane.
The requirement for relative movement between the
workpiece and the surveillance sys~em was presumably no
disadvantage in the Pirlet system, since its purpose was
to examine moving strip. There are, however, many other
application for three dimensional imagining in which either
the workpiece is stationary or there is a need to obtain
~0 the data much more rapidly (e.g. within a small fraction
of a second) than would be possible if the device were
required to wait for the movement of the workpiece to
provide scanning in the second dimension.
The principal characteristic of the present invention,
~S that overcomes this difficulty and represents an important
improvement over the Pirlet system, is that the projected
and detected beams are synchronously scanned in a direction
lying in the beam plane. As a result, the point of beam
intersection is moved in the X direction. This arrangement
facilitates simultaneous synchronous scanning in the second
dimension, i.e. a direction Y perpendicular to both the X
and Z directions, e.g. by means of a simple ~lat mirror
located in the paths of the two beams and rotatable about
an axis parallel to the X direction, a possibility that is
~5 unavailable to the Pirlet system.
-- 6
BRIEF DESCRIPTION OF THE DRAWINGS
__ _____. _ __
Embodiments of the invention are illustrated by way of
example in the accompanying drawings, in which:
Figure 1 shows the basic elements of a known triangu~
lation system;
Figure 2 is a diagram illustrating the manner of
operation of the present invention;
Figure 3 is a portion of Figure 2 on an enlarged scale;
Figure 4 is a diagrammatic front view of a preferred
embodiment of the invention;
Figure 5 is a side view of Figure 4;
Figure 6 is a perspective view showing the geometry of
the arrangement of Figures 4 and 5;
Figures 7 and 8 are a plan and a perspective view,
respectively, of a second embodiment of the invention; and
Figures 9 through 15 are respectively diagrammatic
representations of alternative embodiments.
Figure 1 shows the basic elements of a triangulation
system, consisting of a light source S (conveniently, but
not necessarily, a laser) supplying a beam 10 to a scanning
mechanism M which in turn projects a scanned beam 11 onto
an object 12. ~t a given moment in the scanning process,
the beam 11 strikes a point 13 on the object 12. Although
this point of impact is thought of as a "point," in fact,
due to the diameter of the beam, e.g. about 1 mm, it is
more accurately an "area," albeit a comparatively small
area. Reflection at the point 13 gives rise to a
reflected ("detected") beam 14 that passes through a lens
L to be sensed in a position sensitive detector D at point
Pl. Distance measurement is derived mathematically from
the projection direction, i.e. the angular position of the
scanning mechanism, and the detection direction as deter-
mined by the location of the point Pl on the detector D.
A feature of the arrangement shown in Figure 1 is that the
measurement resolution can be increased by increasing the
angle ~ between the projection axis and the detection axis,
but this can only be done at the expense oE the compactness
of the instrument and the size of the field of view.
Another problem arises in that the shadow efEect increases
with this angle, since a part of the object "seen" by the
proiection mechanism may not be seen by the detector.
This shadow effect, which can prevent continuous profile
recording, becomes more severe as the angle Q is increased.
The present invention provides a new approach that
allows a compact arrangement without compromising the
resolution and the field of view, while also enabling the
undesirable shadow effects to be kept to a minimum.
As mentioned above, a basic requirement of the present
invention is to synchronize the projected and detected
beams in such a way that the detected position in the
detector remains unchanged as the projected beam scans a
flat surface. Figure 2 shows a flat reference plane 20,
(in fact, this plane will have a slight curvature, which
can be ignored for practical purposes, provided the field
of view is relatively small) scanning of the surface of
which will give a zero output, since as the beam 10 is
scanned by the first scanning mechanism Ml to generate
the beam 11, the reflected beam 14 is simultaneously and
synchronously scanned by a second scanning mechanism ~2
to generate a detected beam 15 that passes through the
lens 1I to strike the position sensitive detector D at a
reference point P0. While this synchronized scanning
takes place in the X direction, i.e. in the "beam plane"
defined by the beams 11 and 14, the location of the point
P0 remains unchanged. When, however, the beam 11 strikes
a point 13 of an object 12, the reflected beam 14' becomes
the detected beam 15' which strikes the detector D at
position Pl. This position also remains constant for
a given Z deviation from the reference plane 20 throughout
the scan in the X direction. ~hese considerations are
shown in more detail in Figure 3 where it will be noted
that, provided the angle Q is sma]l, the detected
,~'
~ ~,f!~
position shift, i.e. Pl - P0, in the detector D is
proportional to ~X0 which is related to z by the equation
~X0 ~ z~. If the value of z is negative, i.e. to the right
of the reference plane 20 as seen in Figures 2 and 3, the
shift in the detector D Will be upward, i.e. on the other
side of P0 from Pl.
Since this arrangement synchronously scans both the
projected and the detected beams, it is possible to main-
tain a wide field of view~ while keeping the resolution
along the Z axis independent of the X scan. It is also
independent of any scan in the Y direction. Moreover, the
resolution of the device along the Z axis can be modified
by changing the focal length of the lens L without any
reduction in the field of view, in contrast with previous
arrangements.
Measurement of the positions of the scanners Ml and M2
provides the X and Y coordinates. For a limited field of
view, i.e. up to 5, no trigonometric computation is
required to derive the profile amplitude, i.e. the Z
coordinate, since this coordinate can be assumed to be
coincident with the beam plane. Hence the three coordin-
ates are readily and easily available. On the other hand,
if one accepts a need for computation in the signal
processing, i.e. to allow for deviation of the beams from
the true Z direction, the device can be designed to cover
a very wide field of view, for example up to 90 in each
of the X and Y directions.
In addition, a zoom effect can readily be realized by
using a zoom lens as the lens L. The depth of the volume
sensed is then inversely proportional to the efective
focal length of the lens. On the other hand, the same
number of positions that the detector can resolve are still
available, which means that it is possible to measure small
surface details that are not resolved when operating with
full depth of view.
~ ?,~
_ 9
Various for~s of pos;tion sensitive detectors are now
commercial available. soth unidimensional (linear) and
bidimensional position sensitive detectors are known.
One of 'he advantages of the present arrangement is that
it enables a unidimensional detector to be used, while
nevertheless obtaining three dimensional information.
Since the response time of a unidimensional detector is
about two orders of magnitude faster than that of a
bidimensional detector, the output rate is consequently
enhanced. The type of position sensitive detector chosen
for use in the present invention will depend largely on
the particular application of the device and on such
factors as the resolution and speed of operation required.
Figures 4 to 6 show a preferred embodiment of the
present invention employing an autosynchronising pyramidal
mirror M3 having six facets, a pair of fixed mirrors M4
and M5 and a mirror M6 that can be rota-ted about an X
direction axis 21 by means of a stepping motor 22.
The mirrors M4 and M5 are fixed while the device is in
~ operation, although they can be adjusted initially to set
the position of the reference plane 20 relative to -the
device. As seen from Figure 4, the beam 10 from the source
S initially gives rise to a projected beam 11 and hence to
detected beams 14 and 15. After a small rotation of the
~5 mirror M3, the initial portion 10 of the projected beam
will produce a later beam portion lla. The reference
plane 20 has thus been scanned from point X0 to Xl. The
detected beam portion 14a is reflected by the mirrors M4
and M3 to generate a beam coincident with the previous
detected beam 15. Such detected beam 15 is thus still
received in the detector D at point P0. On the other
hand, a detected beam 14' from the point 13 having a
positive Z coordinate relative to the plane 20 becomes the
beam 15' which is received at position Pl in the detec-tor
D. As seen from Figure 5, scanning in the ~ direction of
~.,
-- 10 --
beam portions llb and 14b is achieved by stepping of the
mirror M6 by the motor 22. In typical operation, the
pyramidal mirror M3 will be rotated anywhere from 10 to
300 revolutions per second and the stepping motor 22 will
be stepped from about 1500 to 2000 steps per second, with
about 1000 steps constituting the full scan in the Y
direction.
Hence it will be seen that the portions of the beams
11, 14 etc. that intersect at the plane 20, or at the
10 object point 13, define a "beam plane" that extends in the
X direction and generally in the Z direction (subject to
some deviation as best shown by the beam portions llb,
14b). Such beam plane always intersects the reference
plane 20 along the X direction.
Figures 7 and 8 show an alternative embodiment in which
the projected and detected beams 10 and 15 are initially
reflected by galvonometer driven mirrors M7 and M8, these
mirrors being electrically synchronized by controllers Cl
and C2 from a ramp signal from a signal source SS. The
Q output signal from the detector D, when seen on an
oscilloscope 0, provides a direct profile 23 of Z values
proportional to displacement along the X axis, such profile
23 being proportional to the Z value of the object 12. In
this embodiment the position of the reference plane 20 can
~5 be set electronically by modifying the phase relationship
of the two excitation signals passed to the mirrors M7 and
M8. As in the previous embodiment, a mirror M6 is used to
deflect the intersecting portions of the projected and
detected beams perpendicularly to the plane of the initial
30 beams, thus providing a surface profile measurement, namely
scanning in the Y direction.
Alternatively, the mirror M6 can be omitted from either
of the embodiments of Figures ~-6 and 7, 8, if Y scanning
can be dispensed with or if one of the other expedients
35 described below is adopted.
Figure 9 shows an alternative to the arrangement of
Figures 4 to 6, in whlch the pyramidal mirror M3 is
replaced by a polygonal mirror M9. Apart from -the conse-
~uent modification to the ~.ocations of the source S and
detector D, this arrangement operakes in the same way as
that of Figures 4 to 6.
Figure 10 shows yet another modification of geometry,
again using the mirror M9, but dispensing with the mirrors
M4 and M5.
L0 Figure 11 is the same as Figure 10, except for ~he
addition between the source S and the mirror M9 of an
acousto-optic device 24. This device 24 is used to pre-
program the projected beam for a predetermined profile of
the object to be scanned. As long as each object scanned
has this predetermined profile, the detector output will
be zero. The detector thus only observes deviations from
the selected profile. This arrangement is particularly
well suited to the inspection of a series of articles that
are intended to have the same shape. The acousto-optic
device 24 can also be used, if desired, to compensate for
errors introduced by a large field of view.
Figure 12 shows yet another geometry employing a
galvonometer driven, i.e. oscil.lating, double-sided mirror
M10, with the projected beam from the source S passing
~S through a hole 25 in the mirror M6.
In the embodiment of Figure 13, the geometry of Figure
10 is modified to dispense with the mirror M6 and to
support the source S and detector D on a common mount 26
with the motor driven mirror M9. This arrangement is
especially applicable to the sweeping out of the contours
of a surrounding space, such as a room, the scanning in
the Y direction being achieved by rotation of the entire
device about the axis 27. As an alternative, as shown in
Figure 14, this scanning can be achieved by rotating ~he
object about an axis 28.
- 12
Figure 15 shows the use of a pair o~ detectors Dl and
D2 each of which could operate basically in accordance
with any one of the foregoing embodiments, although they
have here been illustrated specifically in relation to use
with the rotated pyramidal mirror M3, suitable associated
fixed mirrors and the mirror M6. This arrangement
minimizes the shadow effect problem. The object 12 is
shown obscuring the beam received by the detector D2,
this shortcoming being overcome by the beam received by
the detector Dl. One or other of the detectors, or both,
will receive a beam in more situations than when using a
single detector.
Although any light source can be used, a laser beam
has many advantages. It can be comparatively cheaply
generated and renders the device compact for a brightness
that is orders of magnitude higher than that of a source
of non-coherent light. A large depth of field can be
obtained and an interferometric filter can be used to
improve the signal to noise ratio, whereas a device using
~0 non-coherent light must be operated in a darkened
environment. Both diode and helium-neon lasers can be
used. One of the advantages of a diode laser is that its
infrared emission tends to match well the peak sensitivity
of silicon detectors which are commonly used in the
position sensitlve detector. Another advantage of the use
of a laser source is that its intensity can readily be
electrically controlled, opening up the possibility of
servo loop control of the laser output intensity.
The form of radiation used is not limited to coherent
or non-coherent light, but can include any form of
radiation capable of being effectively collimated and
deflected by available devices as well as being reflected
by the object under examination. Such radiation can
include that in the ultraviolet and X-ray spectra and in
the in~rared and microwave frequencies. It is also
possible to use high frequency sound radiation, provided
an appropriate position sensor and collimated beam are
available.
In addition~ instead of using mirrors to deflect the
beams, the invention includes the use for this purpose of
any scanning mechanism including acousto optic devices .
Such devices are known and examples are described in the
1981 edition of The Optical Industry and Systems Purchasing
Directory, pages E-161 to E-164.
. ~
