Note: Descriptions are shown in the official language in which they were submitted.
CA 02569798 2006-12-01
Attorney Docket No- 19179P0002CA01
FULL-FIELD THREE-DEMIONSAL MEASUREMENT METHOD
FIELD OF THE INVENTION
The present invention relates to three-dimensional surface-
geometry or shape measurement useful in building ,a digital or
mathematical representation of an object or environment for
which the geometry is unknown.
BACKGROUND OF HE INVENTION
Three-dimensional (3-D) surface-geometry or shape
measurement is useful to build a digital or mathematical
representation of an object or environment for which the
geometry is unknown- One of the most widely used techniques to
obtain the 3-D shape of an unknown object is using structured
light- By projecting a known light pattern (stripe, grid, or
more camplex shape) onto an object, the 3-D coordinates of
points on the object surface can be calculated by triangulation
from images acquired from another direction. Figure 2
illustrates this triangulation principle_ In the example shown
in Figure 1, a laser line iB projected onto a scanned object.
The light is scattered from the object surface and the image of
the curved line formed on the object is acquired by a camera
located at an angle to the laser source- The angle and the
position where the scattered light is sensed in the camera
sensor array are related_ The depth information can be computed
from the distorted two-dimensional (2-D) image of the laser
light along the detected profile, based on a calibration of the
measurement system done earlier- ID order to get full range
(depth) information, the laser sheet, formed by projecting the
laser line, has to be moved across the object or scene. Point-
by-point (cf. Parthasarathy, S. et al., 'Laser rangefinder for
robot control and inspection", Robot Vision, SPIE, 336, pp.2-11,
- 1-
'
. ,
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002CA01
1982; and Rioux, M., "Laser range finder based upon synchronous
scanners", Applied Optics 23(21), pp.3837-3844, 1984) and line-
by-line- (of. Popplestone,R. J. et al., "Forming models of plane-
and-cylinder faceted bodies from light stripes", Proc. Int.
Joint Conf- on Artificial Intelligence, pp.664-668,1975; and
Porter II, G. B. and Mundy, J. L., " Noncontact profile sensing
system for visual inspection", Robot Vision, SPIE, 336, pp_67-
76, 1982) scanning use a scanning mechanism equipped with
accurate position sensors. Furthermore, they are slow and not
practical for real-time 3-D shape measurement..
Multiple-stripe methods speed up the data acquisition, but
suf.fer from a correspondence problem (cf. Boyer, K. and Kak, A.,
"Color-encoded structured light for rapid active ranging", IEEE
Trans. _Pattern Analysis and Machine Intelligence, pp.14-28,
1987; Chen, C.. et al-, "Range data acquisition using color
structured lighting and stereo vision". Image and Vision
Computing, Vol. 15, pp.445-456, 1997; and Rusinkiewicz, S. et
al., 'Real-time 3D Model Acquisition", Proceedings of Siggraph,
pp. 438-446, July 2002) of determining which light stripes in
the image correspond with the light stripes actually formed on
the object_ The correspondence problem can be removed by
projecting a multi-frame coded pattern, which carries
information of the coordinates of the projected points, without
considering geometrical constraints. The coded structured-light
approach is an absolute measurement method that encodes all
lines in the pattern from left to right, requiring only a small
number of images to obtain a full depth-image.
A coded structured-light method called intensity-ratio
depth sensing (cf. Carrihill, B. and Hummel, R., "Experiments
with the intensity ratio depth sensor", Computer Vision,
- 2 -
CA 02569798 2006-12-01
Attorney Docket No 19179P0002CA01
Graphics and Image Processing, vol. 32, pp- 337-358. Academic
Press, 1985; and Miyasaka, T. and Araki, K., "Development of
real time 3-D measurement system using intensity ratio method",
Proc. ISPRS Commission III, Vol. 34, Part 3B, Photogrammetric
Computer vision (PCV02), pp. 181-185, Graz, 2002) involves
projecting two patterns, a linear grey-level pattern and a
constant flat pattern, onto the object and capturing the image
of the light pattern formed on the object surface.
An intensity ratio is calculated for every pixel between
the two consecutive frames and the 3-D coordinates of each pixel
are determined by triangulation. This method has the advantage
of fast processing speed, but the accuracy is poor and the
problem of ambiguity arises for measuring objects with
discontinuous surface .shape if the intensity-ratio ramp is
repeated (cf. Chazan, G. and Kiryati, M., "Pyramidal intensity-
ratio depth sensor", Technical Report 121, Center for
Communication and Information Technologies, Department of
Electrical Engineering, Technion, Haifa, Israel, October 1995)
to reduce sensitivity to noise.
Pull-field optical 3-D shape measurement techniques have
been developed to acquire surface-geometry information over a
region of a surface rather than just a point or line. Compared
with other techniques, it has the benefit of fast measurement
speed due to the fact that it does not use scanning to cover the
whole object surface.
Moire interferometry (cf. Takasaki, H., "Moire topography",
Applied Optics, Vol. 9(6), pp.1467-1472, 1970 and fringe
projection (of- Creath, K., "Phase-measurement interferometry
techniques", Progress in Optics, Vol. XXVI, E. Wolf, Ed;
Elsevier Science Publishers, Amsterdam, pp. 349-393, 1988;
- 3 -
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002CA01
Halioua, M_ and Liu, H. C. "Optical Three-Dimensional Sensing by
Phase Measuring Profilometry", Optics and Lasers in Engineering,
Vol. 11(3), pp.185-215, 1989; and Greivenkamp, T. E.. and
Sruning, J. H., Optical Shop Testing, Chapter 4: "Phase Shifting
Interferometry', John Wiley and Sons, Inc., pp.501-598, 1992)
are good representatives of this technique which allows
relatively simple image-processing algorithms to extract the 3-D
coordinate information, high-speed image grabbing, reliable,
quantitative surface measurements, as well as non-contact and
noninvasive characteristics, and potential for real-time 3-D
shape measurement.
The basis of the moire method is that a grating pattern is
projected onto an object. The projected fringes distort
according to the shape of the object. The object surface,
together with the projected fringes, is imaged through a grating
structure called a reference grating as shown in Figure 2.. The
image interferes with the reference grating to form moire fringe
patterns. The moire fringe patterns contain information about
the shape of the object. When the geometry of the measurement
system is known, analysis of the patterns then gives accurate
descriptions of changes in depth and hence the shape of the
object.
Shadow moire (cf. Takasaki, supra; Meadows, D. M. et al.,
'"Generation of surface contours by moire pattern" Appl. Opt.
9(4), pp.942-947, 1970; and Chaing, F. P., "A shadow moire
method with two discrete sensitivities-, Exper. Mech. 15(10),
pp.38-4-385 1975) is the simplest method of moire technique for
measuring 3-D shapes using a single grating placed in front of
the object. The grating in front of the object produces a shadow
on the object that is viewed from a different direction through
- 4 -
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002CA01
the grating. One advantage of this method is that few or
calculations are required to convert image data into profile
information of the measured object.
The fringe projection technique discussed earlier, an
alternative approach to the moire method, uses a fringe or
grating pattern that is projected onto an object surface and
then viewed from another direction. The projected fringes or
grating is distorted according to the topography of the object.
Instead of using the moire phenomenon, however, the 3-D surface
is measured directly from the fringe projection by
triangulation. The image intensity distribution of the deformed
fringe pattern or grating is imaged into the plane of a CCD
array, then sampled and processed to retrieve the phase
distribution through phase extraction techniques, and finally
the coordinates of the 3-D object is determined by
triangulation.
To increase the measurement resolution, phase measuring
interferometry techniques (cf. Takasaki, supra; Creath, zupra;
and Halioua, supra) have been implemented in moire and fringe
projection methods to extract phase information, among which
phase-shifting methods (cf. He, X. Y., et a.2õ, "Phase-shifting
analysis in moire interferometry and its application in
electronic packaging", Opt. Eng. 37, pp. 1410-1419, 1998; and
Choi, Y. B, and Kim, S. W., "Phase-shifting grating projection
moire topography", Opt. Eng. 37, pp.1005-1010, 1998) are the
most widely used.
The principle of this technique is that periodic fringe
patterns are projected onto an object surface and then viewed
from another direction. In general, the minimum number of
measurements of the interferogram that will permit
- 5 -
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002CA01
reconstruction of the unknown phase distribution is three (cf.
Creath, supra), and a sinusoidal fringe pattern is usually used
in this technique.
Traditional phase-shifting systems use hardware such as a
piezoelectric transducer to produce continuous as well as
discrete phase shifts(cf. Creath, supra). In these cases, the
accuracy of the extracted phase is limited by the accuracy of
the mechanical shifting process. The accuracy also depends on
the number of images.. More phase steps usually can generate
higher accuracy in 3-D shape reconstruction. The trade-off,
however, is that longer time is involved both in image
acquisition and processing, which is fairly limited for real-
time analysis.
Another phase measurement technique is using Fourier
transform analysis (cf. Takeda, M., et al., "Fourier Transform
Method of Fringe Pattern Analysis for Computer Based Topography
and Interferometry", journal Opt. Soc. of.Am., 72, pp.156-160,
1982; Kreis, T,, "Digital holographic interference-phase
measurement using the Fourier transform method-, journal of the
Optical Society of America A. Vol. 3, pp. 847-855, 1986;
Freischlad, K_ and Koliopoloulos, C., "Fourier description of
digital phasemeasuring interferometry," Journal Of the Optical
Society of America A. Vol_ 7, pp. 542-551, 1990; Malcolm, A. and
Burton, D., "The relationship between Fourier fringe analysis
and the PFT," Prypntniewicz R., ed., Laser Interferometry
rompni7pr-Add Thtprfernmetry. Pron.. of Scn_ Photo-Ont. Instr. =
Eng. 1553. pp. 286-297, 1991; Gorecki, C., "Interferogram
analysis using a Fourier transform method for automatic 3D
surface measurement", Pure Appl. Opt., Vol- 1, pp. 103-110,
1992; Gu, J. and Chen, F., "Fast Fourier transform, iteration,
- 6
,
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002CA01
and least-squares-fit demodulation image processing for analysis
of single-carrier fringe pattern", journal of the Optical
Society of America A, Vol. 12, pp., 2159-2164, 1995; and Su, X.
and Chen, W., "Fourier transform profilometry: a review", Optics
and Lasers in Zngineering, 35, pp.263-284, 2001).
In this method, only one deformed fringe pattern image is
used to retrieve the phase distribution. In order to separate
the pure phase information in the frequency domain, the Fourier
transform usually uses carrier fringes; this poses difficulty in
practice in trying to accurately control the frequency of the
carrier fringe. Another significant limitation of the Fourier
transform technique is its inability to handle discontinuities.
Moreover, the complicated mathematical calculation of Fourier
transforms is computationally intensive and makes the technique
unsuitable for high-speed 3-D shape measurement_
The phase distribution obtained by applying a phase-
shifting algorithm is wrapped into the range 0 to 2n, due to its
arctangent feature. A phase unwrapping process(cf. Macy, W. W.,
"Two-dimensional fringe-pattern analysis", App/. Opt. 22, pp.
3898-3901, 1983; Goldstein, R. M. et al., "Satellite Radar
Interferometry: Two-Dimensional Phase Unwrapping", Radio
Science, Vol- 23, No. 4, pp.713-720, 1988; Judge, T. R. and
Bryanston-Cross, P. J., "A review of phase unwrapping techniques
in fringe analysis", Optics and Lasers in Engineering, 21,
pp.199-239, 1994; Huntley, J. M. and Coggrave, C. R., "Progress
in Phase Unwrapping", Proc. SPIE Vol. 3407, 1998; and Gniglia,
D. C. and Pritt, M. D., Two-Dimensional Phase Unwrapping:
Theory, Algorithms, and Software, Wiley-Interscience, John Wiley
and Sons, Inc., 1998) converts the modulo 2n phase data into its
natural range, which is a continuous representation of the phase
- 7 -
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002CA01
map. This measured phase map contains the height information of
the 3-D object surface (cf. Halioua, supra).
Therefore, a phase-to-height conversion algorithm is
usually applied to retrieve the 3-D data of the object. This
algorithm is usually related to not only the system setup, but
also the relationship between the phase distribution and the
height of the object surface. Based on geometric analysis of the
measurement system, several phase-to-height mapping
techniques(cf. Zhou, W. S. and Su, X. Y., "A direct mapping
algorithm for phase-measuring profilometry", Journal of Modern
Optics, Vol. 41, No.. 1, pp.89-94, 1994; Chen, X. et a/.7 "Phase-
shift calibration algorithm for phase-shifting interferometry",
Journal of the Optical Society of America A, Vol. 174 No. 11,
.pp. 2061-2066, November, 2000; Liu, H. et al., "Calibration-
based phase-shifting projected fringe profilometry for accurate
absolute 3D surface profile measurement", Optics Communications,
Vol. 216, pp. 65-80, 2003; Li, W. et al., "Large-scale three-
dimensional object measurement: a practical coordinate mapping
and image data-patching method", Applied Optics, Vol_ 40, No.
20, pp_ 3326-3333, July, 2001; Guo, H. et al., "Least-squares
calibration method for fringe projection profilometry", Opt.
Eng. 44(3), pp. 033603(1-9), March, 2005; and Sitnik, R. et al.,
"Digital fringe projection system for large-volume 360-deg shape
measurement", Opt. Eng. 41 (2), pp. 443-449, 2002) have been
developed, all focused on the accuracy of the measurement.
Algorithms that emphasize the speed (cf. Hung, Y. Y- et al.,
"Practical three-dimensional computer vision techniques for
full-field surface measurement", Opt. Eng. 39 (1), pp.143-1491
2000; and Zhang, C. et al., "Microscopic phase-shifting
profilometry based on digital micromirror device technology",
Applied Optic's, Vol. 41, No. 28, pp.5896-5904, 2002) are also
- 8 -
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002CA01
used in some high-speed or real-time systems (cf. Huang, P. S.
et ai., "High-speed 3-D shape measurement based on digital
fringe projection", Opt. Eng. 42 (1), pp.163-168, 2003 ("Huang
No. 1"); and Zhang, S. and Huang, P. S., "High-resolution, Real-
time 3D Shape Acquisition", IEEE Workshop on real-time 3D
sensors and their uses (joint with CVPR 04), Washington DC, MA,
2004).
Mora recently, a new digital fringe projection technique
for 3-D surface reconstruction has been developed using high-
resolution programmable projectors. Compared to traditional
fringe projection and laser-interferometric fringe- projection
techniques, the computer-generated fringe projection technique
has many advantages: (1) any high quality fringe pattern can be
precisely and quickly generated by software; (2) the fringe
pitch can be easily modified to match the object surface, thus
optimizing the range measurement of the object; (3) the phase
can be shifted precisely by software according to the specific
algorithm without a physical phase shifter; (4) the use of a
high and constant brightness and high contrast-ratio projector
improves the accuracy of the 3-D shape measurement; and (5) with
proper synchronization between the projection and image
acquisition, real-time 3-D reconstruction could be achieved.
The most popular methods for computer-generated fringe
projection with phase-shifting technique can be roughly divided
into grey-scale phase-shifting (cf. Hung, supra; Huang, P. S.
and Chiang, F., "Recent advances in fringe projection technique
for 3-D shape measurement", Proc. SPIE, Vol. 3783, pp.132-142,
1999 ("Huang No.1"); Fujigaki, M. and Morimoto, Y., "Accuracy of
real-time shape measurement by phase-shifting grid method using
correlation", JSME International journal, Series A, Vol. 43, No.
- 9 -
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002CA01
4, pp. 314-320, 2000; Hu, Q. et al., "Calibration of a three-
dimensional shape measurement system", Opt. Eng. 42(2), pp.487-
493, 2003; Quan, C. et al., "Shape measurement of. small objects
using LCD fringe projection with phase shifting", Optics
Communications, Vol. 189, pp.21-29, 2001 and Quan, C. et al.,
"Shape Measurement by Use of Liquid-Crystal Display Fringe
Projection with Two-Step Phase Shifting", App1ied Optics, Vol.
42, No. 13, pp.2329-2335, 2003 ("Quan No 2")), color-encoded
phase-shifting (cf. Huang, P. S. et al., "Color-encoded digital
fringe projection technique for high-speed three-dimensional
surface contouring", Opt_ Eng. 38(6), pp.1066-1071, 1999; and
Pan, J.. et a/_, "Color-encoded digital fringe projection
technique for high-speed 3D shape measurement- color coupling
and imbalance compensation", Proc. SPIE, Vol. 5265, pp.205-212,
2004)46,47 and fringe projection based on the Digital Micromirror
Device (MD) method (cf. Zhang, supra; Huang No. 1, supra;
Huang, P. S., and Chiang, F- "Method and apparatus for three
dimensional surface contouring using a digital video projection
system", U.S. Patent No. 64438,272, August 20 2002; and Huang,
P. S. et al., "Method and apparatus for three dimensional
surface contouring and ranging using a digital video projection
system", U.S. Patent No. 6,788,210, September 7, 2004),
The grey-scale phase-shifting method projects a series of
phase-shifted sinusoidal grey-scale fringe patterns onto an
object and then a camera from another direction captures the
images of the perturbed fringe pattern sequentially for
processing. Because the phase map acquired directly is limited
from -n to n, the natural phase distribution of the pixels,
which carries the 3-D surface information, is generated by
applying phase.unwrapping techniques. The 3-D shape information
for each pixel is extracted by use of a phase-to-height
- 10 -
CA 02569798 2006-12-01
Attorney Docket No.119179P0002CA01
conversion algorithm. This approach can potentially increase the
measurement speed.
Instead of projecting a sinusoidal fringe pattern,
Morimoto., Y. et al., -Real-time Shape Measurement by Integrated
Phase-Shifting Method" Proc SPIE, Vol. 3744, pp.118-125, August
19995 and Fujigaki, supra, have proposed an integrated phase-
shifting method which projects a rectangular grey-scale pattern
multiplied by two weighting functions, respectively. Morimoto
built a signal processing board for real-time phase analysis.
The system, which records four frames of the deformed grating on
the object during one cycle of the phase shift, can obtain the
phase distribution every 1/30 seconds. In Fujigaki's method,
thirty-two phase-shifted grey-scale images with rectangular
distribution are projected and captured to determine the phase
difference that corresponds to the height distribution of the ,
object. In the case of worst focus the average phase error is 3%
and the maximum phase error is 5% when the object is stable. For
a moving object, the error increases linearly with the phase-
shifting aberration ratio.
For phase-shifting techniques, sequentially projecting and
grabbing images consume time especially for more phase-shift
procedures. To further increase the measurement speed, unlike
conventional sinusoidal phase-shifting, which uses a minimum of
three phase-shifted fringe patterns, a two-step sinusoidal
phase-shifting method (of. Quaa No. 2, supra; and Almazan-
Cuellar, S. and Malacara-Hernandez, D., "Two-step phase-shifting
algorithm", Opt. Eng., 42(12), pp.3524-3531, 2003) and one-frame
spatial-carrier sinusoidal phase-shifting method (cf.
Kujawinska, M. and Wojciak, J., "Spatial-carrier phase shifting
technique of fringe pattern analysis," Industrial Applications
- 11 -
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002CA01
of Holographic and Speckle Measuring Techniques, Proc. SPIE,
1508, pp. 61-67, 1991) for calculation of the phase -values has
been proposed. Because the phase unwrapping is carried out by
use of an arccosine function for two-step sinusoidal phase-
shifting, or arctangent function two times for spatial-carrier
sinusoidal phase-shifting, the use of these algorithms
simplifies the optical system and speeds up the measurement
compared to the three-step sinusoidal phase-shifting method.
However, the drawback of this method is that the measurement
accuracy is lower because the accuracy is dependent OD the
number of images (of Huzug No... 1, supra; and Morimoto, supra).
The color-encoded method uses a Digital Light Processor
(DL?) / Liquid Crystal Display (LCD) projector to project a
color-encoded pattern onto the object.. Only a single image,
which integrates three phase-shifted images (RGB components), is
captured by a color CCD camera. The image is then separated into
its RGB components, which creates three phase-shifted grey-scale
images. These images are used to reconstruct the 3-D object. The
problems for this technique include overlapping between the
spectra of red, green and blue channels of the color cameras
that make the separation of RGB components difficult and
intensity imbalance between the separated images of the red,
green and blue fringe patterns. The effective separation of the
captured image into its RGB components to create three phase-
shifted images of the object and compensate the imbalance is
non-trivial for this technique.
Fringe projection based on DMD projects a color-encoded
fringe pattern onto the object using a DL? projector. Due to the
particular features of the DLP projector, the RGB color channels
are sequentially projected. With removal of the color filter of
- 12 -
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002CA01
the DLIP projector and the synchronization between the projection
and image acquisition, three grey-scale phase-shifted images are
obtained with high speed (cf. Mang, C., supra, Huang No. 1,
supra; and alai-1g, S., supra). The 3-D shape of the object is
reconstructed using a phase wrapping and unwrapping algorithm
and a phase-to-height conversion algorithm. Considering that
traditional sinusoidal phase-shifting algorithms involve the
calculation of an arctangent function tO obtain the phase, which
results in slow measurement speed, an improved method, called
trapezoidal phase-shifting method (cf. Mang, S., supra) was
proposed for further increasing the processing speed. By
projecting three phase-shifted trapezoidal patterns, the
intensity ratio at each pixel is calculated instead of the
phase- This requires much less processing time.
In fringe-projection techniques, the projected pattern
greatly affects the performance_ Much processing time is spent
on the phase calculation and phase-to-height conversion- To
realize real-time 3-D shape measurement, it is not sufficient
just to speed up the projection and image acquisition. Designing
efficient patterns for fast manipulation are efficient ways of
speeding up the entire 3-D measurement process.
The present invention therefore seeks to provide a novel
full-field fringe-projection method for 3-D surface-geometry
measurement, which is based on digital fringe-projection,
intensity ratio, and phase-shifting techniques, and which uses
new patterns for fast manipulation to speed up the entire 3-D
measurement process.
- 13 -
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002CA01
=MARY OF INVENTION
The present invention provides a full-field fringe-
projection method for 3-D surface-geometry measurement, referred
to herein as "triangular-pattern phase-shifting". The inventive
method is based on two-step phase-shifting, but can also be
extended for multiple-step phase shifting. The inventive method
is based on digital fringe-projection, intensity ratio, and
phase-shifting techniques, discussed above.
In this method, a triangular grey-scale-level-coded fringe
pattern, generated by computer using software, is projected
along a first direction onto an object or scene surface via a
video projector; the projected triangular fringe pattern is
distorted according to the geometry of the surface; the 3-D
coordinates of points on the surface are calculated by
triangulation from distorted triangular fringe-pattern images
acquired by a CCD camera along a second direction.. The 3-D
object may be reconstructed using only two triangular patterns,
which are relatively phase-shifted by half of the pitch. A
triangular-shape intensity-ratio distribution is obtained from
calculation of the two captured distorted triangular fringe-
pattern images.
Removal of the triangular shape of the intensity ratio over
each pattern pitch generates a wrapped intensity-ratio
distribution. The unwrapped intensity-ratio distribution is
obtained by removing the discontinuity of the wrapped image with
a modified unwrapping method such as is used in the sinusoidal
phase-shifting method_ An intensity ratio-to-height conversion
algorithm, which is based on a phase-to-height conversion
algorithm in the sinusoidal phase-shifting method, is used to
reconstruct the 3-D surface coordinates of the object.
- 14 -
CA 02569798 2006-12-01
Attorney Docket No, 19179F0002CA01
The present invention seeks to solve problems, with
existing 3-D surface-shape measurement, that occur in respect of
speed of measurement, ambiguity in determining correspondence
between patterns and the real object or scene surface when
repeated patterns are used, and in depth resolution. The
inventive two-step triangular-pattern phase-shifting method for
3-D surface-shape measurement presents a new efficient
intensity-ratio generation algorithm, which provides faster
processing speed for generating 3-D coordinates of an object or
scene surface with simpler computation and fewer images
required. It improves the depth resolution and has a lower
degree of ambiguity problems with the triangular patterns used.
The two-step triangular-pattern phase-shifting for 3-D
surface-geometry measurement combines the advantages of
conventional sinusoidal phase-shifting method and conventional
intensity ratio methods and only uses two triangular patterns
relatively phase-shifted by half of the pitch to reconstruct the
3-D surface. Compared with the conventional sinusoidal phase
shifting and trapezoidal phase shifting methods, the inventive
method has faster processing speed because of the simple
computation of the intensity ratio and because fewer images are
used to obtain the intensity ratio. It also has a better depth
resolution compared to conventional intensity-ratio based
methods and a lower degree of ambiguity problems when the
intensity ratio ramp is repeated to reduce sensitivity to noise.
,The inventive method therefore has the potential for faster
real-time 3-D surface measurement.
According to a first broad aspect of an embodiment of the
present invention there is disclosed a fringe light projection
- 15 -
,
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002CA01
method for use with a system for measuring three-dimensional
surface-geometry of an object comprising steps of:
a) projecting a first triangular coded fringe pattern
onto a surface of the object at a specified pitch and
along a first direction to produce a first distorted
fringe pattern on the object surface;
b) capturing, from along a second direction from the
projected triangular coded fringe pattern, a first
fringe-pattern image of the first distorted fringe
pattern;
c) projecting at least one additional triangular coded
fringe pattern onto a surface of the object at the
specified pitch having a phase step relative to each
other triangular coded fringe pattern and along the first
direction to produce a corresponding at least one
additional distorted fringe pattern on the object
surface;
d) capturing, from along the second direction from .the
projected triangular coded fringe pattern, at least one
additional fringe pattern image corresponding to the at
least one additional distorted fringe pattern;
e) calculating a distribution of intensity ratios based
on the first and the at least one additional fringe-
pattern image; and
f) calculating a height distribution relative to a pre-
determined reference plane, whereby the three-dimensional
surface geometry of the object may be determined.
- 16 -
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002CA01
According to a second broad aspect of an embodiment of the
present invention there is disclosed a computer-readable medium
in a fringe light projection system for measuring three-
dimensional surface-geometry of an object, the medium having
stored thereon, computer-readable and computer-executable
instructions, which, when executed by a processor, cause the
processor to perform steps comprising:
a) projecting a first triangular coded fringe pattern
onto a surface of the object at a specified pitch and
along a first direction to produce a first distorted
fringe pattern on the object surface;
b) capturing, from along a second direction from the
projected triangular coded fringe pattern, a first
fringe-pattern image of the first distorted fringe
pattern;
c) projecting at _least one additional triangular coded
fringe pattern onto a surface of the object at the
specified pitch having a phase step relative to each
other triangular coded fringe pattern and along the first
direction to produce a corresponding at least one
additional distorted fringe pattern on the object
surface;
d) capturing, from along the second direction from the
projected triangular coded fringe pattern, at least one
additional fringe pattern image corresponding to the at
least one additional distorted fringe pattern;
e) calculating an intensity ratio distribution based on
the first and the at least one additional fringe-pattern
image; and
- 17 -
CA 02569798 2006-12-01
Attorney Docket No- 19179P0002CA01
f) calculating a height distribution relative to a pre-
determined reference plane, whereby the three-dimensional
surface geometry of the object may be determined.
According to a third broad aspect of an embodiment of the
present invention there is disclosed a fringe light projection
system for measuring three-dimensional geometry of an object
comprising:
=a projector for projecting a plurality of triangular
coded fringe patterns relatively spaced-apart by a phase
step onto a surface of the object at a specific pitch and
along a first direction to produce a plurality of
distorted fringe patterns;
an image captor for capturing, along a second direction
from the projected triangular coded fringe patterns, a
plurality of corresponding fringe-pattern images of each
distorted fringe pattern;
an intensity ratio generator for calculating an intensity
ratio distribution based on all captured images; and
a height distribution calculator for calculating a height
distribution relative to a pre-determined reference plane
based on the intensity ratio distribution, to thus
determine the three-dimensional surface geometry of the
object_
BRIEF DESCRIPTION OF TEE DRAWINGS
The present invention will now be described with reference
to the drawings in which:
- 18 -
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002CA01
Figure 1 illustrates the principle of a prior art laser
triangulation measurement system;
Figure 2 is an optical setup for a prior art moire
projection system;
Figure 3 is a schematic diagram of a 3-D measurement
apparatus based on triangular phase-shifting fringe-projection
according to an embodiment of the present invention:
Figures 4(4) and (b) graphically illustrate phase-shifted
input triangular fringe patterns for use in two-step
triangular-pattern phase-shifting in accordance with an
embodiment of the present invention;
Figure 4(c) graphically illustrates an intensity ratio
arising from application of the 3-D measurement apparatus of
Figure 3 to the patterns of Figures 4(a). and (10;
= Figure 4(d) graphically illustrates an intensity-ratio ramp
according to an embodiment of the present invention;
Figure 5(a) graphically illustrates phase-shifted input
triangular fringe patterns for use in three-step triangular-
pattern phase-shifting in accordance with an embodiment of the
present invention;
Figure 5(b) graphically illustrates an intensity ratio
arising from application of the 3-D measurement apparatus of
Figure 3 to the patterns of -Figure 5(e);
Figure 5(c) graphically illustrates an intensity-ratio ramp
according to an embodiment of the present invention;
=
- 19 -
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002CA01
Figure 6 graphically illustrates phase-shifted input
triangular fringe patterns for four-step triangular-pattern
phase-shifting in accordance with an embodiment of the present
invention;
=
Figure 7 graphically illustrates phase-shifted input
triangular fringe patterns for five-step triangular-pattern
phase-shifting in accordance with an embodiment of the present
invention;
Figure 8 graphically illustrates phase-shifted input
triangular fringe patterns for six-step triangular-pattern
phase-shifting in accordance with an embodiment of the present
invention;
Figure 9 illustrates a relationship between an intensity.
ratio of a projected triangular fringe pattern and a height of
an object according to an embodiment of the present invention;
Figure 10 is a schematic representation ofa measurement
system calibration setup according to an embodiment of the
present invention;
Figure 11 illustrates a non-linear mapping of projector
input intensity to camera-captured image intensity according to
an embodiment of the present invention;
Figure I2(a) illustrates an exemplary input triangular
pattern for use in error correction;
Figure I2(b) illustrates a simulated captured triangular
pattern image with measured and ideal gamma curve functions
corresponding to the exemplary input triangular pattern of
Figure 12(e);
- 20 -
,
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002CA01
Figure 13 illustrates a measured and ideal intensity-ratio
with periodic triangular shape according to an embodiment of the
present invention;
Figure 14 illustrates a measured and ideal intensity-ratio
ramp after conversion from the triangular shape of Figure 13;
and
Figure 15 illustrates intensity-ratio error as a function
of intensity ratio according to an embodiment of the present
invention.
'DETAILED DESCRIPTION OF TBE INVSNTION
The present invention will be described for the purposes of
illustration only in connection with certain embodiments_
However, it is to be understood that other objects and
advantages of the present invention will be made apparent by the
following description of the drawings according to the present
invention.. While a preferred embodiment is disclosed, this is
not intended to be limiting. Rather, the general principles set
forth herein are considered to be merely illustrative of the
scope of the present invention and it is to be further
understood that numerous changes may be made without straying
from the scope of the present invention_
Referring now to Figure 3, there is shown a simple
schematic diagram of the inventive triangular phase-shifting
fringe-projection measurement apparatus, shown generally at 30.
It consists of a computer 31, a video projector 32, a CCD camera
33, and a flat reference plane 34 for calibration. The projector
32 is used to project a triangular fringe pattern, digitally
generated by software in the computer 33, onto the surface of an
object 35. The CCD camera 33 is used to capture the image
- 21 -
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002CA01
contour of fringe patterns via a frame grabber (not shown). The
image is processed and a reconstructed 3-D shape 37 is displayed
on the monitor 36..
The measurement apparatus 30 used to demonstrate the
invention is now described, although alternative components or
those with different specifications may be alternatively used.
The computer 31 preferably has a 3.04 GHz processor with 1.0 GB
memory. The digital projection system 32, which is preferably a
model LP600 manufactured by In Focus preferably has a brightness
of 2000 ANSI lumens, a resolution of XGA 1024x768, and a
contrast ratio of 10001. The CCD camera 33, which may be a
model XCHR50 Progressive Scan Black-and-White CCD Camera by Sony
Corporation is used for image capture- Preferably, it delivers
detailed images with an equivalent VGA resolution of 648x494
pixels- The CCD has square pixels, avoiding any aspect-ratio
conversion- The frame grabber (not shown) may be a model
Odyssey XA vision processor board by Matrox and is used to
capture and digitize the images. Those having ordinary skill in
this art will readily appreciate that there may be techniques
that capture images without a framegrabber, such as imaging with
a firewire camera-computer interface, that may be equally
suitable_
In an exemplary scenario, a plastic face-mask was used as
the object 35. The mask had dimensions of 210 mm x 140 TOM x 70
Custom software in the inventive 3-D measurement apparatus
. .
was developed with Visual C++ 6Ø.
Triangular fringe patterns are generated by computer 31 and
projected onto the object 35 by the projector 32. The patterns
- 22 -
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002CA01
are shifted digitally with the different phase steps, and the
camera 33 captures the images of the object 35 via the frame
grabber not shown):
The computer 31 also generates intensity-ratio wrapping,
intensity ratio unwrapping, intensity-ratio difference, and
object-height calculations using calibration parameters obtained
in a prior calibration exerciase. The software also provides
capability to display a reconstructed 3-D object in shaded,
solid or wireframe modes. Median and averaging filters, and
indeed, other computer vision filters known in the art may be
applied to reduce noise and smooth the reconstructed 3-D object
$uxface-
The use ot conventional sinusoidal phase-shifting provides
satisfactory measurement accuracy_ However, calculation of the
arctangent function is time consuming and slows down measurement
speed_ By contrast, the conventional intensity-ratio method has
the advantage of fast processing speed, but may introduce
ambiguity when measuring objects with discontinuous surface
shape if the intensity-ratio ramp is repeated to reduce noise
sensitivity .
In the present invention, a novel high-speed two-step
triangular-pattern phase-shifting for 3-D shape measurement
combines these conventional methodologies in an innovative
fashion to appropriate the relative advantages of each prior art
method.
. Two triangular patterns 41, 42, relatively phase-shifted by
half of the pitch, shown in Figures 4(a) and (In) respectively,
are used to reconstruct the 3-D object in the inventive method.
- 23 -
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002C.401
f0001] The intensity equations for the two shifted triangular
patterns may be formulated as follows.:
2/Nr(Tx, y) x +7 . (x, y) + I'''(x' Y)
2
x
{
ji (x, y) = 2/. (x, y)x i...(x,y) + 31,,, (x, x er. y)
T 2
2/. (x, y) x +/ )
3/. (x, y)
x E1.
T 2 T
4
T 3 , ( 1 )
l.¨ - T ¨ )
4 ' 4
,37. -- aõ )
4
2/÷' (x, y)x+./..(x,y)+ /-E(x, y)---- x e {0,T)
T 2 4
4 (x, y) = 2/.(x, y) x + irni. (x, y) 4 (x, y) x ,. T 3T ,
¨ ¨) ( 2 )
51,õ(x, y)
21,"(x,y)x+1(x,y)-F __________________ x
T 2
= .
.I.,(x, y) = I.(x,y)- I (x, y) (3)
where /,(x,y)and 4(x,y)are the intensities for the two shifted
triangular patterns respectively,
T is the pitch of the patterns,
/.(x,y)is the intensity modulation, and
.7..(x.y)and /..,(x,y) are the minimum and maximum intensities
of the two triangular patterns, respectively.
The intensity ratio ro(x,y) may be calculated by:
. ( 4 )
/,õ (x, Y)
After operation of Equation (4), the triangular patterns
are each divided into four regions R=1 through 4. Each region .
has a different intensity ratio. The shape of the intensity
- 24 - *
CA 02569798 2006-12-01
Attorney Docket No.. 19179P0002CA01
ratio 'Ay) 43 in the full pitch is shown in Figure 4(c), which
has a triangular shape whose values range from 0 to 1. This
triangular shape can be converted to a ramp by applying the
following equation:
r(x, y) =2x round(¨R-1)+ (-1)"ro(x,y) R=1,2,3,4 (5)
2
=
where R is the region number.
The converted intensity-ratio ramp map r(x,y) 44, shown in
Figure 4(d), has the intensity value ranging from 0 to 4.
To generalize two-step triangular-pattern phase-shifting to
N-step triangular-pattern phase-shifting the triangular fringe
patterns can be generated using the following equation:
x+51
2 4
(x, 21-.(x,Y).(x,y)+31.(x,y)
x +5, eFT ¨3T) (6)
x, y) (x + 5,)+1(x,y) 31, x + c5;
2 4
where /Xx,y) is the intensity value of the Ah phase shift at
pixel (x,y); =
Si is the ith phase shift distance n the X direction.
To retrieve the intensity ratio, at least two samples are
used. These samples are taken at:
i=1,2, ............................ .A.T2 (7) =
- 25-
,
CA 02569798 2006-12-01
Attorney Pocket No. 19179P0002CA01
where N represents the number of shifting steps of the
method. In the case of two-step triangular-pattern phase-
shifting, as described above, IN12:
Algorithms to determine the intensity-ratio, which assist
in determination of the 3-D coordinates of the measured object
35, are developed in detail below for different numbers of
phase-shifting steps up to six. Those having ordinary skill in
this art will appreciate that the algorithms for phase-shifting
with more than six-steps can be similarly derived.
For three-step triangular-pattern phase-shifting, three
triangular patterns relatively phase-shifted by one-third of the
pitch are used to reconstruct the 3-D object 35. Figure 5(a)
shows the cross sections of the three-step phase-shifted
triangular fringe patterns 51, 52, 53.
The intensity ratio ro(x,y)can be calculated by:
1,40(x,y)-.L(x,y)+Ii.(x,y)-1.(x,y)
ro(x,Y) _____________________________________________ (8)
Mx,.//)
where 1,10(x,y), i(x,y), and 110,,(x,y) are the highest, median
and lowest Intensities of the three shifted triangular patterns
at the same position in the range of the pitch, respectively.
After operation of Equation (8), the triangular pattern is
divided into six regions R=1 through 6, each with a different
intensity ratio pattern.
The intensity ratio distribution ro(r..,y) 54 over the full
pitch is shown in Figure 5.(b) and is similar to that for two-
step triangular-pattern phase-shifting, shown in Figure 4(e)
with values ranging from 0 to 17 but with three repeated
- 26 -
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002C7101
triangles and six regions over the full pitch. The repeated
triangles can be converted to a ramp 55 by applying Equation (5)
with the region number R ranging from 1 to 6, as shown in Figure
5(0).
For four-step triangular-pattern phase-shifting, four
triangular patterns 62-64, relatively phase-shifted by one-
fourth of the pitch are used to reconstruct the 3-D object 35.
Figure 6 shows the cross sections of the four-step phase-shifted
triangular fringe patterns 61-64,
The intensity ratio ro(rM can be calculated by;
1/1(x,y)-13(x,y)1-1/2(x,y)-4(x,y)11
(9)
./.(x,y)
where /1(x0/),.
.4(x0/), 13(x,y) and I4(x,y) are the intensities for
the four shifted triangular patterns respectively.
After operation of Zquation (9), the triangular pattern is
divided into eight regions, R=1 through 8, each with a different
intensity ratio pattern_ The intensity ratio distribution 4(x,y)
over the full pitch is similar to that for two-step triangular-
pattern phase-shifting, shown in Figure 4(o), with values
ranging from 0 to 1, but with four repeated triangles and eight
regions over the full pitch. The repeated triangles can be
converted to a ramp by applying Zquation (5) with the region
number R ranging from 1 to 8.
For five-step triangular-pattern phase-shifting, five
triangular patterns 71-75, relatively phase-shifted by one-fifth
of the pitch are used to reconstruct the 3-D object 35. Figure
- 27 -
=
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002CA01
7 shows the cross sections of the five-step phase-shifted
triangular fringe patterns 71-75.
[00021 The intensity ratio rAy)can be calculated by:
ro(x,Y)= ________________________________________________ (20)
whete /,40(x,y), 4.4(x ,y), 1,..(x,y)
and jr,,,(c,y) are the
highest, second highest, third highest, fourth highest, and
lowest intensities of the five shifted triangular patterns at
the same position in the range of the pitch, respectively.
After operation of Equation (20), the triangular pattern is
divided into ten regions R=1 through 10, each with a different
intensity ratio pattern_ The intensity ratio distribution r(x,y)
over the full pitch is similar to that for two-step triangular-
pattern phase-shifting, shown in Figure 4(c) with values ranging
from 0 to 1, but with five repeated triangles and ten regions
over the full pitch_ The repeated triangles can be converted to
a ramp by applying Equation (5) with the region number R ranging
from 1 to 10.
For six-step triangular-pattern phase-shifting, six
triangular patterns 81-86, relatively phase-shifted by one-sixth
of the pitch are used to reconstruct the 3-D object 33. Figure a
shows the cross sections of the six-step phase-shifted
triangular fringe patterns.
The intensity ratio ro(x,y)can be calculated by:
Ih,sb(x>y)--1,..di(xpy)+-r(xpY)-L..(x.y)+.r.õ Oe. (x, 31)
(11)
IM ("Y)
- 28 -
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002CA01
where ihmh(x,y) , /..õ(x,y), /.õõ(x,y) and /(x,Y) ,
are the highest, second highest, third highest, fourth highest,
fifth highest, and lowest intensities of the six shifted
triangular patterns at the same position in the range of the
pitch, respectively.
After operation of Equation (il), the triangular pattern is
divided into twelve regions R=1 through 12, each with a
different intensity ratio pattern. The intensity ratio
distribution ro(x,y) over the full pitch is similar to that for
two-step triangular-pattern phase-shifting, shown in Figure
4(c), with values ranging from 0 to 1, but with six repeated
triangles and twelve regions over the full pitch. The repeated
triangles can be converted to a ramp by applying Equation (5)
with the region number R ranging from 1 to 12.
Measurement resolution and precision can be increased by
using more triangular fringes- The intensity ratio is wrapped
into the range of 0 to 4 for two-step phase-shifting; 0 to 6,
for three-step phase-shifting; and 0 to 8, for four-step phase-
shifting, etc.
The unwrapped intensity-ratio distribution may be obtained
by removing the discontinuity of the wrapped intensity-ratio
image with an unwrapping method modified from that commonly used
= in conventional sinusoidal phase-shifting.
Intensity-ratio-to-height conversion, based on phase-to-
height conversion used in conventional sinusoidal phase-
shifting, may be then used to retrieve the 3-D surface
coordinates of the object 35 from the unwrapped intensity-ratio
map.
- 29 -
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002CA01
The following intensity ratio-to-height conversion may be
applicable to intensity ratio-to-height conversion for
triangular-pattern phase-shifting with any number of shifting
steps. The ratio-to-height conversion is similar to that of
conventional sinusoidal phase-shifting in which the phase varies
in the range of 0 to 2n.
The wrapped intensity ratio for triangular phase-shifting
has a sawtooth-like shape. Removing the discontinuity of the
wrapped intensity ratio is possible where the range information
of the object 35 is contained in this unwrapped intensity-ratio
map and permits reconstruction of the 3-D shape of the object
35-
One suitable method to convert the phase map to the height
of the 3-D object surface is phase-measuring profilometry_ As
with conventional sinusoidal phase-shifting, height information
of the 3-D object surface is contained in the measured unwrapped
intensity-ratio map for triangular-pattern phase-shifting_
Therefore, an intensity ratio-to-height conversion retrieves the
3-D surface coordinates of the object 35 for triangular-pattern
phase-shifting. The values of the parameters used in this
algorithm are pre-determined by system calibration.
Figure 9 shows the relationship between the intensity ratio
of the projected triangular fringe pattern and the height of the
object. Point P 91 is the center of the exit pupil of the
projector 32, and point E 92 is the center of the entrance pupil
of the camera 33. The position at z = 0 in the coordinate system
is defined as the reference plane 34. Points F 91 and E 92 are
assumed to be on the same plane with a distance H to the
reference plane 34. The distance between points P 91 and E 92 is
d. The projector 32 projects a triangular fringe pattern with
-30---
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002CA01
pitch p on the reference plane 34. The intensity ratio at point
C 94 is rc, and the intensity ratio at point A 95 is rdi..
In Figure a, because triangles ADPE and ADAC are similar,
the following expression can be obtained-
d
¨= __________________________________ (12)
AC h
where h is the distance of point D 96 on the object surface
with respect to reference plane 34, and
AC is the distance between points A 95 and C 94.
On the reference plane 34, the following expression
holds:
AC r ar
(13)
where p is the fringe pitch on the reference plane 34,
T is the frine pitch of the pattern generated by the computer
31.
AC is the distance between points A 95 and C 94.
On the reference plane 34, the following expression holds:
= AC r-r
= (13)
where p is the fringe pitch on the reference plane 34,
T is the fringe pitch of the pattern generated by the computer
31.
- 31 -
CA 02569798 2006-12-01
Attorney Docket No, 1917920002CA01
If ArtiC = rA r;;. is defined as the intensity ratio difference
between points A 95 and C 94, then by combining Equations (12)
and (13), the height h of the object surface relative to the
reference plane 34 can be calculated by:
h ________________________ (14)
Td
PArow
This according to Equation (14) states that the distribution
of the height of the object surface relative to the reference
plane is a function of the distribution of the intensity ratio
difference. During measurement, the reference plane 34 is
measured first to generate an intensity ratio map for the
reference plane 34_ The measurement result will be used as the
reference for the object 35 measurement- The height h of the
object surface is then measured relative to the reference plane
34. When measuring the object 35, on the CCD array, point D 96
= on the object surface will be imaged onto the same pixel as
point C 94 on the reference plane 34. Because point D 96 on the
object surface has the same intensity value as point A 95 on the
reference plane 34,7:D=r,.. Thus, by subtracting the reference
intensity ratio map from the object intensity ratio map, the
= intensity ratio difference Ar4c=r4-r, at this specific pixel can
be easily obtained- This can be done for the whore intensity
ratio map. Therefore, the height distribution of the object
surface relative to the reference plane 34 is obtained.
Equation (14) describes a non-linear relationship .between the
distribution of the height and the distribution of the intensity
ratio difference_ Non-linear calculations are usually time
consuming. To increase the speed of the 3-D object measurement,
- 32 -
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002CA01
a modified form of Equation J24) using a linear relationship
between the distribution of the height and the distribution of
the intensity ratio difference can be used,. Both linear and non-
linear relationships were considered.
When H is much larger than h (or when d is much larger
than), which is true in general, Equation (14) can be
simplified asl
hmjff.,6x (15
Id
Thus, an approximate linear relationship between the
intensity ratio difference map and the surface height of the
object 35 is derived. However, Equation 414) was obtained only
by considering that the points 2 91, 2 92, C 94, A 95, D 96 are
located in the same X.-Z plane. Typically, however, the object 35
has dimension in the Y direction. This means that the parameter
H is not a fixed parameter, but is a function of the X and Y
coordinates. Therefore, considering the x-y dimensions, the
intensity ratio-height mapping function, Equation (15), for
calculating the surface height of the object relative to the
reference plane can be written as:
h(x, y). K(x, y)ar(x, y) (16)
K(x, y) - PH (x, ,Y)
where
ni
K(x,y)is a coefficient of the optical setup, which is the
function of (x, y).
The intensity ratio difference Ar(x,y)can be calculated by:
Ar(x,y) = r(x,y)-rr(x, y) (17)
- 33 -
=
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002CA01
where r(x,y) is the distorted fringe intensity ratio
distribution of the object surface. =
04y) is the reference fringe intensity ratio
distribution taken from a planar reference plane, and
6,7(xy) is the intensity ratio difference between r(x,y)and
P;(x,Y)-
Both r(x,y) and Oco.,) can be obtained from the calculation of
Equation (5) and an intensity ratio unwrapping method if the
triangular pattern is repeated. If for some points, coefficient
.01c,y) is known, the following equation can be used to calculate
the intensity ratio difference at these points.:
h(x,y)
( 18)
Kfx,y)
To get the non-linear relationship between the intensity
ratio difference map and the surface height of the object,
Zquation (14) can be rearranged as follows:
h
PU
(19)
1-1h
Considering the x-y dimensions, Equation (19) can be
expressed simply as:
Arc; y) = 1-m n(x(,xY,y)17(x(x, 31) (20)
, y)
=
where m(x,y) TQ
P17 (X,Y)
- 34 -
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002CA41
1
faro')
m(x,y) and n(x,y) are system parameters relating to
the optical setup, and
y are the pixel coordinates.
. _
Equation (20) can be rewritten as the following intensity
ratio-height mapping function:
4r(x,Y)
(21)
m(x,y) n(x,y)Ar(x,y)
This is the non-linear relationship between the intensity-
ratio difference map and the surface height of the object 35.
Equations (16) and (21) can be used to calculate the height
of the object surface relative to the reference plane 34 only if
all the system parameters and the fringe pitch on the reference
plane are known. Although the parameters H (in the X - Z plane)
and d in "Equation (15) could be measured, it is difficult to
precisely determine the parameter p, the fringe pitch in the
reference plane 34. Calibration that determines the height of
the object 35 from the intensity-ratio difference values,
without knowledge of all parameters related to the system
configuration, is therefore advantageous.
As the relationship between the intensity-ratio difference
map and the surface height of the object 35 is formulated above
as linear and non-linear, calibration by both of these methods
may be performed to obtain the system-related parameters. Figure
illustrates the measurement system calibration setup.
- 35 -
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002CA01
For linear calibration, to determine the coefficient Kin
Equation (16), in the coordinate system of Figure 10, either the
'calibration object or the reference plane must be translated
through a known distance along the Z direction relative to the
reference plane 34 at 0 101. Here the reference plane 34 is
translated to different positions 102-104 with given depths A.
By applying Equation (16), the intensity ratio-height
relationship for each pixel is determined as follows:
hi(x,y)=K(x,y)Ar,(x,y) (22)
and the intensity-ratio difference ,644y) is obtained by.:
Art(x,Y) ri(x, Y)- rr(x, (23)
where 7;(x,y)is the calibrated intensity ratio distribution
due to the translation 11., of the calibrated plane relative to the
reference plane,
rjx,y)is the intensity ratio distribution of the
reference plane, and
Nis the number of calibration positions.
both '(x,y) and r(x,y) can be obtained by applying
Equation (5) and an intensity ratio unwrapping method if the
triangular pattern is repeated.
Because of the linear relationship, the coefficient K can
be obtained by only one calibration position (N = 1). For the
purpose of increasing the measurement resolution and accuracy,
however, in practice, the system is calibrated by shifting the
calibration plate to several different positions. Applying the
- 36 -
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002CA01
least-squares algorithm to linear Equation (22), the following
equation may be used to obtain the coefficient K.:
,ai7; (x, Y)h1 (X, Y)
K (x, y) --= ___________________ (24)
IAr,7(x,Y)
Non-linear calibration is similar to linear calibration
except that two parameters m(x, y) and n(x,y)are determined as in
Equation (21). The minimum number of calibration positions is
two (N = 2). The intensity-ratio difference can be obtained
using the same method as in linear calibration.
For the purpose of increasing accuracy, more calibration
positions ( N > 2) may be used in practice to perform non-linear
calibration. A least-squares method may be applied to determine
parameters m(x, y) and n(x, y) in Equation (a].), which can be
rearranged as::
Ar(x,y)-= m(x, y)h(x, y)+ n(x, y)h(x, y)Ar (x, y) ( 25)
By choosing h(, y) and h(x, y)r(x, y) as the basis functions, and
applying the least-squares algorithm, the sum of squares is:
=
q = E[Ar,(x,y) m(x, y)121(x,y)-n(x,y)4(x,y)r,(x,y)12, (26)
where q depends on m(x,y)and n(x,y)
A necessary condition for q to be minimum is
ain(x, y)--2Z{.60",(x,Y)-m(x,Y)h,(x,Y)-ri(x,Y)k(x,Y).6,1;(x,Y)Ihi(x,Y)=
/4
(27)
- 37 -
CA 02569798 2006-12-01
Attorney nocket No. 19179P0002CA01
_____ - -.2E[A?: (x, y) - m(x, y)hi(x, y) n(x., y)h,(x, y)Ar,(x, y) (x,
y)Ar,(x, y) 0
n(x , y)
(28)
which can be arranged as:
m(x, y)Eh,2 (x, y)+ n(x, y) (x, y)Ar,(x,y),-- E 12,(x, y)Ar,(x, y)
} (29)
M(X, YE 1.2; (x, y)Art(x, y) + 17(x, .3))Yle(4 Y)Ari2 = 1 (x, y)Ari2 (x, y)
4 14
Equation (29) can be written in matrix form as:
y) a.,(x,y))( m(x, y)) _r bi(x,y))
y) a,(x,y)),;(x, y) )- y) ) (30)
where ,a,(x, y)
;(X $31) = 1= 2 (X, Y)6J;
a3(x, =14= 2 (x, AAril ,
(;y) = 17, (x, y)Ar,(x, , and
The parameters m(x, y) and n(x,y)in Equation (30) can be solved
as:
- 38 -
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002CA01
in(x, = a, (x , y)k(x, y) - a2 (x, y)b,(x, y)
al(x. Y)a)(x, - 4(x, Y)
n(x, - at(x, y)b, (x, y) a,(x, y)k(x,y) (31)
Y)a3(x, 4(x, Y)
The procedure described in respect of linear calibration
may now be applied to obtain the parameter *.x.y)and n(xa).
After completing the calibration and getting the intensity-
ratio difference distribution, the 3-D data of the object can be
calculated from Equation (p.).
The measurement accuracy of the triangular-pattern phase-
shifting method relies on the accuracy of the measured
intensity-ratio.. Gamma curve non-linearity of the projector and
image defocus are major sources of error in digital fringe-
projection techniques-
Intensity-ratio error compensation, motivated by a phase- -
error compensation approach (of- Zhang, S. and Huang, "P.S-
'µPhase error compensation for a 31) shape measurement system
based on the phase-shifting method", Two- and three-dimensional
methods for inspection and metrology XIX, Harding KG, ed.. Proc.
SPTE, 6000, E1-10, 2005) to decrease the measurement error due
to projector gamma non-linearity and image defocus in the
triangular-pattern phase-shifting measurement is described
below.
Without loss of generality, the principle of intensity-
ratio error compensation is introduced for two-step triangular-
pattern phase-shifting; however, intensity-ratio error
compensation as developed below is applicable to all triangular-
pattern phase-shifting with a different number of phase-shifting
steps.
- 39 -
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002CA01
Intensity-ratio error compensation involves estimating
intensity-ratio error in a simulation of the triangular-pattern
phase-shifting measurement process with both real and ideal
captured triangular-pattern images obtained from real and ideal
gamma non-linearity functions. A look-up-table (LUT) relating
the measured intensity-ratio to the corresponding intensity-
ratio error is constructed and used for intensity-ratio error
compensation, and thus shape-measurement error compensation.
The intensity response of an experimental measurement
system is first determined by projecting a linear grey-scale
pattern with-minimum intensity 0 and maximum intensity 255 onto
a white flat plate. An image of the reflected pattern is
captured by the CCD camera '33. An exemplary relationship between
the intensity values input to the projector 32 and those
captured by the CCD camera 33 is shown in Figure 11_ The
intensity values of the captured image are only sensitive to
input intensity values higher than a sensitivity threshold of
about 40, and they have very low sensitivity to input
intensities up to 90. The captured image intensities increase
non-linearly through the mid-range intensities to about 190,
have a nearly linear increase for the higher input intensity
values beyond 190 to about 240; and then increase non-linearly
to about 250.
Theimpact of both gamma non-linearity and defocus of the
projector 32 on the intensity mapping curve is revealed in
Figure 11. It is noted that non-linear response beyond about 240
is mainly generated by defocus of the projector 32..
Todetermine the intensity-ratio error caused by the non-
linearity between the pattern input to the projector 32 and the
image captured by the camera 33, a simulation to retrieve the
- 40 -
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002CA01
intensity-ratio error is carried out. First, a polynomial curve
is fit to the measured gamma curve data (Figure 5), and an ideal
gamma curve is drawn as a straight line between points a(x,y)
111, corresponding to the intensity sensitivity threshold,
discussed above, and b(x0)) 112, the curve endpoint.
Two phase-shifted triangular patterns 41, 42 are then
generated by :Equations (1) and (2) using a miniMum input
intensity value of 40 and maximum input intensity value of 255,
as shown in Figure 12(a). Using these two phase-shifted
-triangular patterns 41, 42 as input patterns to the projector
32, a simulation can be performed to generate simulated captured
triangular-pattern images corresponding to the measured 121, 122
and ideal 123, 124 gamma curves, shown in Figure 1200. The
measured intensity error due to the gamma non-linearity and
image defocus is apparent
'Theintensity-ratio so(x,y) computed using Equation. (4) for
both measured 132 and ideal 131 triangular pattern images is
shown in Figure 13, and the intensity-ratio ramp computed using
Equation 15) for both measured 142 and ideal 141 intensity-
ratios is shown in Figure 14.
In this figure, the ideal intensity-ratio ramp 141 is a
sloped straight line, while the measured intensity-ratio ramp
142 is characterized by a periodic error curve. The difference
between them is the measured intensity-ratio error 151, shown in
_Figure 15.
=
Errors indicated by arrows 143, 144 are mainly due to
defocus while other errors are mainly due to gamma non-
linearity. The two maximum intensity-ratio errors occur where
the intensity ratio r(x,y)has values of 1 143 and 3 144. As can be
- 41 -
"
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002CA01
seen from Figures 4(a), 4(b) and 4(c), these maxima correspond =
to the peaks of the projected triangular-fringe pattern, at T/4
and 3T/4, respectively. This indicates that the major intensity-
ratio error is caused by the projected pattern defocus, where
the sharp peaks are blurred into curves.
The intensity-ratio error, shown in Figure 15, is a single-
valued function of the intensity ratio- By constructing a look-
up table (LUT), which maps the intensity-ratio error as a
function of measured intensity-ratio, intensity-ratio error
compensation can be implemented. A LUT would be constructed
separately for each type of triangular-pattern phase-shifting,
corresponding to different numbers of steps. However, for a
given method, the LUT would only be constructed once, as long as
the positions and orientations of the projector 32 and camera 33
are not changed.
The intensity response of an experimental measurement
system and the mapping of the intensity-ratio error to the
measured intensity-ratio will be specific to the projector 32
and camera 33. The intensity values given above are exemplary
only. Those having ordinary skill in this art will readily
appreciate that other intensity values could be used for other
projectors 32 or cameras 33.
The present invention can be implemented in digital
electronic circuitry, or in computer hardware, firmware,
software, or in combination thereof. Apparatus of the invention
can be implemented in a computer program product tangibly
embodied in a machine-readable storage device for execution by a
programmable processor; and methods actions can be performed by
a programmable processor executing a program of instructions to
perform functions of the invention by operating on input data
- 42 -
=
CA 02569798 2006-12-01
Attorney Docket No. 19179P0002CAP1
and generating output. The invention can be implemented
advantageously in one or more computer programs that are
executable on a programmable system including at least one input
device, and at least one output device. Each computer program
can be implemented in a high-level procedural or object oriented
programming language, or in assembly or machine language if
desired; and in any case, the language can be a compiled or
interpreted language.
Suitable processors include, by way of example, both
general and specific microprocessors. Generally, a processor
will receive instructions and data from a read-only memory
and/or a random access memory. Generally, a computer will
include one or more mass storage devices for storing data files;
such devices include magnetic disks, such as internal hard disks
and removable disks; magneto-optical disks.; and optical disks.
Storage devices suitable for tangibly embodying computer program
. .
instructions and data include all forms of non-volatile memory,
including by way of example semiconductor-memory devices, such
as EPROM, EEPROM, and flash memory devices; magnetic disks such
=
as internal hard disks and removable disks; magneto-optical
disks i CD-ROM disks; and buffer circuits such as latches and/or
flip flops. Any of the foregoing can be supplemented by, or
incorporated in ASICs (application-specific integrated
circuits), FPGAs (field-programmable gate arrays) or DSPs
(digital signal processors).
Examples of such types of computers are programmable
processing systems suitable for implementing or performing the
apparatus or methods of the invention. The system may comprise
a processor, a random access memory, a hard drive controller,
and an input/output controller coupled by a processor bus.
- 43 -
CA 02569798 2013-07-09
It will be apparent to those skilled in this art that various
modifications and variations may be made to the embodiments
disclosed herein, consistent with the present.
Other embodiments consistent with the present invention will
become apparent from consideration of the specification and the
practice of the invention disclosed therein.
Accordingly, the specification and the embodiments are to be
considered exemplary only, with a true scope and spirit of the
invention being disclosed by the following claims.
-44-
