Note: Descriptions are shown in the official language in which they were submitted.
CA 02407918 2002-10-11
METHOD AND APPARATUS FOR IMAGING USING POLARIMETRY AND
MATRIX BASED IMAGE RECONSTRUCTION
FIELD OF THE INVENTION
This invention relates generally to a method of obtaining images of objects
with optical systems using polarimetry and matrix based image reconstruction
of
the object, and more particularly the invention relates to obtaining images
with a
scanning laser optical system using Mueller-matrix polarimetry. This invention
io also relates to a method of obtaining images of the eye using scanning
laser
ophthalmoscopy in combination with Mueller-matrix palarimetry.
BACKGROUND OF THE INVENTION
Image quality in any imaging system can often be limited by noise
is including speckle noise in coherent illumination, or by lower resolution or
a lack
of contrast due to scattered Light or light from secondary light sources.
For more than four decades, confocal scanning laser microscopy has
been used successfully to analyze samples in many diverse fields, ranging from
biology to the characterization of materials2: Webb and co-workers3 presented
2o the confocal scanning laser ophthalmoscope for viewing the ocular fundus,
using
the ocular optics as a microscope objective. Since the optics of the eye
degrade
the image, additional improvements have been made to fundus imaging, such as
adaptative optics4, deconvolution techniques5 or changes in the beam diameter
and its entry position in the pupil of the eye6.
CA 02407918 2002-10-11
The polarization properties of light have been used in conjunction with
imaging techniques in target detection', optical coherence tomographys~9,
ophthalmologic diagnosis'°, remote sensing~~ and microscopy~2. In
general,
optical imaging with polarization has been reported to improve contrast,
reduce
s noise and provide useful information about scenes (nofi available with
polarization-blind imaging). Structural information (for example nerve fiber
layer
thickness'°) obtained from the polarization properties is also useful.
Confocal scanning laser ophthalmoscopy and scanning laser
ophthalmoscopy are used for the diagnosis of eye diseases and disorders that
to affect structures at the rear of the eye and for basic scientific and
biomedical
investigations of these structures. Confocal scanning laser microscopy is used
to
characterize many materials and for biomedical investigations, including the
diagnosis of disorders and diseases of the cornea of the eye. Major
limitations to
the recognition of features Limit diagnosis and evaluation of structures
viewed in
is confocal scanning laser ophthalmoscopy and in confocal scanning laser
microscopy. These limitations include pixel to pixel noise, lowered contrast
and a
lack of resolution. A lowering of contrast and an increase in the size of the
features resolved is partly due to the imperfect optical quality of the
objective that
in the case of ophthalmoscopy is the eye itself. Noise can be increased due to
2o imperfect optics or due to a lower signal reflected from the structures
being
observed, reducing the signal to noise ratio. However, these reductions in
contrast, resolution and signal to noise ratio are a function of the
polarization
properties of the features being imaged.
2
CA 02407918 2002-10-11
Therefore, it would be very advantageous to provide a method which
provides improved image contrast, image resolution and the signal to noise
ratio
of a given image.
SUMMARY OF THE INVENTION
This is achieved in part by providing a method c>f obtaining images of an
object where the object is illuminated by incident beams) of selectively
polarized
light and the images reflected (or transmitted) for each different incident
beam
polarization is recorded using methods which resolve individual image points
to from the object. Matrix methods are used to reconstruct multiple images
from the
recorded image signals and the best image selected.
In one aspect of the present invention there is provided a method for
producing images of an object or region of interest of the object, comprising
the
steps of:
is a) producing an incident beam of light in a pre-selected polarization state
and scanning said incident beam of light point by point across an object or
region
of interest of the object;
b) detecting light intensity signals corresponding to beams of light in a pre-
selected number of polarization states reflected point by point from the
object or
20 region of interest of the object and storing electronic signals
corresponding to the
detected light intensity signals;
c) repeating steps a) and b) for an effective number of pre-selected
polarization states of the incident beam of light;
CA 02407918 2002-10-11
d) constructing a spatially resolved matrix of the object point by point from
the detected light intensity signals and from said spatially resolved matrix
constructing spatially resolved images of the object or region of interest of
the
object for a set of theoretical polarization states of the incident beam of
light in
addition to those input states generated in the incident beam of light, said
matrix
being selected to describe the effect of the object on the polarization
properties
of light;
e) characterizing image quality of each image in accordance with an
effective image quality parameter and based upon said characterization
selecting
to a best image of said object or region of said object; and
f) visually displaying said best image.
The present invention also provides a method for producing images of an
object or region of interest of the object, comprising the steps of:
is a) producing an incident beam of light in a pre-selected polarization state
and scanning said incident beam of light point by point across an object or
region
of interest of the object;
b) detecting light intensity signals corresponding to beams of light
reflected point by point from the objecf or region of interest of the object
and
2o storing electronic signals corresponding to the detected light intensity
signals;
c) repeating steps a) and b) for an effective number of pre-selected
polarization states of the incident beam of light;
4
CA 02407918 2002-10-11
d) constructing a spatially resolved vector of the object point by point from
the detected light intensity signals and from said spatially resolved vector
constructing spatially resolved images of the object or region of interest of
the
object for a set of theoretical polarization states of the incident beam of
light in
addition to those input states generated in the incident beam of light, said
vector
comprised of independent elements of a matrix being selected to describe the
effect of the object on the polarization properties of light;
e) characterizing image quality of each image in accordance with an
effective image quality parameter and based upon said characterization
selecting
io a best image of said object; and
f) visually displaying said best image.
In another aspect of this invention there is provided a method for
producing images of an object or region of interest of the object, comprising
the
is steps of:
a) producing an incident beam of light in a pre-selected polarization state
and illuminating an object or region of interest of the ot~ject with the
selectively
polarized beam of light;
b) detecting an array of light intensity signals reflected from spatially
2o distinct points of the object or region of interest of the object and
storing
electronic signals corresponding to said detected array of light signals;
c) repeating steps a) and b) for an effective number of pre-selected
polarization states of the incident beam of light;
CA 02407918 2002-10-11
d) constructing a vector comprised of independent elements of a spatially
resolved matrix of the object point by point from the detected light intensity
signals and from said spatially resolved vector constructing spatially
resolved
images of the object or region of interest of the object for a set of
theoretical
polarization states of the incident beam of light in addition to those input
states
generated in the incident beam of light, said matrix being selected to
describe the
effect of the object on the polarization properties of light;
e) characterizing image quality of each image in accordance with an
effective image quality parameter and based upon said characterization
selecting
io a best image of said object; and
f) visually displaying said bet image.
In another aspect of the invention there is provided a method for
producing images of an object or region of interest of the object, comprising
the
is steps of:
a) producing an incident beam of light in a pre-selected polarization state
and illuminating an object or region of interest of the object with the
selectively
polarized beam of light;
b) detecting an array of light intensity signals reflected from spatially
2o distinct points of the object or region of interest of the object and
storing
electronic signals corresponding to said detected array ofilight signals;
c) repeating steps a) and b) for an effective number of pre-selected
polarization states of the incident beam of light;
6
CA 02407918 2002-10-11
d) constructing a spatially resolved matrix of the object from the detected
light intensity signals and from said spatially resolved matrix constructing
spatially resolved images of the objecfi or region of interest of the object
for a set
of theoretical polarization states of the incident beam of light in addition
to those
s input states generated in the incident beam of light, said matrix being
selected to
describe the effect of the object on the polarization properties of light;
e) characterizing image quality of each image ire accordance with an
efFective image quality parameter and based upon said characterization
selecting
a best image of said object; and
io f) visually displaying said bet image.
The present invention also provides a method for producing images of an
object using confocal scanning laser microscopy, comprising the steps of:
a) calibrating a confocal scanning laser microscope modified to include a
is polarization generator and a polarization analyzer to obtain a Mueller
matrix
MscN , of the instrument in the incoming direction, wherein a matrix of 16
intensity
values results from intensity measurements with a rotating % wave plate
located
in said generator positioned in each of four positions including 45 degrees, 0
degrees, 30 degrees and 60 degrees, while a'/ wave plate located in said
2o analyzer is placed in each of the same four positions;
b) calibrating said modified confocal scanning instrument to obtain a
Mueller matrix MS N , of the instrument in the outgoing direction, wherein a
matrix
of 16 intensity values results from intensity measurements with a rotating'/
wave
CA 02407918 2002-10-11
plate located in said generator positioned in each of four positions including
45
degrees, 0 degrees, 30 degrees and 60 degrees, while a % wave plate located in
said analyzer is placed in each of the same four positions;
c) placing said object in said modified confocal scanning apparatus and
s focusing light onto said object and recording sixteen gray scale images with
the
object in place for each of four generator states with a % wave plate at 45,
0, 30
and 60 degrees combined with each of the four analyzer states 114 wave plate
at
45, 0, 30 and 60 degrees;
d) placing said sixteen grey scale values for each pixel into a spatially
io resolved matrix, I~"'"~, which is a first element of a Stokes vector, SD n~
reaching
the photodetector;
e) from I~"'"~ calculate Mout from equation 2;
f) from equation 3, calculate M, the spatially resolved Mueller matrix of the
object;
is g) choosing values of an incident Stokes vector, S,N, around a Poincare
sphere in predetermined increments of x and cp which represent, respectively,
the
azimuth and ellipticity of the incident Stokes vector on 'the Poincare sphere;
h) applying equation 4 to reconstruct images, I~°"t~, pixel by pixel
for each
incident Stokes vector;
20 i) for each image, calculate the image quality measure of choice, for
example SNR as defined in equation 5; and
j) display the image with best value of the image quality measure.
s
CA 02407918 2002-10-11
The present invention also provides an optical scanning apparatus for
producing images of an object, comprising:
a) a light sours for producing a light beam;
b) polarization generator means for producing selected polarization states
s in the light beam upon passage of the light beam through said polarization
generator means to produce a selectively polarized light beam;
c) scanning means for receiving the selectively polarized light beam and
spatially scanning the selectively polarized light beam in two dimensions
across
an object point by point;
io d) polarization analyzer means for transmitting light beams of selected
polarization, including directing and focusing optics for directing the
reflected light
beams reflected point by point from the object to said polarization analyzer
means;
e) detection means and light shaping and focusing means for directing
is and focusing the reflected light beams of selected polarization onto said
detection means;
f) computer processing means connected to said detection means, said
computer processing means including image analysis means for processing
signals from said detector due to the reflected light beams of selected
2o polarization detected by said detection means and producing therefrom
images
of the object; and
g) display means for displaying an image of the abject produced by said
processing means.
9
CA 02407918 2002-10-11
BRIEF DESCRLPTfON OF THE DRAWINGS
Preferred embodiments of the invention will now be described, by way of
example only, with reference to the drawings, in which:
Figure 1 shows a block diagram of an apparatus used to perform the
s method of the present invention in which P1 and P2, linear horizontal
polarizers;
QWP1 and QWP2, rotating quarter-wave plates wherein both microscope
configuration (viewing the object) and ophthalmoscope configuration (viewing a
person's eyeball) are shown;
Figure 2 shows elements of the first row of the spatially resolved Mueller
to matrix for a U.S.A.F. chart (4.4 mm square);
Figure 3 shows elements of the first row of the spatially resolved Mueller
matrix for a retinal region (2 degrees) for one sample subject's fundus, with
the
gray level code being shown at the right;
Figure 4 shows results for the target in Figure 2, Figure 4(a) shows the
is worst reconstructed image, Figure 4(b) shows the best original image,
Figure
4(c) shows the best reconstructed image; and
Figure 5(a) shows the worst reconstructed image, Figure 5(b) shows the
best original images, and Figure 5(c) shows the best reconstructed images for
the fundus image in Figure 2 and an second sample subject (a 3 degree field).
io
CA 02407918 2002-10-11
DETAILED DESCRIPTION OF THE INVENTION
Broadly, the present invenfiion provides a method of obtaining images of
an object where the object is illuminated by incident beams) of selectively
polarized light and the images reflected (or transmitted) for each different
incident
s beam polarization is recorded using methods which resolve individual image
points from the object. Matrix methods are used to reconstruct multiple images
from the recorded image signals and the best image selected.
The method may be implemented using an optical scanning system such
as for example a scanning laser system. In one embodiment the method and
io apparatus use Mueller-matrix polarimetry to reduce noise and improve images
recorded with the optical scanning system.
The image improvement includes improvements in the signal-to-noise
ratio, an improvement in contrast across local features and an improvement in
the resolution of features (visibility of small details).
ns Since different objects or different regions of interest of an object have
different polarization properties, the analysis gives an improved image
corresponding to a different incoming polarization state on the Poincare
sphere
dependent on these properties.
Improvements have been shown for both specularly and diffusely
2o reflecting objects and for an object (the fundus) which produces a
combination of
diffuse, specular and directional reflections.
The analysis disclosed hereinafter is spatially resolved, that is, it is
performed on a pixel by pixel basis; so that the improvements in images of
n
CA 02407918 2002-10-11
different areas of the object depend on the local polarization properties of
the
object, and the calculation described herein may produce a different best
image
of each object area of interest.
The analysis depends on the measure of best image quality used, so that
s the best image produced may depend on the measure used. However, in the
examples given below, a signal to noise (SNR) measure across a large area of
the image gives the best image defined by the signal to noise ratio measure
used. It also produced images with higher contrast of local features and
higher
resolution of fine detail. So initially, a SNR measure appears to be the best
image
to quality measure.
The method can be used to give the best image quality for other image
quality measures including those that combine contrast, SNR and resolution
measures if necessary.
One embodiment of the method is based on the incorporation of a
is polarimeter into a laser scanning system combined with a specialized
spatially
resolved calculation and image display. After calculating the spatially
resolved
Mueller matrix of a sample, images for incident light with different states of
polarization are.reconstructed with increments chosen of 1 degree for azimuth
and ellipticity over the Poincare sphere. The increments over the Poincare
2o sphere chosen in the calculation could be varied. The calculation generates
images for incoming polarization states that could not be generated in the
experimental imaging system. The best computed image in both microscope and
12
CA 02407918 2002-10-11
ophthalmoscope modes, are better than any of the original images recorded with
polarimetry. In contrast, the worst computed images are poorer.
Referring to Figure 1, a schematic diagram of a confocal scanning
microscope modified to include a polarimeter ~3 is shown generally at 10 with
the
s polarimeter comprising a generator unit 12 which includes a fixed linear
polarizer
P1 and a rotating quarter-wave plate QWP1. The apparatus includes an analyzer
14 in a symmetric configuration with respect to the generator 12 which
comprises
a fixed linear polarizer P2 and a rotating quarter-wave plate QWP2 in the
analyzer unit. The system 10 may be used in both microscope and
to ophthalmoscope modes. In the microscope mode, the focal length and the
numerical aperture for the objective lens 16 was 90 mm and 0.11 respectively
but
it will be understood that different objective lenses may be used. In the
ophthalmoscope mode; the patient's eye itself acted as a microscope objective.
A 633-nm He-Ne laser 18 is used as the light source and a photo-multiplier
tube
is as the detector but those skilled in the art will appreciate that other
light sources
or photodetectors may be used, either in a modified commeroial microscope or
commercial ophthalmoscope or in a customized design.
An X Y scanning system 30 permits the Gght beam to be scanned across
the sample or inside or outside of the eyeball of the patient in a raster. Any
2o commercially available raster scan system, or application specific system
or any
novel system may be used including resonance scanners, galvonometer
scanners, oscillating mirrors, acosto-optic deflectors, solid state
deflectors, single
facet or polygon rotating mirrors, holographic scanners or micro-elctro
13
CA 02407918 2002-10-11
mechanical scanners. The beam splitter BS directs information bearing light
beam reflected from the sample or eyeball through the collector lens 28
through
confocal pinhole 26 onto the detector 22.
The laser beam is scanned in two dimensions and focused on
s the sample (a target or the retina) by the objective or the ocular. The
light
reflected back from the sample at each point of the scan is recorded by the
detector 22. In studies conducted by the inventors the size of the light beam
entering the objective 16 {and the eye) was 2.5 mm and the confocal pinhole 26
was 600 microns in diameter. The focal length of the collector lens 28 was 50
to mm. The system records images at a rate of 28.5 Hz. However it will be
appreciated that these variables are exemplary only and may be varied by those
skilled in the art.
The sixteen (16) combinations of polarization states required to calculate
the Mueller matrix for each point of the scanned sample correspond to
different
is angles of the fast axes of the two rotating quarter-wave plates QWP1 and
QWP2
as previously described'4. The generator 12 and analyzer 14 may be connected
to computer controlled mechanical actuator system (not shown) for moving the
quarter wave plates QWP1 and QWP2 into the four different positions.
Alternately, fast electro-optical devices {including but not limited to liquid
crystals,
2o photoelastic modulators) could be used to vary the polarization states of
the
generator and analyser'5.
The system is calibrated by taking a measurement in the incoming
pathway. This will give the Mueller matrix , MS's" , of the instrument in the
14
CA 02407918 2002-10-11
incoming direction. A matrix of 16 intensity values results from intensity
measurements with the generator's rotating'/ wave plate in each of four
positions -45 degrees, 0 degrees, 30 degrees and 60 degrees, while the
analyzer's'/ wave plate is placed in each of the same four positions.
s The outgoing direction of the instrument is calibrated. This will give the
Mueller matrix, NIs N , of the instrument in the outgoing direction. A matrix
of 16
intensity values results from intensity measurements with the generator
quarter
wave plate QWP1 in each of four positions described previously while the
analyzer quarter wave plate QWP2 is placed in each of the same four positions.
io Sixteen (16) gray scale images are taken with the object in place for each
of the four generator states (quarter wave plate QWP1 at 45, 0, 30 and 60
degrees) combined with each of the same four analyzer states (quarter wave
plate QWP2 set at 45, 0, 30 and 60 degrees while QWP1 is in each of its fours
states). The generator produces linearly polarized light at a particular
orientation,
is set by the QWP1 and linear polarizer P1. The analyser passes light of a
particular polarization set by the QWP2 and linear polarizes and blocks the
rest.
The 16 grey scale values for each pixel are placed into a spatially resolved
matrix, I~"'"~ which is the first element of the Stokes vector, SD "~ reaching
the
photodetector.
2o For every Stokes vector S~m~ (m=1, 2, 3, 4) produced by the generator, the
intensity reaching the detector for each point of the sample (I~"'"~), is the
first
element of the Stokes vector SDrnn) (n_1, 2, 3, 4) given by:
SDmn) _ ~If An? ' MsZCw ' ~ ' Mslc~av ' s'c'~~ 1
()
is
CA 02407918 2002-10-11
where M=M;~ (i, j=0, 1, 2, 3) is the Mueller matrix of the sample under study,
MA")
is one of the four Mueller matrices of the analyzer unit (each corresponding
to an
independent state), and MscN and Mss are the Mueller matrices of the
experimental system itself (lenses, scanning unit and beam splitter) for first
and
s second passages respectively, previously measured in the calibration
process.
For each generator-analyzer combination, the image is spatially resolved,
giving
a spatially resolved M. Let Mq be the 4x4 auxiliary matrix with each row being
the
first row of every MAn) and Mp~7=[So'~ , So2~ , So3~ , SoUT J be another
auxiliary
matrix with Sour each Stokes vector going into the analyzer unit. These
matrices
to are then related:
1(i_1)1(2y) I(3_,)1(4_1)
1(1_2)1(2_2)1(3_2)1(4_2)
1(1_3)1(2_3)1(3_3)I(4_3)MA ' MOUT
1(1_4)I(2_4)I(3_4)1(4_4)
If M~=[S~') , S~2) , S~3) , SG ) ~, then M is computed by means of:
M - ~MscN ~ i ' MoUT ' ~Mc ~ 1 ' ~Msc v ~ ~ (3)
where MouT is obtained by inversion of equation (2).
is From the spatially resolved Mueller matrix, images of the sample
1(°~) for
any in-coming polarization state SEN can be obtained by:
(ouT)
1 M~ Mol Mo2 Mo3
Si oUT)Mlo Ml M~2 M13 cos(2x~ ~
l cos(2~p~
_
SZOrrr)M2o M21 M22 Mz3 sin~2,~~~ M ~'Sw
cos(2~p~
5300 M30 M31 M32 M33 sin(2g~~
(4)
16
CA 02407918 2002-10-11
where x and cp represent, respectively, the azimuth and ellipticity of the
incident
Stokes vector on the Poincare sphere'B. Using equation 4, we can determine the
Stokes vectors that produce images with both best and worst quality. This
quality
is defined below. This equation gives all output polarization properties. Here
we
consider only the image intensity, 1~°"T~ , to which only the four
elements of the
first row of the Mueller matrix contribute.
One parameter that may be used to characterize image quality is the
speckle noise (SN) or the inverse of the signal-to-noise ratio (SNR) (often
used to
describe speckle') defined as the ratio between the standard deviation and the
io mean intensity across the whole image:
SN = (SNR~-' __ stdv(F~~ura )
mean(I t~"'~ ) (5)
Other measures of image quality are possible including contrast across a
local feature of the image, the size of the smallest features resolved in the
image,
or measures which are combinations of SN, contrast and resolution including
is Fisher's linear discriminant.
The detailed steps followed to obtain improved images according to the
present invention using the preferred Mueller matrix methodology are as
follows.
1 ) Calibrate system 10 in Figure 1 by taking a measurement in the incoming
pathway. This will give the Mueller matrix , MS~N , of the instrument in the
20 incoming direction. For this measurement the analyzer is moved to the usual
position of a sample. An intensity detector is placed behind the analyzer. Any
scanning optics are turned off. A matrix of 16 intensity values results from
17
CA 02407918 2002-10-11
intensity measurements with the generator's rotating % wave plate in each of
four
positions --45 degrees, 0 degrees, 30 degrees and 60 degrees, while the
analyzer's'/ wave plate is placed in each of the same four positions.
2) Continue calibration. A mirror is placed at the plane of the sample. The
s analyzer and the intensity detector are moved behind the last optical
element of
the system. This will give the Mueller matrix , Ms2~.~,,, , of the instrument
in the
outgoing direction. Any scanning optics are turned off: A matrix of 16
intensity
values results from intensity measurements with the generator's °/ wave
plate in
each of four positions -45 degrees, 0 degrees, 30 degrees and 60 degrees,
while
io the analyzer's % wave plate is placed in each of the same four positions.
3) Take the intensity detector out of the system and turn the scanners on.
Record
the 16 gray scale images with the object in place for each of the four
generator
states ('/ wave plate at 45, 0, 30 and 60 degrees) combined with each of the 4
analyzer states (1/4 wave plate at 45, 0, 30 and 60 degrees). The 16 grey
scale
is values for each pixel are placed into a spatially resolved matrix, Itmn)
which is the
first element of the Stokes vector, SD""'~ reaching the photodetector.
4) From I~"'~~ calculate Mo"t from equation 2.
5) From equation 3, calculate M, the spatially resolved Mueller matrix of the
object.
20 6) Choose values of the incident Stokes vector, S,N, around the Poincare
sphere
in predetermined increments of x and cp which represent; respectively, the
azimuth and eHipticity of the incident Stokes vector on the Poincare sphere~6.
rs
CA 02407918 2002-10-11
Applying equation 4, reconstruct images, p"t~, pixel by pixel for each
incident
Stokes vector.
7) For each image, calculate the image quality measure of choice, for example
SNR as defined in equation 5.
s 8) Display the image with best quality.
It is noted that the calibration in a) and b) just above may reduce the effect
of the instrument on the final polarization properties of the image. However,
it is
possible to perform the method without calibration. In this case, the steps
following b) are identical. The Mueller matrix derived is the matrix
corresponding
to to the object plus the instrument.
When system 10 is operated in microscope mode, spatially resolved
Mueller matrices were calculated for two different samples: a U.S.A.F.
resolution
chart (primarily specular reflections) and a grey scale image on white paper
(primarily diffuse reflections, not shown here). Figure 2a shows the spatially
is resolved elements of the first row for the Mueller matrix corresponding to
the
USAF target. The averaged degrees of polarization were 0.87 (nearly Specular)
and 0.18 (highly depolarizing) for this target and the diffuse reflection
respectively. In the ophthalmoscope mode we applied the procedure to retinal
images recorded in a living human eye. In Figure 3 we show the first row of
the
2o Mueller matrix for a retinal fundus region (with blood vessels).
Using these matrices, images were reconstructed for incident light with
Stokes vectors with increments of 1 degree for azimuth and ellipticity over
the
Poincare sphere. Images were obtained for incoming polarization states that
19
CA 02407918 2002-10-11
could not be generated in an experimental system. For each matrix, both the
best and the worst images were reconstructed using SNR as the measure of
quality and the associated Stokes vectors calculated. The higher SNR shows
that the best image calculated is better than the best measured with or
without
s the polarizer in place.
Figure 4 shows the results obtained for the specular reflection in
microscope mode. The best (c) and the worst reconstructed images (a) as well
as one of the original images are shown (a). Results for the retinal fundus
image
of figure 3 are presented on the left in Figure 5. On the right in Figure 5
results
io for the retinal image of a second subject are shown. The original images
presented are the best (lowest speckle noise or highest signal-to-noise ratio)
of
the images experimentally recorded (b). The worst reconstructed images are
shown in (a). The improvement in the images obtained using this method was
noticeable in all cases. Lower noise as well as higher contrast across
features
is and higher resolution are seen in the best reconstructed image (c).
Resolution
improvement is demonstrated by the fact that more structural details and small
features which are not discernible in the original images can be also
observed.
The improvement seen is better than that for frame averaging. Differences in
the
signal to noise as defined in equation 5 between the original and the best
images
2o were 48% for the specular target (70% forthe diffuse target) in microscope
mode
and 45% for the retinal fundus image of Figure 4. The Stokes vectors for the
best
specular image in figure 4 was [1, -0.565, -0.099, -0.819]T ([1; 0.220, 0.604,
0.766]T for the diffuse reflection) arid those corresponding to the optimal
retinal
CA 02407918 2002-10-11
image were [1, 0.719, 0.262, 0.64~jT for a subject measured in Figure 3 and
[1, -
0.969, 0.171, 0.174]T for the second subject with results in Figure 5.
Moreover,
an increase of up to 30% was found in the contrast across the blood vessels of
the subjects presented here.
Improvements in image quality using the present invention have been
obtained for two different confocal scanning laser imaging systems, both
indicated in Figure 1. These include a confocal scanning laser microscope and
a
confocal scanning laser ophthalmoscope (where the optics of the eye is the
final
imaging element before the object of interest (the fundus)). Improvements are
to obtained whether or not a confocal pinhole 26 is used in front of the
hotodetector
and regardless of what size of confocal pinhole is used. Improvements are also
obtained for any wavelength used.
Thus, it will be appreciated by those skilled in the art that the confocal
imaging system shown in Figure 1 is meant to be a non-limiting example of an
is apparatus constructed in accordance with the present invention. The
confocal
imaging system shown in Figure 1 is one that has pinhole 26 conjugate to the
object plane of interest so that apparatus 10 excludes light reflected from
the
object which is not in-focus on the detector 22. This leads to improved
contrast
because of the exclusion of scattered light in addition to the ability to
resolve
2o structures in depth and to reconstruct three-dimensional images is
enhanced. If
depth resolution is important, then system 10 preferably should be used with
confocal pinhole 26. However, scanning laser ophthalmoscopy and microscopy is
performed with a larger pinhole in place of the confocal pinhole or with other
21
CA 02407918 2002-10-11
specialty apertures or without a pinhole and the present invention maybe
implement using an apparatus absent the confocal pinhole using the
methodology described herein which also gives improved image quality.
Similarly, the light source may be an incoherent light source or it may be a
s laser producing coherent light beams as light beams with the different
polarization states can be produced. Diode lasers producing partially coherent
light beams may also be used. Also, Instead of using the fixed linear
polarizer P1
in the generator 12 in apparatus 10 of Figure 1, a Fight source with intrinsic
linear
polarization may be used.
to In this description, scanning the beam iluminating the object which is then
descanned allows a point detector to be used and the image to be recreated
pixel by pixel using timing signals. If the beam is scanned and the light
transmitted by an object is recorded, a stationary single detector cannot be
used.
In this situation, moving single detector or moving linear array synchronized
to
is the scanning beam or an area an-ay of detectors with an exposure equivalent
to 1
frame of the scan (e.g. a CCD array) may be used to record the image. An area
of the object may be illuminated with a scanning beam and reflected images
without descanning could be recorded using a moving single detector or a
moving linear array of detectors synchronized to the scanning beam or an area
2o array of detectors with an exposure equivalent to 1 frame of the scan (eg a
CCD
array) could be used to record the image. If a stationary beam were used to
illuminate the object, any detectorwhich;allowed spatial resolution of the
image
could be used including a single moving detector, a moving linear array of
22
CA 02407918 2002-10-11
detectors or an area array of detectors (e.g. a CCD array). The pixelated
image
that results from any of the above configurations could then be analyzed as
described herein.
Although in the invention disclosed herein the inventors have calculated
s the sixteen elements of the Mueller matrix, just four of them (first row)
are used to
calculate the final improved image. Therefore, an additional methodology based
on the calculation of just these four elements instead of sixteen may result
in the
same improved images.
Therefore, for each independent polarization 'state emerging from the
io generator unit S~'~ (i=1, 2, 3, 4), the Stokes vector associated with the
light
reaching the recording state, SF'' , is given by:
SF's = MT ~ S~'~ = M ~~. - M ~ M (s sT - Sc'~ ( 1 b)
where MT=m"~ ~ (I, m = 0, 1, 2; 3), is the Mueller matrix of the complete
system
and M is the Mueller matrix of the sample under study. Ms'~T and M~~T are the
is Mueller matrices obtained from then calibration procedure.
Let Mp be the 4 x 1 column vector which elements are the first row of the
Mueller matrix MT in a transposed position. When four independent polarization
states, SG'~ (i = 1, 2, 3, 4) are incident, then it is verified:
Soc Sic sic S3c mood ~5~1~~T mood
(2~ (2~ (2) (2~ (T~
I2 SOG SIG S2G S3G m01 (SG2) ~T m01 )
13 .SOG S1G S2G S3G 11102 (SG3~ ~ m02) MG MO
I4 SOG S1G~ S2G S3G m03T ? SG4y T m03 )
23
CA 02407918 2002-10-11
where Ii (i = 1, 2, 3, 4) corresponds to the recorded images for each
independent
polarization states S~'> .
Finally Mp is obtained by inversion of equation (2b):
~o = ~M~ ~ 1 ' IF (3b)
s From the spatially resolved vector Mo, images of the sample 1~ for any in-
coming polarization state SAN can be obtained by:
1
(T) (T) (T) (T) ), ~s~2~,'~ ~ Cos~~~ __ z'
IF~aL = moo moi moa mo3 sin~2;'~~~cos~2~p~
san~2~p}
where x and cp represent, respectively, the azimuth and ellipticity of the
(4b)
io incident Stokes vector on the Poincare sphere and Mo is the transposed
vector
of Mo. Using equation 4b, we can determine the Stokes vectors that produce
images with both best and worst quality using the chosen measure of image
quality.
The detailed steps followed to obtain improved images according to the
~s present invention using apparatus 10 in Figure 1 with the rotatable
polarizers are
as follows. The optical scanning system (shown in Figure 10) is calibrated as
described above
a) calibrating the confocal scanning laser system (microscope or
ophthalmoscope) modified to include a polarization generator and a
polarization
20 analyzer to obtain a Mueller matrix ~1~~,, , of the instrument in the
incoming
direction, wherein a matrix of 16 intensity values results from intensity
24
CA 02407918 2002-10-11
measurements with a fixed linear polarizer and a rotating'/ wave plate located
in
said generator positioned in each of four positions including -4.5 degrees, 0
degrees, 30 degrees and 60 degrees, while a fixed linear polarizer and a'/4
wave
plate (symmetric configuration with respect to the generator) located in the
s analyzer is placed in each of the same four positions; in order to measure
the
above cited matrix, the analyzer and an intensity detector are placed in the
place
of the sample;
b) calibrating the confocal scanning instrument to obtain a Mueller matrix
M~,~, of the instrument in the outgoing direction, wherein a matrix of 16
intensity
io values results from intensity measurements with a fixed linear polarizer
and a
rotating'/ wave plate located in said generator positioned in each of four
positions including -45 degrees, 0 degrees, 30 degrees and 60 degrees, while a
fixed linear polarizer and a % wave plate (symmetric configuration with
respect to
sand generator) located in said analyzer is placed in each of the same four
is positions; in this case a mirror is placed in place of the sample and the
analyzer
unit and the intensity detector are placed in the confocal detection arm;
c) analyzer and detector are removed from the recording pathway and just
the generator is used for the new method.
d) placing the object in the confocal scanning apparatus and focusing light
20 onto the object and recording four gray scale images with the object in
place for
each of four generator states with a'/ wave plate at 45; 0, 30 and 60 degrees;
2s
CA 02407918 2002-10-11
e) placing the four grey scale values for each pixel into a spatially resolved
vector, IF, which elements are the first element of the four SF's reaching the
photodetector;
f) from equation 3b calculate Mo, the spatially resolved auxiliary which
s elements are the first row of the total Mueller matrix, MT;
g) choosing values of an incident Stokes vector, S,N, around a Poincare
sphere in predetermined increments of ~ and cp which represent, respectively,
the
azimuth and ellipticity of the incident Stokes vector on the Poincare sphere;
h) applying equation 4b to reconstruct images; I~~NAL, pixel by pixel for each
io incident Stokes vector;
i) for each image, calculate the image quality measure of choice; and
j) display the image with best value of the image quality measure.
It is noted that the calibration in a) and b) just above may reduce the effect
is of the instrument on the final polarization properties of the image.
However, it is
possible to perform the method without calibration. In this case, the steps
following b) are identical. The first rover derived is the first row of the
Mueller
matrix of-the object +instrument).
In situations where the state of polarization of the input beam cannot be
2o controlled (e.g. astronomical observations), or in situations where the
output
illumination is intrinsic to the sample (e.g. fluorescence microscopy), this
shortened method just described may be modified to use a single input
polarization state and to sample the spatially resolved image for 4 output
26
CA 02407918 2002-10-11
polarization states. Elements of the first row of the Mueller matrix can be
calculated from equations 2b and 3b. Images corresponding to any input
polarization state are then obtained from equation 4b). The detailed
methodology is as follows:
s
Steps a) and b)- calibration as before and is optional:
c) generator and detector are removed from the recording pathway and
just the analyzer is used for the new method.
to d) placing said object in said modified confocal scanning apparatus and
focusing light onto said object and recording four gray scale images with the
object in place for each of four analyser states with a '/4 wave plate at 45,
0, 30
and 60 degrees;
e) placing said four grey scale values for each pixel into a spatially
is resolved vector, IF, which elements are the first element of the four SF's
reaching
the photodetector;
f) from equation 3b calculate Mo, the spatially resolved auxiliary which
elements are the first row of the total Mueller matrix, 11IIT;
g) choosing values of an incident Stokes vector, S,N, around a Poincar~
2o sphere in predetermined increments of x and cp which represent,
respectively,
the azimuth and ellipticity of the incident Stokes vector on the Poincare
sphere;
h) applying equation 4b to reconstruct images, IFINAL, pixel by pixel for each
incident Stokes vector;
i) for each image, calculate the image quality measure of choice; and
2s j) display the image with best value of the image quality measure.
27
CA 02407918 2002-10-11
It will be understood that the present methodology is not restricted to the
calculation of 4 x 4 Mueller matrices from combinations of 4 incoming and 4
outgoing polarization states in-the first method discussed above or to the
first row
of the Mueller matrix or to one input and four output beams in the second
method
s disclosed above. It is noted that any other matrix which describes the
effect of
the object on the polarization properties of light may also be used. Thus,
while
Mueller matrices are always 4 x 4 it is possible to use more combinations of
beam in- and beam out- polarization states.
When Mueller matrices are used, once the spatially resolved 4 x 4 Mueller
to matrix of the object is consfiructed from the detected light signals one
constructs
spatially resolved images of the object for a set of theoretical polarization
states
of the incident beam of light in addition to those input states actually
generated in
the incident fight beams. The images are calculated point by point for a large
number of polarization states (sampled all around the Poincare sphere in 1
is degree steps in one example implementation}: These polarization states can
each be described by a Stokes vector where the Stokes vector characterizes the
polarization of the input light beam. The first element of the vector gives
the
intensity of the beam, the second element gives the degree of vertical or
horizontal polarization, the third element of the Stokes vector gives the
degree of
20 +45 or-45 linear polarization and the fourth element of the Stokes vector
gives
the degree and direction of circular polarization. So for each image
constructed,
one uses the Mueller matrix which has been calculated point by point and a
Stokes vector to generate an image point by point. Once the calculation has
28
CA 02407918 2002-10-11
been done for a chosen number of polarization states, then the best image is
chosen. The calculation would normally include four orthogonal or independent
polarization states that the generator unit is designed to produce as well as
a set
of theoretical input polarization states that are not easily produced
s experimentally.
In conclusion, the present invention demonstrates the use of Mueller-
matrix polarimetry for improving the quality of confocal scanning microscopy
and
ophthalmoscopy images. In another embodiment of the invention the optical
system may use fast electro-optical modulators such as liquid-crystal variable
to retarders or photo-elastic modulators: The polarization state which gives
the best
improvement in image quality differs for the specular and diffuse reflections
in
microscope mode and for the two analyzed subjects in ophthalmoscopic mode.
In general as in imaging with polarized light'8, the Stokes vector
corresponding to
the best image may vary with the characteristics of the object being measured.
is To implement the technique; the improvement in the image may be rapidly
calculated and displayed in software. The best image may be calculated with
respect to a region of the image of interest. An implementation of this
technique
in commercially available microscopy and ophthalmologic
instruments'°''9 or in a
specialized instrument enhances fundus imaging and improves diagnosis
2o techniques.
As used herein, the terms "comprises" and "comprising" are to be
construed as being inclusive and open ended, and not exclusive. Specifically,
when used in this specification including claims, the terms "comprises" and
29
CA 02407918 2002-10-11
"comprising" and variations thereof mean the specified features, steps or
components are included. These terms are not to be interpreted to exclude the
presence of other features, steps or components.
The foregoing description of the preferred embodiments of the invention
s has been presented to illustrate the principles of the invention and not to
limit the
invention to the particular embodiment illustrated. It is intended that the
scope of
the invention be defined by all of the embodiments encompassed within the
following claims and their equivalents.
io REFERENCES
1. J. B. Pawley (Editor), Handbook of Biological Confocal Microscopy, 2"~ ed.
Plenum, New York (1995).
2. A. C. Ribes, S. Damaskinos, A. E. Dixon, K. A. Kellis, S. P. Duttagupta,
and
P. M. Fauchet, Progress in SurFace Science, 50, 295 (1995).
is 3. R. H. Webb, G. W: Hughes; and F. C. Delori, Appl. Opt. 26, 1492 (1987).
4. J. Liang, D. R. Williams, and D. T. MiNer, J. Opt. Soc. Am. A 14, 2884
(1997).
5. I. iglesias and P. Artal. Opt. Lett. 25, 1804 (2000).
6. K. Muth, M.C.W. Campbell, A. J. Roorda, and C. Cui, OSA Technical Digest
Series 1, 56-59 (1997).
20 7. M. P. Rowe, E. N. Pugh, Jr., J. S. Tyo, and N. Engheta, Opt. Lett: 20,
608
(1995).
8. Y. Gang and L. V. Wang, Opt. Lett: 24, 537 (1999).
9. S. Jiao Y. Gang and L. V: Wang; Appl. Opt. 39; 6318 (2000).
10.A. W. Dreher, K. Reiter, and R. N. Weinred, Appl. Opt. 31, 3730 (1992).
2s 11. W. G. Egan, W. R. Johnson, and V. S. Whitehead, Appl. Opt. 30, 435
(1991
12. W. Mickols, I Tinoco, J. E. Katz, M. F. Maestre, and C. Bustamente, Rev.
Sci.
Instrum. 12, 2228 (1985).
CA 02407918 2002-10-11
13. R. A. Chipman, in Handbook of Optics, 2nd ed., M. Bass, ed. (McGraw Hilf,
New York, 1995) Sec. 22.1.
14. J. M. Bueno and J. Jaronski; Opthaf: Physiol. Opt. 21, 384 (2001 ).
15. J. M. Bueno and P. Artal, Opt. Lett. 24, 64 (1999).
s 16. H. G. Jerrard, J. Opt. Soc. Am. 44, 634 (1954).
17. J. W. Goodman, in Laser Speckle and Related Phenomena, 2nd ed., J.C.
Dainty, ed., Vol. 9 of Topics in Applied Physics (Springer-Verlag,1984), 9-75.
18. J. M. Bueno and P. Artal, J. Opt. Soc. Am A 18, 489 {2001 ).
19. B. Pelz, C. Weschenmoser, S. Goelz, J. P. Fischer, R. O. W. Burk, and J.
F.
to Bille, Proc. SPIE 2930, 92 (1996).
31
