Note : Les descriptions sont présentées dans la langue officielle dans laquelle elles ont été soumises.
CA 02338675 2001-O1-26
WO 00/06980 PCT/US99/17004
SPECTRAL TOPOGRAPHY OF MAMMALIAN MATTER
FIELD OF THE INVENTION
This invention relates to clinical pathology and more particularly to systems
and
methods that assist in the automatic analysis and diagnosis of diseased cells
and tissue.
BACKGROUND OF THE INVENTION
Pathologists study samples of tissue and cells for the presence of
malignancies and
other diseases and abnormalities. As described below, recently, microscopy,
spectroscopy and
digital image processing, three traditionally distinct disciplines, have been
coalescing to result
in clinical pathology tools that more rapidly, automatically and accurately
assist the pathologist
to analyze and diagnose these conditions than was possible with conventional
microscopy
alone.
1. Microscopy
In one method of practicing conventional histopathology and cytopathology,
after
a biopsy of tissue or a smear of cells is removed from the suspect area of the
patient and
1 S prepared on one or more slides, the pathologist studies the specimen under
a light, or
transmission, microscope. Tissue biopsies are often sliced into very thin
sections and stained
using one of, or a combination of, well-known staining techniques, such as H
and E
(hematoxylin & eosin) staining. CeII smears are similar:Iy stained. Areas of
abnormality are
visually compared with representations of known disorders to determine whether
and what
disorders are presented.
The fluorescent microscope is another diagnostic tool often used by
pathologists.
Instead of passing incandescent, "white" light through the specimen, the
illuminating "high
energy excitation" light, from a source such as a Xenon or mercury lamp, is
passed through one
or more filter sets that passes only certain wavelengths of light onto (not
through) the specimen.
In response to the light incident upon it, the specimen, usually, but not
necessarily, stained, then
variously fluoresces, or produces lower-energy emissions of measurable
wavelengths (colors)
and intensities, thus providing diagnostically valuable information about the
tissue or cells.
Fluorescent microscopy techniques can be classified into two categories,
namely,
those that involve endogenous (or natural) fluorophores, or reactions, and
those that are
exogenous (originating from outside the specimen). The former includes two
types of
reactions, namely, autofluorescence, whereby tissue compounds, such as elastin
and collagen,
naturally fluoresce without any augmentation from outside the specimen, and
1
CA 02338675 2001-O1-26
WO 00/06980 PCT/US99/17004
immunofluorescence, whereby naturally fluorescing antibodies locate antigen in
the tissues by
combining with the specifc antigen for that antibody (antigen-antibody
reaction) and naturally
fluoresce upon such combination.
The latter implies fluorescent tagging, or labeling, of a cell. In this
technique, a
molecule, such as a peptide, protein or antigen, that attaches to a highly
specific target is
"tagged" with a fluorophore such as fluorscein in order to detect whether
there is a positive
interaction. In one method, for example, a fluorescein conjugated monoclonal
antibody is used
to identify a specific antigen. This technique is often used to assist in the
identification and
diagnosis of nuclear-based diseases, such as viruses, but may also be used to
identify bacterium
and malignant tissue, for example. Another technique in this category,
fluorescent in situ
hybridization ("FISH"), takes advantage of either a) a paired-nucleotide
interaction between a
labeled probe (the "antisense strand") and endogenous mRNA (the "sense
strand"), or b) a
protein-protein interaction, whereby proteins are labeled and incubated with
tissues that contain
target binding proteins or receptors. Exogenous reactiions also includes
general fluorescent
staining, such as the application of a auramine/rhodamine preparation to
smears suspected for
disease, such as tuberculosis. Such staining can highlight a specific feature
such as a nuclear
membrane from cytoplasm.
2. Image Processine
The increased processing power, speed and miniaturization of digital systems
and
computers have revolutionized the field of pathology. V~/hile the pathologist
previously relied
exclusively on the microscope and his or her own f;yesight arid experience to
diagnose
pathological disorders, current imaging and processing technologies have
greatly enhanced the
accuracy and speed with which today's pathologist may diagnose. For example,
the charge-
coupled device (CCD) array has been coupled to the microscope to enable the
high resolution
capturing of data representing microscopic images. These video images may be
digitized by
an analog-to-digital (A-D) converter, then displayed on a display, magnified
or otherwise
manipulated, and stored on other digital media. The video image data may also
be further
processed by a variety of image processing software. Neural networking, which
is a non-linear,
system that sorts out patterns from the data with which it is presented and
learns from
discerning and extracting the mathmenatical relationships that underlie the
data. is one such
processing structure. Applied to pathology, neural nework systems compare
specimen data
collected by a CCD array to learned patterns of data representative of healthy
and diseased
tissue and cells in order to automatically characterize and assess the
specimen for particular
disorders.
2
CA 02338675 2001-O1-26
WO 00/06980 PCT/US99/17004
3. Spectrosconv
Spectroscopy is the study of the spectral characteristics of objects, and more
particularly, the study of the component parts (individual wavelengths) of the
light of objects
and the intensity of those wavelengths. It has long been used as a tool in the
field of chemistry
for identifying elements since each element possesses it own unique spectra.
Since it was first
realized that it can provide detailed information about both the chemical and
physical nature
of matter, spectroscopy has shown great proriiise for the field of medical
diagnostics as well.
For-example, it has been demonstrated thatcertain spectraal characteristics,
such as fluorescence,
indicate the presence of a malignancy or the metabolic condition of tissue.
This inforrriation
can be correlated to the object's location in the target field of view. It is
often possible to
.. determine the boundaries of indistinct edges by correi~tin~; spectra with
"clustering" of spectral
objects. Thus, this field has recently become a valuable partner with
microscopy in order to
analyze the spectral characteristics of a given suspect pathological sample.
Unfortunately, the potential for spectroscopy to enhance, and even
revolutionize,
the field of medical pathology has not yet been fully exploited. First, as
discussed, the
conventional fluorescent microscope uses one filter to block all but a single
wavelength of light
to excite the specimen, and another filter that permits only the reemited
light {and blocks the
higher e, ergy excitation light) to pass to the optical output. Since the
specimen is usually
stained with a fluorophore that fluoresces at a single wavelength in the
spectrum, this method
provides useful diagnostic information about the specimen at this single
wavelength only.
In order to gather more, and more meaningful, information about the specimen,
the
entire image must be captured again at a second wavelength through a second
set of filters.
This process is repeated many times until the desired number of spectral
frames collectively
obtain the "spectral envelope" of the object. Each frame is stored in a
computer and the
composite image can analyzed by the processing software described above and/or
displayed on
a display. This is known as multispectral imaging ("MSI").
However, this particular technique for MSI is impractical for numerous
reasons.
First, it requires the availability of numerous, costly filters. Second,
exchanging filter sets in
and out of a fluorescent microscope is time consuming. Third, the specimen
must normally be
stained with numerous dyes that cause the specimen to fluoresce at each
excitation wavelength,
thus risking the occurrence of specimen "bleaching," a phenomenon that can
ruin or degrade
the diagnostic value of the specimen.
Several new automated f lter systems are available, such as rotatable filter
wheels,
acousto-optic tunable filters (AOTF), liquid crystal tunable filters (LCTF),
and the
3
CA 02338675 2001-O1-26
WO 00!06980 PCT/US99/17004
interferometer, all of which capture an entire image at each wavelength
sequentially until the
entire spectrum has been acquired. Hovvever, these systems are very costly,
and the data
processing of these images is an enormous task considering that a typical S 12
by 512 pixel
array, capturing, for example, 80 wavelengths, results in 21 MB of data per
scene. Due mainly
to complex and time consuming data processing, it is not uncommon for
investigators to feel
compelled to reduce the number of wavelengths acquired. Although 80
wavelengths may
appear to be a large number, it is not uncommon for analytical chemists to
acquire up to 1024
wavelengths simultaneously with off the-shelf CCD detectors in a laboratory
setting. Thus,
acquiring and storing so many consecutive frames is impractical for analysis
of specimens that
are prepared with many fluorophores (which also risks photo-bleaching the
specimens), or
when computation speed is required or desired. In addition, the physical cost
of these
instruments is extremely high and shows little evidence of decreasing.
Further, the above
methods require the target object to be stationary and suffer no chemical
change due to the
environment during spectral acquisition.
Point spectroscopy is one known method of obtaining spectral data from an
object.
This technique captures the entire spectrum of, as its name suggests, only a
single small point
of an object at a time. in order to be practical and meaningful for the field
of medical
pathology, however, a spectroscopy system must be capable of spectrally
dispersing, displaying
and analyzing an entire specimen, or at least substantial sections of a
specimen. Thus, point
spectroscopy is not an ideal solution to the aforementioned drawbacks.
Thus, there is a definite need for a low .cost system capable of rapidly and
efficiently obtaining a meaningful quantity of MSI data of mammalian matter,
including
pathology specimens, and that is capable of automatically analyzing this data
for the
identification of disease in the matter.
SUMMARY OF THE INVENTION
The present invention addresses these need by providing a system and method
that
automatically assesses mammalian matter, notably one or more cells or tissue,
for evidence of
acute disease via multispectral acquisition of images of the matter. In the
broadest
embodiment, the system has three primary components, namely, an image
transmitter, a unique
multispectral imaging spectroscopy subsystem and a processor. The image
transmitter includes
a source of light that illuminates the matter and an optical. output, and is
adapted to transmit an
image of a section of the matter to the optical output. The multispectral
imaging spectroscopy
subsystem is connected to the optical output and substantially, simultaneously
spectrally
4
CA 02338675 2001-O1-26
WO 00106980 PCT/I3S99I17004
disperses the transmitted image into multiple component wavelengths to create
a spectral
image. The processor then processes the spectral image to provide diagnostic
data
representative of the image.
This system provides the advantages of obtaining, in a single acquisition, a
complete multispectral image of, not just a single point, but a substantial
section of the matter,
while eliminating the need for sequential filtering.
In a more detailed embodiment, the multispectral imaging spectroscopy
subsystem
includes an imaging spectrograph and a charged coupled device {CCD) camera.
The
spectrograph has an entrance slit that permits the passage of light from a
slice of the transmitted
image of the section of the matter and a spectrum dispersing prism and mirror
arrangement that
disperses the light passed through the entrance slit into multiple component
wavelengths of a
predetermined spectral range to create a spectral image. The charge-coupled-
device (CCD)
camera is coupled to the spectrograph to acquire, and prepare the spectxai
image. The
preparation of the acquired image typically entails two steps, namely,
digitizing the image and
pre-processing the digitized image with appropriate digital signal processing
algorithms, as is
well known in the art of image processing. The processor resides in a computer
subsystem that
processes the digitized spectral image and provides diagnostic data
representative of the slice
of the image. This system provides several important advantages. The
spectrograph of this
invention is a low cost device that eliminates the time intensive sequential
capture of multiple
wavelengths of a given portion of a specimen in favor of capturing the entire
spectrum
simultaneously. This enables rapid data processing which contributes to the
system real-time
diagnosis capability. The spectrograph, together with the digital camera and
the computer data
processing, provides an extremely simple, efficient and low cost tool for
diagnosis of any
suspect matter whose image can be transmitted to the spectrograph. Further,
the entire system
is a fraction of the cost of other MSI systems, such as the interferometer.
In an even a more detailed embodiment, the irnage transmitter is a lens-based
image
magnification system or telescope, broadly defined as any of various
magnifying optical
instruments. In one of these embodiments, the present also provides a low
cost, efficient
system that automatically assesses the pathology of prepared cytopathology and
histopathology
specimens via multispectral acquisition of images of those specimens whose
images are
illuminated, magnified and transmitted to the optical output via a microscope.
The optical
output of the microscope may also include a standard camera interface, such as
a "C-mount"
connection for rapidly connecting and disconnecting the imaging spectrograph
thereto.
5
CA 02338675 2001-O1-26
WO 00/06980 PCT1US99l17004
In yet a more detailed embodiment, the microscope includes an automatic x-y
stage
capable of controllably translating the slide for sequential MSI acquisition
of the entire
specimen. After one slice is acquired by the system, the computer-controlled x-
y stage
automatically sequences, or moves, the specimen so that the image of an
adjacent slice of the
specimen can pass the image-acquiring, entrance slit for spectral dispersion
and acquisition of
that slice. This process repeats until the entire sample, or as much as is
desired, has been
acquired and investigated.
In yet a further embodiment, a second CCD camera is provided in order to
capture
video images of the magnified sections of the specimen from the microscope and
to provide the
image for display and/or further processing. With this further embodiment, a
beam directing
assembly ~ may be disposed between the microscope: and the spectrograph in
order to
alternatively direct the magnified optical output to either the first CCD
camera or the second
CCD camera. In this way, the system is capable of providing the pathologist
with two
diagnostic tools in one, one being traditional video imaging and the other
being spectral
topography data, the latter provided in the form of graphical display, called
spectral graphs,
tabular form, an actual diagnosis printout, prognosis or suggestions (after
such data is operated
on by the neural network) andlor a combination of all of the above.
In an alternative embodiment, a beam splitter cube may be disposed between the
microscope and the spectrograph in order to simultaneously direct the optical
output from the
microscope to both the spectrograph and the second CCD camera.
In a more preferred detailed aspect of the invention, when the microscope is
set at
40x, the area of the sample submitted to the 5 mm long entrance slit is
approximately 1.25 ,um
wide by 125 ,um long. In this particular embodiment, the spectrograph is
capable of acquiring
spectral data at approximately each 0.5 ,um along the slicE:. Each such 0.5
~cm by 1.25 ,um wide
section is called an "object." With the specific design of the preferred
spectrograph and CCD
array capacity, a maximum of 240 objects can be captured simultaneously from
each slice of
the target sample. Further, each spectrum for each object contains up to a
maximum of 740
wavelength data points in the 380 nm to 800 nm range, with a spectral
resolution of lnm at the
400nm wavelength to approximately 15 nm at 700 nm.
- The neural network algorithm may alternatively comprise an unsupervised
neural
network (USNN), a supervised neural network (SNIP, or a combination of the
two. The USNN
operates to automatically recognize and map (i.e. to train for) the presence
of "f ngerprint"
spectra. The SNN may be implemented to act to automate the system and perform
routine
autocalibration.
6
CA 02338675 2001-O1-26
WO 001069$0 PCTlUS99/17004
One preferred method of spectrally analyzing matter for the presence of
disease
based upon a spectral analysis of the morphologic and physiologic deviation of
the matter from
the norm includes illuminating the matter with a light source, transmitting an
image of the
matter to a multispectral imaging spectrograph, spectrally dispersing the
transmitted image
through a prism and mirror arrangement into multiple component wavelengths of
a
predetermined spectral range; acquiring the spectrally dispersed image of the
multiple
component wavelengths, digitizing the acquired image and manipulating the
digitized image,
and processing the manipulated spectrally dispersed image to provide.a
diagnosis.
In an even more specific embodiment, the irnage transmitter provides a source
of
I O f ltered light to the matter and transmits an image of fluorescent light
reemitted from a section
of the matter to the optical outlet. Typically, the matter comprises.a
prepared pathology sample
and the image transmitter is a fluorescent microscope. In this specific
embodiment, the present
invention eliminates the need for multiple filters by acquiring the entire
spectra of all
fluorophores from a section of the fluorescing matter simultaneously.
A method of practicing this more specific embodiment entails illuminating the
matter with filtered light of a specified wavelength to cau:>e the matter to
fluoresce, transmitting
an image of a section of the fluorescing matter to an optical output,
spectrally dispersing the
transmitted image of the slice into multiple component wavelengths of a
predetermined spectral
range, acquiring and preparing the spectrally dispersed image and processing
data representative
of the image with a processor to classify the slice of the image into one of a
preset number of
categories indicative of the condition of the slice of the image.
The system and method of the present invention is effective in evaluating
diseases
through the analysis of the fluorescent features of mammalian cells and tissue
that fluoresce
either via endogenous or exogenous reaction. The former, also referred to as
natural
fluorescence, includes biological matter having an autofluorescing component,
such as the
elastin compound found in the aorta wall, arteries and other tissue. The
analysis of such matter
using the present invention yields diagnostically valuable information without
any fluorescent
staining preparations. The immunofluorescent method also falls into the
endogenous category.
In such case, the matter is typically prepared with at least one
immunoglobulin antibody labeled
with a tag, such as fluorescein, and is useful for the analysis of the kidney
and heart, for
example.
As an alternative to endogenously reacting matter, the matter, or specimen,
may be
treated with at least one fluorochrome. More particularly, the specimen maybe
treated with a
histochemical probe labeled with a fluorochrome. Even more particularly, a
fluorescent in situ
7
CA 02338675 2001-O1-26
WO 00106980 PCT/US99/17004
hybridization ("FISH assay'') is performed on the specimen and the processor
provides data
indicative of the presence of at least one strain of one of a genetic
disorder, malignancy,
bacteria and virus, such as the Human Papilloma Viru s (HPV). Alternatively,
the specimen
may be prepared with an immunohistochemical stain having irnmunohistological
markers,
S including one of monoclonal and polyclonal antibodies conjugated to a
fluorescent tag. One
example of this is a specimen stained with a direct immunofluorescent assay
(DIF) to identify
the presence of chlamydia trachomatis (CT). Even more beneficially, the system
is capable of
simultaneously quantifying a specimen treated with multiple probes labeled
with different
fluorochrornes, immunohistochemical markers or a cornbination of all of the
above, in order
to diagnose the condition that each stain or fluorophore probe, individually,
is designed to
isolate.
Thus, the present invention can provide a significant advance in the field of
cervical
cytology by providing a tool that automatically assesses a Pap smear or
cervical biopsy for both
various HPV strains and CT bacteria. In particular, a Pap smear specimen may
be prepared
I 5 using a both a FISH assay for HPV identif cation and with a direct
immunofluorescent (DIF)
stain directed to detect CT. in this way a single, dually stained PAP smear
analyzed by the
system of the present invention can provide rapid, robust and low cost
identification of some
of the primary cytological disorders and precursors to cervical cancer.
Conventional staining of biological matter to induce a fluorescent reaction is
another category of reactions applicable to the present invention. For
example, a smear may
be tagged with well-known fluorescent stains, such ass rhodamine/auramine to
produce a
fluorescent reaction that can be exploited by the present invention for the
identification of
several diseases, such as tuberculosis.
Other features and advantages of the present invention will become apparent
from
the following detailed description, taken in conjunction with the accompanying
drawings,
which illustrate, by way of example, the principles of the invention.
BRIEF DESCRIP~'ION OF THE DRAB
FIG. lA is basic schematic showing the automated multispectral topography
system
of the present invention wherein light transilluminates through a mammalian
matter, such as
tissue or cells;
FIG. 1 B is a variant of the image transmitter of FIG. 1 A, wherein the light
incident
on the matter is either reflected off of the matter or is absorbed by the
matter which reemits
light of a lower energy (i.e. fluoresces);
8
CA 02338675 2001-O1-26
WO 00/06980 PCT/US99/17004
FIG. 2 is a schematic of a more detailed embodiment of the present invention
wherein the matter is prepared on a slide and the image transmitter is a
fluorescent microscope;
FIG. 2A depicts the matter prepared on the slide shown in FIG. 2 in greater
detail;
FIG. 3 is a flow chart describing one preferred method of the present
invention;
FIG. 4 is an observed video image of a section of a fluorescing, TB-positive,
sputum sample stained with auramine/rhodamine, with a slice of the section
bounded by a
rectangle;
FIG. 5 is a false color representation of the. spectra present in the slice
shown in
FiG: 4;
FIG. 6 is a more refined version of the spectral image of FIG. S, after
profiling with
an unsupervised neural network;
FIG. 7 is a graph of 25 spectra representing the spectral fingerprint of the
25
channels of the section of the image shown in FIGS. 5 ~u~d 6;
FIG. 8 is a chart showing the unsupervised neural network presentation of the
1 S analysis of, in the two left columns, the calibration smear shown in FIGS.
4-7, and, in the right
two columns, a patient sample;
FiG. 9 is a graph showing spectral curves of autofluorescing specimens, one
being
a portion of a healthy aorta wail and the other a degenerated aorta wall;
FIG. 10 is a graph showing the spectral curves of diseased native and
transplanted
kidneys;
FIG. 11 is a graph showing the spectral curves of three slices of an
immunofluorescing kidney sample, each stained for IgA,-IgG and IgM,
respectively; and
FIG. 12 is a graph showing the spectral curves of sections of two heart
biopsies
whereby no rejection is recorded for either section.
DETAILED DESCRIPTION OF THE PREFhRRED EMBODIMENTS
The invention summarized above and defined by the enumerated claims may be
better understood by referring to the following detailed description, which
should be read in
conjunction with the accompanying drawings. This detailed description of
particular preferred
embodiments, set out below to enable one to build, use and practice particular
implementations
of the invention, is not intended to limit the enumerated claims, but to serve
as particular
examples thereof. The particular examples set out below are preferred specific
implementations
of a system that provides multispectral topography for m~unmalian matter,
i.e,. cells and tissue,
namely, one that provides automated spectral data acquisition, analysis and
diagnosis of images
9
CA 02338675 2001-O1-26
WO 00/06980 PCT/US99/17004
of matter that is suspected to be diseased. The invention. however, may also
be applied to other
types of mammalian matter, such as that which is not suspected to be diseased,
but
identification of its morphological and physiological characteristics is
nonetheless desirable.
Further, in the presently described invention, the matter is illuminated with
a source of light,
with one specific , source providing filtered light to enable analysis of the
fluorescent
characteristics of the matter.
Before describing the invention in greater detail, namely, the method of
multispectral analysis employed by the present invention, its various
components and
experimental results, we now explain the theory behind the application of
spectral topography
as a tool for revealing the "hidden morphologies" and physiologies of tissue
and cells and its
implications for medical pathology.
I. Morphologic Effects On Normal And Abnormal Cells and Tissue
It has long been recognized that the study of the form, such as size and
shape, of
cells and their environments (cytomorphology) and tissue biopsies can reveal
diagnostically
1 S valuable information regarding the state of those stmctures. General
Biologic activity is
reflected best in the cellular structures of the nucleus. JFunctional activity
is reflected mainly
in the morphology of the cytoplasm. The "healthy," baseline morphology of
cells may be
considered a reference level, called euplasia, which is the form and structure
of normal cells
absent stresses from pathologic processes: In euplasia, key nuclear structures
may appear under
a light microscope as round or rounded, uniform and having regular patterns of
nuclear
components such as chromatin, and possess a degree of predictability from one
nucleus to
another. In tissue analysis, predictability from one cell to another cell of
the same type would
be expected.
Numerous inorphalogical effects of processes associated with carcinomas and
precursors to the same have been identified. For example, significant
irregularities in the shape
of nuclear membranes of cells, disarray in the structural orderliness and
shape of chromatin and
parachromatin (the pale areas of the nucleus), the enlargement and
irregularity of the shape of
nucleolus, and importantly, an increased ratio of nuclear area to cytoplasmic
area (N/C ratio)
are some of the recognized morphological factors whiclh favor a finding of
cancer.
Unfortunately, to date, such morphological analysis under a transmission or
fluorescent microscope has, by itself, been of limited value in diagnosing
diseased cells and
tissues for several reasons. First, morphological changes in cells and tissue
occur in varying
degrees. Many changes cannot be recognized (either clearly or at all) through
the eyepiece of
a light microscope even by an experienced pathologist, or even with the
manipulation of video
CA 02338675 2001-O1-26
WO 00/06980 PCTlUS99/17004
images of slides of specimens. These hidden morphologies are due to either the
inherent nature
of the changes, the stage of morphologic activity or a combination of the two.
Second, to date.
there has been no observable absolute morphologic feature of cancer - or a
malignant criteria -
that when present, unequivocally reveals that the particular cell or tissue
under observation is
S cancerous or, when absent, means that there is no cancer.
The present invention is employed to reveal the hidden morphologies of suspect
cell
and tissues, whether in vivo or prepared specimens, by illuminating them with
a light source,
and analyzing their spectral content with the aid of a unique multispectral
imaging spectrograph
and sophisticated processing algorithms.
IL The Method and Components
1. Push Broom Multispectral Im ding
There are several traditional ways of acquiring multispectral information from
a
remote field of view. As discussed above; one method acquires an entire field
through a series
of wavelength filters. The number of filters will be equivalent to the number
of wavelength
I 5 data points needed to identify the spectral signatures of each component
in the field. The field
of view is fixed so data cannot be acquired if the object moves or changes in
any way.
In another method developed for remote eartlh monitoring and implemented by
the
present invention, the system acquires a small slice of the field and passes
it through a
specialized wavelength dispersive spectrograph that acquires the entire
spectrum of the slice
simultaneously. To cover the entire field it is necessary to move on to the
next, adjacent slice.
Concatenating each acquisition enables the entire f eld to be covered. This
method is often
referred to as "push broom" spectral topography because the sample is "pushed"
across the
spectrograph entrance aperture, or slit, and is, in numerous ways, more
versatile and efficient
than the prior described method of multispectral imaging acquisition.
The present invention uses push broom multispectral image acquisition of
suspect
mammalian tissue and cells to reveal, in the case of specimens prepared on
slides, their hidden
morphologies and, in the case of in vivo analysis, their hidden physiologies,
and to ultimately
provide clinical diagnosis of the tissue and cells.
2. The Basic Components and Method
~FTG.1 A is a block diagram showing the basic components of the system. An
image
transmitter 1 includes a light source 2 and an optical output 3. The light
source 2 illuminates
the mammalian matter 4 to be analyzed, whose image is tz~ansmitted to the
optical output 3. An
imaging spectroscopy subsystem 6 substantially simultaneously spectrally
disperses the
transmitted image into multiple component wavelengths of a given range. This
method of
11
CA 02338675 2001-O1-26
WO 00/06980 PCT/US99/17004
spectral dispersion uses a spectrograph originally designed for remote,
telescopic earth
. monitoring and astronomy and is currently in use for both applications in
military and civilian
environments. The original spectrograph was patented to Warren et al. (patent
no. 5,127,728)
and was designed for use in the infrared wavelength range of 3 to 15 ,um, and
is incorporated
herein by reference. For the life sciences applications of the present
invention, the optics were
redesigned for use in the primarily visible 360 to 800 nm wavelength range. A
processor 8,
such as a PC computer loaded with the appropriate software, operates on data
representative
of the spectrally dispersed image to ultimately provide a diagnostic output 8
relating to the
condition of the matter 4. This diagnostic output 10 could be provided at a
computer screen,
at a printout, or at any conventional output device.
FIG. I A depicts an image transmitter arrangement whereby the light source 2
transilluminates through the matter 4. This is typic;alIy the case when, for
example, a
transmission microscope is the image transmitter that provides white Light for
a relarively thin
specimen, such as a cell smear, or a very thin tissue biopsy. FIG. 1B shows an
alternative
i 5 arrangement for the image transmitter 1, which is representative of one of
two optical scenarios.
In one scenario, the Light from the source 2 is reflected off of the matter 4
and to the optical
output 3. Reflection could be used in several situations, one being where the
suspect tissue
biopsy is too thick for the light to meaningfully transillunninate
therethrough, and another being
where the matter to be analyzed is not removed from the patient, but is rather
in vivo. In the
second scenario represented by FIG. I B, the Light from the source is
filtered, and this filtered
light 2 is absorbed by the matter 4. In response, the tissue or cells; reemit
Light of a lower
energy level. This fluorescent light is provided to the multispectral imaging
spectroscopy
subsystem for dispersion and analysis.
FIG. 2 is a schematic of a more detailed embodiment of the present invention
wherein the image transmitter is a fluorescent microscope. The primary
components of this
multispectral topography system 12 include a fluorescent microscope 40, a
prism and mirror
imaging spectrograph 30, a first CCD camera 34, a processor 70 and a
diagnostic output device
80.
The fluorescent microscope 40, found in many laboratories and research
facilities,
has an eyepiece 41 and an optical output 42 having a standard camera
interface, such as a video
port with a "C-type" mount. The light source 43 of the rriicroscope is
filtered by internal filters
and is absorbed by a specimen 18 which has been prepa~~ed on a slide 16. The
specimen then
reemits light of a Lower energy Level and of varying intensities across the
specimen, and lenses
magnify the image of a section of the fluorescing specimen 18 and supply such
image to the
12
CA 02338675 2001-O1-26
WO 00/06980 PCT/I3599/17004
both the eyepiece 41 and optical output 42. The specimen is typically prepared
using
conventional dyes and/or labeled with conventional tags, using conventional
techniques, but
need not be. A spectroscopy subsystem connected to the microscope 40 includes
the prism and
mirror, wavelength dispersive imaging spectrograph 30 having an entrance slit
32. The slit
permits the image of a slice 20, as seen in greater detail in FIG. 2A, of the
specimen 18, the
slice itself comprising many "objects" 21, to pass there~through and into the
spectrograph 30,
and a first CCD camera 34. The novel and modified spectrograph 30 provides
good image
quality over a broad range of operating wavelengths simultaneously, allowing
large spectral
intervals to be surveyed without moving any of the elements of the system.
A beam directing assembly 60, also called a "beam splitter," constructed with
a
"flip" mirror 62, provides both a video image and spectral acquisition. Thus,
when the mirror
is flipped in one direction, the spectrally dispersed light is focused on, and
acquired by, the first
CCD matrix array 36 of the camera 34. As done in conventional CCD cameras, the
analog
image is then prepared for viewing and further processing. In particular, the
image is digitized
by an analog-to-digital (A to D) converter and is then manipulated, or pre-
processed, by the
spectral image pre-processor 37, often called a digital signal processor
(DSP). The prepared
spectral image can then be fed into the processor 70, as discussed in detail
below, and/or
displayed on a display 38 such as a CRT, LCD screen, crr other conventional
display.
When the flip mirror 62 is rotated to a second orientation, the visual,
microscopically magnified image, i.e. a video image, is captured by the matrix
array 56 of the
second CCD camera 54. This image is also digitized b;y an A-to-D converter and
advanced
image processing algorithms 57 (another DSP) manipulate the image for high
quality display
on a conventional display 58. Conventional image pre-processing algorithms in
DSP's include
smoothing, normalization, background substraction, principal component
analysis (PCA) and
partial least squares (PLS). In this way, both spectral imaging data and
visual imaging data of
the specimen can be displayed for the pathologist, stored, analyzed and/or
further manipulated.
In an alternative embodiment not shown, a beam sputter cube replaces the beam
directing
assembly 60 to send light simultaneously to the spectrograph 30 and the
observed-image CCD
camera 54.
The spectrograph 30 uses a prism made of inexpensive flint glass. The support
and
body of the unit may be in either cast aluminum, or ribbed metal plate, or any
other material
that provides for lightweight and for enhanced rigidity. The optical system is
fully ray-traced.
The system, with or without the beam director 62 is easily assembled to the
optical output of
13
CA 02338675 2001-O1-26
WO 00/06980 PCT/US99/17004
a conventional microscope, typically a video port, with the use of a standard
"C-type" mounting
or any other acceptable connecting means.
In one preferred embodiment, when the #luorescent microscope is set at 40x
magnification, each slice 20 captured through the slit 32 and by the
spectrograph 30 is
approximately I.25 ,um wide by 125 ,um long. Further, the system is capable of
acquiring
spectral data at approximately each 0.5 ,um along the slice 20. Each 0.5 ,um
by 1.25 ~cm wide
section is called an "object" 2I. With the specific design of the preferred
spectrograph and
CCD array capacity, a maximum of 240 objects can be captured simultaneously
from each slice
of the target sample. Further, each spectrum for each object contains up to a
maximum of 740
wavelength data points in the 380 nm to 800 nm range, with a spectral
resolution of lnm at the
400nm wavelength to approximately 15 nm at 700 nm. It is understood, however,
that other
slit sizes and CCD cameras having other resolution capacities may be used,
resulting in
different (and greater) object and spectral envelope resolutions.
Sequentially scanning the entire image of the specimen 18 is achieved by
scanning
I 5 the microscope stage under computer control in the histology or
cytopathology setting. The
first CCD matrix array detector 36 collects individual spectra along rows of
pixels from objects
located in the entrance slit 32. This format is sometimes referred to as an
"open image" because
there are no restrictions on the area of the object to be; examined and is
certainly the least
expensive and most flexible method for pathology samples subject to
microscopic examination.
All spectra of all objects in a slice are acquired simultaneously in
milliseconds,
depending on signal strength. Photo-bleaching is reduced especially if objects
such as single
cells or glomeruli are selected by automated pattern recognition and
correlated with signal
strength at certain wavelengths.
The processor 70, which will typically be part of a computer system, such as a
PC,
contains a powerful neural network 72 that provides near instantaneous
recognition of the
spectral fingerprints of the objects 21 of the specimen 18. In one preferred
embodiment, there
are up to 240 bbjects, each as small as 0:5 ~cm, that the neural network can
substantially
simultaneously process. The relatively small file size for each acquisition
greatly enhances
computation speed and simplifies memory management..
'FIG. 3 illustrates how the push-broom methodology is applied to the present
invention for multispectral analysis mammalian matter. In step I00, an image
of the matter,
whether prepared on a slide or in vivo, transmitted to the optical output. It
is understood that,
in the broadest embodiment, the image can be transmitted via any known means,
such as
transillumination through or reflection off the.matter, or fluorescing from
the matters. In the
14
CA 02338675 2001-O1-26
WO 00/06980 PCTNS99/1'1004
case of microscopy (transmission or fluorescent), a particular field, or
section; of a magnified
specimen is presented to the entrance slit 32 of the spectrograph 30. In step
102, a single slice
20 of the section, containing up to 100 objects 21, passes through the slit 32
and, in step I04,
strikes the first curved surface of the prism, is refracted, strikes the
second surface, and exits
to strike the spherical mirror as is described in the Warren et al. patent.
The wavelength
dispersed light then returns through the prism to be focused and stored onto a
first CCD matrix
array, step 106, which acquires and prepares the light. As used herein, the
preparation may
entail digitizing and pre-processing the spectrally dispersed light, but not
necessarily. The pre-
processing, or manipulation, of the digital image is accomplished with
conventional DSP
algorithms to improve its appearance. Over 80% to 90% of all light, over the
entire wavelength
range, is transmitted through the system.
The processor then processes the spectral data representing the spectral
envelopes
of each object in the slice, step 108, using the neural network that sorts out
the morphological
patterns in the objects of the cell(s). The system then inquires into the
status of the acquisitions,
step 110. If only a single acquisition is needed or desired, the process halts
in step 112 and the
diagnostic data may be output for review by the pathologist. However, if as is
usually the case,
an additional slice of the specimen is to be spectrally dispersed and
analyzed, i.e. the answer
to inquiry 110 is "no," and an adjacent slice of the image is transmitted
through entrance slit,
step 114. In the case of microscopy, the specimen slide is moved by the x-y
stage of the
microscope 40 to permit the passage of light representing a slice adjacent to
the previously
dispersed slice. Then, the process reverts to step 104 i:or spectral
dispersal, acquisition and
analysis of the objects in that second slice. This process is repeated until
the entire or enough
of the specimen is analyzed in this fashion. (Once this sequential acquisition
process is
complete and the spectral data for each object is stored in the appropriate
memory "bins;" as
described below, the processor may then synthesize this data to produce a
complete diagnosis
fvr the specimen.
The advantages of utilizing push broom spectral data acquisition for cellular
analysis are numerous. First, the diagnosis is practically instantaneous. To
acquire enough
spectral data using alternative techniques would require 'the creation of huge
digital files prior
to data processing. A full acquisition according one ~oreferred embodiment of
the present
invention includes 240 spectra, each with 740 data points, yet is only 185KB.
A file of this size
is easily handled, resulting in very rapid data processing and decision
making. Further, the
entire system costs much Iess than other methods of analysis, such as the
interferometer.
CA 02338675 2001-O1-26
WO 00/06980 PCT/U599/17004
This method also enables fast low-resolution assessments to determine gross
parameters prior to high-resolution scans. In addition to acquiring
multispectral topographical
maps, the system can "paint" the video image of the specimen with false color
"fingerprint"
spectra indicative of normal and diseased tissue. The system also
differentiates between
chronic and acute disease as well as evidence of anti-rejection drug toxicity.
3. The Neural Network
Traditional mathematical algorithms perform calculations sequentially,
delivering
results based on linear transformations. The neural network (NN) 72 performs
calculations in
parallel, in an analogous way to the human brain; to perform non-linear
transformations. To
date, the system has been powered by an unsupervised neural network (USNN).
However, the
system rnay alteratively be powered by two neural networks: a USNN and a
supervised neural
network (SNN). The USNN collects each spectrum from each object, characterizes
it, sorts it,
and places it into a "bin" of unique spectral signatures. It can thus be
employed to
automatically recognize and map (i.e. to train for) the presence of
"fngerprint" spectra, i.e., to
identify bins containing identical spectral signatures. The SNN may then be
used to determine
the special features that differentiate the spectra in each bin from those in
other bins. Thus, the
system can then compare the presenting spectra of the objects of the matter
under analysis with
the map to identify the presence of the disorder.
The USNN can be thought of as a "fzlter" o~r "sieve" that characterizes and
sorts
2a each spectrum as it is collected and places it into its morphological bin
or "class" of similar
spectral signatures. The process is analogous to alphabetizing the words in a
book by
combining those that are the same or have a common root. By the end of this
procedure, the
number of bins represents the number of spectral objects defined by their
spectxal-
morphological characteristics. The process is referred to as "digital
chromatography" because
the function is almost identical to the process used in analytical chemistry
for the separation of
mixtures of chemical compounds. The big difference is tlhat the USNN program
adds the extra
dimension of providing spatial information by mapping a particular class of
spectrum back to
the sample itself.
The operator can decide to combine some spectral classes, such as background
features, of eliminate others. This is achieved by "thresholding" either by
the user or the
computer. This is a process that enables the USNN to delineate, identify and
separate identical
and non-identical spectra within user defined limits of tlhe application. The
incorporation of
human "wet neurons" to set up the thresholds enables human experience to
control the
operating limits of the software. The USNN becomes a '''white box''' rather
than a "black box"
t6
CA 02338675 2001-O1-26
WO 00/06980 PCT/US99/17004
because the sensitivity and the operating parameters of the network are
available to operator
influence at all times.
Once the USNN has self trained, with initial well-qualified samples, the USNN
compares each newly acquired spectrum against the spectral classes and
categories it
recognizes. The USNN identifies matches, near matches, and no match spectra in
the new
acquisition(s). Each class of spectrum is coded into a few bytes of data and
stored in memory.
Every future spectral acquisition is similarly coded and compared to the
stored data. This
process reduces a spectrum of 740 data points to a block: of memory only a few
bytes in size,
consequently recognition of hundreds of spectra is performed in near real-
time. Training can
take up to two minutes for complex materials and a few seconds for simple
spectra. After
training, recognition is within a second.
The processor 70 may also incorporate a Supervised Neural Net (SNN) that works
in conjunction with the USNN. The SNN would identify the special features of
each bin,
automates the system and performs routine autocalibration. Past experience has
shown that
combining these two neural net architectures is very effective for controlling
many variables,
some or all of which can change with time. Most humans tend to overlook or
"accommodate"
certain changes. In a mufti-parametric system, such as cellular pathology,
this is very dangerous
because a change in one variable can result in non-linear affects elsewhere in
the process and
compromise the accuracy of an assessment. The two neural network systems could
form a
transparent alliance to automatically ensure that conditions necessary for
accurate repeatable
diagnoses are not compromised by changes in temperature, mechanical
maladjustment or
operator error.
One or, more likely, many systems of the invention may be trained and
retrained
by USNN's in order to spectrally characterize the morphologies of the cells of
all presenting
pathological disorders and diseases, thus creating a "library" of spectral
fingerprints
representative of those presenting conditions. It is expected that,
eventually, the entire and
finite universe of categories of pathological disorders will be spectrally
characterized by
USNN's. When this milestone is achieved; systems could then be equipped with
SNN's which
include databases containing the all pre-trained, spectral fingerprints of the
various possible
conditions. When a specimen whose pathology is unknown is presented to the
system of the
present invention, the SNN will rapidly compare and map its spectral
fingerprint with that of
all fingerprints stored in the database or, in the appropriate category of the
database. With the
availability of powerful, fast and low cost personal computer systems with
large amounts of
memory capacity, a single personal computer system will be capable of storing
the an entire
17
CA 02338675 2001-O1-26
WO 00/06980 PCT/US99117004
''spectral library" thereby providing automatic diagnosis of the condition of
any cellular or
tissue specimen presented to it.
It is understood that those skilled in the art may develop protocols to enable
specific
histological assessments to be selected from a computer menu to simplify and
minimize
continuous human interaction with system, perform autocalibratiori,
continuously monitor the
status of the entire system, and record user operations.
4. The Outputs
The system of the preferred embodiment is capable of providing several
diagnostic
outputs. Figures 4, 5, b, 7, and 8, discussed in greater detail below,
illustrate some of these
I0 outputs from analysis of mammalian matter, namely, .a section of a sputum
sample from a
. patient positive for tuberculosis, wherein the samph is stained with the
chemical dye
preparation; auramine/rhodamine. The slide; viewed under epi-fluorescence,
enabled the
system to provide spectral characterization and mapping of the fluorescing
tuberculosis
bacteria. The neural network monitors intensity and when multiple spectra are
presented (while
not necessary for this example wherein only a single fluorophore and thus a
single emitted
wavelength is present) deconvolution requirements and paints back to a
computer monitor the
location on the sample of an identified and processed spectrum.
In particular, FIG. 4 shows a grey scale video, or "observed," image of a
fluorescing
and magnified section of the specimen on the video display 58. It is
understood that the video
display can be any conventional display, such as a CRT or LCD screen, or
equivalent. FIG. S
shows the same image with a false color representation of the spectra present
down the slice.
FIG. 6 is a more refined version of the spectral image of the slice identified
in FIG. 5, but
partitioned into "channels" after profiling with the USNloI. FIG. 7 is a graph
of 25 spectra out
of a possible 240. The y-axis represents the intensity; or magnitude, of the
light emitted from
the object of the slice of the specimen across its spectrum. The entire
baseline-to-baseline
fluorescence envelope between 540 and 700 nm (peaking around 580 nm) is
captured and
relative intensity differences as a function of location on the sample clearly
portrayed. FIG. 8
shows how the USNN associated each spectrum of simiilar nature to an object.
In particular,
it is a chart representing the objects in a training set on the left half and
on the right, objects of
another sample.
All of this visual information may assist the pathologist in analyzing and
diagnosing
the matter. Ultimately, however, the system, via the neural network, is
designed to provide an
actual diagnosis of the condition presented by the matter i.mder observation.
This diagnosis can
18
CA 02338675 2001-O1-26
WO 00/05980 PCT/US99117004
be provided on any conventional output medium. such as on a display, in hard
copy, stored to
memory, transmitted to another computer system, or an:y combination of the
above.
iII. Applying Multispectral Topography to Fluorescing Matter
Many fluorescent techniques have been found very effective in the analysis of
a
variety of types of biological mammalian matter. Some techniques exploit the
endogenous
characteristics of the particular matter {i.e. natural fluorescence),
including autofluorescence
(such as in the aorta wail and arteries) and immunofluorescence {for example,
for analysis of
kidney and heart tissue). Other fluorescent techniques that are not endogenous
include
immunofluorescence using fluorescence-labeled, monoclonal or polyclonal
antibodies
(immunohistologic markers), in situ hybridization techniques, and standard dye
methodologies.
It is thus understood that mammalian matter, as used in this section, includes
any of the
aforementioned types of matter prepared by any of these fluorescent
techniques, and, in the case
of matter that autofluoresces, may have no fluorescent preparation at ail. In
all of these cases,
however, the analysis of such matter is greatly enhanced by the incorporation
of the automated
spectral topography system of the present invention. The remaining discussion
details some
specific applications of the present invention and the experiments conducted
which verify its
efficacy for diagnosis of pathological disorders.
1. Applied to Naturally Fluo~escir~g Matter lEr~dog~~l
Matter that is recognized to contain a substantial autofluorescence compound,
such
the elastin present in the aorta wall and arteries, or collagen present in
other matter, may be
analyzed by the present invention with no fluorescent staining to evaluate
autofluorescent
patterns of diseased and normal aortas and arteries and to yield basic,
diagnostic information.
Further; analysis of the natural fluorescence of cell and tissue matter from
both native and
transplanted organs, such as the kidney, heart and other organs, using an
iinmunofluorescent
staining technique, including fluoresceinated antisera to iunmunoglobulins, in
order to diagnose
both native diseases and rejection, can also be greatly enhanced by the use of
the multispectral
imaging spectrometer system of the present invention.
a. Applied to Autofluorescent FluoronhorgL
FIG. 9 shows the spectral features of two samples of aorta walls, which
possess
autofluorescent elastin layers. The samples were examined under an epi-
fluorescence
microscope excited at 436 nm and exhibited strong blue-green fluorescence. The
spectral
features provided by the spectroscopy subsystem of the present invention and
shown in the
figure clearly differentiate between a normal aorta wall {curve 1 ) and an
aorta wall showing
degeneration (curve 2). Ultimately, once the neural network has fully the
trained the system
19
CA 02338675 2001-O1-26
WO 00/06980 PCT/US99/17004
by many such samples, resulting in the identification and classification of
the spectral
characteristics of the entire universe of normal and degenerated aorta walls,
the system of the
present invention will be capable of analyzing and diagnosing any aorta wall
sample of
unknown diagnosis.
b. Apniied to Im_munoflLOrescence
The diagnosis of disease in some native organs, such as end stage renal
disease
(ESRD), and of some heart diseases, and of rejection of transplanted organs
relies, in part; on
the effective evaluation of immunofluorescence findings in frozen sections of
biopsy
specimens. For example, the ESRD population is growing progressively older.
Both dialysis
and kidney transplantation are effective techniques for prolonging life in
ESRD. The
commonly employed diagnostic tests (i.e., renal transplant ultrasound and
hippuran scintigram)
are helpful in differentiating rejection from other causes of graft
malfunction. Pulsed-Doppler
is a good tool for studying vascular complications involving renal transplants
and helps
differentiate vascular rejection from other complications;. The diagnostic
value of quantitative
Duplex Doppler Sonography (DS) in renal allograft evaluation is being viewed
increasingly
critically. Therefore, renal biopsy to establish specific diagnosis by
histopathology evaluation
of allograft dysfunction remains mandatory.
In fact, specific renal parenchyma) disease, such as acute or chronic
rejection, "de
novo" or recurrent glomeruiar disease, immunosuppression nephrotoxicity,
(mainly by
cyclosporine A) that contribute to graft malfunction, can typically only be
diagnosed by renal
histopathologic study.
Recognizing the importance of effective and efficient histopathologic analysis
for
these conditions, the system of the present invention was thus used to
evaluate fluorescence
patterns in organ specimens stained for immunoglobulins, complement
components, and plasma
proteins and was correlated with light microscopic and standard
immunofluorescence findings
and clinical data. It was found that spectral analysis of the following
histopathological
specimens using the system of the present invention provided qualitative as
well as quantitative
information that is extremely valuable in the diagnosis of these specimens.
i. Applied to Kidne, Biopsies
The system of the present invention was tested and was found to assist in the
early
diagnostic evaluation of biapsies from patients with renal) disease, and with
routine monitoring
of patients who have undergone transplantation.
In particular, immunofluorescence studies of renal allografts with rejection
usually
reveal non-diagnostic patterns of immunoglobulin deposits with existing
fluorescence .
CA 02338675 2001-O1-26
WO 00/06980 PCT/US99/17004
microscopes. The application of the spectral topography microscope of the
present invention
has been shown to aid in the detection of immunoglobulin deposition as a
function of their
spectral fingerprints and their spatial correlation within the tissue and adds
a powerful digital
advance to current methods of observation.
In patients with native kidneys, spectral images from the system can be
compared
to clinical findings and standard light and immunofluoreacence studies to
categorize the disease
state and quantif es its severity. Currently, immunofluarescence is graded
crudely (0 to +4).
The system adds quantitative as well as qualitative new insights into the
structural
abnormalities underlying immune and non-immune system based renal diseases.
The experiment entailed analyzing archival glides of patients with suspect
renal or
heart transplants or native organs presenting with evidence of disease. The
samples had been
treated with immunofluorescent stains specific to IgA, IgG or IgM. They were
also treated with
Hematoxylin/Eosin (H&E) staining to observe and analyze the samples under
transmission
(white light) microscopy. The system operating parameters were as follows:
The microscope objective: 40X
Entrance slit to the spectrometer: 50 ,um
Average acquisition time of spectra: 9 Sec
Average acquisition time of Observed image:1 Sec
Number of objects per acquisition: 240
Size of each object: 0.5 ,um by 1.25
,um
Spectral range per acquisition: 400 to 800 izm
Number of Renal disease patients: 7
Number of repeat tests per slide:
The excitation wavelength was 405 nm and spectra were acquired through a long
pass filter that allowed all light of greater than 470 ~n to pass, and blocked
the excitation
wavelength and all wavelengths shorter than 470 nm. An Olympus BH2 epi-
fluorescence
microscope was used.
All slides had been previously examined and: were photo-bleached to some
extent,
and grossly in some cases. Nevertheless, as shown below, the system provided
evidence that
spectral topography is able to differentiate between native kidney with
chronic disease and
21
CA 02338675 2001-O1-26
WO 00/06980 PCT/US99117004
transplanted kidney showing evidence of chronic rejection or cyclosporine
toxiciri~. It was also
possible to differentiate between immunofluorescent staining for IgA, IgG and
IgM.
( 1 ). Comparison of native kidney with chronic disease and
transplante idney with chromic rejiection
FIG. 10 shows that spectral topography differentiates between native kidney
with
chronic disease, (curve 1) and transplanted kidney showing signs of rejection
with and without
cyclosporin toxicity, (curves 2 and 3). All these lbiopsy samples were stained
with
fluoresceinated anti-sera to human IgA:
Curve 1: (14058) Diabetic, chronic diabetic nephropathy, and
glomerulosclerosis.
I Curve 2: (12815) Diabetic, 7-year-old transplant with chronic cyclosporine
toxicity
and signs of rejection.
Curve 3: (13487) 1 year transplant, chronic transplant rejection,
glomerulopathy and
segmental sclerosis.
(2). Differentiation between IgA I~<'' nd M
FIG. 11 shows clear differentiation between. the spectra of immunofluorescence
staining for IgA, IgM and IgG in the spectral region from SSOnm to 650 nm:
Curve 1: (14058A3) Patient with chronic diabetic nephropathy and
glomerulosclerosis.
Stained for IgA.
Curve 2: ( 14058G4) Patient with chronic diabetic nephropathy and
glomerulosclerosis.
~ Stained for IgG.
Curve 3: ( 14058M2).Patient with chronic diabetic neplr~ropathy and
glomerulosclerosis. Stained for IgM.
(3). ~ummary of results
In general, the results were most encouraging, even with low signal strength,
and
demonstrate the robustness of the technique. These tests provide strong
indications that it is
possible to differentiate between acute and chronic renal transplant rejection
and cyclosporine
toxicity.
ii. Applied to Heart Transplant Biop,~es
Although traditional medical treatment modalities and newer surgical
interventions
may provide short-term relief, cardiac transplantation may still be the only
intervention that can
alter the natural history and poor prognosis for patients with end stage
cardiomyopathy. In spite
22
CA 02338675 2001-O1-26
WO 00/06980 PCT/US99/17004
of medical advances. rejection and infection of transplanted hearts still
result in significant
morbidity and mortality, and the development of graft coronary occlusive
disease limits even
longer-term survival. A particularly lethal from of rejectiion, called humoral
rejection, usually
occurs early after transplantation. The diagnosis that leads to correct
therapy is best determined
from an immunofluorescence evaluation of frozen sections of heart biopsies
from the
transplanted heart.
Non-rejection pathology is frequently seen post transplant to include ischemia
or
catecholamine effects, interstitial fibrosis, myocardial calcification, and
cyclosporine-associated
endocardial i~ltrates called the Quiity effect. It is therefore very important
to be able to
automatically diagnose evidence of immunosuppressive disorders even if there
is no evidence
of cellular rejection.
The International Society of Heart Transplantation (ISHT) provides criteria to
enable grading of heart rejection ranging from Grade 0 to +4. Figure 12 shows
two spectra
from two patients, both negative for humoral rejection with an ISHT grate 0
and also negative
for Quilty effect. Both were of right ventricular septal endocardial biopsies
of heart transplant
patients. The slides were stained with FITC conjugated antisera to
irnmunoglobulins, the third
component of complement and fibrinogen. Both spectra are fundamentally the
same. The
results summarized are:
Curve 1: {16757) Negative Quilty effect, negative rejection, ISHT Grade 0.
Curve 2: ( 16471 ) Negative Quilty effect, negative rejection, ISHT Grade 0.
This example provides a good indication that spectra are consistent between
patients with
similar findings.
In general, immunofluorescence studies with antibodies to immunoglobulin G,
immunoglobulin A, immunoglobuiin M, Clq, C'3; HLA-DR, and fibrinogen and
immunoperoxidase staining for endothelial cells (factor VTII-related antigen)
and macrophages
(KP 1 [CD6$]) are regularly performed, and are enhanced with the present
invention, because
spatially resolved immunofluorescence features can be quickly, accurately and
inexpensively
studied and evaluated. The spectrograph, in conjunction with neural network
powered data
analysis, provides an immediate and digitally objective interpretation of
spectral objects present
in such tissue to aid the pathologist in making an assessment. Thus, the
spectroscope/microscope combination enhances the speed and specificity of
early indications
of organ rejection.
23
CA 02338675 2001-O1-26
WO 00/06980 PCT/US99/17004
2. Applied to Other Fluorescent Techniques
. As stated above, biological matter be stained with a fluorescent dye or
labeled with
a fluorescent tag. ImmunohistochemicaI reactions comprise one category of
reactions and
include mononclonal and polyclonat antibody staining with immunohistologic
markers.
Fluorescent in situ hybridization (FISH) is another category and can be
directed to genetic
analysis or protein analysis. In the former, hybridization takes place between
a nucleotide label
probe (called the "antisense strand") and an endogenous nucleotide (e.g. mRNA,
called.the
"sense strand"). This is referred to as a paired-nucleotide interaction. In
the latter, proteins are
labeled and incubated with tissue that includes the target binding protein in
what is called a
protein-protein interaction. Examples of the application of the present
invention for the
diagnosis of disorders resident in such matter is now set forth.
a. ~pp~ied To Cervicovaginal Disorders
Effective diagnosis of two particular disorders, namely Human Papilloma Virus
(HPV) and Chlamydia Trachomatis (CT), has become a subject of increasing
importance to
pathologists. The first disorder is believed to be a precursor of cervical
cancer, and the second
disorder is the most common sexually transmitted bacterial pathogen in the
U.S., and is
recognized to cause substantial morbidity. However, the diagnosis of both
disorders are subject
to the aforementioned limitations in technology. PAP smears currently screen
for the early
detection of cervical cancer and other abnormalities by preparing slides of
stained, exfoliated
cervical cells for analysis on a light microscope. While a valuable screening
tool, FAP smears
detect only 50-80 % of the abnormalities subsequently found by
histopathological examination
of biopsy specimen. Further, these two disorders are not successfully tested
for with the
conventional PAP smear screening.
Siadat-Pajouh et al. and others have showmthat cancer-associated HPV genotypes
in the nucleus of cervicovaginal cells are detectable by DNA analysis of the
cells in a PAP
smear slide via use of a fluorescence based in situ hybridization (FISH)
assay. Images of the
cells acquired (filtered) with a fluorescent microscope acre then digitally
captured by a CCD
camera, and then analyzed using algorithms which detect all cell nuclei from
images of the
DNA counterstain. The images of the nuclei can then be used as a mask and
mapped over the
FISH image of the same microscopic scene to quantify the corresponding
fluorescent HPV
signal from each nucleus. (See Siadat-Pajouh et al., "Detection of HPV Type
16/18 DNA in
Cervicovaginal Cells by Fluorescence Based Ln Situ Hybridization and Automated
Image
Cytometry," Cytometry, 15:245-257 (1994)). This method can be described as a
single
wavelength examination of fluorescent signals with a single wavelength filter.
To gain more
24
CA 02338675 2001-O1-26
WO 00106980 PCTNS99/17004
data about the DNA structures of the nucleus resulting; in enhanced speed,
specificity and
accuracy of HPV identification, multiple fluorescent tags must be used which
would require the
use of multiple filters for the fluorescent microscope. iHowever, as discussed
above, single
wavelength examination with multiple filters is time consuming; risks sample
bleaching, and
S throws away useful mufti-wavelength data.
Further, while detection of Chlamydia Trachomatis (CT) , the most common
sexually transmitted disease in the U.S., has been difficult using standard
PAP smear
techniques, direct immunofluorescence (DIF) stainings oi" smears with
fluorescein-conjugated
monoclonal antibody has been shown to be effective in its diagnosis. (See
Garozzo et al.,
"Chlamydia Trachomatis diagnosis: a correlative study of pap smear and direct
immunofluorescence,", Clin. Exp. Obst. Gyn., vol. 20(4):259-266 (1993).)
The present invention has utility as a low co:>t, rapid and efficient
diagnostic tool
and method that could assist in the identification of these "early risk"
disorders together with
(in addition to) the PAP smear screening, all in a single test. In particular,
the present invention
can automate the analysis of normal and diseased cells that are identifiable
using a sensitive
FISH assay and DIF staining, to more objectively, cost effectively, and
rapidly identify the
presence of HPV and CT from a PAP smear.
i. Identification of Human Parailloma Virus PV
One primary goal of new cytology instruments is to reduce the false-negative
ratio
(FNR) of screened PAP smears: Of the various technique:c offered to increase
the detection rate
of cytologicaIly undetected false-negative cases, screening simultaneously for
HPV in the DNA
of epithelial cells along with PAP smears has been shown to be one of the most
effective.
When used together, they appear to complement each other and offer a high
detection rate.
The reason is that substantial evidence has accumulated associating specific
HPV's
with human anogenital disorders, most notably cervical cancer. To date, more
than 60 HPV
genotypes have been described, and about 20 of these are associated with
anogenital lesions.
These 20 can be characterized into "high risk" and "low risk" groups, in
accordance with their
association with benign and malignant tumors. As detailed in the Siadat-Pajouh
et al. article,
referred to above, a FISH assay has been developed to detect HPV 16 and 18 in
cervical smears,
these two sequences being most identified with malignant: lesions. The speed,
specificity and
accuracy of multiple HPV type identification can be greatly enhanced by the
use of multiple
fluorescent tag probes. While the conventional FISH assay is one relatively
low cost method
of HPV identification, single wavelength examination of multiple fluorescent
signals, with an
CA 02338675 2001-O1-26
WO 00/06980 PCT/US99/17004
equal number of wavelength filters (filters), is time consuming, risks
bleaching the sample, and
discards the mufti-wavelength data that define a fluorescent feature.
The multispectral topography system is a tool that enables the use of such
multiple
tagging by collecting entire spectra and using deconvolution algorithms to
accurately
differentiate the various spectral signals that contribute to an observed
fluorescent envelope.
ii. Identification of Chlamydia Trachomat~i;s,~~
As discussed above, among the techniques i:~sed for identifying CT, DiF has
been
found to be a very effective technique. In particular, CT specific monoclonal
antibodies,
conjugated to fluorescein isothiocyanate provides highly specific fluorescein-
staining. This test
is well suited to sampling and handling similar, and, as with the FISH assay,
is consistent with
that of a~PAP smear.
iii. The Dual Staining Method
Conventional FISH and DIF assays require a high quality fluorescent microscope
equipped with special objectives and multiple filters. Testing for these
conditions thus makes
I5 these techniques inappropriate for most office settings. However, using the
multispectral
imaging system of the present invention, a single cervical PAP smear.slide
that is dual-stained
with FISH labeling and DIF staining can be automatically and simultaneously
diagnosed for
both HPV sequencing and CT identification by collecting all of the wavelength
data that define
these conditions.
b. Tissue Infected with Tuberculosis Bacilli
The analysis of matter stained with fluorescing dyes such as
rhodamine/auramine
(exogenous fluorphores) is greatly enhanced with the application of the
present invention. The
following is a description of one such experiment.
Slides of tissues from patients either known to be infected with tuberculosis
(TB)
and also free from TB were analyzed. The slides were prepared with
Auramine/Rhodarnine
stain in preparation for observation under an Olympus BlEI2 epi-fluorescence
microscope. The
excitation wavelength was 510 nm and spectra were acquired through a long pass
f lter that
allowed all light of greater than 540 nm to pass, and blocked the excitation
wavelength and all
wavelengths shorter than 540 nm.
FIG. 4 shows the video image of a calibration smear known to be positive for
TB
coded cTB 11299sp3. The "observed" image camera uses false color enhancement
far easy
identification of areas in the smear with high intensity emission. FIG. 5
shows the spectral file
acquired by the spectrograph. also with false color enhancement, as observed
on the computer
monitor. Each acquisition captures a slice of the smear, as shown by the
rectangular box in
26
CA 02338675 2001-O1-26
WO 00%06980 PCT/US99117004
FIG. 4. Objects marked i and 2 correlate with the spectral image in FIGS. 5
and 6 and the
spectra in FIG. 7. The video camera of the system had no difficulty in
observing areas of
intense light for those samples with TB and a noticeablE: absence of light in
samples without
TB. FIG. 6 shows the spectral image partitioned into "channels". Each channel
is the spectrum
of an area in the slice. For simplicity in presenting this concept, only 25
channels are shown
representing an area on the smear of 8 by 2.5 hem.
Under normal operating conditions there are a maximum of 240 channels, each
channel corresponding to one row of pixels. At this resolution, each of 240
objects is 1 ,um by
2.5 ~cm when submitted with a 20X objective, and are all .acquired
simultaneously. Partitioning
is set-up either by the operator or automatically by the computer. If a filter
set is in the system,
blocking wavelengths below a certain value, it is usual to establish channels
to eliminate the
bulk of absent wavelength data. Each spectrum corresponds to a specif c
location on the
sample smear that correlate to channel number. The Y-axis provides signal
intensity that can
be calibrated to quantify the signal present against the object emitting the
signal. The bands in
FIG. 6 indicate the location of identical spectra that can be: painted back to
the original observed
image. Each of these spectra indicate the presence TB and were identified with
the USNN
described in the following section.
In most fluorescence experiments, fluorescent signals are composed of natural
fluorescence as well as from "tags". It was found that signa.t intensity
levels of a smear positive
for TB would greatly exceed the signal from natural fluorescence (also
referred to as
autofluorescence). The system produces a very large nur~zber of spectra
simultaneously, many
of which can be different from its neighbor, and indicative of differences in
condition or state
at a particular location.
To illustrate how the USNN concept works for the identification of TB, the
calibration smear ctb11299sp3 was used to train the USNN to recognize the
spectral
characteristics of a smear presenting TB, as previously shown in FIGS. 4, 5,
6, and 7. Spectral
and observed images were then captured from a smear known to be positive for
TB, coded as
tb 121 l Osp2. The same channel selection was used as for the calibration
acquisition.
FIG. 8 shows the USNN presentation of the analysis of the calibration smear,
ctb11299sp3, and the patient sample coded tb121 lOsp2.. The top left indicates
that the first
acquisition was a training set and there were 9 spectral features identified.
If the threshold
(sensitivity) of the USNN were adjusted; then it would. locate either fewer
spectra, by only
looking for gross differences, or more spectra by looking for subtle
differences.
27
CA 02338675 2001-O1-26
WO 00/06980 PCT/US99/17004
The first column shows channel number, the second the spectral object
associated
with that particular channel. The third and fourth columns show the search for
recognized
spectral objects in tb121 lOsp2. A pathologist confirmed that the area-
emitting object 1 was
clearly that of TB. The fourth column indicates that tlxere are 12 recurrences
of object 1, 3
occurrences of object 8 and some spectra that tend tovvard object 8. The
numbers such as
NM0.0607823 indicate the percent similarity to the object indicated.
Considering the signal
strength of object 8, it is probably a small amount of natuxal fluorescence
that is expected to be
present. The spectral intensity and image data provides enough information for
quantification
and digitization.
Having thus described .the basic principles and exemplary embodiments of the
.. invention, it will be apparent that further variations, alterations,
modifications, and
improvements will also occur to those skilled in the art. lFor example, it is
understood that the
system may be designed with modified optics to capture a wider or different
range of
wavelength spectra than that identified above. Further, it is understood that
the present
invention is not limited to the use of microscopes. The spectrography
subsystem may operate
with any type of light image transmitter capable of transnutting an image
mammalian cells and
tissue, such as any lens based, telescopic, system, or a fiber optic based
imaging system, such
as an endoscope. Further, the matter is not limited to prepared slides. For
example, the system
could automatically analyze the spectral characteristics of cervico-vaginal
tissue during an
actual gynecological examination by connecting the spectroscopy subsystem and
computer to
a colposcope, for example. Finally, this system is not limited to use as a
primary diagnostic
tool. It may also be used as a tool to improve quality control in the
pathology lab. For example,
during screening of PAP smears, in the event that a cyto-technician observes a
specimen from
a particular patient with abnormal or atypical cell characteristics, while
previous specimens
were reported as "negative," all previously screened samples should be
reviewed by a
cytopathologist to ascertain that no "false negative" reporting has occurred.
Further, in some
labs, a certain percentage of "negative" samples may be submitted for
rescreening by a senior
cytopathologist. The system of the present invention may be used to rescreen
such samples,
thus improving quality assurance by increasing objectivity of analysis (by
reducing dependence
on human judgment) as well as increasing the speed and lowering the costs of
the
screeninglrescreening process. Accordingly, the foregoing discussion is
intended to be
illustrative only; the invention is limited and defined only by the various
following claims and
equivalents thereto.
28
