Note: Descriptions are shown in the official language in which they were submitted.
CA 02182793 2004-12-02
i
-1-
Title: AUTOMATED CYTOLOGICAL SPECIMEN CLASSIFICATION
SYSTEM AND METHOD
TECHNICAL FIELD OF THE INVENTION
This invention relates generally to cell classification, particularly to
cytology,
and, more particularly, to a semi-automated method and apparatus for quickly
and
accurately classifying cells based on cellular morphology. This invention also
relates
to use of and the combining of the aforesaid classifying technique and results
with
immunochemical staining techniques to obtain a quantifiable index or number
value for
use in diagnostics or other purposes.
BACKGROUND OF THE INVENTION
In the medical industry there is often the need for an experienced laboratory
technician .to review a specimen of biological matter for the presence of
cells of a
certain cellular type. An example of this is the need to review a Pap smear
slide for
the presence of malignant or premalignant cells. A Pap smear often contains as
many
as 100,000 to 200,000 or more cells and other objects, each of which a
technician must
individually inspect in order to determine the possible presence of very few
malignant
or premalignant cells. Pap smear tests, as well as other tests requiring
equally
exhausting cell inspection techniques, have therefore suffered from a high
false negative
rate due to the tedium and fatigue imposed upon the technician.
Several thousand women die each year in the United States alone from cervical
cancer; a cancer from which a woman theoretically has a high probability of
survival
if detected in its early in situ stages. If not detected early, however, the
chances of
survival may decrease. If a malignant cell in a Pap smear is missed, by the
time the
woman has another Pap smear performed the cancer may have advanced.
Consequently, the importance of detecting the presence of only one or a few
malignant
or premalignant cells among the hundreds of thousands of cells in a smear
cannot be
overstated. Unfortunately, present manual screening methods are inaccurate. In
fact,
CA 02182793 2004-12-02
-2-
recently some laboratories have been found to have incorrectly classified as
benign up
to 30 % of the specimens containing malignant or premalignant cells. Also
unfortunate
is that many prior attempts to automate the cell inspection or classification
have been
unsuccessful.
Predominately, these prior attempts at automation have relied on feature
extraction, template matching and other statistical or algorithmic methods
alone. These
attempts have required expensive and time-consuming cell preparations to
distribute the
cells and other objects over a slide so that none of the cells or objects
overlap.
However, even then these attempts have been unsuccessful at accurately
classifying
specimens in a reasonable time frame.
These difficulties have been overcome by combining an algorithmic or
statistical
primary classifier with a neural network based secondary classifier as
disclosed in U.S.
Patent Nos. 4,965,725 and 5,257,182,
A commercially available automated
Pap smear screener, using a primary classifier in conjunction with a
neurocomputer
based secondary classifier is produced by Neuromedical Systems, Inc.~ of
Suffern,
New York~under trademark PAPNET'~.
The use of immunochemical staining techniques in cytology and in histology
fields is known. However, in the past there have not been satisfactory
techniques for
quantifying the examination results of an immunochemical stained specimen.
Also,
there is no automated or semi-automated analytic technique available for
diagnostics
assessment of immunocytochemical specimens or immunohistochemical specimens.
SUMMARY OF THE INVENTION
2~ The present invention provides a method and apparatus for semi-automating a
cell classification process using at least primary and secondary
classifications followed
by review by a skilled technician. Specifically, the process analyzes a
specimen which
has been stained, such as with an immunochemical marker which indicates
information
regarding the cell or cells, such as cell proliferation. The specimen is
classified and
objects within the specimen are ranked according to the likelihood that each
represents
malignant cells or cell clusters. A display of the highest ranked individual
cells and
a display of the highest ranked cell clusters is made available for review and
W0 95/227;9 PCTIUS95/00853
-3-
manipulation by a cytologist. With the aid of a computer and mouse the
cytologist can
compute various diagnostically significant information in the displays.
In accordance with an aspect of the invention, a semi-automated method of
classifying a specimen for the presence of premalignant or malignant cells,
includes the
steps of treating a specimen with an immunochemical marker to provide a visual
indication of cell proliferation or other biologic characteristic, ranking
individual
objects in the specimen in an order according to the likelihood that each
object has
attributes consistent with a premaIignant or malignant cell or cell cluster,
selecting for
display a set of said objects according to said ranking, and displaying images
of said
selected objects to facilitate review by an operator.
Preferably, the invention includes the step of permitting the operator to
select
or to eliminate cells, images of one or more cells, sometimes referred to as
or included
in "tiles", etc., thereby to edit the display to designate or to focus on
those cells of
particular diagnostic interest. The invention may also include the steps of
integrating
the stained regions or areas of certain designated displayed cells. Such
integration may
take the form of classical integration, image processing techniques, counting
of number
of cells, etc.
The present invention combines morphology with immunochemical staining
techniques as applied to cytology; and the invention also may be applied with
respect
to cell blocks or tissue sections, which also is known as or the same as an
histological
specimen. Thus, the invention relates to the field of cytopathology and to the
field of
histopathology. Where description is presented herein concerning cytological
samples
or specimens, it will be appreciated that the description also will be
appropriately
applicable to histological samples or specimens which are responsive or
susceptible to
an appropriate immunochemical stain, such as a MiB-1 stain.
Immunochemical specimens have not been amenable in the past to automated
or semi-automated quantitative techniques. Rather, such stains were used to
"work up"
a specimen. In the past there was no automated or semi-automated correlation
made
between cytological or histological objects that were stained, e.g., with an
itnmunochemical stain, and cytological or histological objects having
morphological
abnormality or some other characteristic of interest. In contrast, the present
invention
relates to obtaining a quantitative index of an immunochemical stain of a
cytological
CA 02182793 2004-12-02
-4-
specimen or an histological specimen that may correlate clinically, e.g., be a
prognostic
indicator.
According to an aspect of the invention, a cell classification technique, such
as
a classification technique of the type currently known as the 1'APNET system
and
technique
is used to find or to help find cells of interest,
e.g., those for which further diagnostic or medical consideration would be
desired or
warranted; and an immunochemical staining technique is used to provide
information
about such cells of interest. Since the stained cells are pre-selected based
on the
aforesaid classification technique, a quantification of the results of the
staining of such
cells can be made. Such quantification, then, can be based on the selected
cells, rather
than on the entire specimen.
The present invention in a sense combines morphology with immuno-stain
process analysis of cytological specimens. As a result, it is possible to
obtain a
quantification of activity, proliferation, or other processes or
characteristics of the
cytological specimen.
In the past a meaningful quantitative index or value relating to immuno-
staining
could not be obtained in an automated or semi-automated process. Rather, in
the past
the positive-staining areas of an entire specimen on a slide could be
integrated; but this
did provide useful quantification. In the present invention postivt-staining
cells or cell
clusters are selected by the classifier and from those selected a meaningful
quantitative
index can be generated.
These and other objects, advantages, features and aspects of the present
invention will become apparent as the following description proceeds.
2$ To the accomplishments of the foregoing and related ends, the invention,
then
comprises the features hereinafter fully described in the specification and
particularly
pointed out in claims, the following description and flue annexed drawings
setting forth
in detail a certain illustrative embodiment of the invention, this being
indicative,
however, of but one of the various ways in which the principals of the
invention may
be employed.
y1'O 95122749 ~ PCT/US95IOD853
-5-
BRIEF DESCRIPTION OF THE DRAWINGS
In the annexed drawings:
Figure 1 is a schematic illustration of a cytological classification or
screening
device in accordance with the present invention;
Figure- 2 is a -diagrammatic illustration of the scanning passes which the
sct'eening device performs;
Figure 3 is a ~chematic illustration of the screening device of Figure i with
particular emphasis on the processing system; and
Figure 4 is a flow diagram schematically illustrating the flow of image data
through the classification device.
DETAILED DESCRIPTION OF THE INVENTION
With reference to the several figures in which like reference numerals depict
like items, and initially to Figure 1, there is shown a semi-automated cell
classification
device 10 in accordance with the present invention. Briefly, the device 10
includes an
automated optical microscope 12 having a motorized stage 14 for the movement
of a
slide 16 relative to the viewing region of the viewing portion 18 of the
microscope, a
camera 20 for obtaining electronic images from the optical microscope, a
processing
system 22 for classifying objects in the images as likely to be of a
predetermined type,
and a memory 24 and a high resolution color monitor 26 for the storage and
display
respectively of objects identified by the processing system as being likely to
be of that
predetermined type.
In its preferred embodiment the classification device 10 is semi-automated. In
other words, the processing system 22 performs a classification of cells and
cell
clusters in a specimen and ranks the cells and clusters in an order according
to the
likelihood that each cell and cell cluster is, for example, a malignant or
premalignant
cell or cell cluster. The cells and cell clusters may be cytological
specimens; also, the
cell clusters may be a cell block, tissue section, otherwise known as an
histological
specimen, etc.
The cells and cell clusters which are ranked as most likely to be a
premalignant
or malignant cell or cell cluster are displayed on the monitor 26 for review
by a skilled
technician who can then make a final determination regarding whether each
displayed
cell and cell cluster is premalignant, malignant or otherwise indicative that
further
21~2~93
WO 951227.19 . PCTlUS95100853
~ ' 3
-6-
medical attention to the patient from whom the specimen was obtained is
justified.
Preferably. the display of cells classified as likely to be malignant, for
example, is
separate from the display of cell clusters classified as likely to be
malignant, such as
by presenting the groups of cells or cell clusters one after another on the
same monitor
26.
In view of the semi-automated nature of the classification device 10, the
microscope 12 will preferably include, in addition to the motorized stage 14,
automated
apparatus for focussing, for changing lens objectives between high and low
power, and
for adjustment of the light incident of the slide, as well as circuitry for
controlling the
movement of the motorized stage, typically in response to a command from the
processing system. The microscope may also include an automated slide
transport
system for moving the slides containing the specimen to be classified on to
and off of
the motorized stage, a cell dotter for marking relevant areas of the slide,
and a bar code
reader for reading encoded information fiom the slide. An example of a
microscope
performing at least some of these functions is manufactured by Carl Zeiss,
Inc. of
Germany, and a suitable motorized stage is manufactured by Ludl Electric
Products,
Ltd. of Hawthorne, New York.
In accordance with the invention the automated microscope 12 preferably
performs three scans of the slide having the specimen disposed thereon, as
shown
diagrammatically in Figure 2. The first scan of the slide is performed at a
relatively
low magnification, for example 50 power, and is called the low resolution scan
(30).
The second scan is performed at a higher magnification, for example 200 power,
and
is called the high resolution scan (35). The third scan is referred to as the
high
resolution rescan and is also performed at a high magnification (40).
During the first scan (30) of the slide, approximate focal planes for the
specific
areas of the slide are found and it is determined whether that area of the
slide contains
a portion of the specimen. Once a low resolution scan (30) has been performed
of the
whole slide, and the focal planes and areas of the slide containing the
specimen have
been logged, the high resolution scan (35) is performed.
The high resolution scan (35) is performed only on the areas of the slide
found
in the low resolution scan (30) to contain a portion of the specimen.
Consequently, the
comparatively long high resolution scan (35) is performed only on relevant
areas of the
W0 951227-19 PCTlUS95100853
slide and the processing time is greatly reduced. During the high resolution
scan (35),
the automated microscope 12 scans the relevant areas of the slide, and the
camera 20
takes electronic images of these areas and sends the images to the processing
system
22. The processing system 22 performs a primary classification of the image
which
fords the centroids of biological objects having attributes typical of the
cell class for
which screening is being performed, such as malignant cells. Using a smaller
sub-
image centered around these centroids, the processing system 22 performs a
secondary
classification which assigns each centroid a value indicative of the
possibility that the
object having that centroid is a cell of the type for which classification is
being
performed. Simultaneously, the centroids are also ranked based on the value
assigned
through the secondary classification.
Upon completion of the high resolution scan (35), the high resolution rescan
(40) is performed for, preferably, the highest ranked sixty-four cells and
highest ranked
sixty-four cell clusters, although other numbers of cells and cell clusters
may be
selected by the system. During the rescan (40) the automated microscope 12
will move
to each of the highest ranked centroids of the cells and cell clusters and the
camera 20
will obtain a high resolution color image of the object having that centroid
as well as
the contextual image surrounding the object. These high resolution images are
then
stored in the memory 24 which may be a removable device, such as an optical
disk or
a tape, etc., or a fixed storage device such as a hard disk. Alternatively,
the images
may be transferred to another computer via a network or through transportation
of the
data on a removable storage media.
The sixty-four cell images and sixty-four cell clusters make up two separate
summary screens, one for the cell images and one for the cell clusters. The
number
of images, e.g., 64, is exemplary only; other numbers can be used. The number
may
be uniform or it may be a function of the ranking, e.g., reflective of the
degree of
abnormality or interest of the specimen or cells) in the specimen. Thus, if
there are
many cells of interest, the number may be greater than 64; if there are only a
few cells
of interest, then the number may be less. Each summary screen is preferably
arranged
~ 30 as an 8 x 8 matrix of high resolution color images, or tiles, featuring a
suspect cell or
cell cluster in the center of each image. It will be appreciated, however,
that other
numbers of images may be displayed concurrently to produce a summary screen,
such
WO 95/227.19 ~ ~ ~ PCT/US95100853
_g.
as a 4 x 4 matrix. These summary screens are displayed on the high resolution
color
monitor 26 for tertiary analysis and classification, for example, by a
cytologist (or if
an histological sample, for example, an~histologist). This analysis may take
place at
anytime after the classification process by the processing system 22 has been
completed. Further, through the use of a removable memory device or a network
connection, the images of the summary screens may be transferred to a work
station
remote from the microscope 18, camera 20 and processing system 22 for display
and
analysis. In such an instance a separate graphics processor 41 may be employed
to
drive the high resolution color monitor 26 and provide a suitable interface
with the
cytologist.
Herein the screening method and apparatus of the present invention will be
described as used in screening a Pap smear for the presence of cervical cancer
cells.
However, it will be apparent to a person of ordinary skill in the art that
this is only an
illustrative use and that the present invention may be used in screening
samples of other
biological matter taken by a variety of cell sampling techniques, such as
aspiration and
exfoliation to name but two. Further it will be apparent that while the
illustrative
example screens for malignant or premalignant cells, the screening may be
performed
for the detection of other cell classes or types. As is mentioned elsewhere
herein, the
specimen may be a cell block or tissue section, also known in the art as an
histological
specimen.
Turning now to a more in-depth discussion of the present invention with
specific
reference to Figure 3, the screening device 10 is shown with particular
emphasis on the
classification elements embodied in the processing system 22. The processing
system
22 preferably includes an image processor and digitizer 42, neurocomputers 44
and 45,
and a general processor 46 with peripherals for printing, storage, etc.
The general processor 46 is preferably an Intel~ 80486 microprocessor or
similar microprocessor based microcomputer although it may be another computer-
type
device suitable for efficient execution of the functions described herein. The
general
processor 46 controls the functioning of and the flow of data between
components of
the device 10, may cause execution of additional primary feature extraction
algorithms
and handles the storage of image and classification information. The general
processor
46 additionally controls peripheral devices such as a printer 48, a storage
device 24
W O 9512279 ~,,~, ~ ~ ~~~1 PCT/US95/00853
-9-
such as an optical or magnetic hard disk, a tape drive, etc., as well as other
devices
such as a bar code reader S0, a slide marker 52, autofocus circuitry, a
robotic slide
handler, the stage 14, and a mouse 53.
The image processor and digitizer 42 digitizes images from the video camera
~ S 20 and performs a primary algorithmic classification on the images to
filter out
unwanted information. The image processor and digitizer 42 (hereinafter
collectively
referred to as the image processor) performs two separate filtering
operations. In one
J
operation, the images of the specimen taken through the microscope 12 by the
video
camera 20 are filtered to fmd images representative of individual cells which
have
attributes of a premalignant or malignant cell. In a separate operation, which
is
preferably performed simultaneously with the first filtering operation, cell
clusters
having attributes of premalignant or malignant cell clusters are found.
Secondary cell classification is performed by the neurocomputers 44 and 45.
Each neurocomputer is =rained to handle the specific types of images provided
by one
of the filtering operations of the image processor 42. For example, the
neurocomputer
44 is trained to recognize individual premalignant and malignant cells among
the output
of one of the filtering operations of the image processor 42, while the
neurocomputer
45 is trained to recognize premalignant and malignant cell clusters among the
images
output by the other filtering operation. Alternatively, secondary cell
classification
functions could be performed using a single neurocomputer, or a template
matching
algorithm designed to identify shapes known to be typical of a pathological
cell or cell
cluster. A template matching or other group processing algorithm could be
efficiently
implemented in a parallel distributed processing network, for example. Another
alternative secondary classification embodiment is a holographic image
processor
designed to perform group based class~cation.
The image processor 42, the neurocomputers 44 and 45, and the general
computer 46 may each access read-only and/or random access memory, as would be
readily apparent to one skilled in the art, for the storage and execution of
software
necessary to perform the functions described relative to that processing
component.
Further, each component 42, 44, 45, 46 includes circuitry, integrated circuit
chips, etc.
for the control of communication or data transfer over the data bus 54 as well
as other
functions typical of similar processors as would be appreciated.
CA 02182793 2004-12-02
-1~-
Referring to Figure 4, there is shown a data flow diagram for the processing
system 22. As shown, the image digitizer and image processor 42 is divided
into three
functional elements, 42a, 42b and 42c, to best represent the functional flow
of image
data through the image processor 42. When the automated microscope 12 focusses
on
S an image in the specimen being analyzed during the high resolution scan
(35), the
image is presented to the image digitizer 42a which converts the image into a
digital
format. The digital image data is then subjected to two separate
classification processes
42b and 42c designed to perform different filtering operations on the image
data. The
algorithmic classification functions 42b and 42c perform the primary
classification.
The algorithmic classifier 42b is analogous to a band-pass filter which allows
individual cells having morphological attributes consistent with a
premalignant or
malignant cell to pass through the classifier 42b. The classifier 42b filters
out other
objects and fragments, including cells which are too small to be a malignant
cell, such
as a neutrophil, cells which are too large, cell clusters, and cells which
have other
features distinguishing them from malignant cells, for example red blood cells
which
are distinguishable by their intensity or brightness in the representative
image.
Many types of malignant cells tend
to be dissociative and thus the algorithmic classifier 42b is optimized to
find these
dissociative cells which have the appropriate morphological characteristics of
a
premalignant or malignant cell and to pass those cells on to the neurocomputer
44 for
further classification.
The algorithmic classifier 42c is analogous to a band-pass filter which passes
cell groups or clusters and rejects individual, or dissociated, objects and
cells. Some
types of cancers tend to form cell clusters, for example adenocarcinomas,
while other
cell clusters, for example, multicellular epithelial fragments and glandular
clusters, may
have other diagnostic significance to a reviewer, as is discussed more fully
below. The
algorithmic classifier 42c is optimized to pass images of significant cell
clusters to the
neurocomputer 45 for further classification.
Preferably the algorithmic classifiers 42b and 42c, as well as the image
digitization function 42a, are combined in a single processor 42 which permits
both
sequential and simultaneous processes. In the preferred embodiment, the image
CA 02182793 2004-12-02
-11-
processor and digitizer 42 is a low level morphological feature extraction
image
classifier which includes among other things an image digitization function
and an
ISMAP (Iconic to Symbolic Mapping) board.
Alternatively, the image processing and digitization
functions could be separated into two or more components.
The secondary classification is performed by the neurocomputers 44 and 45.
Each neurocomputer 44, 45 is trained to recognize cells or cell clusters,
depending
upon its training, which are most likely to represent premalignant or
malignant cells or
cell clusters. The neurocomputer 44 and 45 are provided image data from the
algorithmic classifiers 42b, 42c, respectively, which is consistent with its
training. For
example, the neurocomputer 44 is trained to recognize individual premalignant
or
malignant cells and is provided with image data which the algorithmic
classifier has
identified as having the morphological attributes of individual premalignant
or
malignant cells. Similarly, the neurocomputer 45 is trained to recognize
premalignant
or malignant cell clusters and is provided with image data which the
algorithmic
classifier 42c has identified as having the morphological attributes of
premalignant or
malignant cell clusters.
As noted above, the neurocomputer 44 is provided with image data from the
primary algorithmic classifier 42b which consists of images of cells and other
objects
which the algorithmic classifier has identified as having attributes
consistent with
dissociated premalignant and malignant cells. In accordance with its training,
the
neurocomputer 44 assigns images in the data which are likely to represent
premalignant
or malignant cells with a relatively high value and assigns images that to the
neurocomputer are less likely to represent malignant cells, for example benign
cells,
with a lesser value. The sixty-four highest ranked cell images, i.e., those
cell images
most likely to represent premalignant or malignant cells are provided to the
general
computer 46.
The neurocomputer 45 is provided with image data representing cell clusters
identified by the primary algorithmic classifier 42c. The neurocomputer 45, in
accordance with its training, assigns images in the data which are likely to
represent
premalignant and malignant cell clusters with a relatively high value while
assigning
~1~2'~~:~
R'O 95122749 PCTIUS95/00853
-12-
images which are likely to be benign with a lesser value. This time, the
highest ranked
sixty-four images of cell clusters are provided to the general computer 46.
The neurocomputers 44 and 45 are each preferably a computer embodiment of
a neural network trained to identify suspect cells or cell clusters. In this
embodiment
the parallel structure of a two or three-layer backpropagation neural network
is
emulated with pipelined serial processing techniques executed onany of a
number of
commercially available neurocomputer accelerator boards. The operation of
these
neurocomputers, for example, is discussed in Hecht-Nielsen, Robert,
"Neurocomputing:
Picking the Human Brain"> IEEE Spectrum, March, 1988, pp. 36-41. The
neurocomputers are preferably implemented on an Anza Plus'" processor, which
is a
commercially available neurocomputer of Hecht-Nielsen Neurocomputers. Such a
neurocomputer could be easily configured to operate in a manner suitable to
perform
the secondary classification functions by one of ordinary skill in the art
through
reference to corresponding manuals, etc.
Preferably each neurocomputer 44, 45 is trained with images which have been
classified by the corresponding primary algorithmic classifier and then
reviewed by a
person skilled in analyzing cytological specimens for the presence of
malignant cells
and cell clusters. Such a person can readily review the cells and cell
clusters which
have been classified by the primary algorithmic classifiers 42b and 42c and
determine
whether the cells or cell clusters are benign or malignant. Operating in a
training
mode, the neurocomputer 44 is trained with images of individual objects
classified as
having morphological attributes of a malignant cell by the algorithmic
classifier 42b
which a skilled technician has determined to be malignant or benign. The
images are
fed through the neurocomputer and the neurocomputer is informed by the
reviewer of
the correct classification of the image as benign or malignant. Based on a
series of
such inputs the neurocomputer can organize itself to associate a known benign
image
with an output of .1 and a known malignant image with an output of .9. Such
outputs
represent, for example, the degree of certainty that a cell is normal or
abnormal,
respectively. When the secondary classifier is presented with new, unlrnown
cells, it
generalizes from its training and attaches a net value to the image. The
closer that the
secondary classifier is able to categorize the unknown image into the benign
category,
the closer is its net value equal to .1. Conversely, the more closely that the
unknown
2:1~~'~~~
W 0 95/22749 PCT/U595100853
-13-
image appears to resemble the nonbenign images of its training set, the closer
is the net
value assigned to that image equal to .9.
The neurocomputer 45 is trained in the same way but with images of cell
clusters which the primary algorithmic classifier 42c has identified as having
~ 5 morphological characteristics of premalignant or malignant cell clusters.
These images
are further classified by a skilled person who then assigns the images a value
between
.1 and .9 depending on whether the cell cluster is benign or malignant, and
the
neurocomputer is provided with the known images and their respective values
assigned
by the skilled reviewer.
High resolution images of the cells and cell clusters ranked by the
neurocomputers 44 and 45 as most likely to be representative of malignant
cells or cell
clusters are obtained through a rescan (40) of the automated microscope of the
relevant
areas on the specimen. The general computer 46 then stores the image in the
memory
24, which may include an optical disk or tape, and makes the images available
for
viewing on the monitor 26 in separate summary screens or pages of information
or for
some other use. As discussed above, the first summary screen is preferably a
display
of the sixty-four cells classified by the neurocomputer 44 as most likely to
be malignant
cells. These cells and surrounding material on the slide, called tiles, are
displayed in
an arrangement, such as an eight by eight matrix, which facilitates review by
a
technician. A cytologist can easily visually scan this summary screen in
search of
images of cells having the attributes for which classification is being
performed. If the
system is being used to screen a Pap smear for the presence of cervical
cancer, the
cytologist would typically examine visually the first screen for cells having
attributes
of a malignant or premalignant cervical cell, such as a comparatively large,
dark
nucleus. The cytologist is preferably permitted to interact with the general
computer
46 through a mouse 53 to identify certain cells as malignant, to make other
determinations or to cause the general computer to execute other commands or
display
different material.
The second screen, which includes images of cell clusters, is typically
examined
- 30 for grouped glandular cells, endocervical cells, endometrial cells, and
cell groupings
peculiar to certain types of cancers, for example, adenocarcinomas. The
presence of
adenocarcinoma or any other malignant cells or cell clusters is of course
direct evidence
W0951227.19 ~ PCTlUS95100853
-14-
of cancer. However, other material on the second summary screen may also be of
relevance to the cytologist. For example, the presence of endocervical cells
facilitates
a determination that the specimen contains an adequate sampling of the
transitional zone ,
within the cervix where evidence of cancer is most likely to occur in the
earlier stages
of cancer. Further, the presence of endometrial cells in certain patients,
such as post
menopausal patients, may also indicate the presence of abnormal conditions
including
cancer. Manipulation of the screen display as well as interaction with the
general
computer 46 is again permitted through use of a mouse 53 or other interactive
devices
including a keyboard.
The Pap smear specimens used herein as an illustrative specimen are obtained
by staining a cervical smear with Papanicolaou stain or a similar stain for
highlighting
the nuclei of the biological matter by staining the nuclei a purple color so
that they are
easily discernible from the surrounding cytoplasm. Other stains, such as
immunolabeling stains, are specific to certain activity or occurrences within
a cell and
can detect, for example, cell mitosis or division. One such stain is MiB-1
which is a
monoclonal antibody to certain antigens expressed in a nucleus of a cell
around the time
of mitosis. Consequently, the MiB-1 stain cari be used to provide a visual
indication
of cell proliferation. As it is known that malignant cells tend to proliferate
at a much
faster rate than benign cells, an indication of a region in a specimen
undergoing
significant proliferation can provide important diagnostic information to the
person
reviewing the sample. This is especially true in multicellular fragments or
clusters
where the occurrence of proliferation is high.
The MiB-1 stain effectively dyes the nucleus of a cell brown while not
significantly coloring the cytoplasm. In the instance of a Pap smear specimen
the MiB-
1 stain is useful in providing diagnostic information on the specimen by
itself or in
conjunction with conventional Pap smear staining. When used in connection with
a
conventional Pap smear staining, the stained Pap smear is destained using
known
techniques and then restained with the MiB-1 stain in a procedure such as that
described below. When it is not necessary that the specimen be first screened
with a
Pap smear stain or when a different type specimen is used for which screening
with a
conventional stain is not desired, the MiB-1 stain may be applied to the
specimen
following a procedure such as that below without first destaining the
specimen.
~18~'~~~
W0 95122749 - PCT/US95/00853
-15-
A suitable procedure for applying the MiB-1 stain to a previously unstained or
destained specimen includes retrieving the antigens in the specimen such as
by:
1) placing the specimen slide in plastic jar containing lOmM citrate buffer,
pH 6.0;
2) placing the plastic jar containing the specimen and citrate buffer in a
microwave oven;
3) microwave heating the specimen to 100 degrees Celsius and maintaining
the specimen at 100 degree Celsius for approximately 20 minutes;
4) cooling the specimen in the citrate buffer below a maximum temperature
of 50 degrees Celsius; and
5) rinsing the specimen in TBS for approximately 2 minutes.
Once the antigen retrieval procedure is completed, the specimen is stained
with
the MiB-1 stain such as through the standard SAB protocol using an antisera
dilution
of 1:200 and counter staining with Haematoxylon. The standard SAB protocol is
discussed in detail in Kok & Boon, "Microwave Cookbook for Microscopists: Art
and
Science of Visualization," Third edition, Coulomb Press Leyden, Leiden (1992).
Subjecting a MiB-1 stained specimen to the classification device 10 described
above will result in the system producing two summary screen of objects for
review by
a cytologist. The first summary screen again contains images of individual
objects and
their contextual surrounding matter that the classification system ranked as
most likely
to represent premalignant or malignant cells that also stain positively for
the
immunochemical stain. The second screen contains cell clusters, such as
multicellular
epithelial fragments which the classification system ranked as most likely to
represent
premalignant or malignant cell clusters that also stain positively for the
immunochemical stain. As a result of the MiB-1 staining, and the selectivity
of the
stain to proliferating cells and their expressed proteins, the summary screens
will
contain individual cells and cell clusters which may be undergoing, preparing
for or
having completed mitosis as well as a number of cells that stain positive but
are not
malignant or involved in malignant processes, for example, Herpes Virocytes,
cells
from lymph follicles (follicular cervicitis) and cells from tissue repair. A
cytologist can
readily distinguish the premalignant and malignant cells and cell clusters
from other
material and benign processes which are not of interest.
~~82 ~~~
W0 95122749 PCTIUS95100853
,,
=16-
The result of subjecting the MiB-1 stained specimen to the classifrcation
device
is that, not only are morphologically significant cells detected, but also
those bearing
evidence of proliferation. Consequently, the cytologist examining the specimen
can ,
determine the presence of malignant cells and the degree of proliferation of
malignant
5 cells. Such information allows a doctor to determine how aggressive of a
treatment
program is necessary for the patient from whom the specimen was obtained.
In some instances the immunostaining may increase the sensitivity of the
neurocomputers 44 and 45 in recognizing premaIignant and malignant cells. The
neurocomputers 44 and 45 alternatively may be trained with specimens which
have
10 been stained with an immunochemical stain.
The cytologist can use the mouse 53 to select cells or cell clusters from the
display 20 for grouping on the summary screen with other cells or clusters, to
delete
cells or clusters from the display as well as to command the general computer
46 to
perform certain processing or calculations on the information in the summary
screen.
For example, a count of the number of proliferating cells in a specimen is a
diagnostically significant measurement. The cytologist can review the summary
screens, edit the display to remove certain objects or to designate others for
counting,
and then instruct the general processor 46, such as through use of the mouse
53 or a
keyboard, to count the number of proliferating cells in the display. The
cytologist can
also mark cells in a tile of the summary screen, for example individual cells
in an
epithelial cell fragment, as positive-staining or negative staining and
instruct the general
computer 46 to determine the percentage of positive-staining nuclei in a tile
or
fragment. The classification can also be tailored to provide a total count of
the brown
stained cells in the specimen, and in conjunction with the neurocomputers
andlor the
cytologist, a relative count of stained benign cells to stained malignant
cells.
Another diagnostically significant measurement is the area of the display that
is
stained. This is particularly relevant for cell clusters, such as epithelial
fragments,
which are displayed on the second summary screen. Again, using the mouse 53
for
example, the cytologist can review the images on the second screen, edit the
screen by
removing images, selecting images to form a separate screen of for example 16
significant images, and command the general computer 46 to determine the area
of the
display that is stained by the MiB-1 stain. The general processor 46 then
calculates the
NO 95f22749 PCT/US95/00853
-17-
stained area, such as a percentage of the total screen area, using
conventional
algorithmic techniques, and provides the result to the cytologist.
Through the interaction of the cytologist with the general processor 46
through
use of a mouse 53 or similar interactive tool, and the ability to edit the
images on the
screen and identify characteristics of cells in the display or within
individual tiles in the
display, the technician can instruct the system to provide a large amount of
diagnostic
information which is specifically tailored to the specimen reviewed and the
cellular
material found"in the specimen.
While the example given here of an itnmunolabeling dye is MiB-1 other types
of immunochemical markers can be used in equivalent ways to indicate the
desired
characteristics in the specimen, such as proliferation or other
characteristics. One
supplier of immunochemical labels is ICN Biochemicals although many other
suppliers
are available. Further, the classification performed by the image processor 42
and
neurocomputers 44 and 45 can be tailored to select images for display based on
the
detection of a feature made detectible by the immunochemical marker.
Another example of the invention includes the classifying of a cytological
specimen or an histological specimen. Such classification determines cells or
groups
of cells of interest for further consideration, e.g., medical or diagnostic
consideration.
According to the invention, a technique described above using the PAPNET
system
provides primary and secondary classification steps and includes use of a
neurocomputer or neural net in the classification process. The classification
method
may be based on morphology. The classification methodology may be expanded or
extended beyond morphology to include criteria related to the immunochemical
staining
of the cells or groups of cells in a cytological specimen or histologicaI
specimen.
An example of such expanded processing includes using information relating to
the amount of stained material in the objects, tiles, cells, or groups of
cells, etc. which
were selected by the classificr° wn process. Such use may include
counting the number
of stained objects in the tile:; .::at were selected for display. The expanded
processing
may include integrating the stained regions or area of the selected tiles that
were
selected for display. Another type of such expanded processing may include the
counting of the number of tiles that were selected for display which tiles
also include
some or at least a predetermined amount or proportion of stained material,
stained
WO 951227x9 PCT/US95100853
-IS-
region or stained area. These are but several of the ways that the cells,
areas where
selected cells are located, tiles, etc., determined in'the classification
process, can be
processed to quantify the immunochemical staining in conjunction with selected
cells
or groups of cells of morphological interest. In this expanded processing
technique,
it is possible that the system determines the quantifiable value automatically
by ..
conventional image processing techniques, image analysis techniques and/or
area
integration algorithm techniques without intervention by an operator.
Another example of such expanded processing involves a semi-automated
approach. In this approach an operator views the display and the cells or
groups of
cells selected in the classification process. The operator then can edit the
displayed
sample/specimen for determining what is to be further considered for the
quantification
process. The operator undesignates (de-selects), e.g., by pointing a mouse
cursor at
or in a particular displayed tile and clicking the mouse button, cells, groups
of cells,
objects, tiles, etc. which are not to be included in the expended processing
of the type
IS described above, such as image processing/analyzing and integration of the
stained
areas, etc.
An alternate example of such expanded processing involves a modified semi-
automated approach similar to that described just above. In this approach an
operator
views the display and the cells or groups of cells selected in the
classification process.
The operator designates (selects), e.g., by pointing a mouse cursor at or in a
particular
displayed tile and clicking the mouse button, cells, groups of cells, objects,
tiles, etc.
which are to be included in the expanded processing of the type described
above, such
as integration of the stained areas, etc.
STATEMENT OF INDUSTUrar. APPLICATION
The present invention is applicable to cell classification in general and is
particularly applicable to the classification of cells in a cervical smear.
