Note: Descriptions are shown in the official language in which they were submitted.
1(36QlZ'~ ~ ~
Thc prcsent lnvention rclates to subJect analysis ~ ~ -
methods and systems in general and, more particularly, to
a method and apparatus for partlcle analysis which utilizes ~-~
color algebra and image processing techniques.
The preferred cmbodiment for this application i9
very similar to that in applicant's copending application ;~
Serial No. 179,798, filed August 28, }973. However, to
avoid confusion and numerous referencing, much of the descrip-
tive material in the original application will be repeated ;
here.
The need for an accurate, fass and relatively inexpeusive
system for anlyzing particulate matter entrained in a gas
or liquid exists in many fields of current technology. ~or
exflmple, recent activities in the area of pollution nnalysis and
. ~ .
control have emphasized the need for a means for particle
identification, classification and morphology analysis. A
similar need also exists in the field of medical technology
for automating labor intensive medical laboratory procedures,
such as blood analysis.
The recent spiraling rise of medical care cost have ;~
raised the hope that these costs could be reduced by the appli-
cation of automation technology to the labor intensive procedures
used in the medical field. One of the most fundamental tests
performed in the most cursory examination or treatment of a
patient is blood analysls. Blood has three maJor particle ;
components: red blood cells (RBC), white blood cells (WBC) !'~
and platelets, suspended in a fluid tPlasma). An analysis of ~; ;
the relative and absolute quantities of these particles,
and additional information regarding their morphology (form
and structure) provide considerable insight into the state of
health of the patlent.
2 ~ -
~vb/sl
: ' ,.;''.. .. ' "'' :.:' ~:, ' : ` '' .: ' ' ' : ::,. ::, .:.' '" ,' : ' ~ ' ` ': : '
.: . . " . . , :: . ~'' ` ' '. ,' ' ' .'. :' ' ' .-: ' ~ ' :' '
":' ' ~ ' ' :: `: :' ' ' ' ' ' ' '
:: '-:' . . :
- ' 10~0~
At the present time, several companies have successfully
developed and marketed instruments for automated blood analysis.
Technion's SM~ system for plasma analysis ancl Hemaloy System
for cell analysis, and Coulter Electronics' Model S cell counter
are well known examples. Using different technologies, the
Hemalog and Coulter S generally provide a cost-competitive
count of the various particle constituents present in a blood
sample. The basic concept, common to both techniques, involves
flowing a thin column of diluted blood past a sensor which ~ '~
detects whether a solid particle is present in the liquid medium.
This concept, commonly called "flow-through", provides a count
of the particles present, but does not provide any qualitative
information regarding the identity of these particles or of
their morphology. ;
Therefore, it is necessary to pre-segregate the sample to
determine whether the instrument is counting a RBC or a WBC.
These two types of cells have significantly different chemical
properties, so they can be separated relatively easily. Howeuer,
it is not possible to further automatically differentiate these
cells according to their individual morphological differences
using currently available commercial technology. ` ;
Nevertheless, such a differentiation is extremely important
in about 25 percent of all hospital patients, and it is hlghly
desirable in 50 percent of the patients. This is particularly
!
true of the numerous types of WBC's whose relative concentration
26 and individual morphology are extremely important. Of lesser ~;~
Ç~ 3
~ ()12'~ ~importance, but still signi~icant is the detection of abnormal
red cell morphology. These measurements, commonly known as
the "Differential" count, are currently performed by manual
labor. `
There are two basic approaches to dif~erentiating a single
cell by morphology; a direct or pattern recognition approach ;~
and, an indirect approach. The latter relies on there being . -~:
indirect signatures of chemical differences which have a high
degree of correlation with the direct signature of morphological
differences in the basic WBC's type.s. Technion's Memalog-D, 't ~ ~ '
employs this approach, using enzymatic stains as the chemical
signature to separate five basic WBC types. `` ~ ;
As in all indirect techniques, there are both theoretical .:
and practical sources of error. For example, abnormal varia-
tions within any of the five basic WBC groups cannot ~e
detected. In a high risk hospital population, 10 percent
to 20 percent of the patients may have relatively normal . ~.
distribution among the five chemical groups, but still have
morphological abnormalities indicative of a pathology. In ~.
other words, the morphological/chemical correlation is incom-
plete, resulting in false negatives, the most serious type of
error. Furthermore, a percentage o~ any healthy population :
will have unusually low enzyme levels with no accompa~ying
morphological abnormalities or clinical symptoms thus resulting
in uneconomical false positives. In addition, ~ ;
~b! ~ 4 ~ ~
. .: . . ; ::: ::: : .. . :~ : .
the important RBC morphology is not provided by the indirect
technique.
In the direct approach, the morphology of the particles
or cells is examined direc-tly using computer pattern recognition
techniques. Performing the blood cell differential measurement .
using pattern recognition techniques is within the current state
of the Corning, Geometric Data Corp., and Coulter have announced
instruments using these techniques. However, these early
instruments, in order to be practical from a cost standpoint
utilize designs which are too slow and do not automate the analysis ~ :
of abnormal white blood cells or red blood cells. The same
general problems exist in other fields of technology employing ;.
particle analysis techniques.
STATEMENT OF INVENTION
~ . . . . . . . . .
This application endeavors to disclose a method of .:
analysis which comprises the steps of illuminating a particle
., :
containing sample with a light modified by the sample; producing
a first signal representing a first predetermined wavelength band : :
of the sample modified light at a region in the sample; producing :
a second signal representing a second predetermined wavelength
band of the sample modified light at said region, the f.irst and
second wavelength bands being selected to produce a differential
contrast between said region and at least one other region in
the particle sample; and alyebraically combining with thresholding
the first and second signals to classify the sample region in at
least one of a predetermined number of categories. : ~:
It is, accordingly, a general object of the invention . .`.
to provide an improved system for subject analysis ~ :
It is a specific object of the invention to provide a ~-
mb/~ 5 ~
Z,~Z
particle analysis system which employs color algebra in conjunction ;~
with image processing techniques to analyze scene regions. ~
It is another specific object of the present invention ~ -
to provide, as one embodiment thereof, a commercially feasible
automated blood differential measurement system. ;~
It is a further object of the present invention to ~;;
employ color algebra techni~ues which permit the use of simplified
algorithms to analyze scene regions.
It is still a further object of the present invention
to provi;de an automated blood differential measurement system `
which employs scanning and data processing components which, in ;~
conjunction with color algebra techniques, drastically reduce
both the computer capacity requirement and the processing time.
. ,
'
. ~; . .
mb/~iJ'~ - 5a -
: ' ' . , ' ,', ~ , ' :
1~0~22
It is a feature of the present invention that the auto~
mated blood differential measuremenk system embodiment provides
increased accuracy over existing systems due to the inherent
superiority of a direct measurement technique over an indirect
S measurement technique together with the additional ability
to make finer distinctions between WBC's in any one of the - .
five basic types, the ability to recognize abnormal v. normal
morphology; the ability to provide RRC measurements; and the
ability to distinguish between cell regions and other scene .~
regions. :
It is still another feature of the blood analysis embodi~
~, . . .
ment of the present invention that ~onventional blood staining .
procedures can be employed with the color algebra technique of
the invention. . .
These objects and other objects and features of the present
inv~ntion will best be understood from a detailed description
of a preferred embodiment thereof, selected for purposes of
illustration, and shown in the accompanying drawings in which:
Figure lA is a functional block diagram of the blood
analysis embodiment of the invention; :~
Figure lB is a more detailed functional block diagram ~:
of the invention showing data flow;
Figures 2A through 2C are representative histograms of ~ ;
a blood sample; and
Figures 3 through 9 depict in partial block and diagrammatic ~ ~
form the blood analysis embodiment of the invention. ~.
The particulate matter analysis system of the present
invëntion can be used for analyzing many different types of
29 particulate matter. However, for purposes of illustration ~;.; :
~ ;~
c~/ - 6 -
.
~o~z~
and ease of description, the following discussion will be ,
directed to the blood analysis embodiment of the particulate
matter analysis system as shown in functional block diagram form
in Figure 1. ; ;~
The present invention utilizes color al~ebra techni~ues
to reduce both the computer capacity requirement and the pro-
cessing time. Before proceeding with the detailed description
of the present invention, it will be helpful to briefly review
some basic information with respect to "color".
The perception of color is a complex physiological pheno- ` ;
menon which occurs in response to variations of the spectral
components of visible light impinging upon the retina. The
quantitative description of "color" is complicated by the fact
that the same perceived color can be produced by numerous com-
binations of different spectral components.
In order to standardize the description of colors in
scientific work, a system of "chromaticity" measurements was
developed by the C.I.E. (Commission Internationale de l'Bclairage)
in 1931. The chromaticity measurements are obtained by convolving
the spectral components of the illumination with three specific
spectral distributions to produce "Red", "Green", "Blue" inten-
sities. The percenk fraction of each of these intensities is
expressed as X, Y, and Z coordinates, respectively, where:
X = R
R + G + B -
y a ( ~ ,
R ~ G ~ B
Z = B
R + G + B
29 ~
a1,~ ~ 7 ~ ;~: -
, : : : ~ ,, - ~ ,
:-. .
601;ZZ ~ ~
The three spectr~l distributions have been established so that
any combination of wavelengths which produces the same subjective
color will also produce the same chromaticity coordinates.
The three components X, Y, and Z, generally corresponding
S to the fraction of Red, Green and Blue light in the illumination, ~ `
can be plotted on a two dimensional graph. Chromaticity coor-
dinates have been used in the past as one or more features in
. . :
multi-dimensional feature space pattern recognition systems to
recognize and classify, among other things, blood cells.
Biological specimens are stained to improve contrast of
the normally transparent tissues, and render various structures
more recognizable. Blood cells are normally stained with a
Romanovsky type stain, e.g., Wright's stainj a two component
stain system comprising a red and blue dye. The blue stain com-
ponent stains cell nuclei, the cytoplasm of lymphocytes, and
certain granules in the cytoplasm of some of the other cells,
in particular the basophilic granules of the basophils. The
red stain component is absorbed by the red cells, lightly by the
cytoplasm of most white cells, by eosinophil granules and to some -
extent cell nuclei. These staining patterns are not absolute
or mutually exclusive because almost every cell part absorbs
both stain components to some extent. However, usually one or
the other stain component is predominant and this predominance
forms the basis of a functional analysis system utilizing the ~;
color differences. Thus, the cytoplasm of most cells, with the ~
exception of lymphocytes, is stained light violet to red-orange, ~ -
the cytoplasm of lymphocytes is stained a pale blue, the nucleus ;
of the cells is stained a deep purple, the eosinophil granules
29 are stained a deep red to orange, and the basophil granules
are stained deeply blue.
8 -
.. ~
,
,. . . : ~ , . , ~:: . , . . :
~: . . , ~ . ,,. ,. ,, . : , . , .::
~L0~012Z
For blood cells that have been stained with Wright's stain, ~ -
the "red" absorption peak of methylene blue and its derivatives
occurs at ahout 570-600 n.m., the "blue" absorption peak of the
Eosin-Y stain component occurs at about 500-530 n.m. and finally,
the "blue-violet" natural absorption peak of hemoglobin occurs
at about 400-420 n.m.
The present invention utilizes this color in~ormation to
generate information with respect to the "differential contrast"
between and/or among various points or regions in the cell.
The color information ts reduced to "differential contrast"
information by illuminating the sample with white light with
subsequent filtration by narrow wavelength band filters.
Alternatively, the "differential contrast" information can be
produced by illuminating the blood sample with selected narrow
wavelength bands of light.
Each point or region in the cell will modify the light
in accordance with its absorption, transmission and reflectivity
characteristics. The term "contrast" refers to a substantial
difference in the modification of the light by two or more cell
points orregions at one wavelencJth band. The term "differential
contrast" refers to a dissimilar pattern of contrasts at two
or more wavelength bands.
The appropriate wavelength bands are selected with respect
to the spectral content of the stain or dye system's light modify- -
ing characteristics and/or with respect to the light modifying
characteristics of the natural material, e.g. hemoglobin. W~th
appropriately selected wavelength bands, the desired "differential
contrast" of the various cell points or regions to be recognized
29 is established by their marked density and/or reflectivity
Cb! _ ~ _
~, .... . .. . ~ - , - :
,
,
- , , .
"
1~6012Z
differences. Thus, when the wavelength bands are properl~
chosen, a particular cell region, such as WBC cytoplasm will
be very dense at one wavelength band and relatively transparent ~ ..
at another. Another re~ion such as, ~BC c~toplasm will display .~ .
a different contrast pattern at the same wavelength bands. ~. :
The differential cont.as-t of the cell components established
by the choice of the various wavelength bands permits the
identification and classification of cell components or regions
by means of a "color al~ebral' illustrated below.
The color algebra can be implemented by sampling and
digitizing the signal representing the sample modi~ied illumina- ;
tion at each of the ~avelength bands to produce a digitized ~
serial data stream, and then histogramming the digitized values ::
as shown in Figure 2, Characteristically, the histograms of
the points in the scanned blood sample exhibit two or more
groups of points, or peaks at diferent density levels. For
example, as shown in Figure 2, the peaks may correspond to a
group of background points at about the same densit~, or to
another group of somewhat denser cell cytoplasm points or
possibly to a third group of very dense cell nucleus points.
Several types of cellular components may be combined into a
peak at one ~avelength, but will be separated at another wave~
length. For example, in Figure 2, WBC and RBC nuclei, basophil
granules and l~mphocyte cytoplasm are combined in peak 3 of
histogram A, but are separated into peaks 5, 6 and 7 of histo- -
gram B.
In practice, histogramming has pro~ed to be a feas~ble ~
method for establishing thresholds. However, it should be ~:
29 understood that the color algebra also can be implemented b~
c~ - 10 -
.~.,. ., .. ..... , . . , . . . .. ,., . .. ~:
:, . . :~ ,,, ,, ., - . ~ ~ . .
:,.
.
~ 0122 ~ ~ ~
arbitrarily establishing thc thresholds without sampling, diyit-
izing or histogramming. For example, a suitable color algebra
can be used to detect sample regions of blood cells flowing in a
liquid stream past a sensor. In this situation, no scanning, -~
S sampling, digiti~ing or histogramming is employed.
,~ ,
Thresholds are established to separate the peaks of the histo-
grams. The thresholds are shown as TA, TB, and Tc, with T'B illus-
trating the use of multiple thresholds. Any point in the digitized
data stream can then be characterized as a thresholded signal in
,:.:
binary form as exceeding or not exceeding the various thresholds.
The thresholded signals can be combined to produce the
following color algebra~
.; ~ - .
. _ . .. ..
A B' B C
. ~::
Background 0 0 0 0
RBC Cytoplasm 0 0 1 1 ;~
RBC Neucleus 1 1 1 1
WBC Neucleus 1 1 1 0 ;~
Monocyte
cytoplasm 0 0 1 0
Neutrophil
EosinophiLGranules 0 1 1 0
Basophil Granules 1 0 1 0
Lymphocyte Cytoplasm 1 0 0 0 ~;
. - _ ' '
This color algebra i9 applicable for the previously discussed
example of a Wright's stained blood sample and the wavelength bands
set forth above. Other color algebra can be employed to classify
cell components stained with other dye systems or using the char- ~ ;
acteristic absorption of other natural cellular constituents the
wavelength bands again being selected to provide differential con-
trast between at least two differen~ regions in the sample.
cb~
,~ .
,............ - . ~ . - -: , .
: .. : .. . .. - . - . , ~ ' :': " i -:' `: ' '
... , , : -
:: - , . ~ : .. .. . . .
. , ~ ,. : . .: . . . . , ,.: . ,. : . :: .. ,~: . ~ :: ............ :
.:...... : . : .. . , ,. ... : . . : . ~
~L060~L2~ ~
It can be seen from the table that the color algebra character-
izes a particular point or region as being in one o a number of
cell component classifications. In addition, the color algebra also
permits differentiation betw~en cell components and background areas
in the blood sample. Thus, the thresholded signals can be alge-
braically combined to produce sample region c:Lassification signals.
The preceding example of a color algebra illustrates the
classification of the cell components and bac]cground by algebraic ~`~
combination of thresholded signals. Alternatively, the signals
10 can be algebraically combined and then thresholded with or without ~
further algebraic combination to produce the Sample Region ~ ~;
Classification Signals.
From the preceding description, it will be appreciated
that the use of color algebra in the present invention permits the
lS separation of an image into a number of categories o cell components
or background without requiring the normal procedure of chromaticity
coordinate calculations and subsequent complicated pattern recogni~
tion data processing. It also will be appreciated that it is not -
~ : .
necessary to stain the cells to use the differential contrast and
color algebra features of the present invention. Alternatively,
native constituents of the cells may be utilized to provide the
necessary contrast patterns. For instance, in addition to the natur-
al absorption of hemoglobin near 400 n.m., the absorption peak of
DNA ~normally found in cell nuclei) at 258 n.m. and the absorption
peak of proteins (normally found predominently in the cell cyto-
plasm) at 230 n.m. can be used as the wavelength bands. Because
of the partial overlap of absorption waves of these two cellular
constituents and the presence of some other constituents which also
29 absorb at these wavelengths, the resulting color algebra is somewhat
c`')/ - 12 -
.
.. . ,~. ,.. ,.... , .. " ... ...
1~16(~Z~ ~
more complicated than that employed in Wright's stain. Further-
more, in using these threc wavelength bands, the long experience of
the medical community with Wright's blood stain would be lost.
For this reason, the Wright's stain system is the one of choice.
It will further be appreciated that the color algebra fea-
ture of the present invention is no-t limited to three wavelength
bands of the preferred embodiment. Any two or more wavelength
bands which will produce differential contrast between at least
two regions in the subject of interest can be utilized to produce
an appropriate color algebra.
Having described the differential contrast and color algebra
concepts as they relate to the present invention, I will now pro-
ceed with a description of the general systems concept of the ;~
preferred embodiment.
Returning now to Figures lA and lB, there i5 shown in block
form the general systems concept, the principles of operation
and the data flow of the blood analysis embodiment.
In Figure lA, the blood sample is prepared for analysis by ;;
being spread in a thin layer on a glass slide or other suitable
surface and stained with a suitable blood stain. Normally, the
prepared slide is magnified by an optical system ~microscope) and
a portion of the magnified image is scanned and digitized at several
wavelength bands. Details of this process will be presented in Fig-
ure 3. The magnifi~d image is then embodied in two or more streams
of numbers (the digitized serial data-signals~ which represent the
transmission or density of the image over the raster of points.
There are three basic stages in the process of analysis of -~
the scanned and digitized image~ the regions are located
or localized; (2? quantitative "features" which characterize
the cells in some desirable way are extracted from the localized
ch~ - 13 -
~:
,"",, , ,, , "" , . . .... . . . .................. .... .....
::: : . : . :, : :: : . - . -: : :::.:. ~ , :: . . .: :
i(360~2Z
regions; and (3) using these features the cells are further classi- ;
fied as normal, abnormal, neu~rophil, lymphocyte, etc. i~
The previous state of the art method for performing these
tasks was to store the stream of numbers representing the image
density at various points in a computer memory. Then, algorithms
stored in the computer would localize the cells, eXtraGt the fea~
tures and classify the cells. As an image contains a large number ~ ~ ;
of points, a large memory was required to store it. Also, since all
three stages of the analysis were performed by the computer processor,
10 it was of necessity fast and powerful. Both o~ these factors ~;
required the use of a computer so costly that to actually analyze
blood smears in this way would be prohibitively expensive.
The preferred embodiment does not use storage of any of the
,.. .
stream of digitized image points in a computer memory. It makes ;~
use of a combination of color algebra and simple processing cir-
cuitry to reduce computer memory requirements to that just suficient
to store only the compiled features of the cells in the image. At
the same time, the work the computer must perform is reduced to
classification of the cells using the compiled features. Both of
20 these characteristics permit the use of a relatively simple and ~ ;
inexpensive computer. Even so, by relieving the computer of the ~ ;
tedious localizing and feature extraction tasks, the present embodi-
ment is able to operate much faster with a small inexpensive com- ~ ;~
puter than a previous state of the art design which used a ].arge
expensive computer.
This combination of color algebra and preprocessing of the
stream of sampled and digitized points is further illustrated
in the block diagram in Figures lA and lB. The points of each
color representation of the digitized image ~the Digitized ~ ;
Serial Data Signals) are histogrammed and thresholdPd to produce
cb/ - 14 - ;~
,~.
: ::. . ~ . -:: -,.... . . .. .
Z
Thresholded Signals. The background density is subtracted from
the image density to produce a "Data" signal. Using color
algebra, each image point is then classi~ied as either background,
neucleus, WBC cytoplasm or RBC to produce Sample Region Classifi-
cation Signals. In the preferred embodiment, a line delay is
employed to re-establish the vertical connection of two adjacent
image lines. The sample Region Classificatic)n Signals are then
used to derive Control Signals for identifying all segments on
each scan line and for compiling the cell features for each
cell segment on a line-by-line basis. Details of these Control
Signals w.ill be discussed and elaborated in Figure 5.
To keep the features for each encountered cell separate,
eaeh seene region in the field is given one of two region numbers
or"tags" according to the type of region. Thusj each eell region ~:
is given a cell number or tag and eaeh baekground region is given
a baekground number or tag. Cireuitry to assign these tags,
and eorreet errors which might occur, are discussed and elaborat~
ed upon in Figures 6 and 7.
The aetual compilation of the partial features for each . - :
eell segment on a line-by-line basis is performed by the speeial
eireuitry shown in Figures 8 and 9. Using the Figure 5 Control
Signals, this eireuitry operates on the Data Signals and produees
the proper measures of size, shape, density and eolor of the
individual eells and eell inelusions. The complete cèll features
are stored in a section of the eomputer memory reserved for
each "tagged" cell number, or nonzero baekground number. `
At the end of the sean of the image, the pre-proeessor has
eompleted the eompilation of the eomplete eell features for ;~
29 eaeh eell encountered in the image and these features are stored
.
cb/ - 15 - ~:
, ~ .
,. ... .,.. ,.. . . ... . .... .-. ., . ~,
i~)6C) 1 ;2Z
under appropriate numbers or "tags" in the main memoxy. The ~ -
computer then has only to further classi~y the cells, usually ,;
by multi-dimensional feature space analysis familiar to the art,
to produce the differential count data output. The instructions
which perform this further classification and perform overall
system monitoring are shown residing in a separate "control ~ ,
memory". System monitoring functions include monitoring the ; ~ ,
histograms and the compiled features to insure that the sample
has been properly stained and that the systemis performing within
predetermined operating parameters, keeping track of the patient's
identification, monitoring the focus control, summarizing data
over a large number o~ cells, and averaging and outputting the' ~ ,
summarized data.
It will be appreciated from the oregoing and following
description that the pr~erred embodiment is one specific example
of a more general method and apparatus or subject analysis
characterized by the compilation o partial cell features rom
a scanned signal representing the sample. The preferred embodi~
ment comprises a sophisticated analysis system which isolates
and analyzes each cell in the scene containing many blood cells.
In order to accomplish this sophisticated analysis of a complex
scene, a number o types o control signals are generated from
both normal and delayed signals, the partial cell features are
compiled from identified cell and background segments in each
scan line and then the complete cell features are compiled
from the partial features utilizing cell and background "tags" -
which have been assigned to each region in the scene. However,
a less complex version of the invent}on can be employed to analyze
29 a scene containing only one complete cell (or one cell of a
' cb~ - 16 -
, .. , .. .:: , . : . , ~ . .. .. . .. .. .
1~ 22 ~`
particular type, such as a WBC). In this case, a single type of
control signal is derived from undelayed signals and are used
to compile the partial and complete cell features from the
single cell without using cell "tags".
Having described the o~erall systems concept and general
operating princiPles of -the preferred embodiment, I will now
discuss in detail the specific circuitry of the embodiment.
Referring to Figure 3, there is shown in diagrammatic
and partial blockform an optical-to-electrical input stage for
1~ the blood cell analysis system which is indicated generally by
the reference numeral 10. An optical scanner 12 scans in
raster fashion a field 14 which contains a blood cell sample 16.
The sample 16 comprises a blood film composed of red cells, white
cells, and platelets spread on a monolayer 18 on a standard glass
slide 20.
The blood layer 18 is stained with a suitable stain which
enhances the morphologicalcomponents of the blood cells. A
typical example of such a stain is the previously mentioned
Wright's stain. The stained blood layer 18 is scanned within
field 14 by means of the optical scanner 12. For purposes o
illustration, the spacing between the scan lines shown in Figure
3 has been greatly exaggerated and the relative movement of the
field 14 across the blood sample 16 has been indicated by rela- ;
tive movement arrows 22. Furthermore, the optical system within ;~
scanner 12 has been generalized in the drawings. It will be
appreciated that suitable magnification stages and focusing
control systems~ e,g., a microscope input to scanner 12 can be
and normally would be, employed in the blood analysis embodiment
29 o~ the invention. `
c~/ - 17 -
.;:: . ~, : - - : :
. :.~ . . - : ~ : . . . : . . . . . .
- ~ :, . . . -, , . . . ~.
~(~6012d~
Thc blood sample 16 is illuminated by light from an illumina
tion source 11. The sample can be illuminated directly to pro~
vide reflective modification of the light by the blood sample or
from beneath to provide transmissive modification of the light.
It will be appreciated that a fluorescent stain can be employed
to provide the desired modification of the illumination.
The scanned output beam 24 from scanner 12 is passed through
a beam splitting prism 26 which divides the output beam 24 into
three separate beams 28a, 28b, and 2~c. Each beam 28 passes
through the previously mentioned color filters 30a, 30b, and
30c and impinges upon photo tubes 32a, 32b, and 32c. Alter-
natively, dichroic coatings can be used on the beam splitting
prism 26 to achieve the desired color separation. The electrical
signal from the photo tube 32 on output lines 34a, 34b, and
34c represents in electrical form the optical transmission of
each segment of the scanned field 14. The optical transmission
(linear) is converted to optical density (logarithmic) by means ~;~
of log-converters 36a, 36b and 36c. The analog output of the
log-converters 36 is converted into a Digitized Serial Data . ;~
Signal at a specified sampling interval by means of A/D con~
verters 38a, 38b, and 38c. The outputs from A/D converters 38a,
;
38b, and 38c are identified in Figure 1 as Digitized Serial
Data Signals labeled A~, B', and C
Looking now to Figure 4, the three channel data A', Bl
and C' is applied as an input to a histogrammer 40 and to corres-
ponding signal level comparators 42a, 42b, and 42c. During
the first pass of scanner 12 through field 14, the histogrammer
40 collects the histographic lnformation within the field for
29 each signal, i.e., the density distribution of the points within
'~ ~
cb/ - 18 - ~ ~
, , , ,, . " . . . . . . .
~o~;o~
the field 14. The three histograms are thresholded and durin~
the second scan of the field the thresholded outputs TA~ TB'
and TC are applied to outpu-t lines 44a, 44b, and 44c as the
second input to the corresponding comparators 42a, 42b and 42c.
The magnitude of the optical density data A', B', and C' is ~ .
thus compared with the preset thresholds T~, TB, and TC to pro~
duce thresholded signals. The potential for thresholding a
data signal more than once is illustrated in Figure 2 by the
label T'C and comparator 42c'.
The output fr~m each of the comparators 42a, 42b and 42c
is a ONE if the corresponding input is equal to or.greater.
.
than the preset threshold TA, TB, or TC (an "over-threshold" ;~
signal) and ZE~O lf less than the threshold (an "under-thresho~d"
signal). The thresholded signal output from each of the three :~
channel comparators on output lines 46a, 46b, and 46c is a
one-bit datum representing the presence or absence o~ an over-
threshold signal.
For purposes of clarity ln the drawlngs, relative shadlng
has been used on input and output lines to deslgnate the type
of signals thereon. Thus, loo~ing at Flgure 4, a multiple - :
num~er of bits ls indicated by a heavy line, such as, the out-
put lines 44 from the histogrammer 40 while a one bit date line
i5 indicated by a relatively light line such as lines 46a, 46b.
The thresholded signals on comparator output line 46a is
applied to a 3 x 3 shift register array 48a. Selected outputs
from the 3 x 3 array are inputted to line delays 50a and 52a.
The line delays can be implemented in a variety of ways includ-
ing delay lines, shift registexs, etc, The outputs from
2~ the li~e delays 50a and 52a are ~ed back to the 3 x 3 shift
cb~ ~ 19 ~
: ,
.
l~Q~
register array 48. The separate sections within the 3 x 3 array
are identified by the lettcr "A" with suitable subscripts 1
through 9. The timing of the 3 x 3 array and the line delays
50a and 52a is designed to provide a total delay of two scan
lines through field 14 plus the time delay represented by shift- ~ ;
ing the one bit data signal through three of the blocks in the
3 x 3 array 48a. Thus, a one line delay for field 14 corresponds
to the delay produced by Ag, A8, A7 and line delay 50a.
Given this delay configuration for the 3 x 3 array 48a and
the corresponding line delays 50a and 52a, it will be appreciated
that the 3 x 3 array 48a restores the vertical connection of
points in three adjacent lines within the scanned field by
delaying two lines. The signals within the 3 x 3 array blocks
Al through Ag are applied to corresponding input lines identified
collectively by the reEerence numeral 54a to a logic circuit
shown in block form in Figure 4 and identified by the reference
numeral 56a. The logic circuit 56a performs a spatial filtering
function with respect to the center element A5 in the 3 x 3
array. Normally, the output signal A from logic circuit 56a is ;~
the same as the center element A5 in the 3 x 3 array 48a. How-
ever, if the center element ~5 is ZERO and all or most of the
surrounding elements Al through A4 and ~6 through Ag are ONE,
the logic circuit 56a will change the value of the output signal
A to a ONE. Conversely, if all or most of the elements surround-;~
ing a ONE center element are ZEROS, then the value of the center - ;
element A5 is changed to ZERO for the output signal A from
logic circuit 56a. The same filtering is performed for the ~
signals on input lines 46b and 46c, For purposes of clarity, ~ ~`
29 the Same ~e~erence numerals have been used in Figure 4 with the
~, ~, . .
cb/ - 2 0 -
'
... . . . .: ,, .:,, . , , -
. ... , ~, .. .. : . . . , ~ . ~ ;
~0601Z~
correspondiny small letter designations ~or the "b" a~d "c"
. ~ ~
channels. ~n example of such filtering follows~
. NUMBER OF SURROUNDING O ' s . _ :
Value of 01 2 3 4 S 6 7 8
5IElement 5
I O 1 1 _1 0 0 ,0 ,O O (
1 11 1 1 1 1 0 _0, ~
The A, B, and C outputs from the corresponding circuits 56a, 56b,
and 56c are filtered versions of the data in array blocks A5,
B5 and C5, respectively.
The spatial filtering provided by the 3 x 3 arra~ 48a, its
corresponding line delays 50a and 52a and the logic circuit 56a
is optional in the present invention. If a very clean signal
with no noise is available, filtering is not necessary. How-
ever, since most practical electronic systems are noisy, thepreferred embodiment of the present invention includes the
filtering circuit just described. The spatial filtering func-
tion can be performed at several points in tne data analysis
For example, it can be performed after the color algebra as
well as before.
Referring now to Figure 5, the thresholded and spatially
filtered signals Aj B, and C from the three logic circuits
shown in Figure 4 are applied as inputs to a color logic
circuit 5~ shown in block form in Figure 5, The color logic ;~
circuit 58 processes the A, B, and C signals to produce sample
xegion classif~cation signals~ In the preferred embodiment, ~-
W these sxgnals represent points in the nuclei, white cell
,;~, ' ,
- 21 - ~
'; -:
~06~
cytoplasm, red c~lls and background in the scanned image.
The color algebra performed by color logic circuit 58
is not as complicated as the generalized color algebra des-
cribed previously. Thelogic circuit 58 per~orms the following
color algebra:
Cell Component Type A B C
_ - ~ :
Background 0 0
~ucleus (N~ 1 1 0
~BC Cytoplasm 1 0 0
0 1 0 ~
RBC (R) Cytoplasm 0 ~ 1
= don't care
The color logic circuit 58 produces three output or ~ -
"sample region classification" signals which indicate when a ;~
point is part of a cell's nucleus, white cell's cytoplasm
or a red cell. These three outputs appear, respectively, ~;
on output lines 60, 62, and 64, and are inputted to correspond-
ing five block arrays 66, 68, and 70. Each array is provided
with a line delay 72. The purpose of the line delay is to ;
delay the signal and thereby re-eskablish the vertical connection
of the points within the array. Note that a delay of a single ; ;
line was produced by the signal transition in the 3 x 3 array
48a shown in Figure 4 as the signal progressed ~rom block Ag
to A5. The line delay 72 shown in F~gure 5 then produces
another single line o~ delay. It also should be noted that
the point A5 in the 3 x 3 arxay shown in Figure 4 and the point
shown in the ~ive blork array in Figure 5 correspond to the
same point in the scanned field 14
c ~ - 22
:: :.. , :,,, .. .. , . , : ~ , . ; , .~ . . . . ..
:-:. ,, : ~ .
:, , :: . , , . : ,- . .: - , - : -:
The outputs from the four arxay blocks ~Nl, N2~ N4 and N5
are applied as inputs, on leads identified collectively by the
reference numeral 74, to a nucleus perimet~rcontrol logic cir-
cuit 76.
The control logic circuit 76 is designed to produce control
signals for the system with respect to the perimeters of each ~
detected nucleus. The control circuit 76 generates four control ~:
signals: straight perimeter, nucleus (SPN); diagonal perimeter,
nucleus (DPN~, previous row perimeter, nucleus (PRN); and, ;~
store previous row, nucleus (SPRN). The truth table for generat-
ing these four control signals is:
~.
:.
.
~ ,
; ;,
- ~3 -
cb~
. , . ; . . , , , .. : .... .
~o~zz
4 INPUT ELEMENTS OF LOGIC 76, 82 ~ 84
USING ARRAY ~6 ~S AN EXAMPLE
.
__ _ : '~
Nl N2 -
. _ . ;
N4 5 : :
CONTENTS OF 4 ELEMENTS OF 5 BLOCK ARRAY
0 1 2 3 4
. , , _ . ,
O O O O . O O . O O O 1 . '
O O O 1 . 1 O 1 1 O O
. .
6 7 8 9
o j1 . o ~1 ~1~' 1--o _ o
~
__ . l .
O ~ . 1 O 1 11 , O O O 1 .
10 11 12 13 14 :
m 1l 1 ol 1l 1 11 T~I 1l 1 11
Il I I 11 L 1 1 lo I ! ! I 1 ! 11 1 o I
~ . . . ~ , .
Output o~ logic 76, 82, & 84~
O: 0, 1, 2, 4, 15, (6, excepting logic 76) ~.
STRAIGEIT PERIMETER: 3, 5, 10
D~AGON~L PERIMBTER: 7, 11, 13, 14, ~6, 9, logic 76 only) ~`
ALTE~N~TE PERIMETER: 12 ~:
',; :~
STORE ALTERNATE PERIMETER: 8, (9, excepting.logic 76) ~ -
.~ . .
cb~p - 24 - :
`' ~ .
~. :. : ~ ::: . ..... , . .. -
: ~ : : - . . - - . . .: . . : .
~(360~L2Z
Similar logic is also applied with respect to the out-
puts from the white cell cytoplasm ~ive block array 68 and the
red cell ~ive block array 70. The respective outputs from these ~;
arrays are applied through input lines 78 and 80, respectively,
to corresponding control logic circuits 82 and 84. The white
cell cytoplasm control logic circuit 82 generates four output
signals: straight perimeter, white cytoplasm (SPWC); diagonal --
perimeter, white cytoplasm (~PWC); previous row, white cytoplasm
(PRWC~; and, store previous row, white cytoplasm (SPRWG). Similarly,
the red cell control logic 84 also produces four outputs namely,
straight perimeter, red cell (SPR~; diagonal perimeter, red cell
(DPR); previous row, red cell ~PRR); and, store previous row, :
red cell CSPRR).
An additional control logic circuit 86 develops control
signals based upon input signals from the nucleus, white cyto- : ;
plasm, red cell and cell tag five block arrays 66, 68, 70, and ~ i;
92, respectively. The input signals to logic array 86 on input
leads 88 comprise the signals from the N4 and N5 bloc~s of the
nucleus array 66: signals from the WC4 and WC5 blocks of the
white cell cytoplasm array 68 and, signals from the R4 and R5
blocks of the red cell array 70 and signals from the T~ and T5 -
blocks of g2 in Figure 6. The control logic circuit 86 gener- ~
ates ~ine output signals in accordance with the truth table . ;
~s follows:
~; :
~ . ~
cbj - 25 ~
' ' ' ,. , : . '; ' ': , . .. ' , ' :' ., . :' ' ., , :
::: ':.' : ' ,., ' , , : -
:,, '': . , '. . ' ' '., ,.; ,, ,' : ~
' ': ' . '' . . . , ' ~ . :
~LO~Ol~ ~
INPUT
INTERMEDI~TE - --
CLASSIFICATIONS N WC R T
Background, zero (0) 0 0 0 Zero '
Background, non-zero (NZB) 0 0 0 Non-Zero :
WBC Nucleus (WN) 1 0 0 NA -
WBC Cy~oplasm (WC) 0 1 0 NA ;~
RBC Nucleus (RN) 1 0 1 NA
RBC Cytoplasm (RC) 0 0 1 NA
NA = not applicable
:; .
TRANSITION OF INTERMEDIATE CLASSIFICATIONS IN ELEMENTS :~
#4 and #S IN ARRAYS NOS. 66, 68, 70 and 92 _
`' :'
. .
Transition Count Store :
_ :', ' ,'
4 5 N WC R B W R B _ LINK ~1~1 .
_
0 0 0 0 0 0 0 0 Q 0 0
WN 1 ~ :
WC WC 1 ` " "'~
RN RN 1
RC RC 1
_ . '
0 ~7N 1 :'`"
O WC 1
O RN 1 1
O RC l
~ . _ ..
~N O 1 ::
WC O 1 .~ .
RN O 1
RC O 1 :
WC RC 1 1 :.
R~ ~C 1 1
W~ WC 1 1 ~:
WC ' ~ . I `', ~
~! - 26 - ;
:, . , . , . . .. .. : .. . . . .. . . .
.: . ., . . : ,: ., -, : : : : -
;ZZ
TR~NSITION OF INTE~MEDIATE CLASSIFICATION IN
ELEMENTS #4 and #S IN ~RI~YS NOS. 66, 68, 70,
and 92 contd.
.
_ _ _
Transition Count Store ::~
4 5 N WC R B W R B LINK PINH ::
: ~ :
RC RN 1 1 1
RN RC 1 1
RC WN 1 1 ~ :
WN RC 1 1 .
RN WN 1 ~
RN 1 .
RN WC 1
WC RN 1 1
, _ : ~
RN NZB 1 1 1 1
WN NZB 1 1 1 1 :
NZB RC 1 1 1
NZB WN
RC 1 1 1
NZB WC 1 . 1 1 :
_ I . .
'` ,'''
The "count" (or "compile partial features"~ output control signals
from logic circuit 86 for the nucleus, cytoplasm,red cells and ~ .-
non-zero background (NZB) are identified in Figure 5 by the
respective abbreviations, I'CNTN", i'CNTC", "CNTR" and "CNTB".
.
~ r 27 -
,: - : ,- : : ~: : ~
~0~0~22 ~ ~
Three l'store" control siynals are generated to control ~
thestorage of partial white cell feature data ~STW), the storage . !,,
o~ the partial red cell feature data (STR) and the storage of
non-zero background data (STB). The final two output signals
from the control logic circuit 86 are a "lin};" signal (LINK),
and a perimeter inhibit signal ~PINH). The purpose of these
two signals will be explained subsequently. The count nucleus -`
signal (CNTN) and the count cytoplasm signal (CNTC) are applied
to an OR gate 90 which produces an output signal for indicating -
that white cell partial features are being compiled ~CNTW).
It can be seen ~rom Figure 5, that the Perimeter Control ;
Signals from logic circuits 76, 82 and 84 are derived, inter
alia, from signals which are delayed by means of line delays
72. Howevex, in a simpler embodiment of the invention, the line
delays 72 can be omitted if the sample analysis does not require
perimeter information and the concomitant use of perimeter ;~
control signals. In such a simpler embodiment, there is also
a reduction in the complexity of the cell "tagging" logic which
will be discussed below in connection with Figures 6 and 7.
The control signals generated by the logic circuits shown
in Figure 5 are employed to identify an encountered cell
segment and to control the compilation of the partial and complete
features of the various components of the cells. The partial
cell features, such as size, density, shape, perimeter length,
inclusions, etc., are complied on a line-by-line basis for
each identi~ied cell segment or non~zero background segment.
~c~ scene region is assigned an appropriate number or tag in
order to pxoperl~ control the compilatIon of the complete features
29 ~xom the partial ~eatures for a particular cell- The Fesion
c~/ - 28 - -
.
: . : . ,: : : ~ . . .
:: - : - - . . .. .
~0~ 22 ~
identification numb~r or tay is passed from one row to the next
when there are vertlcally connected points in a region.
~ s will be elaborated shortly, the maln scene background
region which is simply connected in the topological sense to
S the scene border is normally assigned a zero background tag
or number. A "Non-Zero Background'i is a background region
which is not or appears not to be "simply connected" in a topo- -
logical sense to the main scene background, and which is thus
assigned a "non-zero" tag or number.
In practice, a true non-zero background region represents ;~
an inclusion in a cell (such as the central pallor in a red
cell or a vacuole in a white celll ~hich is physically as pale ~ ;
as the main background and thus is identified as background by
the color algebra. Occasionally a portion of the main scene back-
lS ground region will be incorrectly assigned a non-zero background
number because no simple connection to the main background has
been encountered at the time the assignment is made. For
example, this condition will occur when an inverted U-shaped cell ~ ~
is encountered during a raster scan. However, this error is ~ ;
~0 corrected by the "Equate" and CHG signals when the simple
connection is subsequentl~ detected.
The circuitry shown in Figures 6 and 7 is emplo~ed to
generate and assign the appropriate number or tag ko the region.
From a ~unctional standpoint, the circuitry must assign a ne~
cell or background number to the cell or background if the cell
or b~ckground region has not been encountered preYiously in the
scan o the f~eld 14. Con~ersel~, the circuitry must assign
the appropr~ate old cell or background number i~ the cell or
29 back~round has bee~ encountered previously. In some s~tuations,
cb! - 29 -
~(~6QlZ;~
the initial data may indicate that a cell or backyround segment
from a new region has been encountered when in fact the segment
actually is part of a previously encounterecl and identified
region. ~hen this situation is recogni~ed, the new number musk . ~
be removed from the segment and the segment tagged with the ; -
appropriate old number.
The circuitry wh.~ch accomplishes the region identification
or tagging function is shown in partial block and schematic form ~ ~ :
in Figure 6 and ~n block form in Figure 7. Referring now to ~;
Figure 6, there is shown a five block tag array 92 and a line
delay 94. The ~ive blocks of the tag array are identified
as T1 through T5. These blocks correspond to the same portion
of the scanned image as Al-A5, Bl-B5 and Cl-~C5 in Figure 4.
From a ~unctional standpoint, the purpose of the tag array 92
and its associated circuitry is to determine if there is any ;.
point in the scanned p~cture of the same type as point T5 which
has previously been assigned a number and which is touching
point T5. If this is the case, then the point in T5 should be
assigned the same number~ ~ ;
~he red and white blood cell and background numbers or
"tags" are obtained from corresponding UP-DOWN White and Red
Blood Cell and Background counters 95, 97, and 97b, respectively.
The operation of these counters will be described below.
Looking at Ei~ures 5 and 6, the outputs from N3, WC3 and R
o~ the arrays 66, 68, and 70, respectively, are applied as
inputs to a logic c~rcu~t 96 which is also ~dentified in Figure
6 by t~e designat~on "S3 !~, The logic circuitry shown in S3
2s dupl~cated ~n logic circuits 98, 100, and 102, which are -
29 des~gnated respect~vel~ as "Sl",l'S2", and llS4llo These four
cb/ 30
logic circuits Sl-S~ determine whe-ther each o~ the points rep-
resented by Tl through T4 are oE the same region type as the
point represented by T5. The inputs to the logic circuits Sl-S4 ~ ;
correspond to the same nu~bered blocks in -the nucleus, white
cell cytoplasm and red cell arrays 66, ~8, cmd 70, respectively,
shown in Figure 5. Thus, for the S3 logic circuit the inputs
comprise the signals from the N3, WC3 and R3 blocks of the corres-
ponding arrays and the count white (CNTW) signals. For purposes
of clarity, the count red and count white signals input lines
haYe been omitted from Sl, S2 and S4.
In each of the logic circuits Sl-S4, and as shown in detail
in S3, the nucleus and white cell cytoplasm signals are ORed by
O~ gate 104 to produce a white cell output. The output of OR
gate 104 is ANDed with the signal count wh~te ~CNTW, Fig. 5)
in AND gate 106 to indicate that T5 and T3 are both white cell
points. The R3 and count red cell signal (CNTR, Fig. 5) are
also ANDed by an AND gate 108 to indicate that T5 and T3 are
both red cell po~nts. The output of gate 104 is also ANDed
through inverting inputs ln gate 108b with the CNTR, CNTC and R3
signals to indicate that both points are background points.
If either "both" red cell points, "both" white cell points,
or "both" background points are indicated, OR gate 110 will
produce a ~igh output.
The same basic logic is performed by log~c circuit S1, S2, ~;
and S4. A high output from an~ one of the logic circuits Sl -
through S4 indicates that the corresponding point in the tag
array 92 i.e., points Tl through T4 are o~ the same cell type ~;
as T5. Assuming that one or more of the points Tl through T
~- are of the same type as T5, the precedence of the point or
~,
cb/ - 31 -
, . :, .
~36~LZ2
points must be determined. ~ precedence logic circuit shown
by the dashed lines in Fig. 6 and identified by the re~erence
numeral 112 determines the precedence of tl~e points in the tag ~ -
array in the ~ollowing order: T~ (from the present cell segment) ~.
Tl, T2 and T3 (from the previous cell segment). -~
The precedence logic shown within block 112 is employed ~-~
to handle the specific situation in which more than one of the
outputs from the logic circuits Sl through S4 is high. In this
situation, it is necessary to determine the first one in pre-
cedence.
The output from the precedence logic circuit 112 on output
line 114 is ONE (high~ if there is no point in T4, Tl, T2, or
T3 which is of the same t~pe as that of T5 and ZERO (low~ if
there is a point which is the same as T5. However, if T4 is
lS the ~irst point which is the same type as T5, the precedence loyic
eircuit 112 produces a high "ONE" output on output lead 116 which
actuates a corresponding gate 118. With gate 118 actuated, the
particular tag or number in T4 is gated onto bus 120 and back
into point T5 in the tag array.
If the particular point in T4 was not the same type as that
in T5, the preeedence logic circuit 112 next examines the type
of the point in Tl. A corresponding circuit is provided for the
T1 point in the tag array with gate 122 being actuated by the
output from the precedence logic circuit 112 on output line 124.
Thus, if the points Tl and T5 are of the same type, and Tl and
T4 are not of the same type, the number of part;cular tag in Tl
is gated through gate 122 onto bus 120 and then into tag T5. A
similar ~rrangement is also provided for the tag array point T2
~9 through output l~ne 126 and gate 128 and for tag array point T3 ~;
e~/ - 32 - ~
. ~ , . `, ~ ~, , !
., ' , ' . ' .' ` ' . ' , `' . ' ' . ' 1.
, '` ` ~ ~ ' ~ ' , ' , . ,,, '
10~i012Z ` ~: ~
through output line 130 and gate 132.
I~ there is no point in the tay array which is of the same
type as T5, the output on lead 114 from the precedence logic
circuit will be high and this oukput is fed to red and white
cell and background counter AND gates 134, 136 and 136b, res-
pectivel~. The second input ~or each AND gate is the correspond~
ing count red signal count white signal or count background
signal ob-tained from the circuitry shown in F~igure 5. If the ~
count red signal is present, AND gate 134 produces a ONE output ~ -
on line 138 which is used to increment the red blood cell counter
to the next number. The outpu~ from ~ND gate 134 is also used
to actuate a gate 140 which ~ates this next red blood cell
number from counter 97 onto bus 120 and thus into tag array
point T5. A similar arrangement is provided for the white
blood cell counter 95 through AND gate output line 142 and gate
144, and background counter 97b through AND gate output line
142b and gate 144b.
The output from the precedence logic circuit 112 on line
114 is also applied to a NEW number logic circuit shown by the
dashed lines in Fig. 6 and identified by the reference numeral
146. The NEW number logic circuit 146 maintains a record of ;
the assignment of a new number to a string of points on the
"present" line of analysis. The "present" line is repxesented
in part within the tag array by points T5 and T4 while the "pre-
~5 vious" line appears in part in the tag array points T3, T2, and
T
The ~EW logic circuit 146 is used to distinguish between -
two cases in which points in the same object have been assigned
29 different numbers. The two cases can be thought of in general
cb/
., ~ :: . . . : . . ~............... . .
~S36(3~2f~ ~
terms as the "sloping line" c~se the'~shaped" case.
In the first case, a portion of the particular region
under analysis slopes gently upwardly in the direction o~ the
scan. The slope is gradual enough so that three or more points ~, ,
S are encountered which are not contiguous to any point of the same
type,in the prev~ously scanned line. Since the present line ; ~
points ~at least three or more) are not contiguous wlth the , ~ ,
points of the same type in the previous line, the precedence
logic, tag array, and the appropriate red or white cell or back~
ground counters will assign a "new number" to the present line
points. However, in actuality the present line points are a ;
part of the same region as the previous line points. Thus, we
have a situation in which the previous line points have been
assigned one number while the present line points have been
assigned another number althou~h in fact all of the points are
part o~ the same region.
In the second or "U-shaped" case, the first encountered ,~
upstanding leg portion of the "U~shaped region or object is
assigned one number and the second encountered upstanding leg
of the "U"-shaped ragion or object is assigned another number.
Upon subsequent scans, the system will recognize that the two
upstanding legs which have been assigned individual numbers `;
are in fact all part of the oneparticular region or object.
In both cases recognition occurs when points of the same ,
cell type but having different numbers appear in points T3 and
T5 in tag axray 92. Although the "sloping line" and "U-shaped"
- cases appear the same to the tag array 92, it is expedient
to distinguish them and treat them differently. In the case ,
29 of the 'Isloping line" object the "Present" line tag or number ;~-
cb~ -.34 ~
. '' ~ ' ' ' "` . , "' . ' ' ' ' '" ' ' . ' ':
' ' . ' ' , ' ' ' ' ' , , . ' ' ' . , , ' ' , . . .. ' ' . ' . ' ' ' ' ' ' '
" ~L06~Z~ :
wlll be changed to the "previous" line tag or number by means
of the circuitry shown in Figure 7 and the appropriate xed or
white cell or background counter will be decremented. In the ~ ~;
case of the "U"~shap~d object, the two tags or cell numbers
will be EQUATED with each other for purposes of subsequent
identification and incorporation of the features stored under
each tag or number. - ~;
These two cases are distinguished by means of the N~W
number logic circuit 146 which comprises OR gates 148 and 150
- 10 AND gates 152, 154, and 156, and Flip-Flop 158. The inputs
to the NEW number logic circuit 146 are: count red (CNTR)
on line 160; count white (CNTW) on input line 162 and count
background (CNTB~ on line 162b; the output from the precedence
logic circuit on line 114 which represents an "Assign New-
Number" si~nal; and, finally, a "cl1ange" signal (CHG) on line
164. The "change" signal is derived ~rom a logic circuit 166
shown in Fig. 7 in accordance witll the following truth table:
IWPUTS OUTPUTS
_ _
Same Same T5 ~ T3 NEW CHG EQUATE DCNT
Type No.
1 0 0 0 ~ 1 0 ; ;~:
1 0 1 0 1 1 0
1 0 ,0 1 1 0 1 '
_ .~.
~ - don't care
The NEW Flip-~lop 158 is set whenever the "Assi~n-New-Number"
~s~ l on t~e ~recedence output line~il4 ~s 0N~ or high. The
~s~ Ne~-~umber ~ al ~s ~ppl~ed as one ~nput on ~ND gate 152.
~,
c~! ~ 35 ~
- ,, ,
.. -
10~;()122
The second input to the ~ND gate 152 is provided by the output
from OR gate 148. This i.nput is ONE (high) whenever a new number
is assigned because either the coun-t red signal, count white
signal, or count background signal on OR gate input lines 160,
162 and 162b, respectively, is also high. The output from AND
gate 152 is applied as one input to OR gate 150. Thus, if the j;~
output from AND gate 152 is high the output from OR gate 150
will also be high. The outputs from OR gate 148 and 150 are
ANDed by AND gate 154 thereby producing a high output on line
168 which sets the NEW Flip-Flop 158.
The Flip-Flop 158 maintains itsel~ in the set condition
as long as either a count red, a count white or count background
signal is present on lines 160, 162 and 162b. If an ob~ect-to- ~-
zero background (or cell to zero background) transition occurs,
it can be seen that the count red, count white and count back-
ground signals will be low on OR gate input lines 160, 162 and
162b thereby allowing the NEW Flip-Flop 158 to reset. The Flip-
Flop 158 also can be reset by a change signal (CHG) on line 164.
For certain complex scenes, the NEW number logic circuit
146 preferably should be triplicated with input OR gate 148 omitt-
ed and the inputs 160, 162 and 162b going directly into the
triplicated AND gates 152 and 154, the outputs from the triplicated ;`
NEW FFs being ORed together into logic circuit 166.
Referring now to Figure ~ there is shown in block ~orm
additional circuitry that operates in conjunction with the tag
array 92 and line delay 94. For purposes of clarity, this
circuitry was omitted from Figure 6 and tS shown in Figure 7.
~or the"sloping ~i~e" case, the circuitr~ shown in Fi~ure
29 7 ~ncludin~ the previously d~scussed log~c circuit }66) per~orms
cb/ - 36 -
,. . . . : ~
: ~ . - ., .. : . ~ , :
1~)601Z2
the follo~ing operations~ changes the tag or region number ~ ~.
in T5 to the tag or region n-lmber in T3; (2~ decxements the
appropriate red or white blood cell or background counter; and,
(.31 when appropriate,changes, the tag or region n~lmbers in T4
and in the line delay 94 to the tag or region number in T3.
In the case of the "U"-shaped cell or object, the logic cir-
cuit 166 produces an "EQUAI'E" signal and, when appropriate, changes ;~
tag or region numbers in T5, T4, and the line delay 94 to the
tag or region number in T3. The EQUATE signal causes the cell
tags in T3 and T5 to be pushed on toan equate stack (not shown)
~n the main memory (Fig. 9).. The "change" signal (CHG) from
logic circuit 166 on line 170 gates the T3 tag or region number -
on bus 172 through gate 176 onto T~. The tag or region number
on T3 hus 172 is also gated into T~ through gate 178. Operation .
of gate 178 is controlled by means of a logic circuit 180. The
truth table for logic circuit 180 is as ~ollows:
II~PUTS
T5 - T~ CHG
~ :,~ .. ..
OUTPUT :~
.
1 1 1
0 1 0
0 0 0 , ' ~
It can be seen from the truth table that if T5 is equal to
T4, the T3 number on bus 172 is gated through gate 178 into T4. ~ .
A similar logic circuit 182 controls another gate 184 which gates
the T3 number on bus 172 into t~e ~irst element of theline delay
9~. Since the line delay 94 comprises a shift register having a
,
X ~b/ ~ 37 ~ ~
:. : . :.:: .- : : : : . .
: . : . : ~ . : .: , , : - : :: . . .: .: : . :
: . .. . .. . , . : , , , . ..
~L06~
predetermined number of storage elements, the lo~ic182 and gate
184 is duplicated ~or a prcdetermined number of adjacent storage
elements in the shift register 94. This adclitional circuitry is
represented in Figure 7 by the continuiny three dots. The pur-
pose of the logic associated with the shift register line delay
94 is to correct the improperly numbered "present" line points
in T5, T~, and the line delay 94. ~ote that: these present points
~ctually should have the same number as the previous line point ;
in T3- ;~.
The logic circuit 166 also generates a "down count" or
counter decrement signal ~DCNT) in accordance with the truth
table set forth above. The "down count" signal is applied as one
input to two AND gates 186 and 188 shown in Figure 6. AND gate
186 controls the operation of the red blood cell counter 97.
The second input to ~ND gate 186 is the count red signal (CNTR~.
In a similar manner AND gate 188 decrements the white blood cell
counter 95 AND gate 188b decrements the background counter 97b.
In a simplified version of the preferred embodiment, the ~:
NEW Flip-Flop and its associated circuitry, the DCNT signal,
and the CHANGE signal operating upon the elements of line delay
94 can be eliminated, any occurrence o the same region type
but different numbers in T3 and T5 being EQUATED. However,
this can result in the features for a particular cell being
.
stored under a number of cell number tags. This in turn event- ;~
.25 ually increases the work of the computer in sorting out these
~umeric EQUATES.
There remains one speclal case which should be provided :~
ior the~lcase when one or more cells is touching or overlapping
29 ~n ed~e of the field. A cell which overlaps the edge o~ the
., ~ . .
cb~ - 38 -
... , - :
: . .: . ,
~..
-: : : . . .
.~ :; . .,. . . . , ~ .
,, : : :
: . . . . . : . ,.. . :
: :, : . ~ - . .. .
~O~Ol~Z
field will be incomplete and thus not suit~ble for analysis.
This case is provid~d for by causing the scan and digitize cir-
cuitry to output black points during its horizontal and vertical
retrace intervals. These points are the first that are encounter-
ed at the beginning o~ a scan of the ~ield, and being black,they look like an object. These points are given the tag n~ber
ZERO. Any cell touch.~ny the field edge will appear to be part
of the same object, and thus will also be assigned tag number
ZERO. In addition, background points touching the edge of the
field are also assigned a zero number. Thus any background
region with a non-zero number is not simply connected through
background points to the edge of the field. To simplify data
handling, special circuitry (not shown) prevents the storage
of any data in main memory when the tag number is ZER0. Thus,
all xegions touching the field's eclge are ignored.
Having described in detail the operation of the circuitry
. .
which assigns a "tag" to each of the identified segments in `
response to the control signals and sample region classification -~
signals as shown in Figures 6 and 7, I will now discuss the utiliza~
tion of these identification numbers with respect to the scanned
image data. Referring back to Figure 4 for a moment, the ~ -
Digitized Serial Data Signals A', B', and C' are applied to corres-
ponding storage shift re~isters l90a, 190b, and l90c. Each shift
re~ister has a corresponding line delay 192a, 192b and lg2c.
The output from each dela~ is fed back tnto the corresponding -~
sh~ft register, The delay prov~ded by the signal transit -~
through the lower portion, as viewed in the drawing of shift
~e~isters 190 and the line delays 192 correspond to oneline width
29 of the scanned ~m~ge 14. ~h~s dela~ is emplo~ed to s~nchronize j;
:;
cb~ - 39 ~
' ' , ' : ' , ' ', ' " ' , ' .. , ,' ' . ' . . ' :' ~' ' ' , ' : '. . ' , ': ' : ' '' .
o~z ~ .i
the image data s~gnal with the prevlousl~ discussed control
signals. -
The output from each shift register on lines 194a, 194b,
and 194c is applied as one input to a background subtract circuit
196a, 196b, and 196c. The second input to -the background circuit
is the associated background density output from histogrammer 40.
The output from each o~ the background subtract circuits 196 is
a six-bit digitized signal representing the scanned image data
with the bac~ground density subtracted therefrom. These output
are identified as PATA-A, DATA~B and DATA-C.
Re~erring now to Figure 8, the partial cell features are
compiled ~or each of the identified and tagged cell and non-zero
background segments. The full data signals DATA-A~ DATA-B~ and -
PATA-C are inputted to white and red blood cell density summing
circuits. As shown in Fiyure 8, a separate accumulator 198a,
198b, and 198c is provlded for each data channel to sum the
densities of the whIte blood cell nucleus D~TA-At DATA-B~ DATA-C~
Corresponding accumulators 200a, 200b, and 200c are provided for
the white blood cell cytoplasm data. Red blood cell density
summation is pro~ided for data channels ~ and C by accumulators
202a, and 202c. The DATA-A~ DATA-B~ and DATA-C information is
gated into the appropriate accumulators in accordance with the
gatin~ control signal count nucleus (CNTN), count cytoplasm (CNTC)
and count red (CNTR). These signals are derived from the control
logic circuit 86 shown in Figure 5.
The control si~nals are also used to gate either the a~pro- ~-
priate tag number ~rom the tag array block T5 ~nto white blood
cell tag xeg~ster 204 or red blood cell tag cell register 206.
29 In ~ddit~on~ these control signals are ~lso used to ~ncrement
.,,
C~7 ~ 40 -
" ' `~ ' ' ' "' ' . ,, , ' ' ~ ' ', ' ' ' ' '
.` ' ' " : ' ~ ` . . I
1~0~;2Z`
either nucleus, cytoplas~ or rcd blood cell size counters 208,
210, and 212, respectively. -
Looking now at the bottom portion of Figure 8 there are
shown three dual perimeter counters 21~, 216, and 218 ~or the
S white bloocl cell nucleus perimeter, white blood cell c~toplasm
perimeter, and red blood cell perimeter, respectively. Each
eounter sums the number of straight and diac~onal perimeter ~::
signals in each cell component type. The dual white blood .
cell nucleus perimeter counter 214 is incremented by the straight -.: :
perimeter control signal (STN~ and by the diagonal perimeter
nucleus eontrol signal (DPB) which are obtained from control ~ .
logie eireuit 76 shown in Figure 5. ::.
The dual eytoplasm perimeter eounter 216 is ineremented
by the output from two AND gates 218 and 220. AND gate 218
has as its input the straight perimeter, white eytoplasm signal
(STWC~ whieh is derived from eontrol logie 82 shown in Figure 5
and the in~erted perimeter inhibit signal (PINH) whieh is derived
from eontrol logie eircuit 86 shown in Figure 5. ~;
Referring baek to the truth table $or eontrol logie eireuit ~;
86, it ean be seen that when the perimeter inhibit signal is
low or ZERO and the straight perimeter white eytoplasm signal
is present, AND gate 218 will produee an output whieh inerements
the straight perimeter segment eounting portion o~ the dual
eytoplasm perimeter eounter 216. AND ?20 also utilizes the peri-
meter ~nhibit signal together wit~ the diagonal perimeter, white
cytoplasm control signal ~DPWCl wh~eh is derived from the eontrol
logic circuit 82 shown ~n Figure 5. Similar circuitry is also
used ~ox the dual xed perimeter counter 218 thxough AIJD gates 222
29. ~nd 224, The coxresponding control sign~ls str~ight perimeter `
e~/ - 41 -
. .
~(~6~ 2
red (SPR) and diagonal perimeter red (DPR) are obtained from
control logic ci.rcuit 84 shown in Figure 5.
Referring back f~ a moment to the tag array shown in Figures
6 and 7, if there are no cell points in the tag array blocks T4
and T5 and there are cell points in tag array Tl and T2, the
configuration reflects the existence of a perimeter segment from
a previous cell on a previous line that was not detected by the
system. This situation is handled by the circuitry s~own toward -
the bottom of Figure 8. The cell tag or number from the T2 block
of the tag array 92 ~s gated into an appropriate nucleus alternate
number register 226, a cytoplasm alternate number register 228
or a red blood cell alternate number register 230. The gating
signals for the nucleus and cytoplasm alternate number register
226 and 228 comprise the control signals previous row perimeter,
lS nucleus (PRNl and previous row, white cytoplasm (PRWC) which
are obtained from control logic circuits ~6 and 82, respectively,
shown in Figure 5.
The red blood cell alternate number register number 230
is controlled by the gating signals previous row, red cell (PRR)
which is derived from control logic circuit 84 shown in Figure 5.
These control signals are also used to increment corresponding
alternate perimeter counters 232, 234, and 236
At the bottom of Figure 8 is the circuitry for compiling
non-zero background features for red cell control pallor and
white cell inclus~ons. In this embodiment only the area is com-
piled; other more complex features can of course be included.
Non-zero bac~round size ~s accumulated in counter 212b
wh~ch i$ incremented by control s~gnal CNTB ~rom log~c ~6 in
2~ Figure 5. ~n ~ddition, the background tag is gated from T5
cb! - ~2 -
:.,: :-. .- :, . ..... . : : :- :
:.: ~ . , . : : ~. : . . , . .:: : . : . , : : :
; .: : , ~. , : :: : . .
... -: ~ . . . .
in array 92 into register 206b.
Looking now at F~yure 9, the white blood cell portion of the
nucleus and cytoplasm counters, accumulators and registers have
been duplicated in Figuxe 9 with the same reference numerals
being used to identify like components. Figure 9 illustrates the
outputs ~rom each of these circuit components. Note that the
inputs shown in Figure 8 have been omitted from Figure 9. Further-
more, the entire red blood cell and non-zero background portion
has been omitted from Figure 9. However, it should be understood
that the same basic circultry is employed for the handling of
the red blood cell data and non-zero background.
Figure 9 illustrates the use of each region "tag" to
sequentially compile complete cell features from the partial cell
features of each identified cell segment having the same cell
"tag". The outputs from the white blood cell nucleus si~e
counter 208, cytoplasm slze counter 210, density accumulators 198a
through 198c and 200a through 200c, nucleux and cytoplasm peri-
meter counters 214 and 216, respectively, are shifted into a
buffer memory 238 in response to a store white cell signal (STW).
The appropriate tag or cell number from the white b].ood cell
register 204 is also shifted into the buffer memory at the same
time. The contents of the buffer memory are added into a main
memory 240 (which includes a controller) in locations determined
by the cell tag. In this way all the partial ~eatures haying
the same cell tag are added to the same locations to produce ths ~'
complete features for the tagged cell. The main memory controller
controls the gating of the buffer memory data into the main
memor~ and adds the buffer contents to the previous contents '~
in the main memory. After a short delay the WBC counters and
accumulators are cleared by a "clear"signal produced by delay
_ q3 _
,
, - , ., , ,, ~ ; ~ .
, ` ` ' . . ' , ' ' , ,
circuit 242.
It should be noted at this point that the xed blood cell
and non-zero background tnformation is processed in the same ~`
manner through a buffer memory (not shownl into the main memory
and controller 240.
The contents of the alternate perimeter counters 232 and
234 for the nucleus and cytoplasm, respectively, are also shifted
into another buffer memory 244. In a similar manner, the tag
or cell numbers contained in the alternate number registers 226
and 228 are shifted into the buffer memory 244. The alternate ~
perimeter and alternate numher data is shifted into the buffer ;~-
memory 244 in response to the store previous row, nucleus (SPRN) ~
signal or the store previous row, white cytoplasm (SRWC) signal ~;
which are obtained from the Flgure 3 logic circuits 76 and 82,
respectively. These two signals are applied as one input to an
AND gate 246 whose output controls the shifting of the alternate
perimeter and alternate number data into the buffer memory 244.
The second input to AND gate 246 is provided by the output o an
OR gate 248 whose inputs comprtse the outputs of the nucleus
alternate number register 226 and the cytoplasm alternate number
register 228.
The operation of the alternate perimeter circuitry shown in
the bottom of Figure 9 can best be understood by looking back
for a moment at Figures 5 and 6. Assume that the ive block
delay arrays 66, 68, and ~0 in Figure 5 and the tag array 92
shown ~n Figure 6 conta~n nuclear points in blocks numbers 4 and
S~ e.~. T4 and T5,whi~1e the bloc~ numbers 1, 2, and 3 contain no
nuçlear p~rllts~ In th~s s~tuatton, ~t ~s clear t~at a perimeter
29 segment has been encountered. ~Iowever, let us assume that all ~
'` ~.~, '
cb/ ~ s
lO~O~Z'~
five blocks h~ve nuclear points, but thc points in the scanned
image just below points 4 and 5 have backyround points ~this will
be recoqnized on the next line scan). The perimeter segment will
be recognized only when the points in TS and T4 are shited
through to the, tag array T3 and ~2 and Tl and the background
poxnts just below points T5 and T4 are placed in T5 ~nd T~.
~t will be appreciated that at this time it is too ]ate to recog-
nize this special case for the perimeter segment by means of
the regular circuitry, The additional alternate perimeter
,10 circuitry shown in Figures 8 and 9 is employed to determine and ,
compile th,e extra perxmeter segments produced in this specific
situation. ~,
Referring bac~ to Figure 9, the contents of the buffer
memory are added into the main memory in response to the main
memory controller. After a suitable delay produced by delay
circuit 250, the alternate perimeter counters are cleared by
the "clear-A" signal.
It remains to descri,be the operation of a one-bit LINK ~ ~
registers 252, 252b in Figure 8. The LII~K 252 is set by logic ~;
86 in Figure 5 when it appears that a nucleated red blood cell ~- ~
has been encoun~ered, or when it is not possible to tell whether ~ ,
a nucleated RBC or a WBC which touches an RBC has been encountered.
Nucleated RBC's have the property of having both a nucleus and
hemoglobin in their cytoplasm. Thus, parts of the cell will
be analyzed by the RBC portion of the hardware in Figure 8, and
part by the WBC nardware. When the data is stored for this type
cell, both the ~BC and ~BC port~ons of the data must be stored.
T~u~ a ~ n the link register 252 causes a "STW" and "STR"
29 s~gnal when e~ther is present thereby effecting the desired --
~, ' . ''', ~ '.
cb! ` - q5 -
; '.
z;z
dual stora~e. In addltion the link xegis-ter 252 set causes the
two region tags ~rom 204 and 206 to be entered on a link push
down stack in computer ma~n memory.
The LII~K 252b is set by the LINK signal from logic ~6 in
S Figure 5 ANDed with the CNTB s~gnal, when a region of non-zero
background is adjacent to a cell region. It: is not always
possible to determine at that time whether the non-zero back-
ground is a real red cell central pallor or white cel~ inclusion,
or whether it will subsequently touch zero background and thus
become part of the true scene background. To save the information
for future reference, the link register 252b set causes the two
region tags from T4 and T5 to be entered intothe same link push
down stack in the computer main memory 240, In this way the cell
region is "associated" with the non-zero background region for
the purpose o~ compiling complete feature for the cell and its
inclusions.
Features for various part of different objects stored under
different tags (red cell, white cell, and non-zero background)
are compiled using the information in the "Equate" and "Link"
stacks, and these cells are then further classified using the
compiled features~ This classification is performed by the
computer CPU (Fig. 1) using instructions in the control memory,
while the scanner is moved to a new field on the sample. After
the classification is completed, the area in the main memory
reser~ed for complete features is zeroed in readiness for the
features from the cells in the next field to be analyzed. This ;
process is repeated until sufficient cells have been examined,
at Which time a summary of the data is output on a data output
2~ dev~ce.
... . . .
~b! - 4 ~ -
, .
It will be appreciated from the fore~oing descript.ion that
the pre~erred embodiment is one specific example of ~ m~re general
method and apparatus for subject analysis characterized by com- :
pilation of partial features from one or more signals represent- :
ing the sample. In the preferred embodiment, partial features ~ :~
are compiled from a raster scanned signal representing the sample ~:
using control signals derived from the previously mentioned
color algebra. However, an alternative version of the inv~ntion ~
can be employed to compile features for the various regions of :.
a sample from a signal representing a sample entrainedi in a gas
or liquid flowing past a fixed sensor, using control signals
derived from that signal, or from a color algebra.
In addition, the preferred embodiment incorporates the .
compilation of complete features from partial features which .~
represent the size, perimeter, and density of the cell at the ~`
various wavelength bands. From these measurements can be
derived features representing the average color and shape
of the various cell regions. However, these features are ;:~
only a few of the many features representing shape, color
and density which can be compiled using my invention. Features
representing cell characteristics other than size, shape, peri- :
meter length, density and color also can be compiled with my .
invention, depending on the des.ires of the user. It should
be understood that the particular set of features described
in connection with the preferred embodiment was chosen for ~ .
purposes of illustration and should not be considered as limit-
ing the scope of the invention. .
Having described in detail a preferred embodiment of
29 my invention 6 it will be apparent to those skilled in the art ~.;.~.
cb/ 47
. .. ,, . : , , . : : - -
L0~ 2
that numerous modifications can be made therein without depart- .
ing from the scope of the invention as dcfined in the following
claims.
- :
- , :
~.
c~p/ - 48 -
~`'.
