Language selection

Search

Patent 2195565 Summary

Third-party information liability

Some of the information on this Web page has been provided by external sources. The Government of Canada is not responsible for the accuracy, reliability or currency of the information supplied by external sources. Users wishing to rely upon this information should consult directly with the source of the information. Content provided by external sources is not subject to official languages, privacy and accessibility requirements.

Claims and Abstract availability

Any discrepancies in the text and image of the Claims and Abstract are due to differing posting times. Text of the Claims and Abstract are posted:

  • At the time the application is open to public inspection;
  • At the time of issue of the patent (grant).
(12) Patent Application: (11) CA 2195565
(54) English Title: INSPECTION DEVICE AND METHOD
(54) French Title: DISPOSITIF ET PROCEDE DE CONTROLE
Status: Dead
Bibliographic Data
(51) International Patent Classification (IPC):
  • G06K 9/00 (2006.01)
  • G01N 15/14 (2006.01)
  • G02B 21/26 (2006.01)
  • G02B 21/36 (2006.01)
  • G06T 5/00 (2006.01)
  • G06T 7/00 (2006.01)
(72) Inventors :
  • TJON-FO-SANG, ROBERT (United States of America)
  • RUTENBERG, MARK R. (United States of America)
(73) Owners :
  • AUTOCYTE NORTH CAROLINA, L.L.C. (United States of America)
(71) Applicants :
  • NEUROMEDICAL SYSTEMS, INC. (United States of America)
(74) Agent: GOWLING LAFLEUR HENDERSON LLP
(74) Associate agent:
(45) Issued:
(86) PCT Filing Date: 1995-07-25
(87) Open to Public Inspection: 1996-02-08
Examination requested: 2001-12-04
Availability of licence: N/A
(25) Language of filing: English

Patent Cooperation Treaty (PCT): Yes
(86) PCT Filing Number: PCT/US1995/010006
(87) International Publication Number: WO1996/003709
(85) National Entry: 1997-01-20

(30) Application Priority Data:
Application No. Country/Territory Date
08/280,293 United States of America 1994-07-26

Abstracts

English Abstract




A device for the visual inspection of a specimen (21), comprising a first
microscope (20) for obtaining a magnified view of different areas of a
specimen (21), a display monitor (14) for displaying images of at least a
subset of the different areas of the specimen (21), selection means (18) for
enabling the selection of an image displayed on the monitor, a second
microscope (20a) for review of an area of the specimen (21) corresponding to
the selected image, a motorized stage for positioning said specimen (21) with
respect to the field of view of the second microscope (20a), and a processor
(16a) for determining the image selected and instructing the motorized stage
to position the specimen (21) so that the area of the specimen (21)
corresponding to the selected image is in the field of view of the second
microscope (20a).


French Abstract

L'invention concerne un dispositif permettant d'effectuer le contrôle visuel d'un spécimen (21). Ce dispositif comprend un premier microscope (20) pour obtenir une vue agrandie de différentes zones d'un spécimen (21), un écran d'affichage (14) pour visualiser des images d'au moins un sous-ensemble des différentes zones du spécimen (21), des moyens de sélection (18) pour permettre la sélection d'une image affichée sur l'écran. Il comprend, en outre, un deuxième microscope (20a) pour contrôler une zone du spécimen (21) correspondant à l'image sélectionnée, un palier motorisé pour positionner ce spécimen (21) par rapport au champ de vision du deuxième microscope (20a), et un processeur (16a) pour déterminer l'image sélectionnée et demander au palier motorisé de positionner le spécimen (21) de sorte qu'il place la zone du spécimen (21) correspondant à l'image sélectionnée dans le champ de vision du deuxième microscope (20a).

Claims

Note: Claims are shown in the official language in which they were submitted.


12

What is claimed is:
1. A device for the visual inspection of a specimen, comprising:
a first microscope for obtaining a magnified view of different areas of a specimen;
a display monitor for displaying images of at least a subset of said different areas
of said specimen;
selection means for enabling the selection of a image displayed on said monitor;a second microscope for obtaining a view of an area of the specimen
corresponding to the selected image,
a motorized stage for positioning said specimen with respect to the field of view
of said second microscope; and
a processor for determining the image selected and instructing said motorized
stage to position said specimen so that the area of the specimen corresponding to said
selected image is in the field of view of said second microscope.
2. The device of claim 1, said selection means including a mouse.
3. The device of claim 1, said selection means including a light pen.
4. The device of claim 1, said selection means including a track ball.
5. The device of claim 1, said selection means including a keyboard.
6. The device of claim 1, said processor including means for classifying
images in said specimen and providing to said display monitor images having
predetermined attributes.
7. The device of claim 1, wherein said specimen is a biological specimen.
8. The device of claim 1, wherein said specimen is a cytological specimen.
9. The device of claim 1, wherein said specimen is a histological specimen.
10. The device of claim 1, wherein said specimen is affixed to a slide.
11. A device for the visual inspection of a specimen, comprising:
a storage memory for storing images and locations of different areas of a
specimen;
a display monitor for displaying at least a subset of said stored images;
selection means for enabling the selection of a image displayed on said monitor;a microscope for obtaining a magnified view of the specimen;
a motorized stage for positioning said specimen with respect to the field of view
of the microscope; and
a processor for determining the image selected and instructing said motorized
stage to position said specimen in accordance with the stored location of the area on the
specimen corresponding to said selected image.

13

12. The device of claim 11, said processor including means for classifying
images in said specimen and providing to said display monitor images having
predetermined attributes.
13. The device of claim 11, including a second microscope for obtaining the
images stored in said storage memory.
14. A method for facilitating the inspection of a specimen, comprising the stepsof:
a) positioning a specimen in the field of view of a microscope to obtain views
of plural different parts of a specimen;
b) displaying images of said views;
c) detecting the selection by an operator of at least one of said images; and
d) repositioning said specimen so that the selected image is in the field of
view of a microscope.
15. The device of claim 14, including the step of classifying images in said
specimen and displaying images of said display having predetermined attributes.

Description

Note: Descriptions are shown in the official language in which they were submitted.


WO 96103709 PCT/US95~10006
~ 21 ~5565




TITLE: INSPECT110N DEVICE AND METHOD

FIELD OF THE ~ NTION
The present invention relates to an apparatus and method for the visual inspection
of an image, and, more particularly, to a device and method for facilitating the review
of a selected specimen or area of interest with a lll;~.lU~-,U~-,.

R~l UND OF THE ~ ON
Often objects requirmg an inspection calmot be inspected by the naked eye.
C~ , the inspection process must be ~ d by viewing an image of the
object, such as an image displayed on a high resolution video monitor. This may be
necessary when the object to be inspected is very small and requires ~ - to be
properly inspected or when the object is located in a hazardous or otherwise i - . ~ . hlr
~.IIYUUIIIII~IIL.
IS A couple of examples of such a situation is the inspection of cellular matter on a
slide for the presence of malignant or I 'i,, cells, as in a Pap smear screeningprocess, and the inspection of ~ t.,. chips. In these instances a number of
objects or a smgle object havmg a number of areas of interest, are displayed on a video
screen. An inspection i ' ' ,, or inspection technician then inspects the individual
objects or areas for flaws, defects, or certain criteria indicative of an event or condition.
Many inspection processes require a skilled technician to review hundreds or thousands
of images per day for the detection of only a few flaws or O~,~.U11~,111,~,5. These inspection
procedures can become quite tedious, tendmg to detract from the overall quality of the
inspection performed by even the most r of tff~h~ Further, the non-
detection of a certain condition, such as the pre~nce of a smgle malignant cell among
thousands of benign cells, can have very severe adverse , While the display
of the image of the object facilitates inspection, it does little to ensure that the inspection
is adequate.
It would be desirable to provide a method or a device which controls or audits the
inspection process of a displayed image of the object in such a way as to increase the
probability of an adequate inspection.

SUMMARY OF TIIE INVI~TION
The present invention provides a device and method for ~ ily
an area of a specimen in the field of view of a Ul;l,IU:>-,U~C to facilitate a
technician directly viewing an area of the specimen which was found to be significant in

WO 96103709 ' ~ 2 1 9 5 5 6 5 PCTIUS95110006




a review of a number of images of different areas of the specimen. The device mcludes
a display upon which a number of images are presented and a mouse, for example, which
allows the technician to select an image on the display for view through a .l~ u~upC.

In accordance witb one aspect of the present invention, a device for the visual
inspection of a specimen includes a first Illl~lU~Uy~i for obtaining a magmfied view of
different areas of a specimen, a display monitor for displaymg images of at least a subset
of the different areas of the specimen, a selector for enabling the selection of a image
displayed on tbe monitor, a second III;~IV:~UIJ~ for obtaining a view of an area of the
specimen ~,UllC:-pUIII~ to the selected image, a motorized stage for positioning the
specimen with respect to the field of view of tbe second Illl~lU:~Up~, and a processor for
.1.~..",;, ,~, the image selected and instructing the motorized stage to position the
specimen so that the area of the specimen cu..c r ~' _ to the selected image is in the
field of view of the second IIPI~IU~U~
In accordance witb anotber aspect of the invention, a device for tbe visual
inspection of a specimen includes a storage memory for stormg images and locations of
different areas of a specimen, a display monitor for displaymg at least a subset of the
stored images, a selector for enabling the selection of a image displayed on the monitor,
a IIII~IU~U~ for obtaining a magnifed view of the specimen, a motorized stage for
positioning the specimen with respect to the field of view of the I~ ,IU:~UI)~;, and a
processor for ~' _ the image selected and instructing the motorized stage to
position the specimen in accordance with the stored location of the area on the specimen
c~" ,~p" ~ g to said selected image.
In accordance with a further aspect of the invention, a method for facilitating the
inspection of a specimen mcludes the steps of positioning a specimen in the field of view
of a .. ;.. u,.o~e to obtain views of different parts of a specimen, displaying images of the
views, detecting the selection by an operator of at least one of the images, and,, the specimen so that the selected image is in the field of view of a
;CIu~u~.
These and other objects, auv _ features and aspects of the present mvention
will become apparent as the following description proceeds.
To the ~ pli~ of the foregoing and related ends, the invention, then
comprises the features hereinafter fully described in the r ~r '' and palLi~ulally
pointed out in claims, the following description and the almexed drawings setting forth
m detail a certain illustrative ~ ~ " of the invention, this being indicative, however,
of but one of the various ways m which the principals of the invention may be employed.

wo 96103709 PC~JUS95110006
~ ' : ; 2i 95565




It will be .I~Jrcc;at d that the scope of the invention is to be determined by the claims and
the equivalents thereof.

BRIEF D~;S~ IIV.; OF THE DRAWINGS
In the annexed drawings:
Figure 1 is a schematic illustration of a viewing and inspection auditing device in
accordance with the present invention;
Figure 2 is an illustration of an exemplary inspection display screen;
Figure 3 is a close-up view of a portion of the display screen of Figure 2;
Figures 4 is a flowchart of an inspection auditing algorithm in accordance with the
present invention; and
Figure 5 is a schematic illustration of a review station.

DETAILED D~ ~ IIVN OF THE INVENnON
With reference now to the several figures in which like reference numerals depict
like items, and initially to Figure 1 there is shown an exemplary inspection apparatus 10
employing the features of the present invention. The apparatus 10 includes a camera 12
for capturing an image of an object, a monitor 14 for displaying the captured image for
inspection purposes, and a general processor 16 and mouse 18 for facilitating the auditing
featnres of the invention. The apparatns 10 may include a ~lu~,u~/e 20 to magnify
small or Illi~,lU:~l,UlJ;, objects to a size suitable for inspection and display on the monitor
14. The Illi.,lU~.,U~JC 20 would thus have particular use when the object to be inspected
is a cell or group of cells. Preferably, the l~_IU~.,U~/C 20 is automated in that it is
associated with a motorized stage 21 which is capable of presenting different areas of an
object or specimen witbin the field of view of the llli.,lU~Cu~ as determined by the
general processor 16. The motorized stage 21 may be a part of the Illi~,lU~.,U~C 20 or
may be a separate element ~ ~ in concert with the U~l,U~C. An optional
image processor 22 may also be included to perform various image processing functions,
such as antomated inspection and/or classifying functions, on the image captured by the
camera 12 prior to display on the monitor 14.
To facilitate discussion of the present invention the following description willfocus on an exemplary automated Pap smear screening process. It will be a~
however that tihe imvention is not limited to the auditing of a Pap smear screening
process, but has broad application in any visual inspection process carried out by a person
~ on a displayed image of an object.

WO 96/03709 - ~ 2 1 9 5 5 6 5 PCT/US9~/10006


In the exemplary i ' ~ ' the objects to be inspected are a number of cells
contained among, for example, 200,000 cells im a Pap smear specimen disposed on a
slide. The inspection apparatus is tailored to ;~r~, ""~ locating and displayingrelatively few ~ u~.,ul!i., cells of the many cells of the specimen. The displayed cells
would be those, for example, which appear most likely to be malignant or ~
Such an automated Pap smear screening system is marketed by N.,u., ' I Systems,
Inc., of Suffern, New York under the trademark PAPNET~. The PAPNET~ screening
system and related screening systems and methods are further disclosed in U.S. Patent
Nos. 4,965,725 and 5,287,272 and in U.S. Patent Application Serial Nos. 07/425,665,
07/502,611, 08/196,714 and 08/196,982 all of which are ;III,UIAU~ ' ~ by this reference.

Preferably the inspection and auditing appaMtus 10 is automated to at least somedegree. The Illi~,lU~CU~)C 20 is prefeMbly automated and is capable of scanning the
specimen while the camera 12 grabs an image of the view provided by the Illi~.lU:I~,UlJC.
The image is then digitized amd prefeMbly provided to the image processor 22.
The image processor 22 may be e~uipped to perforrn its own automated inspection
of the imaged objects im addition to the inspection of the displayed images performed by
the inspection technician. PrefeMbly, the image processor 22 also includes functions
directed to ~' ~ ~ specific locations of mterest in an object, such as the centroid of
a cell, to aid in the auditing functions of the mvention.
In the exemplary ' ' ûf the mvention, the image processor 22 performs
a lllul~hu1O2;;~1 filtering of the image received from the cameM 12 to determine a
number of objects in the image which are of the same ~IJYl~ ~ size as a malignant
or ~,, li, cenical cell. A secondary ~ r~ i.. is also performed by a neuMI
network to further reduce the number of images which must be inspected by a
~;y ' ~ The resultant images amd their CUllC:I~/Ulldillg CO~ ' ' in the specimenare then transferred to the geneMI processor 16 for stoMge m the memory 24 and for
display by the monitor 14.
It has been foumd that the screening f~mctions performed by the primary and
secondary classifiers m series will, for a specimen having at least one ,ul~ or
malignant cell, Mnk at least one l ~ or malignant cell among the highest 64
ranked cells in the specimen. For this reason it is geneMlly sufficient that the.,.yl..t~ - closely examines only the 64 highest Mnked cells. It has further been
fonnd that it is desirable that the 64 highest Mnked cell images, or tiles, be presented in
an 8 x 8 matrix 30 of tiles 32, as is shown in Figure 2 or in some other sequence,
: _ t, etc., for example, four screens of a 4 x 4 matrix of tiles. Cull~c~ ly,

wos6/03709 21 95565 PCr/USss/10006


the ~ya~ , will individually examine each of the sixty-four displayed cellula}
images 34 centered in a tile 32, and identify any possibly malign;mt images for storage
in memory 24 and later analysis by a p rhnlngi~t A ~ u~Liv~ tile is illustrated in
Figure 3 with a cell centered within the tile. The ~ y ;ut~ will then command th~e
device to display the next screen of sixty-four images, and an inspection of these images
will be performed.
The commands to the general processor 16 to store the image of a possibly
malignant cell (e.g., fo} subsequent review by a p~~hnlngict), the commands to display
the next screen of images, amd other commands to the processor 16 may be ~ l 'h ~1
through entering the a~,l,., . sequence of keystrokes in the keyboard or through the
use of a Cull~ mouse-driven cursor. The mouse 18 may be a mechanical mouse
or optical mouse. In the case of a mouse activated system, the ~,J i ' may move
a cursor appearing on the display to the area of the possibly malignant image and depress
a key on the mouse 18 which would command tbe general processor 16 to store thatimage m the memory 24. Alternatively, tbe system may employ a mouse driven menu
;L~ ~;"" such as is known in the art, to command the general processor 16 to
perform a certain fumction, or tbe mouse could be equipped with~ multiple keys for tbe
selection of separate functions. The use of mouses is well known in the art and there are
several 'Iy available mouses and software drivers to operate the mouse for a
number of different computer ~ Variations on the manner in which the
cytu,~ l : (inspection technician) interfaces with the general processor 16 will be
apparent to those of ordinary skill in the art and are within the scope of the invention.
Other manners in which the technician can interface with the general processor 16 include
a light pen, a track ball and a keyboard, for example.
While most C,Yl ~ ~ ~ attempt to .hl.l;Y ~ l-y inspect each cell in a specimen,
the economic pressures on a ~,y ' to examme an increasing number of slides a
day are ~ J~. While United States federal regulation of the Pap smear inspectionprocess mandates that a ~"Y ~ ~ inspect only a prescribed number, for example,
eighty, specimen slides per day to maintain quality of inspection andlor to minimr~e
fatigue, this is no guarantee that each possibly malignant cell will be adequately
inspected.
The present invention, however, provides a means requiring the ~;yl~1t~. l~..;. ;~ to
direct his or her attention and focus to each and every cellular image on the display to
alleviate this problem. Using the mouse 18, the ~;y. ' must drag the display
cursor 36 to within a certain tolerance of the a~ center 38 of each image 34 of
the sixty-four tiles 32 that are presented and maintam the mouse in that respective center

WO 96103709 PCTIUS9~/10006
~ ~ ' 21 95565




for a period of time sufficient for a cyl~A~ to perform an adequate ~ r" : i""
of the imaged cell. In the cytological classifier e.l.bodil.~l.t of the invention a view of
primarily only a single cell is ~ ., 'y centered in the image area (also referred to
as tlle herein). The position ;~f~ ;.. of the cursor, such as its X and Y .uu.,''
is transferred to the general processor 16 such as &ough a port, which is cu.l~ Liu-~l.
If the positional i.~ matches, to within a certam tolerance, the center of the
image, then the general processor 16 notes that the image has been examined. Theindication that the image has been examined, as evidenced from the cursor 36 being
.II),Ul~ ~ ' ty centered on an image 34 for a sufficient period of time to perform an
adequate inspection, may be mdicated on the display 30 such as by l.:ghl;gh:; c, the
perimeter of the tile 32 or by some other means.
Recalling that there are a number of discrete images on the display, preferably an
8 x 8 matrix 30 of sixty-four tiles 32, with the suspect cell 34 pre-centered in each
image, the cy~ - must devote at least a substantial amount of attention to the
suspect cell in order to correctly center the cursor 36 in each tile in order to move to the
next screen or specimen slide. Only after the cursor 36 has been 1~ 'y centered
on each image 34 on the screen 30 for a sufficient period of time will the general
processor 16 allow the next set of images 30 to be displayed on the monitor 14. The
display of the next set of images 30 may be automatic or may be the result of the
cyi ' , ' ~, the cursor 36 to a set location on the screen and clicking the
mouse 18 or &ough a certain sequence of keystrokes.
Referring to Figure 4, there is shown a relatively detailed flowchart
illustrating the step-by-step functions preferably performed to accomplish the features
described above. In the discussion below reference numeMls contained with ~
designate like numbered steps in the flowchart of Figure 4. After a screen 30 consisting
of, prefeMbly, sixty-four tiles 32 has been displayed on the monitor 14, the mouse
routine 95 will begin ~ ~ whether the cursor 36 has appeared within a certain
number of prxels of a tile center 38 for a sufficient duMtion of time for a ~
to perform an adequate inspection of the image 34 centered in that tile. Initially, the
routine 95 will determine whether the screen 30 has recently been changed to display a
new set of sixty-four images (100). If not, meaning that a new screen 30 has not been
requested by the cyi ' ' since all the tiles 32 of the last screen have been
inspected, the routine 95 will not perform any cursor checking. If the screen 30 has been
changed, then further screen changing is disabled (105), thus preventing a new screen
from being displayed by the general processor 16, regardless of whether such a change
is requested by the ~- ' A sepaMte ~ r~m~n~l timer is then initialized for

Wo 96/03709 2 1 9 5 5 6 5 pcT/us9sllooo6


each of the sixty-four tiles 32 in the screen 30 (110). These decremental timers are set
at a number sufficiently high that number multiplied by the time period between obtaining
cursor 36 positions from the port will equal a time which has been determined to be
sufficiently long in duration for a ~,JL~ ~ to adequately inspect a tile 32 to
determine whether the image 34 centered in that tile should be tagged for further analysis
by a cytologist. The X and Y . ' of the screen cursor 36 as determmed by the
mouse 18 are then obtamed from a port, as is cu.l.~,.ltiu~l (115). Note that while this
port is contimually refreshed with the current position of the screen cursor 36, the routine
will access that port and obtain the X and Y ~o, of the screen cursor only once
every certain duration as determined by a timing circuit in the general processor 16 or
the time it takes to execute a pass through the routine 95.
The fl~rr~m~nf~l timers (counters) may be set to a count of one in which case the
simple passing of the mouse controlled screen cursor through the ~ JIUl ' ' place in a
tile for a time that is long enough to be detected there will be an acceptable time.
After the c ~- of the screen cursor 36 have been obtamed, the X and Y
uo, ~ are compared against pl~ ~ ' values to determine whether the cursor
36 lies within a certain distance of the tile center 38 (120). If the maximum tolerance
dist~mce is held coDstant at all positions around the tile center 38, a circle is formed
within which the screen cursor 36 must lie in order for the Cullc_, ' ~ d~ tld
timer to be dc~l~ ' However, as this is somewhat c , lly intensive,
preferably the: ' of the screen cursor 36 are checked to determine whether they
lie within a square or tolerance box 40 formed around the tile center. This facilitates
d ~ v whether the cursor is within a certam boxed tolerance of the C~llt~
C A 'Iy~ to define the tolerance box 40 ~ " ~ the center 38 of each tile 32,
only four values need be ~l~ ' ~ ' and stored, for example, the X coordinate at the
top left corner 42 of the box, the Y coordinate at the top right corner 44 of the box, the
Y coordinate at the bottom left corner 46 of the box, and finally the X coordinate at the
bottom right corner of 48 the box. By comparing whether the X coordinate of the screen
cursor 36 lies between the top left X coordinate and the bottom right corner X coordinate,
and whether the Y coordinate of the screen cursor is between the top right corner Y
coordinate and bottom left corner Y coordinate, it can be determined whether the screen
cursor lies within this tolerance box 40 (125).
If the cursor 36 is not within the tolerance box 40 ~UII~ , a tile center 38,
then no further action is taken until the next cursor position is obtained from the port.
If the cursor 36 lies within the tolerance box 40 centered around a tile center 38, the
~lrrrl~mrnt~l counter for that tile 32 is d~l~ .l~ by one (130). It is then determined




~ . ~ . .. . . .

WO 96/03709 , ,, PC'rlUS95110006
21 95565

whether the d~ C ' ' counter for each of the sixty-four tiles 32 has been d~
to zero (135). If not, then no further action in the routine 95 is taken until the timed
period has elapsed to obtain a new cursor position from the port. If, however, the
de~...l.._lli~l timer for all tiles 32 have been i l . .' ;1 to zero, then the screen change
is enabled (140). ('~ , when the ~ ' requests that the screen 30 be
updated to display a new set of sixty-four images, the general processor 16 will~,hlu .. Icdl7_ the request and display the next screen 30 of images 34. When the routine
95 now checks to determine whether the screen 30 has been changed (100) it will
determine that it has, and the ability to change the screen to display a new set of sixty-
four images will be disabled until those images have been inspected (105).
It is noted that the flowchart of Figure 4 illustrates only those functions related
to the cursor checking routine. The general processor 16 will, of course, perform other
functions in between the time periods in which the cursor 36 position is obtained, and that
position is checked to determine whether it lies within the tolerance box 40 of a tile
center 38. Also note that, if the ~,Ji ' has determined that a certain tile 32
merits further review by a cytologist, that tile is stored for future review, and the
' I timer for that tile is " l~, see to zero, as if that tile had been inspectedfor the ~ duration of time.
Each time the d~ I timer for a certain tile 32 has been d ' to zero
an indication will be made on the screen that tile has been examined for an acceptable
duration of time. Such an indication may be through ~ ;h ;--p the perimeter 50 of the
tile 32, or making some other visual notation witbin that tile.
Inthepreferred~ I ' t, itisnotneoessarythatthec~. ' ~ ~ manipulate
the screen cursor 36, via the mouse 18, to the center 38 of a tile 32 and leave it there for
the complete time duration. It is only necessary that the screen cursor 36 appear within
the tolerance box 40 around a tile center 38 and that the routine 95 confirms the presence
of the cursor within the tolerance box a certain number of times. ('r---, '.~/, the
Ly ' ' ~ ~ may compare two different tiles 32 on the screen 30, and move the screen
cursor to the al~yl~ center 38 of each tile back and forth a couple times if he or she
so desires. It is only necessary that the composite time that the screen cursor 36 spends
within the tolerance box 40 is sufficient to indicate an adequate inspection of that tile.
While the above routine refers to an ' -' wherein tiles may be
inspected ~ '~, it would be obvious to one of ordinary skill in the art that theroutine could be simply modifled to require that each tile be inspected separately for a
set period of time before moving the cursor to the next tile, or that the timing functions
could be removed altogether requiring only that the routine confirms that the screen

wo 96/03709 2 1 9 5 5 6 5 rcTIusg~lloao6
~ .

~ . .
cursor 36 has been moved tbrough the tolerance box 4û ~ UU~ Ulg each tile cenoer 38.
This latter ~ v~ without the timing constraints would have particular application
when a person ,u.lrulllPIll~ the inspection would recognrze a flaw or that a cell was
malignant very quickly, or as quickly as he or she could position the cursor near the
S cenoer 38 of a tile 32 and the routine 95 could confirm that the cursor has been moved
witbin the required tolerance of the tile center.
Given the description above and a reasonable amount of time and effort, one of
ordinary skill in the art of ~UIuL ~ "u~ could wrioe the software code
in the h~lUl ~ language following the flow chart of Figure 4 to inoerface with the
lû mouse and an inoernal or external timing circuit to reduce the features above into a format
suitable for execution by the general processor. The resultant code would then be loaded
into memory 26 accessible by the general processor 16. It would also be apparent to one
of skill in the art tbat equivalent devices, such as a light pen, for example, could be
substituoed for tbe mouse, with , ' ~ changes to the inoerface hardware and the
software drivers, to ~rr~p~ the same results as the mouse.
In some instances i~ may be desirable tbat the technician review not only the
images presented on the display 14, but that the oechnician review through a ~lu~"ù~,u~,
actual views of an object or specimen that has been identified on the display as being of
concern. This may be the case when reviewing and classifying a Pap smear for the2û presence of malignant cells. In wch a review and l~ ;. . it is of oen beneficial for
the oechnician to view not only a cell in isolation, but also to view the contextual
wrround of the cell, i.e., the maoerial on the slide in the vicinity of the suspect cell. The
~u,.~ ' g maoerial can confum to the technician whether the wspect cell is malignant
or benign or whether the cell is part of an overall grouping of ceils, in which case the
oechnician may also be able ~o diagnose certain 'i,, ~ types. Further, a uli~lu~U~ue
view is often betoer in color, contrast and in resolution than a recreation of the image on
a display.
Review of the actual view of the object or cell is facilitaoed by employing a
u~u~,c system 20 having a motorized stage 21, as shown in Figure 1, with the
motorized stage bemg in with the general processor 16. The motorized
stage 21 positions an area of a specimen or slide mounted thereon with respect to the
field of view of ~ uscu~, 20 based on h~rul~u~l~iùll, such as X and Y .:o~ ,
received from the general processor 16. An exemplary u,i.,.u ,~u~,c is ~ ura~,Lu~cd by
Carl Zeiss, Inc. of Germany, and a suitable motorized stage is ll~urh~luled by Ludl
Electric Products, Ltd. of Hawthorne, New York.

WO 96103~09 ' ~ ~ , PCT/US9~/10006
21 95565

When a technician detects on the display 14 a cell image which is suspected of
being a malignant cell, the technician selects the cell image for review through the
Illil,lUD(,U~e 20 such as by positioning a screen cursor on the cell image and depressing
a button on a mouse, for example. The selection of the cell image is transferred to the
general processor 16 which in turn sends dU~I~ r ' ' coordinate ~ such as X
and Y ~,OVI~" of the center of the image, to the motorized stage 21. The motorized
stage 21 thus repositions the area of the slide or specimen mounted thereon wnich
CUll~ UnllD to the selected image in the field of view of the Illil,lUD~,O~JC 20. When the
technician then views the specimen through the lUiClU~U,U~, 20, the technician will see the
area of the specimen containing the suspect cell, preferably with the suspect cell centered
in the field of view. The technician may also scan the area and Dulluuudill6 areas with
the UUclUDCuyc 20 using the motorized stage 21 to affect the lrl~ ,. .g of the
specimen with relation to the field of view of the Illi,lUD~,U~lC.
In some I ' it is ad~l .6~11D for the scanning of the specimen and the
review of the scanned images to take place at separate stations. In such an instance the
scanning station 60, which scans a specimen and stores images for later review by a
technician, would be ~ "y as shown in Figure 1, although the video display 14
may not be necessary if no review of images were to take place at the scanning station.
A separate review station 62 for the review of previously scanned images is shown in
Figure 5. The review station 62, like the scanning station, preferably includes a general
processor 16a with memory 26a and a mouse 18a, a storage memory 24a for storing
images to be reviewed, a video display 14a for the display of the stored images, a
~ui~luDuulJe 20a and a motorized stage 21a. The review station 62 nced not include a
dedicated image processor or a camera smce the images have already been created and
possibly partially classifed by the scanning station 60. In this; ' - ' t, the scanned
specimen is taken from the scanning station 60 along with the storcd images and their
coordinate locations on the specimen, which may be stored on elcctronic media or an
optical disk, for instance, to a review station 62. The specimen is then placed on the
motorized stage 21a and the images are loaded into storage memory 24a. The technician
then calls up the images on the display 14a and reviews the images as discussed above.
If the technician detects an image on the display 14a which is suspicious, the image is
selected, such as by positioning a screen cursor on the image and depressing a certain
button on a mouse 18a. The general processor 16a then determines the could' ' ofthe area of the specimen Cullc r ~ to the selccted image and transfers the coordinate
i,.r",.. - ;".. to the motorized stage 21a which positions the a,u~JIUl area of the
specimen in the field of view of the lUiClUDCu~, 20a. The technician can then examine

W0 96/03709 21 9 5 5 6 5 PCTIUS95110006
.



11
the actual magnified view of the suspicious cell or other object as well as the contextual
surround through the IIIiCIU~CUAUC 20a.
It will be d~U~UI~ ' ' ' that while the example depicted is that of inspecting cells
that are suspected to be malignant or ~ the routine could be applied to any
inspection technique. For example, the routine could be used to determine whether a
' inspection of a microchip was being performed adequately, or for any other
c.. ~u- t .;, ~ ~rplir~tinn such as the inspection of the image of a human heart, or a
simple . ' I gear. Additionally, while in the described example the location to
which the techniciam's interest was being directed was the center of the image, the routine
could be applied to a location other than the center of the image or to several locations
within the image.
While it is still possible to circumvent the inspection checks of the invention, a
skilled cyt~ ' would have to practically intentionally ignore the ~~u-~hùlo~y of
the cellular images being displayed to avoid ~ , at least a cursory inspection of
the images.

Representative Drawing
A single figure which represents the drawing illustrating the invention.
Administrative Status

For a clearer understanding of the status of the application/patent presented on this page, the site Disclaimer , as well as the definitions for Patent , Administrative Status , Maintenance Fee  and Payment History  should be consulted.

Administrative Status

Title Date
Forecasted Issue Date Unavailable
(86) PCT Filing Date 1995-07-25
(87) PCT Publication Date 1996-02-08
(85) National Entry 1997-01-20
Examination Requested 2001-12-04
Dead Application 2005-07-25

Abandonment History

Abandonment Date Reason Reinstatement Date
1997-07-25 FAILURE TO PAY APPLICATION MAINTENANCE FEE 1997-10-23
2004-07-26 FAILURE TO PAY APPLICATION MAINTENANCE FEE

Payment History

Fee Type Anniversary Year Due Date Amount Paid Paid Date
Application Fee $0.00 1997-01-20
Registration of a document - section 124 $100.00 1997-04-08
Reinstatement: Failure to Pay Application Maintenance Fees $200.00 1997-10-23
Maintenance Fee - Application - New Act 2 1997-07-25 $100.00 1997-10-23
Maintenance Fee - Application - New Act 3 1998-07-27 $100.00 1998-07-10
Maintenance Fee - Application - New Act 4 1999-07-26 $100.00 1999-07-19
Maintenance Fee - Application - New Act 5 2000-07-25 $150.00 2000-07-25
Registration of a document - section 124 $50.00 2000-10-05
Maintenance Fee - Application - New Act 6 2001-07-25 $150.00 2001-07-12
Request for Examination $400.00 2001-12-04
Maintenance Fee - Application - New Act 7 2002-07-25 $150.00 2002-06-18
Maintenance Fee - Application - New Act 8 2003-07-25 $150.00 2003-06-25
Owners on Record

Note: Records showing the ownership history in alphabetical order.

Current Owners on Record
AUTOCYTE NORTH CAROLINA, L.L.C.
Past Owners on Record
NEUROMEDICAL SYSTEMS, INC.
RUTENBERG, MARK R.
TJON-FO-SANG, ROBERT
Past Owners that do not appear in the "Owners on Record" listing will appear in other documentation within the application.
Documents

To view selected files, please enter reCAPTCHA code :



To view images, click a link in the Document Description column. To download the documents, select one or more checkboxes in the first column and then click the "Download Selected in PDF format (Zip Archive)" or the "Download Selected as Single PDF" button.

List of published and non-published patent-specific documents on the CPD .

If you have any difficulty accessing content, you can call the Client Service Centre at 1-866-997-1936 or send them an e-mail at CIPO Client Service Centre.


Document
Description 
Date
(yyyy-mm-dd) 
Number of pages   Size of Image (KB) 
Representative Drawing 1997-06-10 1 6
Cover Page 1997-05-05 1 11
Abstract 1996-02-08 1 37
Description 1996-02-08 11 478
Claims 1996-02-08 2 52
Drawings 1996-02-08 3 38
Cover Page 1998-06-11 1 11
Assignment 1997-01-20 19 704
PCT 1997-01-20 7 362
Prosecution-Amendment 2001-12-04 1 51
Correspondence 1997-02-18 1 38
Prosecution-Amendment 2003-06-03 1 36
Fees 1997-09-30 2 128