Note: Descriptions are shown in the official language in which they were submitted.
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
METHOD AND SYSTEM FOR GENERAL PURPOSE ANALYSIS OF
EXPERIMENTAL DATA
CROSS REFERENCES TO RELATED APPLICATIONS
This applications claims priority from U.S. Provisional Applications No.
60/135,481, filed on May 24, 1999, and 60/140,061, filed on June 21, 1999.
COPYRIGHT AUTHORIZATION
~o A portion of the disclosure of this patent document contains material,
which is
subject to copyright protection. The copyright owner has no objection to the
facsimile
reproduction by anyone of the patent disclosure, as it appears in the Patent
and
Trademark Office patent files or records, but otherwise reserves all copyright
rights
whatsoever.
15 FIELD OF THE INVENTION
This invention relates to analyzing experimental data. More specifically, it
relates to methods and system for general purpose analysis of images from
experimental data collected with automated feature-rich, high-throughput
2o experimental data collection systems.
BACKGROUND OF THE INVENTION
Historically, the discovery and development of new drugs has been an
expensive, time consuming and inefficient process. With estimated costs of
bringing
25 a single drug to market requiring an investment of approximately 8 to 12
years and
approximately $350 to $500 million, the pharmaceutical research and
development
market is in need of new technologies that can streamline the drug discovery
process.
Companies in the pharmaceutical research and development market are under
fierce
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
pressure to shorten research and development cycles for developing new drugs,
while
at the same time, novel drug discovery screening instrumentation technologies
are
being deployed, producing a huge amount of experimental data.
Innovations in automated screening systems for biological and other research
are capable of generating enormous amounts of data. The massive volumes of
feature-rich data being generated by these systems and the effective
management and
use of information from the data has created a number of very challenging
problems.
As is known in the art, "feature-rich" data includes data wherein one or more
individual features of an object of interest (e.g., a cell) can be collected.
To fully
~o exploit the potential of data from high-volume data generating screening
instrumentation, there is a need for new informatic and bioinformatic tools.
Identification, selection, and validation of targets for the screening of new
drug compounds is often completed at a nucleotide level using sequences of
Deoxyribonucleic Acid ("DNA"), Ribonucleic Acid ("RNA") or other nucleotides.
15 "Genes" are regions of DNA, and "proteins" are the products of genes. The
existence
and concentration of protein molecules typically helps determine if a gene is
"expressed" or "repressed" in a given situation. Responses to natural and
artificial
compounds as indicated by changes in gene expression are typically used to
improve
existing drugs, and develop new drugs. Changes in binding between proteins are
also
2o used to screen compounds for biological activity. However, it is often more
appropriate to determine the effect of a new compound on a cellular level
instead of a
nucleotide or protein level.
Cells are the basic units of life and integrate information from DNA, RNA,
proteins, metabolites, ions and other cellular components. New compounds that
may
25 look promising at a nucleotide or protein level may be toxic at a cellular
or organism
2
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
level. Florescence-based reagents can be applied to cells to determine ion
concentrations, membrane potentials, enzyme activities, gene expression, as
well as
the presence of metabolites, proteins, lipids, carbohydrates, and other
cellular
components.
There are two types of cell screening methods that are typically used: ( 1 )
fixed
cell screening; and (2) live cell screening. For fixed cell screening,
initially living
cells are treated with experimental compounds being tested. After application
of a
desired compound the cells are incubated for a given time and then "fixed" to
preserve a final cell state for later analysis. Live cell screening usually
requires
~o environmental control of the cells (e.g., temperature, humidity, gases,
etc.) since
before, during and after application of a desired compound, the cells are kept
in a
controlled environment until data collection is complete. Fixed cell assays
allow
spatial measurements to be acquired, but only at one point in time. Live cell
assays
allow both spatial and temporal measurements to be acquired.
As is known in the art, a "cell assay" is a specific implementation of image
processing methods used to analyze images of cells and return results related
to the
biological processes being examined. As is known in the art, a "cell protocol"
specifies a series of system settings including a type of analysis instrument,
a cell
assay, dyes used to measure biological markers in cells, cell identification
parameters
2o and other general image processing parameters used to collect cell data.
The spatial and temporal frequency of chemical and molecular information
present within cells makes it possible to extract feature-rich cell
information from
populations of cells. For example, multiple molecular and biochemical
interactions,
cell kinetics, changes in sub-cellular distributions, changes in cellular
morphology,
changes in individual cell subtypes in mixed populations, changes and sub-
cellular
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
molecular activity, changes in cell communication, and other types of cell
information
can be acquired.
The types of biochemical and molecular cell-based assays now accessible
through fluorescence-based reagents is expanding rapidly. The need for
automatically
extracting additional information from a growing list of cell-based assays has
allowed
automated platforms for feature-rich assay screening of cells to be developed.
For
example, the ArrayScan System by Cellomics, Inc. of Pittsburgh, Pennsylvania,
is one
such feature-rich cell screening system. Cell based systems such as FLIPR, by
Molecular Devices, Inc. of Sunnyvale, California, FMAT, of PE Biosystems of
~o Foster City, California, ViewLux by EG&G Wallac, now a subsidiary of Perkin-
Elmer Life Sciences of Gaithersburg, Maryland, and others also generate large
amounts of data and photographic images that would benefit from efficient data
management solutions. Photographic images are typically collected using a
digital
camera, but can also be generated by scanning systems such as confocal light
~5 microscope systems. A single photographic image may take up as much as 512
Kilobytes ("KB") or more of storage space as is explained below. Collecting
and
storing a large number of photographic images ads to the data problems
encountered
when using high throughput systems. For more information on fluorescence based
systems, see "Bright ideas for high-throughput screening - One-step
fluorescence
2o HTS assays are getting faster, cheaper, smaller and more sensitive," by
Randy Wedin,
Modern Drug Discovery, Vol. 2(3), pp. 61-71, May/June 1999.
Such automated feature-rich cell screening systems and other systems known
in the art typically include microplate scanning hardware, fluorescence
excitation of
cells, fluorescence emission optics, a microscope with a camera, data
collection, data
25 storage and data display capabilities. For more information on feature-rich
cell
4
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
screening see "High content fluorescence-based screening," by Kenneth A.
Guiliano,
et al., Journal of Biomolecular Screening, Vol. 2, No. 4, pp. 249-259, Winter
1997,
ISSN 1087-0571, "PTH receptor internalization," Bruce R. Conway, et al.,
Journal of
Biomolecular Screening, Vol. 4, No. 2, pp. 75-68, April 1999, ISSN 1087-0571,
"Fluorescent-protein biosensors: new tools for drug discovery," Kenneth A.
Giuliano
and D. Lansing Taylor, Trends in Biotechnology, ("TIBTECH"), Vol. 16, No. 3,
pp.
99-146, March 1998, ISSN 0167-7799, all of which are incorporated by
reference.
An automated feature-rich cell screening system typically automatically scans
a microplate with multiple wells and acquires mufti-color fluorescence data of
cells at
io one or more instances of time at a pre-determined spatial resolution.
Automated
feature-rich cell screening systems typically support multiple channels of
fluorescence
to collect mufti-color fluorescence data and may also provide the ability to
collect cell
feature information on a cell-by-cell basis including such features as the
brightness,
size and shape of cells and sub-cellar measurements of organelles within a
cell.
The collection of data from high throughput screening systems typically
produces a very large quantity of data and presents a number of bioinformatics
problems. As is known in the art, "bioinformatic" techniques are used to
address
problems related to the collection, processing, storage, retrieval and
analysis of
biological information including cellular information. Bioinformatics is
defined as
2o the systematic development and application of information technologies and
data
processing techniques for collecting, analyzing and displaying data acquired
by
experiments, modeling, database searching, and instrumentation to make
observations
about biological processes.
The need for efficient data management is not limited to feature-rich cell
z5 screening systems or to cell based arrays. Virtually any instrument that
runs High
5
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
Throughput Screening ("HTS") assays also generate large amounts of data. For
example, with the growing use of other data collection techniques such as DNA
arrays, bio-chips, microscopy, micro-arrays, gel analysis, the amount of data
collected, including photographic image data is also growing exponentially. As
is
known in the art, a "bio-chip" is a stratum with hundreds or thousands of
absorbent
micro-wells on its surface. A micro-well includes a specific point of
attachment that
may or may not have any depth. A single bio-chip may contain 10,000 or more
micro-gels. When performing an assay test, each micro-well on a bio-chip is
like a
micro-test tube or a well in a microplate. A bio-chip provides a medium for
analyzing
~o known and unknown biological (e.g., nucleotides, cells, etc.) samples in an
automated, high-throughput screening system.
Although a wide variety of data collection techniques can be used, cell-based
high throughput screening systems are used as an example to illustrate some of
the
associated data management problems encountered by virtually all high
throughput
~5 screening systems. Collecting feature-rich cell data from a microplate
plate used for
feature-rich screening typically includes 96 to 1536 individual wells. As is
known in
the art, a "microplate" is a flat, shallow dish that stores multiple samples
for analysis.
A "well" is a small area in a microplate used to contain an individual sample
for
analysis. Each well may be divided into multiple fields. A "field" is a sub-
region of a
2o well that represents a field of vision (i.e., a zoom level) for a
photographic
microscope. Each well is typically divided into one to sixteen fields, or more
Each field typically will have between one and six photographic images taken
of it, each using a different light filter to capture a different wavelength
of light for a
different fluorescence response for desired cell components. In each field, a
pre-
25 determined number of cells are selected to analyze. The number of cells
will vary
6
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
(e.g., between one and one hundred or more). For each cell, multiple cell
features are
collected. The cell features may include features such as size, shape,
brightness,
pattern, etc. of a cell.
There are a number of problems associated with analyzing experimental data
collected from feature-rich cell screening systems. One problem is that a
biologist
may desire to create his/her own cell assay to analyze biological processes
associated
with cells. However, most biologist do not have the expertise required to
implement
image processing methods necessary to complete his/her cell assay.
Another problem is that a biologist may desire to develop two or more
~o different cell assays run at the same time to focus on different cell
information. For
example, for a first cell assay it may be necessary to collect cell feature
data including
cell shape, cell size and cell diameter data for a desired experiment by
analyzing cell
image data. For a second cell assay, it may be desirable to collect skewness
and
kurtosis for a desired cell feature by analyzing cell image data. However,
analysis
tools known in the art do not allow a biologist to select his/her own image
processing
techniques to create a cell assay outside of a fixed list of image processing
techniques
available with the analysis tool. That is, a biologist may desire to analyze
skewness
and kurtosis, but hislher analysis tool may only provide image processing
techniques
for analyzing cell shape, and cell size.
2o Another problem is that many image processing tools can not be easily
interfaced with existing feature-rich cell screening systems. Many image
processing
tools known in the art are proprietary and are not adaptable for general use
with
existing feature-rich cell screening systems. This also limits the ability of
a biologist
to create a cell assay for a desired experiment.
7
CA 02359467 2001-07-18
WO 00/72258 PCT/iJS00/14246
Another problem is that even if image processing packages known in the art
are used, a biologist or other scientist, has to select not only image
processing routines
to accomplish an assay feature measurement, but also choose from a large
number of
image processing options for the image processing routines. This may create
additional confusion or frustration on the part of the biologist as the
biologist may not
know what image processing options are the most appropriate for a give assay
feature.
Thus, it is desirable to provide a general purpose analysis tool that allows
virtually any cell assay to be created by a biologist. The general purpose
tool should
provide image processing techniques for a cell assay created by a biologist,
without
~o requiring the biologist, other scientist or analyst have any in-depth
knowledge of
image processing techniques.
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
SUMMARY OF THE INVENTION
In accordance with preferred embodiments of the present invention, some of
the problems associated with analyzing image acquired from feature-rich
experimental data are overcome. Methods and system for general purpose
analysis of
images acquired from experimental data are presented.
One aspect of the invention includes a method for presenting assay features
associated with a pre-determined set of image processing routines for
analyzing
experimental data including images. The pre-determined set of image processing
routines includes only a limited set of options available for processing an
image.
~ o Another aspect of the invention includes a method for analyzing
experimental data
including images using a set of selected assay features selected from a set of
pre-
determined assay features to help analyze image data. The set of selected
assay
features are processed in a pre-determined order appropriate for analysis of
image
data.
A pre-determined set of general assay features is presented. An assay feature
includes one or more measurements for an object in a digital photographic
image
acquired from the experimental data. The set of general assay features
includes object
features, aggregate features and general purpose image processing features. A
set of
desired assay features is selected from the pre-determined set of general
assay
2o features. A set of images is processed using the desired assay features
from the
selected set of general assay features. Such general assay features (e.g.,
length, width,
height, etc.) are common image processing features that are useful for
virtually any
assay or protocol that may be developed to obtain measurements from
experimental
data. The general assay features presented typically include only a few of the
many
possible image processing options that could be used to take such measurements
from
9
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
a digital image, thereby helping to reduce confusion associated selecting such
image
processing options.
The methods and system may help provide a general purpose assay
development tool. The methods and system may allow a biologist, other
scientist or
lab technician not trained in image processing techniques to quickly and
easily design
protocols and assays to analyze images acquired from experimental data (e.g.,
cells).
The methods and system may improve the identification, selection, validation
and
screening of new experimental compounds (e.g., drug compounds). The methods
and
system may also be used to provide new bioinformatic techniques used to make
~o observations about experimental data including multiple digital
photographic images.
The foregoing and other features and advantages of preferred embodiments of
the present invention will be more readily apparent from the following
detailed
description. The detailed description proceeds with references to the
accompanying
drawings.
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
BRIEF DESCRIPTION OF THE DRAWINGS
Preferred embodiments of the present invention are described with reference
to the following drawings, wherein:
FIG. 1 A is a block diagram illustrating an exemplary experimental data
storage system;
FIG. 1B is a block diagram illustrating an exemplary experimental data
storage system;
~o FIG. 2 is a block diagram illustrating an exemplary array scan module
architecture;
FIG. 3 is a flow diagram illustrating a method for selecting assay features
for
experimental data.
FIG. 4 is a flow diagram illustrating a method for selecting assay features
for
~5 images acquired from experimental data;
FIG. 5 is a block diagram illustrating an exemplary graphical user interface
for
selecting object features;
FIG. 6 is a block diagram illustrating an exemplary graphical user interface
for
selecting general image processing operations; and
2o FIG. 7 is a block diagram illustrating a screen display for graphically
displaying images processed using a desired set of assay features.
11
CA 02359467 2001-07-18
WO 00/72258 PCT'/US00/14246
DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS
Exemplary data storage system
FIG. 1 A illustrates an exemplary data storage system 10 for preferred
embodiments of the present invention. The exemplary data storage system 10
includes
s an analysis instrument 12, connected to a client computer 18, a shared
database 24 and a
data store archive 30 with a computer network 40. The analysis instrument 12
includes
any scanning instrument capable of collecting feature-rich experimental data,
such as
nucleotide, protein, cell or other experimental data, or any analysis
instrument capable of
analyzing feature-rich experimental data. As is known in the art, "feature-
rich" data
~o includes data wherein one or more individual features of an object of
interest (e.g., a
cell) can be collected. The client computer 18 is any conventional computer
including a
display application that is used to lead a scientist or lab technician through
data analysis.
The shared database 24 is a mufti-user, mufti-view relational database that
stores data
from the analysis instrument 12. The data archive 30 is used to provide
virtually
~5 unlimited amounts of "virtual" disk space with a mufti-layer hierarchical
storage
management system. The computer network 40 is any fast Local Area Network
("LAN") (e.g., capable of data rates of 100 Mega-bit per second or faster).
However, the
present invention is not limited to this embodiment and more or fewer, and
equivalent
types of components can also be used. Data storage system 10 can be used for
virtually
2o any system capable of collecting and/or analyzing feature-rich experimental
data from
biological and non-biological experiments.
FIG. 1B illustrates an exemplary data storage system 10' for one preferred
embodiment of the present invention with specific components. However, the
present
invention is not limited to this one preferred embodiment, and more or fewer,
and
25 equivalent types of components can also be used. The data storage system
10' includes
12
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
one or more analysis instruments 12, 14, 16, for collecting and/or analyzing
feature-rich
experimental data, one or more data client computers, 18, 20, 22, a shared
database 24, a
data store server 26, and a shared database file server 28. A data store
archive 30
includes any of a disk archive 32, an optical jukebox 34 or a tape drive 36.
The data
store archive 30 can be used to provide virtually unlimited amounts of
"virtual" disk
space with a multi-layer hierarchical storage management system without
changing
the design of any databases used to stored collected experimental data as is
explained
below. The data store archive 30 can be managed by an optional data archive
server 38.
Data storage system 10' components are connected by a computer network 40.
~o However, more or fewer data store components can also be used and the
present
invention is not limited to the data storage system 10' components illustrated
in FIG. 1B.
In one exemplary preferred embodiment of the present invention, data storage
system 10' includes the following specific components. However, the present
invention is not limited to these specific components and other similar or
equivalent
~5 components may also be used. Analysis instruments 12, 14, 16, comprise a
feature-
rich array scanning system capable of collecting and/or analyzing experimental
data such
as cell experimental data from microplates, DNA arrays or other chip-based or
bio-chip
based arrays. Bio-chips include any of those provided by Motorola Corporation
of
Schaumburg, Illinois, Packard Instrument, a subsidiary of Packard BioScience
Co. of
2o Meriden, Connecticut, Genometrix, Inc. of Woodlands, Texas, and others.
Analysis instruments 12, 14, 16 include any of those provided by Cellomics,
Inc. of Pittsburgh, Pennsylvania, Aurora Biosciences Corporation of San Diego,
California, Molecular Devices, Inc. of Sunnyvale, California, PE Biosystems of
Foster City, California, Perkin-Elmer Life Sciences of Gaithersburg, Maryland,
and
25 others. The one or more data client computers, 18, 20, 22, are conventional
personal
13
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
computers that include a display application that provides a Graphical User
Interface
("GUI") to a local hard disk, the shared database 24, the data store server 26
and/or the
data store archive 30. The GUI display application is used to lead a scientist
or lab
technician through standard analyses, and supports custom and query viewing
capabilities. The display application GUI also supports data exported into
standard
desktop tools such as spreadsheets, graphics packages, and word processors.
The data client computers 18, 20, 22 connect to the store server 26 through an
Open Data Base Connectivity ("ODBC") connection over network 40. In one
embodiment of the present invention, computer network 40 is a 100 Mega-bit
~o ("Mbit") per second or faster Ethernet, Local Area Network ("LAN").
However, other
types of LANs could also be used (e.g., optical or coaxial cable networks). In
addition, the present invention is not limited to these specific components
and other
similar components may also be used.
As is known in the art, OBDC is an interface providing a common language
for applications to gain access to databases on a computer network. The store
server
26 controls the storage based routines plus an underlying Database Management
System ("DBMS")
The shared database 24 is a mufti-user, mufti-view relational database that
stores summary data from the one or more analysis instruments 12, 14, 16. The
2o shared database 24 uses standard relational database tools and structures.
The data
store archive 30 is a library of image and feature database files. The data
store
archive 30 uses Hierarchical Storage Management ("HSM") techniques to
automatically manage disk space of analysis instruments 12, 14, 16 and the
provide a
mufti-layer hierarchical storage management system. For more information on
data
2s storage system 10 and 10' see, co-pending application number 09/437,976,
entitled
14
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
"Methods and System for Efficient Collection and Storage of Experimental
Data,"
assigned to the same Assignee as the present invention, and incorporated
herein by
reference.
An operating environment for components of the data storage system 10 and
s 10' for preferred embodiments of the present invention include a processing
system
with one or more high-speed Central Processing Units) ("CPU") and a memory. In
accordance with the practices of persons skilled in the art of computer
programming,
the present invention is described below with reference to acts and symbolic
representations of operations or instructions that are perfornled by the
processing
~o system, unless indicated otherwise. Such acts and operations or
instructions are
referred to as being "computer-executed" or "CPU executed."
It will be appreciated that acts and symbolically represented operations or
instructions include the manipulation of electrical signals by the CPU. An
electrical
system represents data bits which cause a resulting transformation or
reduction of the
i5 electrical signals, and the maintenance of data bits at memory locations in
a memory
system to thereby reconfigure or otherwise alter the CPU's operation, as well
as other
processing of signals. The memory locations where data bits are maintained are
physical locations that have particular electrical, magnetic, optical, or
organic
properties corresponding to the data bits.
2o The data bits may also be maintained on a computer readable medium
including magnetic disks, optical disks, organic memory, and any other
volatile (e.g.,
Random Access Memory ("RAM")) or non-volatile (e.g., Read-Only Memory
("ROM")) mass storage system readable by the CPU. The computer readable
medium includes cooperating or interconnected computer readable medium, which
25 exist exclusively on the processing system or be distributed among multiple
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
interconnected processing systems that may be local or remote to the
processing
system.
Array scan module architecture
FIG. 2 is a block diagram illustrating an exemplary array scan module 42
architecture. The array scan module 42, such as one associated with analysis
instrument 12, 14, 16 (FIG. 1 B) includes software/hardware that is divided
into four
functional groups or modules. However, more of fewer functional modules can
also
be used and the present invention is not limited to four functional modules.
The
~o Acquisition Module 44 controls a robotic microscope and digital camera,
acquires
images and sends the images to the Assay Module 46. The Assay Module 46
"reads"
the images, creates graphic overlays, interprets the images collects feature
data and
returns the new images and feature data extracted from the images back to the
Acquisition Module 44. The Acquisition Module 44 passes the image and
interpreted
feature data to the Data Base Storage Module 48. The Data Base Storage Module
48
saves the image and feature information in a combination of image files and
relational
database records. The client computers 18, 20, 22 use the Data Base Storage
Module
48 to access feature data and images for presentation and data analysis by the
Presentation Module 50. The Presentation Module 50 includes a display
application
2o with a GUI as was discussed above.
Selecting features for images acquired from experimental data
FIG. 3 is a flow diagram illustrating a Method 52 for selecting assay features
for experimental data. In FIG. 3 at Step 54, multiple pre-determined assay
features
for analyzing images acquired from experimental data are presented. An assay
feature
includes one or more measurements for an object in an image acquired from the
experimental data. At Step 56, a set of desired assay features selected from
the
16
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
multiple presented assay features are received. At Step 58, one or more image
processing routines from a library of image processing routines are selected
for an
assay feature from the set of desired assay features. The one or more image
processing routines are used to accomplish the selected assay feature. At Step
60, the
one or more image processing routines are associated with the assay feature.
At Step
62, a loop is entered to repeat steps 58 and 60 for assay features in the set
of selected
assay features.
Method 52 is illustrated with one specific embodiment of the present
invention. However, the present invention is not limited to such an embodiment
and
~o other embodiments can also be used.
In such an embodiment, at Step 54 multiple pre-determined assay features for
analyzing digital photographic images (hereinafter "images") acquired from
experimental data for an assay are presented by analysis instruments 12, 14,
16 (FIG.
1 B) or by client computers 18, 20, 22 (FIG. 1 B). In one embodiment of the
present
~5 invention, the multiple pre-determined assay features include object
features (See,
e.g., FIG. 5). An "object" feature operates on an individual object (e.g., a
cell) or an
object component (e.g., cell membrane, cell nucleus, etc.) In another
embodiment of
the present invention, the multiple pre-determined assay features include
object
features and aggregate features. An "aggregate" feature includes assay
features that
20 operate on multiple objects (e.g., number of objects, average value of a
feature,
standard deviation value of a feature, etc.). In another embodiment of the
present
invention, the multiple pre-determined assay features include only aggregate
features.
In one specific embodiment of the present invention, the multiple pre-
determined assay features presented at Step 54 include general assay features
that can
25 be used by virtually any biologist, other scientist or analyst to analyze
measurements
17
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
from objects (e.g., cells) in images collected from experimental data. Such
general
assay features (e.g., length, width, height, etc.) are common image processing
features
that are useful for virtually any assay or protocol that may be developed to
obtain
measurements from experimental data. In such an embodiment, the general assay
features presented typically include only a few of the many possible image
processing
options that could be used to take measurements from a digital image.
For example, an assay feature for a simple measurement such as determining
an object's length, may include multiple different types of image processing
thresholds (e.g., a number of pixels, types of pixels, type of object
components
~o in/around a desired object, etc. to be included for the object to determine
its length).
In one embodiment of the present invention, two image processing thresholds
(e.g., a
minimum and a maximum) value may be presented to a user for determining an
object's length. Other image processing thresholds are handled internally
without
presenting such information to a user.
The general assay features and limited image processing options for the
general assay features presented allow a biologist, other scientist or analyst
without
much image processing experience to easily and quickly create assays and
protocols.
Since general assay features and limited image processing options are
presented,
instead of specific assay features with many different options, a user with
limited
2o image processing experience is less likely to get confused when he/she is
creating an
assay or protocol.
In one specific embodiment of the present invention, the general assay
features
associated with image processing options are presented in a specific ordering.
However, the present invention is not limited to such an embodiment with such
a
18
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
specific ordering. This specific ordering may also help a user with limited
knowledge
of image processing select the appropriate options for a desired assay or
protocol.
Typically an assay will include two or more channels. A "channel" is a
specific configuration of optical filters and channel specific parameters that
are used
to acquire an image. In a typical assay, different fluorescent dyes are used
to label
different cell structures. The fluorescent dyes emit light at different
wavelengths.
Channels are used to acquire photographic images for different dye emission
wavelengths.
Given a digitized image including one or more objects (e.g., cells), there are
~o typically two phases to analyzing an image and extracting feature data as
feature
measurements. The first phase is typically called "image segmentation" or
"object
isolation," in which a desired object is isolated from the rest of the image.
The second
phase is typically called "feature extraction," wherein measurements of the
objects are
calculated. A feature is typically a function of one or more measurements,
calculated
15 SO that it quantifies a significant characteristic of an object. Typical
object
measurements include size, shape, intensity, texture, location, and others.
For each measurement, several features are commonly used to reflect the
measurement. The "size" of an object can be represented by its area,
perimeter,
boundary definition, length, width, etc. The "shape" of an object can be
represented
2o by its rectangularity (e.g., length and width aspect ratio), circularity
(e.g., perimeter
squared divided by area, bounding box, etc.), moment of inertia, differential
chain
code, Fourier descriptors, etc. The "intensity" of an object can be
represented by a
summed average, maximum or minimum grey levels of pixels in an object, etc.
The
"texture" of an object quantifies a characteristic of grey-level variation
within an
25 object and can be represented by statistical features including standard
deviation,
19
CA 02359467 2001-07-18
WO 00/72258 PCT/I1S00/14246
variance, skewness, kurtosis and by spectral and structural features, etc. The
"location" of an object can be represented by an object's center of mass,
horizontal
and vertical extents, etc. with respect to a pre-determined grid system. For
more
information on digital image feature measurements, see: "Digital Image
Processing,"
by Kenneth R. Castleman, Prentice-Hall, 1996, ISBN-0132114674, "Digital Image
Processing: Principles and Applications," by G. A. Baxes, Wiley, 1994, ISBN-
0471009490 , "Digital Image Processing," by William K. Pratt, Wiley and Sons,
1991, ISBN-0471857661, or "The Image Processing Handbook - 2nd Edition," by
John C. Russ, CRC Press, 1991, ISBN-0849325161, the contents of all of which
are
~ o incorporated by reference.
In one exemplary preferred embodiment of the present invention, Method 52
is used to analyze cell image data and cell feature data from "wells" in a
"microplate."
In another preferred embodiment of the present invention, Method 52 is used to
analyze cell image and cell feature data from micro-gels in a bio-chip. As is
known in
the art, a "microplate" is a flat, shallow dish that stores multiple samples
for analysis
and typically includes 96 to 1536 individual wells. A "well" is a small area
in a
microplate used to contain an individual sample for analysis.
Each well may be divided into multiple fields. A "field" is a sub-region of a
well that represents a field of vision (i.e., a zoom level) for a photographic
2o microscope. Each well is typically divided into one to sixteen fields, or
more. Each
field typically will have between one and six photographic images taken of it,
each
using a different light filter to capture a different wavelength of light for
a different
fluorescence response for desired cell components. However, the present
invention is
not limited to such an embodiment, and other containers (e.g., varieties of
biological
chips, such as DNA chips, micro-arrays, and other containers with multiple sub-
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
containers), sub-containers can also be used to collect image data and feature
data
from other than cells.
In one embodiment of the present invention, Step 54 includes presenting a set
of static assay features in a uniform manner on a graphical user interface for
every
s user. In such an embodiment, the set of static assay features cannot be
modified by a
user. In another embodiment of the present invention, Step 54 is optionally
split into
two sub-steps. In a first sub-step, a user first selects a desired set of
assay feature
names from a list of assay features. In a second sub-step the desired set of
assay
feature names is dynamically presented on graphical user interface
specifically for the
~o user. In such an embodiment, a user can dynamically modify the set of assay
features
that will actually be presented and used instead of receiving a set of static
assay
features that cannot be modified by a user. Any assay features selected by a
user from
a list of assay features are also associated with one or more image processing
routines
as is described for Step 58 below.
15 As was described above, an assay feature includes one or more measurements
for an object in an image acquired from experimental data. In one exemplary
embodiment of the present invention, objects in the images acquired from
experimental data include, but are not limited to, cells. Exemplary object
features for
cells are illustrated in Table 1. However, other object features and can also
be used
2o and the present invention is not limited to the cell features illustrated
in Table 1.
Virtually any object feature can be presented at Step 54.
Copyright ~ 1999, by Cellomics, Inc. All rights reserved.
CELL SIZE
CELL SHAPE
CELL INTENSITY
CELL TEXTURE
CELL LOCATION
CELL AREA
21
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
CELL PERIMETER
CELL SHAPE FACTOR
CELL EQUIVALENT DIAMETER
CELL LENGTH
CELL WIDTH
CELL INTEGRATED FLUORESCENCE INTENSITY
CELL MEAN FLUORESCENCE INTENSITY
CELL VARIANCE
CELL SKEWNESS
CELL KURTOSIS
CELL MINIMUM FLUORESCENCE INTENSITY
CELL MAXIMUM FLUORESCENCE INTENSITY
CELL GEOMETRIC CENTER
CELL X-COORDINATE OF A GEOMETRIC CENTER
CELL Y-COORDINATE OF A GEOMETRIC CENTER
T
Step 54 also includes presenting aggregate features. Aggregate features are
features associated with a collection of objects such as a population of
cells. In one
exemplary embodiment of the present invention, the aggregate features include,
but
are not limited to, any of the well summary data for a microplate including
cells
illustrated in Table 2. However, the present invention is not limited to
presenting
aggregate features for the well summary data illustrated in Table 2. Virtually
any
summary data for aggregate features can be presented. In Table 2, a "SPOT"
indicates a small region of fluorescent response intensity as a measure of
biological
~ o activity.
ht ~ 1999, by Cellomics, Inc. All rights reserved.
WELL CELL SIZES
WELL CELL SHAPES
WELL CELL INTENSITIES
WELL CELL TEXTURES
WELL CELL LOCATIONS
WELL NUCLEUS AREA
WELL SPOT COUNT
WELL AGGREGATE SPOT AREA
WELL AVERAGE SPOT AREA
WELL MINIMUM SPOT AREA
WELL MAXIMUM SPOT AREA
WELL AGGREGATE SPOT INTENSITY
WELL AVERAGE SPOT INTENSITY
WELL MINIMUM SPOT INTENSITY
WELL MAXIMUM SPOT INTENSITY
WELL NORMALIZED AVERAGE SPOT INTENSITY
WELL NORMALIZED SPOT COUNT
WELL NUMBER OF NUCLEI
22
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
WELL NUCLEUS AGGREGATE INTENSITY
WELL DYE AREA
WELL DYE AGGREGATE INTENSITY
WELL NUCLEUS INTENSITY
WELL CYTOPLASM INTENSITY
WELL DIFFERENCE BETWEEN NUCLEUS AND CYTOPLASM INTENSITY
WELL NUCLEUS BOX-FILL RATIO
WELL NUCLEUS PERIMETER SQUARED AREA
WELL NUCLEUS HEIGHT/WIDTH RATIO
WELL CELL COUNT
Table 2.
The aggregate features can also include, but are not limited to, microplate
summary data for cells illustrated in Table 3. In Table 3, "MEAN" indicates a
statistical mean and "STDEV" indicates a statistical standard deviation, known
in the
art, and a "SPOT" indicates a small region of fluorescent response intensity
as a
measure of biological activity.
MEAN SIZE OF CELLS
MEAN SHAPES OF CELLS
MEAN INTENSITY OF CELLS
MEAN TEXTURE OF CELLS
LOCATION OF CELLS
NUMBER OF CELLS
NUMBER OF VALID FIELDS
STDEV NUCLEUS AREA
MEAN SPOT COUNT
STDEV SPOT COUNT
MEAN AGGREGATE SPOT AREA
STDEV AGGREGATE SPOT AREA
MEAN AVERAGE SPOT AREA
STDEV AVERAGE SPOT AREA
MEAN NUCLEUS AREA
MEAN NUCLEUS AGGREGATE INTENSITY
STDEV AGGREGATE NUCLEUS INTENSITY
MEAN DYE AREA
STDEV DYE AREA
MEAN DYE AGGREGATE INTENSITY
STDEV AGGREGATE DYE INTENSITY
MEAN MINIMUMSPOT AREA
STDEV MINIMUM SPOT AREA
MEAN MAXIMUM SPOT AREA
STDEV MAXIMUM SPOT AREA
MEAN AGGREGATE SPOT INTENSITY
STDEV AGGREGATE SPOT INTENSITY
MEAN AVERAGE SPOT INTENSITY
STDEV AVERAGE SPOT INTENSITY
MEAN MINIMUM SPOT INTENSITY
STDEV MINIMUM SPOT INTENSITY
MEAN MAXIMUM SPOT INTENSITY
STDEV MAXIMUM SPOT INTENSITY
MEAN NORMALIZED AVERAGE SPOT INTENSITY
23
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
STDEV NORMALIZED AVERAGE SPOT INTENSITY
MEAN NORMALIZED SPOT COUNT
STDEV NORMALIZED SPOT COUNT
MEAN NUMBER OF NUCLEI
STDEV NUMBER OF NUCLEI
NUCLEI INTENSITIES
CYTOPLASM INTENSITIES
DIFFERENCE BETWEEN NUCLEI AND CYTOPLASM INTENSITIES
NUCLEI BOX-FILL RATIOS
NUCLEI PERIMETER SQUARED AREAS
NUCLEI HEIGHT/WIDTH RATIOS
WELL CELL COUNTS
At Step 56, a set of assay features selected from the presented assay features
are received on the analysis instruments 12, 14, 16 or client computers 18,
20, 22. For
example, set of assay features selected from the multiple presented assay
features may
include object features for "cell perimeter," "cell width" and "cell length."
(e.g., from
Table 1 ).
At Step 58, one or more image processing routines from a library of image
processing routines are selected for an assay feature from the set of selected
assay
features. The one or more image processing routines are used to accomplish the
io selected assay feature. To accomplish the "cell length" feature, one or
more image
processing routines are called from a library of image processing routines to
accomplish the "cell length" feature. For example, image processing routines
including "select object( )," "object boundingbox ( )," "object rotate180 (
)," and
"object longest side ( )" (e.g., see length feature in Table 6) may be
selected from a
15 library of image processing.
As is known in the art, there are many libraries of image processing routines.
See for example, AnVisilog (Image Processing/Analysis Library), by
NoesisVision,
Inc. at the Universal Resource Locator ("URL") "www.noesisvision.com," MIL
(Matrox Imaging Library) by Matrox Electronic Systems Ltd. At the URL
20 "www.matrox.com," ImagePro ( Image Processing/Analysis Library) and Optimas
24
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
Image Processing/Analysis Library) by, MediaCybernetics, at the URL
"www.mediacy.com," and others. Any of these image processing libraries or
others
known in the art can be used with the present invention.
At Step 60, the one or more image processing routines are associated with the
selected feature. For example the "cell length" feature is associated with the
image
processing routines "select object( )," "object boundingbox ( )," "object
rotate180
)," and "object longest side ( )" (e.g., see length feature in Table 6).
At Step 62, a loop is entered to repeat steps 58 and 60 for assay features in
the
selected set of assay features. For example, after the cell length feature is
associated
~o with the image processing routines, the cell width and cell perimeter
features are also
associated with image processing routines by repeating steps 58 and 60.
Method 52 allows a biologist, other scientist or analyst not trained in image
processing to create assays and protocols to analyze experimental data. Method
52
can be used to analyze images collected from feature-rich cell experimental
data
~ 5 generated by HTS systems.
Processing selected assay features for images acquired from experimental data
FIG. 4 is a flow diagram illustrating a Method 64 for selecting assay features
for images acquired from experimental data. At Step 66, a set of images is
acquired
from experimental data on an analysis device. At Step 68, a set of assay
features is
2o selected from a set of multiple presented assay features to analyze the set
of images.
An assay feature includes one or more measurements for an object in an image
acquired from the experimental data. A presented assay feature is associated
with one
or more image processing routines from a library of image processing routines
to
accomplish the assay feature. At Step 70, processing of the set of images
using the
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
selected set of assay features is requested. At Step 72, results are received
from the
processing of the set of images using the selected set of assay features.
Method 64 is illustrated with one specific embodiment of the present
invention. However, the present invention is not limited to such an embodiment
and
other embodiments can also be used.
In such an embodiment, at Step 66, a set of images (e.g., for cells or
components of cells acquired from cell experimental data) is acquired on
analysis
instruments 12, 14, 16 or client computers 18, 20, 22 (e.g., FIGS. 1 A and 1
B). In one
embodiment of the present invention, there are two ways to acquire images: ( 1
) from
~o prepared samples; or (2) from stored image sets.
Images are acquired automatically from a feature rich array scanning system
(e.g., using array scan module 42 of FIG. 2) as an experiment is being
conducted.
Images are acquired from stored images sets after a desired experiment has
been run
by a feature rich array scanning system and the results have been saved in a
shared
i5 database 24 or a store archive 30, or local hard drive.
FIG. 5 is a block diagram illustrating an exemplary graphical user interface
74
presented on the analysis instruments 12, 14, 16 or client computers 18, 20,
22 for
selecting object features at Step 68. The graphical user interface 74 includes
graphical entities such as graphical check boxes or graphical buttons to
select object
2o features.
FIG. 5 illustrates, for example, graphical check boxes to select object
features
including size, shape, intensity, texture, location, area, perimeter, shape
factor,
equivalent diameter, length, width, integrated fluorescence intensity, mean
fluorescence intensity, variance, skewness, kurtosis, minimum fluorescence
intensity,
25 maximum fluorescence intensity, geometric center, x-coordinate of a
geometric center
26
CA 02359467 2001-07-18
WO 00/72258 PCT/LJS00/14246
or y-coordinate of a geometric center. FIG. 5 illustrates a set including some
of the
most commonly used object features used to measure objects in an image.
However,
the present invention is not limited to the object features listed in FIG. 5
and more,
fewer or equivalent object features can also be used. FIG. 5 also illustrates
graphical
radio buttons for selecting fluorescence channels for desired dyes. Aggregate
features
are selected with a similar graphical user interface.
Returning to FIG. 4, at Step 68, a set of assay features is selected from
multiple pre-determined assay features to analyze the set of images. In one
embodiment of the present invention, Step 68 includes creating a protocol for
an assay
~o by selecting multiple pre-determined assay features (e.g., selecting
multiple graphical
buttons from FIG. 5). A "protocol" specifies a series of system settings
including a
type of analysis instrument, an assay, dyes used to measure biological
markers, cell
identification parameters and other general image processing parameters used
to
collect data. An "assay" is a specific selection of image processing methods
used to
~5 analyze images and return results related to biological processes being
examined. For
more information on the image processing methods used in cell assays targeted
to
specific biological processes, see co-pending applications 09/031,217 and
09/352,171,
assigned to the same Assignee as the present application, and incorporated
herein by
reference.
2o For example, for an exemplary assay-X, FIG. 5 illustrates selection of
graphical check boxes for a perimeter 76, length 80 and width 82 object
features for
fluorescence channel zero, Dye-0 84. Radio button for DYE-0 84 is illustrated
as
selected in FIG. 5. Thus, assay-X would include obtaining object measurements
for
perimeters, lengths and widths of objects in images from fluorescence channel
zero
25 for a desired dye.
27
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
The assay features presented at Step 68 are associated with one or more image
processing routines from a library of image processing routines to accomplish
the
assay feature measurement (e.g., at Step 60 of Method 52, FIG. 3). Thus, a
user
selecting the assay features presented at Step 68 does not have to understand
how the
assay feature is accomplished, but only how to choose desired assay features
of
interest to accomplish his/her own desired analysis (e.g., for a desired
assay). If a
new library of image processing routines was used, the assay features
presented at
Step 68 typically would not change, even though a whole new set of image
processing
routines might be used to accomplish an assay feature measurement.
~o Returning to FIG. 4, at Step 70 processing of the set of images using the
selected set of assay features is requested. In one embodiment of the present
invention, Step 70 includes selecting a series of general image processing
operations
in addition to selecting object and/or aggregate features. The image
processing
operations are applied before receiving the results at Step 78. The image
processing
is operations may include filtering, object segmentation or mask modification
(See, FIG.
6).
In one embodiment of the present invention, processing of the set of images at
Step 70 includes applying general image processing routines to an image
acquired
from experimental data in a pre-determined order using a set of desired assay
features
2o selected from a graphical user interface (e.g., FIG. 6). However, the
present invention
is not limited to such an embodiment. In such an embodiment, pre-determining
the
order of applying the general image processing routines relieves a user of
another
image processing detail when he/she is creating an assay or protocol. Assay
features
are presented on a graphical user interface (e.g., FIG. 6) in the order that
they are
25 processed. For example, before segmenting an image, it is usually important
to filter
28
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
the image to improve the efficiency of the segmentation. The filters may
smooth and
sharpen an image. Providing a pre-determined order helps make the creation of
an
assay or protocol simpler than if a user had to also determine a processing
order
himself/herself. The pre-determined processing order may also help a user more
easily compare his/her results between or among several different experiments.
In one embodiment of the present invention, processing the set of images at
Step 70 with selected object and aggregate features may include both
independent and
dependent processing of fluorescence channels. "Independent processing" refers
to
the creation of "independent masks" for each of the fluorescence channels. As
is
~o known in the art, a "mask" is one or more binary values used to selectively
screen out
or let through certain bits in a data value. Masking is typically performed by
using a
logical operator (AND, OR, XOR, NOT) to combine the mask and the data value.
"Dependent processing" refers to the use of a mask from one channel to derive
a mask for analysis in another channel. This "derived mask" may be a simple
copy of
15 the parent mask or further processing may be applied to the parent mask.
Feature
extraction in the second channel occurs based on the derived mask.
For example, an approach to analyzing the cytoplasm-to-nucleus translocation
of a transcription factor in a cell can be performed using derived masks.
First, labeled
nuclei are used to establish a mask. Second, a Transcription Factor ("TF")
channel is
2o setup to use a derived mask. The TF channel is defined as dependent on the
nucleus
channel. This copies the nuclei mask to the TF channel. The mask can be
applied
directly to measure a mean nuclear intensity of the TF, which is proportional
to the
amount of TF in the nucleus. Next, the mask is dilated a number of times and
the
binary exclusive OR/XOR function applied to the pair of masks. This leads to a
ring
25 shaped derived mask positioned over the peri-nuclear cytoplasm. Analysis
within this
29
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
mask provides an estimate of the amount of TF in the cytoplasm. By calculating
the
ratio of the mean intensity within the nuclear mask in the TF channel and the
mean
intensity within the cytoplasmic ring mask in the TF channel, a measure of the
cytoplasm-to-nucleus translocation can be established.
s In one embodiment of the present invention, at Step 70, images from selected
fluorescent channels are typically processed through a series of general image
processing operations before analysis. Such general image processing steps are
used
to remove noise and help improve feature interpretation. The general image
processing steps may include filtering, segmentation, etc. as is discussed
below.
~o Table 4 illustrates independent general image processing operations.
However, other independent image processing operations can be used and the
present
invention is not limited to the independent image processing operations
illustrated in
Table 4.
- The ability to perform smoothing, noise reduction, or local contrast
adjustment
such as edge enhancement processing on the images as a preliminary step to
segmentation,
depending on the image quality and the task.
~ Smoothing - The smoothing method is based on a uniform, low pass 3 X 3
kernel.
~ Sharpening - The sharpening method is based on a common, high pass 3 X 3
kernel.
Segmentation - Segmentation allows separation of an image into separate
objects.
~ Separate Grey - This method can be applied to segment a grayscale image into
objects.
There is one input parameter for the method, which relates to the contrast of
the input image.
The output of this method is a binary image that is overlayed on a grey scale
image to show
the object division.
~ Threshold (Fixed) - A single user specified threshold can be used for images
with very stable
backgrounds and relatively good SNR. This is an alternative to the Separate
Grey operation.
The output of this method is a binary mask.
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
~ Threshold (Auto) - A histogram-based method where the minimum intensity
between two
peaks can be determined automatically and then optionally corrected before
applying. The
output of this method is a binary image.
~ Threshold - Threshold is setup interactively via a slider or by typing in a
threshold value.
When using a fixed threshold, the threshold value will be applied throughout
the scan. When
using an auto threshold, the auto threshold is computed for the current image
and the
correction coefficient is determined to make it match the one set manually.
This coefficient
will be applied to every threshold value determined during the scan.
~ Fill Holes - This method provides a means of filling holes in binary masks
that may occur
during segmentation.
~ Remove Border Objects- This method removes objects that touch the border of
the image.
Masks that touch the border often represent objects that are only partly
within the image. The
features extracted from such objects may not be non-representative of a
complete object.
Mask Modification - Masks from the segmentation process may be modified by
multiple
cycles of erosion and dilation. This is useful for smoothing the outlines of
the masks as well as
creating masks that may be impractical from just the segmentation methods. The
sequence of
erode and dilate, or dilate and erode, helps to remove noise from a mask
outline.
~ Erode - Masks may be reduced in size by binary erosion for any number of
cycles. Each
erosion is a reduction in the size of the mask by removing perimeter pixels.
~ Dilate -Masks may be expanded in size by binary dilation for any number of
cycles. Each
dilation ads an additional outline of 1 pixel in width.
~ Remove Small- Small objects can be pieces of debris or they may form due to
the
segmentation operations. These objects may be removed. The size value is
related to half of
the width. It is the number of erosions needed to erase the object.
Separate Binary- Provides a means of separating binary object masks.
Table 4
Table 5 illustrates general image processing operations that are useful to
apply
to a dependent mask. However, other image processing operations can be used
and
31
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
the present invention is not limited to the image processing operations
illustrated in
Table 5.
ent Masks
~ Erode - Masks may be reduced in size by binary erosion for any number of
cycles.
~ Dilate- Masks may be expanded in size by binary dilation for any number of
cycles.
~ XOR- Masks can be combined by application of the exclusive OR binary
operation. Thus
creating a ring around an original nuclear mask. The ring can be expanded or
contracted
relative to the original nuclear mask while the width of the ring stays
unchanged.
Table 5.
FIG. 6 is a block diagram illustrating an exemplary graphical user interface
86
s for selecting general image processing operations. These operations,
illustrated in
Tables 4 and 5, are selected by inputting a number in the graphical box
displayed, or
by checking a graphical check box. If a graphical box has a value of zero, or
a
graphical check box is not checked, the general image processing operation is
not
executed. For example, as is illustrated in FIG. 6, no filtering is requested.
However,
~o grey scale segmentation 88 is selected, a value of 50 is used for the grey
scale
threshold 90. In addition, an independent mask is selected for dilating the
mask for 2
cycles 92, and the XOR operation 94 is selected for a dependent mask.
In one embodiment of the present invention, processing at Step 70 includes
obtaining measurements for selected object and aggregate features. Table 6
illustrates
one possible implementation of the object features from Table 1 using the
independent masks from operations in Table 4. However, the present invention
is not
limited to this implementation and other implementations can also be used.
Object Feature Description
(Independent Mask)
Area Number of pixels inside an object
(mask).
Perimeter Number of pixels in an outline.
32
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
Object Feature Description
(Independent Mask)
Equivalent DiameterDiameter of the circle with circle
area = Area.
Length, Width Longest and shortest sides of a bounding
box that fits an
object the best (after rotating it
180 degrees).
Area Length * Width
Shape Perimeterz / 4~ *Area (this feature
is not simply a
combination of Area and Perimeter).
Integrated IntensitySum of intensities within an object
(mask).
Mean Intensity Integrated Intensity / Area.
Variance Variance of intensities within an
object (mask).
Skewness Third statistical moment for intensities
within an object
(mask).
Kurtosis Fourth statistical moment for intensities
within an object
(mask).
Min Intensity Minimum intensity within an object
(mask).
Max Intensity Maximum intensity within an object
(mask).
Geometric Center X coordinate of a geometric center
X of an object (mask)
within a field (image).
Geometric Center Y coordinate of a geometric center
Y of an object (mask)
within a field (image).
Table 6.
The feature set for dependent or derived masks is more limiting than the set
for independent masks. One reason for this is that dependent masks are not
necessarily related to a form of a signal in a dependent channel. Thus, for
example, a
perimeter or shape of a derived mask is typically more related to a primary
channel
rather than the dependent channel.
Table 7 illustrates one implementation of object features for dependent masks
created using the aggregate operations from Table 5.
Object Feature Description
(Dependent mask)
IntegrlntlndMask Integrated intensity under independent
mask applied
to current channel.
AveIntIndMask Average intensity under independent
mask applied to
current channel.
IntegrlntRingMask Calculated only if XOR is selected:
Integrated
intensity under ring mask applied
to current channel
AveIntRingMask Calculated only if XOR is selected:
Average intensity
under ring mask applied to current
channel
IndMask2RingRatio Calculated only if XOR is selected:
Ratio of average
intensity under independent mask
applied to current
33
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
Object Feature Description
(Dependent mask)
channel to average intensity under
ring mask applied
to current channel
Table /.
In one embodiment of the present invention for processing at Step 70, a
primary mask is applied and desired object features are extracted, a derived
mask is
s applied and aggregate features are extracted. In one embodiment of the
present
invention, object features represent cell data and aggregate features
represent the well-
level or microplate level data for a population of cells in a well. However
the present
invention is not limited to such an embodiment and aggregate features for
other types
experimental data can also be used.
~o In one embodiment of the present invention, object and aggregate features
are
calculated and constrained by settings of aggregate "feature gates." "Feature
gates"
are provided to define sub-set of an object population that will contribute to
an object
or aggregate feature set. The feature gates include selection of a range
including a
lower and upper limit on the range. For example a feature gate for the object
feature
~5 area may be set with a lower limit of zero and an upper limit of 2000.
Thus, only
objects (e.g., cells) that have an area between zero and 2000 pixels will be
included.
Returning to FIG. 4 At Step 72, results are received from the processing of
the
set of images using the selected set of assay features. In one embodiment of
the
present invention, the results are written to a local database associated with
the
2o analysis instruments 12, 14, 16 or client computers 18, 20, 22. In another
embodiment of the present invention, the results may also be propagated to the
shared
database 24 and/or the store archive 30.
In one embodiment of the present invention, results may be displayed using
one of three display options illustrated in Table 8. However, the present
invention is
34
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
not limited to three display options and more or fewer display options can
also be
used.
Display Option Description
Every ProcessingImages will be redisplayed after
Step every step in the
processing sequence for each
channel.
Final Labeled Images for labeled independent
Field channels will be
Mask displayed after all processing
steps.
Masked Field Gray scale images will be displayed
Image without a
background.
fable S.
In one very specific embodiment of the present invention, Method 64 can be
used in an automatic manner. In such an embodiment, a protocol is created to
automatically accomplish the steps of Method 64 and store results in a
database for
later analysis. Such a very specific embodiment may used in conjunction with a
HTS
system. When a desired experiment is completed, a protocol may be
automatically
~o initiated and used to automatically accomplish the steps of Method 64.
FIG. 7 is a block diagram illustrating an exemplary screen display 96 for
graphically displaying information acquired from images processed using a
desired
set of assay features. However, the present invention is not limited to this
screen
display and other screen displays, and more or less information can also be
displayed,
~5 and the information can be displayed in different formats.
The screen display 96 includes a portion of an image of interest 98 for an
object (i.e., a cell) acquired from an image 100 including multiple objects
(i.e., a
population of cells). The screen display 96 includes object feature data 102
measured
from the image of interest 98, and aggregate data 104 and 106 measured from
image
20 100 and nine other images (not displayed). The object feature data 102 and
the
aggregate data 104 and 106 displayed includes object and aggregate features
selected
at Step 68 of Method 64 (FIG. 4).
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
The image of interest 98 includes a magnified image of an individual cell
identified by 98' in the image 100 including multiple objects. Screen display
96
illustrates exemplary assay feature data only for well A-3 illustrated by the
blacked
well 108 in the graphical illustration of a microplate 110 including 1536
wells.
These methods and system described herein may allow experimental data from
high-throughput data collection/analysis systems including images to be
analyzed.
The methods and system can be used for, but is not limited to analyzing cell
image
data and cell feature data collected from microplates including multiple wells
or bio-
chips including multiple micro-gels in which an experimental compound has been
io applied to a population of cells. If bio-chips are used, any references to
microplates
herein, can be replaced with bio-chips, and references to wells in a
microplate can be
replaced with micro-gels on a bio-chip and used with the methods and system
described.
The methods and system help provide a general purpose assay development
15 tool. The methods and system allow a biologist, other scientist, or lab
technician not
trained in image processing techniques to quickly and easily design protocols
and
assays to analyze images acquired from experimental data (e.g., cells). The
methods
and system may improve the identification, selection, validation and screening
of new
drug compounds that have been applied to populations of cells. The methods and
2o system may also be used to provide new bioinformatic techniques to
manipulate
experimental data including multiple digital photographic images.
It should be understood that the programs, processes, methods and systems
described herein are not related or limited to any particular type of computer
or
network system (hardware or software), unless indicated otherwise. Various
types of
36
CA 02359467 2001-07-18
WO 00/72258 PCT/US00/14246
general purpose or specialized computer systems may be used with or perform
operations in accordance with the teachings described herein.
In view of the wide variety of embodiments to which the principles of the
present invention can be applied, it should be understood that the illustrated
embodiments are exemplary only, and should not be taken as limiting the scope
of the
present invention.
For example, the steps of the flow diagrams may be taken in sequences other
than those described, and more or fewer elements may be used in the block
diagrams.
While various elements of the preferred embodiments have been described as
being
~ o implemented in software, in other embodiments in hardware or firmware
implementations may alternatively be used, and vice-versa.
The claims should not be read as limited to the described order or elements
unless stated to that effect. Therefore, all embodiments that come within the
scope
and spirit of the following claims and equivalents thereto are claimed as the
invention.
37
