Note: Descriptions are shown in the official language in which they were submitted.
3~
Technical Field
This invention relates to optical displays and i.mage enhancement.
Background Art
Drama~i.c improvements have ~een made in recent years in the field of
image enhancement. Most of these improvements have been made in the aerospace
industry where, for instance, an image improved picture is made fron, a satellite
picture where the image improved picture is vastly clearer than the original
satellite picture to the extent that it seems to contain information absent from
the original. These image enhancement techniques are performed as electronic
manipulations of signals which represent the picture rather than by manipulation
of real optical scenes themselves.
In accordance with this invention image enhancement is performed with
all of the versatility of the known electronic digital image enhancement tech-
niques, 6ut employing manipulation of the real optical scenes so that character-
istics of the original scene participate throughout the image enhancement process
and in the final composite optical display.
While some devices in the past, such as document veri-fication devices~
have employed techniques where an image is projected onto a mask or pattern,
like a negative, which is a parameter of some predetermined valid document, none
of those devices~ like this invention, projected together an image and a param-
eter of itself. Thus, these prior devices have used projection to compare a
real optical scene to a standard, but they have not produced the unique results
which are produced with this invention hy the interaction o~ a real optical
scene and its parametric image.
Dlsclosure of Invention
Thi.s invention relates to optical systems and more particularly to
arrangen~entg Oe optical systetlls including feedback circuits in which a parameter
3~
of a real scene is projected onto the live scene or an image of ~he live scene.
In its si.mplest sense the invention can be viewed as taking a picture
of a real scene and projecting the plcture back on the scene itself. The result
is generally emphasis of all information content of the scene increasing contrasts
and color intensities.
In more complex forms of the invention, the picture of the real scene
is su~jected to change to produce a picture which is a parameter of the real
scene and the parametric image is projected back on the original scene or on the
image. The simple form of the invention is really just a limiting case of these
complex forms ~here the parametric relationship be~ween the original scene and
parametric image i5 a one to one relationship.
A variety of changes may be employed in forming the parametric image
from the original scene and these different parametric relationships produce
many useful results in the composite optical display when the parametric image
is recombined with the original scene or with an image of the original. For
instance, the parametric image may be a negative or time delayed or subject to
spatial modulation or color modulation.
W~ere the parametric image is a negative of the original scene, the
combined optical display will have reduced contrast from the real scene with the
contrast reduction extending all of the way to a null picture ~uniform gray)
depending upon the relative intensities of the illumination in the original
scene and the projection of the parametric image and the absorption and reflectiv-
ity of materials where recombination takes place. Many situations exist where
the composite display of this type with reduced contrast produces useful results.
~or instance, in many industrlal inspection situations where microcircuitry and
the like are inspected, these composite displays can increase the visibility of
dull features which are normally obscured because of their proximity to bright
133~
featurcs, and features which differentially absorb light of a particular wave-
length can be distinguished from each othe-r.
The use of a negative parametric image is particularly advantageous
when combined with time delay. Thus, where the parametric image is a negati-ve
image of a real sccne projected back Oll the real scene after a time delay, the
composite display is a reduced contrast, more or less null image, in which any
moving article is emphasized. In the ultimate case of a pure null image only
moving components are visible and all stationary components in the real scene
are invisible. Component displays of this type are useful in watc'ning an article
undergo a processing change during manufacture.
Many desirable results can be produced in the composite display where
the parametric image is produced with spatial modulation such as defocusing.
Thus, a de~ocused parametric image combines with the real subject to obscure
generalities and emphasize major transitions in the original subject. This can
viewad generally as tending to convert a half-tone picture into an outline or
line drawing picture. This technique is important where it is desirable to pro-
duce a composite display which emphasizes the outlines of objects. Consider, for
instance, the problems in microscopy where different shaped cells are to be ex-
amined. A defocused image is projected onto the cell sample in a microscopic
field of view producing emphasis of the edges of the cells in the focal plane of
the microscope. Note that t~e parametric defocused image might also be a partial
negative image to increase illumination in the peripheral areas of v-iew where
normal illumination of the microscope field is weak.
S~me of the most powerful benefits of the invention are obtained when
projection of the parametric image and the real scene together is performed con-
tinuously with real images. Thus, some of the techniques mentioned above may be
used on a finite basis where recombination of the parametric image and real
3~
scene ends there. This is the case where the parametric image is a vir~ual
image or where the combined optical display ls not further photographed, para-
metrically modified and recombined. In other situations unique advantages are
obtained where the parametric image is a real image and the composite display
is a real combination which is further and continuously photographed, parametri-
cally modified and recombined. This technique can be performed most conveniently
with optical equipment such as television cameras, digital processors ~nd tele-
vision projectors which can parametrically modify and recombine real images con-
tinuously in real time.
The powerful results of these continuous systems may be appreciated
from these examples. Where a semen sample is examined under a microscope by a
television camera, the output of which is converted into a negative image time
delayed slightly and reprojected by a television projector onto the sample, a
technician examining the sample to perform a count of live spermatozoa will see
only the live spermatozoa with dead spermatozoa and stationary debris washed out
of the image.
Similarly~ a negative time delayed image of a security area combined
with the original image and displayed on a monitor will instantly call a guard's
attention to intrusion into khe security area.
2U Where the real optical scene is a transparent tube or passageway with
a clear fluid containing particulata matter flowing through it~ ~he real scene
combined with a negative time delayed image will show a picture of the particles
which were in motion. Note that the fluid medium was in mo~.ion too, but it does
not appear in the composite image because the fluid is uniform so that different
parts of it appear to be the same optically. Careful adjustment of the time de-
lay in the parametric image can produce very useful results. For instance,
slower moving particles near the edges of the s~ream may be weaker optical com-
~L~8~3~
ponents in the combined di3play while faster moving particles in the center of
the stream will be brighter components Such a composite display carri.es i.m-
portant f:Low information, and if a constant time delay is used corresponding to
overage ~low rate, the composite display gives a live picture of flow for uses
like wind tunnels.
In the example given above for combining a defocused parametric image
of cells with the original, the addition of continuous sweeping of the focal
plane of the microscope an.d continuous parametric modification and reprojection
can permit the technician to see the cell surface in topological projection.
Continuous parame~ric modification of and recombination with the com-
posite display has particular advantages and produces surprising results where
the real optical scene contains components with non-uniform reflectivi~y at
different colors, ~nd these results may be enhanced by detection of the composite
display and reprojection of the parametric image through differen~ color filters.
A variety of additional applications of th.is method will be apparent
considering the variety of modifications which can be employed singly and in
combination in preparing the parametric image. These changes include changes
in color, time, size, intensity and all of the other manipulations which have
developed in the field of image enhancement. Additionally, the method of this
2n invention can be practiced with a variety of forms of apparatus.
The preferred apparatus of the invention for practice of the method
of the invention uses the television camera, processor and projector mentioned
above, and there are a number of ways that these can be arranged for particular
desired results.
Bri_f Desc_iption of Drawings
In the accompanying drawings:
Figure 1 is a schematic view of one form of the a.pparatus of thi.s in-
3~
ventlon which ls a~apted ~or direct microscopic viewing o~` an optical display
prod~ced in accordance with the method and apparatus o this invention.
Pigure 2 ls a schematic view of an alternative form of the apparatus
of the invention in which the parametric image is recombined with the real op-
tical scene at the carnera stage of the feedback circuit instead of on the real
subject itself so that the real subject can still be viewed unaltered.
~ igure 3 is a schematic view of another alternative form of the ap-
paratus of the lnventi.on ln which the parametrlc image is recombined wlth an
image of the original scene so that the original scene remains unaltered and the
parametric modlfication is finite instead of progressive. Viewed from the eye
of the observer he sees a virtual image of the parametric image superimposed on
an image of the real scene.
Best_Mode For Carrying Out The Invention
Referring now in detail to the drawings and particularly to Figure 1,
the system illustrated therein consists of an optical block containing a main
b,~re 10 with branch bores 12 and 14. The main bore supports a magnifying lens
16 which is adapted to be focused on a specîmen 18 supported at a stage 20. As
lndicated above the stage 20 may be a flat support zone or in the alternative
may bo a conduit through whlch the specimen 18 moves.
2Q Posltioned at the opposite end of the bore 10 from the magnifying lens
16 is a televIsion camera which is preferably a charge coupled device, for in-
stance~ RC~ CCD camera TC ll60. Mounted in the main bore 10 opposite the branch
bores 12 and 14 are beam splitters 24 and 26, respectlvely, which may be conven-
tional products such as Natlonal Photocolor Corporation Beam Splitting Pellicle
standard dlcroic coatlng ~ST-SQ-DC). The beam splitter 24 permits direct view-
ing of the specimen 18 through eye piece 28, and the beam splitter 26 pe~mits
transmlssion o t'he image of specimen 18 to CCD camera 22 while at the same time
13~
reflecting a parametric image ~rom a projection cathode-ray tube 30 through
projection lens 32 back onto the specimen 18.
A television monitor 34 is connected to the camera 22 to perrnit rnoni-
toring of the camera output, and a digital processor 36, such as an Intel ~ulti-
bus 8080 is connected between the CCD camera 22 and projecting CRT 30 for mani-
pulating the camera output to produce at the CRT a parametric image of the speci-
men 18.
This apparatus thus provides a closed circuit optical system for pro-
duction of optical displays in accordance with the method described above. The
closed circuit system includes the support stage 20 where the specimen is viewed,
~he recording stage at camera 22 where an optical representation of the specimen
is converted into an electronic signal output, the parametric processing sta~e at
digital processor 36 where the electronic output is converted into a parameter
of the camera output, the projection stage 30 where the processor output is pro-
jected as an optical parametric image of the original specimen 18 back onto the
specimen 18 where the combined optical display of the original specimen with its
superi.mposed parametric image is continuously viewed by the camera 22, further
parametrically modi~ied by the processor 36 and reprojected by the CRT 30.
As indicated above, the parametric processing stage consists of a
2~ suita~,le digital processor which may be programmed in conventional manner to
first convert the output o CCD cam0ra 22 into a digital representation of the
camera picture with an analog to digital converter and then perform an~ of the
p~rametric manipulations described above on the digital picture and finally re-
convert the digital picture to an analog signal for the projection CRT 30.
In place of the digital processor 36 a variety of simp'le circuits may
be employed. For i.nstance, the outpu~ of camera 22 may be passed through simple
circuits which invert the intensity component of the picture signal and pass
f~
that signal through a delay line to provide the negative delayed parametric
image describcd above. While the simpler circuits may be employed, ~he digital
processor is preferred for its fle~ibility in adapting the system to a variet~
of the parametric images described above.
The apparatus of Figure 2 is similar to the apparatus of Figure 1
except that the beam splitter 26 of Figure l is replaced by the beam splitter 38
of Figure 2. This change changes the closed loop path for reprojection of the
parametric image so that the parametric image from CRT 30 is recombined with the
real image of the specimen 18 at the input of the CCD camera 22. As a result a
technician employing the instrument may simul~aneously view theunaltered specimen
through eye piece 28 and the combined optical display on monitor 34. Additional-
ly~ where desired, an additional eye piece and beam splitter can be employed
between the beam splitter 38 and the camera 22 to permit real viewing of the com-
~ined optical display.
In the apparatus of Figure 3, the beam splitter 24 of Figure 1 has been
replaced by a pair of beam splitters 40 and 42 and a mirror 44 which deflect an
image of the specimen 18 to the eye piece 28, and a beam splitter 46 directs the
parametric image from CRT 30 and beam splitter 26 to eye piece 28 where images of
the specimen and parametric image are viewed as the combined optical display.
2~ Note that here the image transmission path for the paramctric image is no~
"closed" so that the parametric image is continuously recombined wi~h an image of
the specimen, ~ut the parametric modification of the optical display is not pro-
gressive since the camera does not see the parametric image.
-- 8 --
