Note: Descriptions are shown in the official language in which they were submitted.
CA 02403483 2002-09-16
WO 01/69942 PCT/IBO1/00347
METHODS AND APPARATUSES FOR SUPERIMPOSITION OF IMAGES
FIELD OF THE INVENTION
The field of the invention is image projection in general, and electronic
image
projection in particular.
BACKGROUND OF THE INVENTION
U.S. Patent No. 5,386,253 to Fielding, incorporated herein in its entirety by
this reference, discusses exemplary projection systems utilizing one or more
spatial
light modulators (SLMs). As noted in the Fielding patent:
Spatial light modulator devices include so-called "active matrix"
devices, comprising an array of light modulating elements, or "light valves,"
each of which is controllable by a control signal (usually an electrical
signal)
to controllably reflect or transmit light in accordance with the control
signal.
A liquid crystal array is one example of an active matrix device; another
example is the deformable mirror device (DMD) developed by Texas
Instruments . . . .
See Fielding, col. 1, 11. 13-21. Of course, yet other types of light
"engines," or
sources, and projectors exist, and various of them may be used in connection
with the
inventions described herein.
Regardless of the type of projector used, audiences frequently desire to see
images high in detail and richness and low in objectionable artifacts. High
resolution
and image quality in particular facilitates suspension of disbelief of an
audience as to
the reality of the projected images. Such quality indeed often is an important
factor in
the overall success of the motion picture viewing experience among today's
audiences.
Providing high resolution images to audiences can be prohibitively expensive
in terms of producing the software, and in terms of the hardware necessary to
show
high resolution images. Imax Corporation, for example, the intended assignee
of this
application, utilizes not only specialized cameras and projectors, but also
seventy
millimeter, fifteen perforation film to increase the resolution and quality of
projected
images.
CONFiR;JlATION COPY
CA 02403483 2002-09-16
WO 01/69942 PCT/IBO1/00347
In some venues, it is desirable to be able to display high resolution moving
picture images that are non-film based, such as computer generated graphics,
or
material captured with electronic cameras. It is particularly prohibitive to
display
these kinds of high resolution images using conventional electronic projectors
(and
especially those utilizing SLMs) because it is not technically or economically
feasible
to produce the necessary spatial light modulators (SLM) at sufficient
resolution to
match the high resolution of the source material. As well, such electronic
projectors
frequently fail to furnish the dynamic range and overall brightness of images
provided
by large-format films.
In one solution to achieve the desired resolution, conventional electronic
projection systems have employed "tiling" techniques. Tiling involves the
useof
multiple projection displays of sub-images that are displayed adjacent to each
other to
form a composite image. The use of multiple projection displays allows for
greater
resolution than is available with a conventional single projection display.
Thesub
images can be blended inside a single projector or if multiple projectors are
used, the
sub-images are blended on the screen. For example, when two projectors are
used
one projector projects a first sub-image on a screen. A second projector
projects a
second sub-image on a screen. The first and second projectors are positioned
such
that the first and second sub-images are projected onto a screen adjacent to
each other.
It is difficult to align the projectors exactly and therefore undesirable
seams
between the first and second sub-images are often apparent to the viewer. To
improve
the appearance and continuity of the composite image, the first and second
projectors
are conventionally positioned such that the first image slightly overlaps the
second
image. Mere overlapping of sub-images typically is insufficient, however, as
the
additive intensity of the images in the regions of overlap in some scenes
likewise may
be noticeable to audiences. General methods of reducing brightness in these
regions
require careful matching of the displays at the seam area(s), both
geometrically and
photometrically.
Another approach is to combine or superimpose two or more sub-images by
off setting two or more SLMs by, for example, one half of a pixel. With this
approach, the sub-images are simultaneously displayed and the pixels of one
spatial
light modulator are positioned to lie between the spaces of the pixels of
another SLM.
This approach is discussed in U.S. Patent No. 5,490,009. A disadvantage of
this
2
CA 02403483 2002-09-16
approach is that it requires twice the number of SLM devices while the
resulting
combined resolution of the two SLMs is limited to being less than a factor of
two
horizontally Or vertically. This is because there is always Some overlapping
of
superimposed pixels since for reasons of ~m(formity and efficiency it its
desirable that
the pixels be as nearly equal to 100% of the space allowed by their pitch as
possible.
This effectively limits the gain in resolution to about the square root of two
horizontally or vertically, which produces an overal l increase in the number
of pixels
of about 1.4 times.
There are also times when it is desired to produce stereoscopic or three
dimensional (3D) images with an electronic projector. Typically the projection
of
stereoscopic or 3D images requires two separate image projectors, one
dedicated to
projecting left c~ye images, and the other dedicated to projecting right eye
images,
This requirement. when combiacd with a superimposition technique that doubles
the
number of required SLMs in order to produce the necessary high resolution can
be
cyst prohibitive.
SUMMARY OF THE 1NVENT10N
In one embodiment of this invention, two sub-images for superimposition are
created using a single spatial light modulator. A first sub-image is projected
vvith the
SLM at a first position and, during the same frame, a second sub-image is
projected
using the same SLM. In one embodiment, micro-actuators are used to move the
SLM
from the fjrst to the second position. The SLM is subsequently moved back to
the
first position for the projection of the next image frame_ T'he first and
second position
of. the SLM are such, that the two resulting sub=images are offset by one half
of a pixel
is both horizontal and vertical directions, allowing the two subimages to
combine to
product a final Image having a greater resolution than that provided by the
actual
pixels contained in the SLM.
The first and second projection positions may be discreet static positions, or
they may be continuously varying dynamic positions, such as the crest and
trough
portions of a sinusoidal motion profile.
In another embodiment, high resolution, stereoscopic images arc created using
the principle oftemporal superimposition and an eleetmnie projection system
having
a minimum of low resolution SLMs. The Invention alternately projects og set
image
3
1,o0~LSO~ W e.i
CA 02403483 2002-09-16
WO 01/69942 PCT/IBO1/00347
sub-fields to each eye, which are then combined by the human visual system
into a
single, integrated high resolution image. The human visual system similarly
integrates the separate left and right eye images into a single, 3D image.
BRIEF DESCRIPTION OF THE DRAWINGS
Figs. 1 to 3 are schematic block diagrams illustrating the general structure
of
an active matrix projection system.
Fig. 4 is an illustration of a spatial light modulator in accordance with the
invention at a first display position.
Fig. 5 is an illustration of the spatial light modulator in accordance with
the
invention at a second display position.
Fig. 6 is a close up of the pixels of a spatial light modulator illustrating
the
superimposition of pixels to create a higher effective resolution.
Fig. 7 is a schematic illustrating the means by which a SLM may be moved
from one position to another in accordance with the invention.
Figs. 8 and 9 illustrate two motion profiles of the SLM.
Figs. 10 and I 1 illustrate two path profiles of the SLM.
Fig. 12 is a schematic of the arrangement of spatial light modulators and
optics
of the inventive method and apparatus.
Fig. 13 is a timing diagram of the sub-images projected by the novel
projector.
Fig. 14 is a timing diagram of the state of polarization of each of the lenses
in
the pair of electronic glasses.
Fig. 15 is a timing diagram of the sub-images projected by the projector in an
alternate embodiment.
Fig. 16 is a timing diagram of the alternate eye glasses associated with the
alternate embodiment depicted in Fig. 1 S.
Fig. 17 is a schematic of an embodiment of a projector incorporating an
electrically controllable wave plate.
Fig. 18 is a diagram of the polarization of light produced by the projector of
Fig. 17.
Fig. 19 is a timing diagram of the sub-images projected by the projector of
Fig. 17.
4
CA 02403483 2002-09-16
WO 01/69942 PCT/IBO1/00347
DETAILED DESCRIPTION
Referring to Fig. l, a projection system comprises a reflective screen (for
example a cinema screen) 10 and a projector 12, positioned and aligned
relative to the
screen so as to generate a focused image on the screen 10.
The projector 12 comprises a lamp 13, typically rated at several kilowatts for
a
cinema application, generating a light beam which is directed onto a planar
active
matrix display device 14 comprising, for example, a DMD array of 512 x 512
individual pixel mirrors. Each mirror of the display device 14 is individually
connected to be addressed by an addressing circuit 15 which receives a video
signal in
any convenient format (for example, a serial raster scanned interlaced field
format)
and controls each individual mirror in accordance with a corresponding pixel
value
within the video signal. The reflected and modulated beam from the active
matrix
device 14 (or rather, from those pixels of the device which have been
selectively
activated) is directed to a projector lens system 16 which, in a conventional
manner,
focuses, magnifies and directs the beam onto the screen 10 as shown
schematically in
Fig. 2.
For a color system, three separate active matrixes as shown in Fig. 3 or three
separate lamps with one SLM and a combining prism can be used. Other color
systems are also known.
Referring now to Fig. 4, there is illustrated a spatial light modulator (SLM)
30
having a plurality of pixels 32 arranged in a grid of rows and columns. SLM 30
could
be a deformable mirror device, (DMD) such as that sold by Texas Instruments,
in
which each of the pixels is actually micro-steerable mirrors which can be
toggled
between an off state and an on-state in rapid succession, as is necessary to
display an
image onto a projection screen. The total number of pixels in a DMD device is
typically limited by technological and economic factors, and commercially
available
DMD chips are not capable of projecting very high resolution images such as
those
that are associated with 70 mm motion picture film.
In one embodiment of this invention, a single SLM is used to project two sub
images during a single frame where the sub-images are offset by a some portion
of a
pixel. Fig. 5 shows SLM 30 in the two projection positions. Position 33 is
indicated
by ghost outline, whereas position 34 is indicated by the solid black lines.
Position 34
5
CA 02403483 2002-09-16
WO 01/69942 PCT/IBO1/00347
is an offset of position 33 by, for example, slightly less than one pixel
horizontally 35
and vertically 36.
Fig. 6 is a close up of pixels in the two positions illustrating how the
pixels at
the second position are positioned to be in the spaces between the pixels at
thefirst
position. The dark pixels, 51 are indicative of the pixels at the second
position,
whereas the lighter cross-hatched pixels 41 are indicative of the pixels at
the first
position. The two sub-images created by projection images at the two different
positions, even though displaced in time, are combined by the human visual
system
into a single coherent image, in a manner similar to that in which separate
images,
projected rapidly are perceived as a smoothly moving image.
In Fig. 7, a SLM 30 is schematically shown to be connected with two linear
actuators, AH and A,, and to two springs, SH and 5,~. The springs, Sti and Sv,
act to
bias SLM 30 in position 33 - SH in the horizontal direction and Sv in the
vertical
direction. Actuator AH acts to move SLM 30 in the horizontal direction and
actuator
Av acts to move SLM 30 in the vertical direction. Actuators AH and Av act
together
to move SLM 30 from position 33 to position 34. Actuators AH and Av may be
piezoelectric actuators, such as those supplied by Physik Instrumente GmbH of
Germany, which are capable of precise positioning down to the subnanometer
range.
This example is illustrative only, and other means know to those skilled in
the
art may be used to move the SLM from a first position to a second position.
Additionally, the sub-images could be generated by moving other components
within
the projection system, other than the SLM. For example, a mirror or a group of
optical elements such as a 1:1 relay carrying the image from the SLM within
the
projector could be moved between two positions thereby creating two
complementary
sub-images when projected onto the screen.
In Fig. 8 a timing diagram is shown illustrating linear motion of a SLM 30
from a first position indicated by 70 to a second position ndicated by 72. At
70 and
72 the SLM 30 is stationary for the duration of the sub-frame projection
period. The
periods 71 and 73, represent the time required for the SLM 30 to travel from
the first
position to the second position, and back again. The sum of the periods 70 to
73 is
equivalent to one normal frame in motion picture projection - typically I/24
of a
second or approximately 41 milliseconds. A projector incorporating the
inventive
6
CA 02403483 2002-09-16
WO 01/69942 PCT/IBO1/00347
method should be capable of displaying images at twice the normal frequency,
or
frame rate.
In Fig. 9 a timing diagram is shown illustrating a sinusoidal motion profile
in
which the SLM 30 never comes to a discreet stop, but is in continuous motion
from
one position to the other. The motion profiles are designed so as to maximize
the
time when the SLM is essentially stationary (T1 and T2 in the diagram) without
requiring the mechanical system to bring it to a complete stop.
Figures 10 and 11 illustrate two possible motion paths for moving the SLM
from one position to the other. In Fig. 10 a single pixel is shown in each of
the two
extreme positions and a linear path of motion for the pixel is shown. A linear
path is
produced, for example, by the actuators, AH and Av, in Fig. 7 moving in the
respective directions at the same time and at the same rate. Figure 11
illustrates an
elliptical path of motion, which may be desirable for reasons of mechanical
durability.
This elliptical path is produced, for example, by the actuators, Ai, and Av,
in Fig. 7
moving in their respective directions at varying rates and times.
Referring now to an alternative embodiment illustrated in Fig. 12, a projector
100 is depicted schematically and is comprised of six separate SLMs, grouped
in two
sets of three, each group having its own combining prism. Prism 102 has
separate red
1038, green 1036 and blue 103B SLMs. Prism 102 combines the light of each of
the
three separate SLMs into one full color light beam, which exits in the
direction
indicated by arrow S. Similarly, prism 104 has separate red 1058, green 1056,
and
blue 105B SLMs. Prism 104 combines the light of each of the three separate
SLMs,
which exits in the direction indicated by arrow P.
The light from both prisms 102 and 104 is directed towards a polarizing beam
splitter, 106, as seen in Fig. 12. The light from prism 102 becomes linearly
polarized
in an "s" orientation, and the light from prism 104 becomes linearly polarized
in an
orthogonal, or "p" orientation.
Prisms 102 and 104 are offset slightly in relation to each other, so that
images
formed by each can be superimposed on the screen thereby creating composite
images
that have a higher overall resolution than one generated by either prism
alone.
Typically, the prisms and/or SLMs are oriented so that the output ofone prism
is
offset by one half of a pixel vertically, horizontally or both.
7
CA 02403483 2002-09-16
WO 01/69942 PCT/IBO1/00347
Electronic glasses, 107, as seen in Fig. 12, are provided to audience members
in order to decode the spatial and temporal multiplexing of the images as
produced by
the projector.
The glasses have liquid crystal lenses, 108 and 109, which can be alternately
switched between two orthogonal states of polarization. Such liquid crystal
lenses are
similar to those used in alternate eye 3D electronic glasses, such as those
used by
Imax Corporation, except they lack a front polarizer, which is commonly
included
with liquid crystals to enable them to operate as alternately transmissive and
opaque
shutters.
A timing diagram is depicted in Fig. 13, which shows the sequencing of
images produced by the two separate prisms within projector 100. Referring now
to
the output of prism 102, a first right (R) eye sub-field is projected onto the
screen
during the first portion of frame 1. The duration of one frame is typical 1y 1
/24 second
(or 40.3 milliseconds). The output of prism 102 is then switched to provide a
sub-
I 5 frame intended for the left (L) eye. Similarly, the output of prism 104
alternates
between a first left (L) eye sub-field, followed by a right (R) eye sub-field.
The
polarization of the images from prism 102 is "s" and the polarization of the
images
from prism 104 is "p".
Figure 14 depicts a timing diagram which indicates the state of polarization
of
the lenses in the glasses worn by viewers. During the first half of a frame
period, the
left eye lens transmits the light produced by prism 104, and blocks the light
produced
by prism 102. As shown in Fig. 14, this is accomplished by setting the
polarity on the
left eye lens to "p". Thus, letting in all the light polarized in the p
direction and
keeping out all of the light polarized in the s direction. In the second half
of the frame
period, after the polarization of the left lens has been switched, it
transmits the light
produced by prism 102, and blocks the light produced by prism 104. Similarly,
this is
accomplished by changing the polarity on the left eye lens to "s". Thereby,
the left
eye lens lets in all the light polarized in the s direction and blocks light
polarized in
the p direction during the second half of the frame. As can be seen in Fig.
14, the
right eye lens in the glasses is operated out of phase with the left eye lens-
letting in
light polarized in the s direction during the first half of the frame and
letting in light
polarized in the p direction during the second half of the frame. The
operation of the
8
CA 02403483 2002-09-16
WO 01/69942 PCT/IBO1/00347
lens allows each eye to see the images intended only for it, thus allowing the
human
visual system to integrate the two sets of images into a three dimensional
image.
Since the light output by prisms 102 and 104 are offset relatively, the
composite image can be temporally fused by the human visual system, resulting
in the
perception of a higher resolution image than the images produced by either
prism
alone. Experiments have shown that temporal fusing can occur if the switching
between sub-images is fast enough. Typically the overall resolution can be
improved
by a factor of about 1.4.
In another embodiment, the timing profile is changed so that the frequency of
subframes is increased, for example by a factor of two, so that each sub-frame
is
displayed for a period of about 10 msec.
In yet another embodiment, the offset sub-fields are presented simultaneously
to one eye, while the other eye is blocked by an opaque shutter. Here the
polarizing
beam splitter is replaced by an alternative method that does not rely on
polarization to
combine the two images. The eyeglasses act to direct the light from both sub-
fields to
the appropriate eye. In the subsequent time period, the first eye is blocked
by a
shutter, and the other eye is presented with two offset sub-fields
simultaneously. The
eyeglasses required by this embodiment are standard alternate-eye electronic
liquid
crystal shutter glasses. This embodiment is illustrated in Figures 15 and 16.
In an alternative embodiment, viewers wear passive glasses in which the
lenses are mutually orthogonal linear polarizers. An active alternate phase
'/4 wave
plate (such as a Ferroelectric Liquid Crystal) is located at the projector and
switches
the polarization of the light by 90 degrees every half frame (approximately 20
msec.)
Fig. 17 depicts a projector I 10 with lens 112 incorporating an electrically
controllable wave plate 111 located prior to the lens. The wave plate could
alternatively be located after the lens as illustrated by the dashed lines
113. This
projector produces the two overlapped images from prisms 102 and 104 (not
shown in
Fig. 17, but shown in Fig. 12) onto screen 114. Fig. 18 illustrates how the
switching
of the polarization of 111 (or 113) causes the light that reaches the screen I
14 to
alternate in polarity, corresponding alternately to the images from prisms 102
and
104.
Fig. 19 illustrates the switching arrangement for the sub-images presented to
prisms 102 and 104, and the switching of the polarity of 111 (or I I 3). The
9
CA 02403483 2002-09-16
WO 01/69942 PCT/IBO1/00347
controllable wave plate 111 (or 113) switches at two times the frame rate
(approximately 20 msec. for 24 frames per second) and prisms 102 and 104 carry
the
appropriate eye sub-image at each time.
In all cases it should be noted that the while a frame rate of 24 fps is
typical
for motion picture films, other frame rates are commonly employed and may be
used
without departing from the spirit of the invention. It should also be noted
that visual
fusion ofthe sub-images is improved by higher frame rates, and this will
contribute to
an improvement in the quality of the results obtained from the temporal
superimposition.
The foregoing is provided for purposes of explanation and disclosure of
preferred embodiments of the present invention. For instance, a preferred
embodiment of this invention involves using a deformable mirror device as the
spatial
light modulator. It is expected that such capabilities or their equivalent
will be
provided in other standard types of spatial light modulators, in which case
the
preferred embodiment ofthis invention may be easily adapted for use in such
systems.
Further modifications and adaptations to the described embodiments will be
apparent
to those skilled in the art and may be made without departing from the scope
or spirit
of the invention and the following claims.
