DISPOSITIF OMNIDIRECTIONNEL SERVANT A CAPTER DES IMAGES

AN OMNIDIRECTIONAL IMAGING APPARATUS

Abrégé français

L'invention concerne un dispositif omnidirectionnel servant à capter l'image d'une scène (130) depuis un point de vue unique. Ce dispositif comprend un réflecteur (135) tronqué sensiblement parabolique placé de façon à réfléchir orthographiquement des rayons principaux de rayonnement électromagnétique émis par ladite scène, ledit réflecteur possédant un point focal coïncidant avec ledit point de vue unique dudit dispositif ominidirectionnel, y compris ledit réflecteur sensiblement parabolique. Ce dispositif comprend également des moyens télécentriques (112, 113) couplés optiquement au réflecteur et servant à filtrer pratiquement les rayons principaux de rayonnement électromagnétique qui ne sont pas réfléchis orthographiquement par ledit réflecteur. Ce dispositif comporte également un ou plusieurs capteurs d'image (111) placés de manière à recevoir lesdits rayons principaux réfléchis orthographiquement du rayonnement électromagnétique depuis ledit réflecteur parabolique, ce qui permet de capter l'image de ladite scène.

Abrégé anglais

The invention disclosed herein is an omnidirectional imaging apparatus for
sensing an image of a scene (130) from a single viewpoint. The omnidirectional
imaging apparatus includes a truncated, substantially paraboloid-shaped
reflector (135) positioned to orthographically reflect principal rays of
electromagnetic radiation radiating from said scene, said paraboloid-shaped
reflector having a focus coincident with said single viewpoint of said
omnidirectional imaging apparatus, including said paraboloid-shaped reflector.
The omnidirectional imaging apparatus also includes telecentric means (112,
113), optically coupled to said paraboloid-shaped reflector, for substantially
filtering out principal rays of electromagnetic radiation which are not
orthographically reflected by said paraboloid-shaped reflector. The
omnidirectional imaging apparatus also includes one or more image sensors
(111) positioned to receive said orthographically reflected principal rays of
electromagnetic radiation from said paraboloid-shaped reflector, thereby
sensing said image of said scene.

Revendications

37
CLAIMS
1. An omnidirectional imaging apparatus for sensing an image of a scene
from a single viewpoint, comprising:
(a) a truncated, substantially paraboloid-shaped reflector
positioned to orthographically reflect principal rays of electromagnetic
radiation
radiating from said scene, said paraboloid-shaped reflector having a focus
coincident
with said single viewpoint of said omnidirectional imaging apparatus,
including said
paraboloid-shaped reflector;
(b) telecentric means, optically coupled to said paraboloid-shaped
reflector, for substantially filtering out principal rays of electromagnetic
radiation
which are not orthographically reflected by said paraboloid-shaped reflector;
and
(c) one or more image sensors positioned to receive said
orthographically reflected principal rays of electromagnetic radiation from
said
paraboloid-shaped reflector, thereby sensing said image of said scene.
2. An omnidirectional imaging apparatus according to claim 1, wherein
said paraboloid-shaped reflector is convex.
3. An omnidirectional imaging apparatus according to claim 1, wherein
said paraboloid-shaped reflector is concave.
4. An omnidirectional imaging apparatus according to claim 1, wherein
said paraboloid-shaped reflector comprises a substantially paraboloidal mirror
having
a surface which substantially obeys the equation expressed in cylindrical
coordinates:
<IMG>

38
z being an axis of rotation of said surface, r being a radial coordinate, and
h being a
constant.
5. An omnidirectional imaging apparatus according to claim 1, wherein
said one or more image sensors comprise one or more charge-coupled devices.
6. An omnidirectional imaging apparatus according to claim 1, wherein
said one or more image sensors comprise one or more charge injection devices.
7. An omnidirectional imaging apparatus according to claim 1, wherein
said one or more image sensors comprise photographic film.
8. An omnidirectional imaging apparatus according to claim 1, wherein
said one or more image sensors comprise one or more video cameras.
9. An omnidirectional imaging apparatus according to claim 1, wherein
said one or more image sensors has a curved surface that matches a field-
curvature of
said image.
10. An omnidirectional imaging apparatus according to claim 1, wherein at
least one of said one or more image sensors has a non-uniform resolution.
11. An omnidirectional imaging apparatus according to claim 1, wherein
said one or more image sensors are positioned along an axis passing through
the
vertex of said paraboloid-shaped reflector and through said focus of said
paraboloid-shaped reflector.
12. An omnidirectional imaging apparatus according to claim 1, further
comprising one or more planar mirrors positioned between said paraboloid-
shaped
reflector and said one or more image sensors, wherein said one or more planar
mirrors

39
optically couple said paraboloid-shaped reflector to said one or more image
sensors.
13. An omnidirectional imaging apparatus according to claim 1, wherein
said paraboloid-shaped reflector comprises a mirror truncated at a plane which
includes said focus of said paraboloid-shaped reflector.
14. An omnidirectional imaging apparatus according to claim 1, wherein
said paraboloid-shaped reflector comprises a mirror truncated at a plane that
is
substantially perpendicular to an axis passing through the vertex of said
paraboloid-shaped reflector and through said focus of said paraboloid-shaped
reflector.
15. An omnidirectional imaging apparatus according to claim 1, wherein
said paraboloid-shaped reflector comprises a normal paraboloidal mirror.
16. An omnidirectional imaging apparatus according to claim 1, further
comprising a transparent support coupling said paraboloid-shaped reflector to
said one
or more image sensors to thereby maintain the relative positions thereof.
17. An omnidirectional imaging apparatus according to claim 1, further
comprising a fixed base and a movable base, wherein said paraboloid-shaped
reflector
is mounted on said fixed base and said one or more image sensors are mounted
on
said movable base, whereby movement of said one or more image sensors produces
a
changing field of view.
18. An omnidirectional imaging apparatus according to claim 17, further
comprising a zoom lens positioned between and optically coupling said one or
more
image sensors and said paraboloid-shaped reflector.
19. An omnidirectional imaging apparatus according to claim 1, further

40
comprising a fixed base and a movable base, wherein said paraboloid-shaped
reflector
is mounted on said movable base and said one or more image sensors are mounted
on
said fixed base, whereby movement of said paraboloid-shaped reflector produces
a
changing field of view.
20. An omnidirectional imaging apparatus according to claim 19, further
comprising a zoom lens positioned between and optically coupling said one or
more
image sensors and said paraboloid-shaped reflector.
21. An omnidirectional imaging apparatus according to claim 1, wherein
said one or more image sensors generate an image signal representative of said
image
of said scene, further comprising an image signal processing apparatus coupled
to said
one or more image sensors and receiving said image signal for converting said
image
signal into image signal data.
22. An omnidirectional imaging apparatus according to claim 21, wherein
said image signal processing apparatus maps said image signal data into a
Cartesian-coordinate system to produce a perspective image.
23. An omnidirectional imaging apparatus according to claim 21, wherein
said
image signal processing apparatus maps said image signal data into a
cylindrical-coordinate system to produce a panoramic image.
24. An omnidirectional imaging apparatus according to claim 21, wherein
said image signal processing apparatus further includes interpolation means
for
providing interpolated image data, whereby said interpolated image data and
said
image signal data are combined to from a digital image.
25. An omnidirectional imaging apparatus according to claim 24, wherein

41
said image processing apparatus further includes means for zooming in on a
preselected portion of said digital image to thereby provide an enlarged image
of said
preselected portion from a predetermined focal distance.
26. An omnidirectional imaging apparatus according to claim 1, wherein
said telecentric means comprises a telecentric lens.
27. An omnidirectional imaging apparatus according to claim 1, wherein
said telecentric means comprises a telecentric aperture.
28. An omnidirectional imaging apparatus according to claim 1, further
comprising at least one lens optically coupling said one or more image sensors
and
said paraboloid-shaped reflector.
29. An omnidirectional imaging apparatus according to claim 28, wherein
said at least one lens has a focal plane between said one or more image
sensors and
said at least one lens, and wherein said telecentric means is a telecentric
aperture
positioned along said focal plane.
30. An omnidirectional imaging apparatus according to claim 28, wherein
said telecentric means comprises a collimating lens optically coupling said
paraboloid-shaped reflector and said at least one lens.
31. An omnidirectional imaging apparatus according to claim 1, further
comprising a zoom lens optically coupling said one or more image sensors and
said
paraboloid-shaped reflector.
32. An omnidirectional imaging apparatus according to claim 1, further
comprising a microscope objective optically coupling said one or more image
sensors
and said paraboloid-shaped reflector.

42
33. An omnidirectional imaging apparatus according to claim 1, further
comprising a field-flattening lens optically coupling said one or more image
sensors
and said paraboloid-shaped reflector, said field-flattening lens having a
field curvature
approximately opposite to the field curvature of said paraboloid-shaped
reflector.
34. An omnidirectional imaging apparatus according to claim 33, wherein
said field-flattening lens comprises a piano-concave lens which is positioned
closely
to said one or more image sensors.
35. An omnidirectional imaging apparatus according to claim 33, wherein
said field-flattening lens comprises a meniscus lens having aplanatic sides.
36. An omnidirectional imaging apparatus according to claim 1, wherein
said scene is a substantially hemispherical scene, and further comprising:
an additional truncated, substantially paraboloid-shaped reflector
positioned to orthographically reflect principal rays of electromagnetic
radiation
radiating from an additional hemispherical scene, said additional paraboloid-
shaped
reflector having a focus coincident with a single viewpoint of said additional
hemispherical scene;
additional telecentric means, optically coupled to said additional
paraboloid-shaped reflector, for substantially filtering out principal rays of
electromagnetic radiation which are not orthographically reflected by said
additional
paraboloid-shaped reflector; and
additional one or more image sensors positioned to receive said
orthographically reflected principal rays of electromagnetic radiation from
said
additional paraboloid-shaped reflector, thereby sensing said additional
substantially
hemispherical scene.
37. An omnidirectional imaging apparatus according to claim 36, wherein
said additional hemispherical scene and said hemispherical scene are
substantially

43
complementary to one another so that the combination thereof is a
substantially
spherical scene, and wherein said paraboloid-shaped reflector and said
additional
paraboloid-shaped reflector are normal convex paraboloids, positioned back-to-
back
along their planes of truncation, and having a common paraboloidal axis and a
common focus point.
38. An omnidirectional imaging apparatus according to claim 36, wherein
said additional hemispherical scene and said hemispherical scene are
substantially
complementary to one another so that the combination thereof is a
substantially
spherical scene, and wherein said paraboloid-shaped reflector and said
additional
paraboloid-shaped reflector are normal concave paraboloids, positioned such
that their
vertexes coincide and they share a common paraboloidal axis.
39. An omnidirectional imaging method for sensing an image of a scene
from a single viewpoint, comprising the steps of:
(a) orthographically reflecting principal rays of electromagnetic
radiation radiating from said scene on a truncated, substantially paraboloid-
shaped
reflector such that said single viewpoint of said omnidirectional imaging
method
coincides with a focus point of said paraboloid-shaped reflector;
(b) telecentrically filtering out a substantial portion of any principal
rays of electromagnetic radiation which are not orthographically reflected by
said
paraboloid-shaped reflector; and
(c) sensing said orthographically reflected principal rays of
electromagnetic radiation from said paraboloid-shaped reflector with one or
more
image sensors to thereby sense said image of said scene.
40. The method of claim 39, wherein step (c) comprises sensing said
image of said scene from a position along an axis passing through the vertex
of said
paraboloid-shaped reflector and through said focus of said paraboloid-shaped
reflector.

44
41. The method of claim 39, further comprising the step of optically
coupling said paraboloid-shaped reflector and said one or more image sensors
with
one or more planar mirrors positioned between said paraboloid-shaped reflector
and
said one or more image sensors.
42. The method of claim 39, further comprising the steps of providing an
image signal which is representative of said image of said scene and
converting said
image signal into image signal data.
43. The method of claim 42, further comprising the step of mapping said
image signal data into a Cartesian-coordinate system to produce a perspective
image.
44. The method of claim 42, further comprising the step of mapping said
image signal data into a cylindrical-coordinate system to produce a panoramic
image.
45. The method of claim 42, further comprising the steps of interpolating
said image signal data to define approximate values for missing image data,
and
forming a digital image from said mapped image data and said interpolated
image
data.
46. The method of claim 45, further comprising the steps of zooming in on
a preselected portion of said digital image to thereby obtain an enlarged
image of said
preselected portion from a predetermined focal distance, interpolating said
image data
to define approximate values for missing image data, and forming a digital
image
from said mapped image data and said interpolated image data.
47. The method of claim 39, wherein said scene is substantially
hemispherical and further comprising the steps of:
orthographically reflecting principal rays of electromagnetic radiation
radiating from an additional substantially hemispherical scene on an
additional

45
truncated, substantially paraboloid-shaped reflector such that a single
viewpoint of
said additional hemispherical scene coincides with a focus point of said
additional
paraboloid-shaped reflector;
telecentrically filtering out a substantial portion of any principal rays of
electromagnetic radiation which are not orthographically reflected by said
additional
paraboloid-shaped reflector; and
sensing said orthographically reflected principal rays of
electromagnetic radiation from said additional paraboloid-shaped reflector
with
additional one or more image sensors to thereby sense said additional
hemispherical
scene.
48. A method for omnidirectionally sensing images of a scene from a
single viewpoint, the method comprising the steps of:
(a) mounting a truncated, substantially paraboloid-shaped reflector
on a fixed base;
(b) mounting one or more image sensors on a movable base;
(c) orthographically reflecting principal rays of electromagnetic
radiation radiating from said scene on said substantially paraboloid-shaped
reflector
such that said single viewpoint of said omnidirectional imaging method
coincides
with a focus point of said paraboloid-shaped reflector;
(d) telecentrically filtering out a substantial portion of any principal
rays of electromagnetic radiation which are not orthographically reflected
from said
paraboloidal-shaped reflector;
(e) moving said movable base to a first position;
sensing a first image of said scene having a first field of view
by sensing said orthographically reflected principal rays of electromagnetic
radiation
from said paraboloidal-shaped reflector with said one or more image sensors;
(g) moving said movable base to a second position different from
said first position; and
(h) sensing a second image of said scene having a second field of

46
view by sensing said orthographically reflected principal rays of
electromagnetic
radiation from said paraboloidal-shaped reflector with said one or more image
sensors.
49. A method for omnidirectionally sensing images of a scene according to
claim 48, further comprising the step of optically coupling said substantially
paraboloid-shaped reflector and said one or more image sensors with a zoom
lens.
50. A method for omnidirectionally sensing images of a scene according to
claim 49, further comprising the steps of:
locating an area of interest within said scene with said zoom lens set at
a first power of magnification; and
magnifying said area of interest by setting said zoom lens at a second
power of magnification greater than said first dower of magnification.
51. A method for omnidirectionally sensing images of a scene from a
single viewpoint, the method comprising the steps of:
(a) mounting a truncated, substantially paraboloid-shaped reflector
on a movable base;
(b) mounting one or more image sensors on a fixed base;
(c) orthographically reflecting principal rays of electromagnetic
radiation radiating from said scene on said substantially paraboloid-shaped
reflector
such that said single viewpoint of said omnidirectional imaging method
coincides
with a focus point of said paraboloid-shaped reflector;
(d) telecentrically filtering out a substantial portion of any principal
rays of electromagnetic radiation which are not orthographically reflected
from said
paraboloidal-shaped reflector;
(e) moving said movable base to a first position;
(f) sensing a first image of said scene having a first field of view
by sensing said orthographically reflected principal rays of electromagnetic
radiation

47
from said paraboloidal-shaped reflector with said one or more image sensors;
(g) moving said movable base to a second position different from
said first position; and
(h) sensing a second image of said scene having a second field of
view by sensing said orthographically reflected principal rays of
electromagnetic
radiation from said paraboloidal-shaped reflector with said one or more image
sensors.
52. A method for omnidirectionally sensing images of a scene according to
claim 51, further comprising the step of optically coupling said substantially
paraboloid-shaped reflector and said one or more image sensors with a zoom
lens.
53. A method for omnidirectionally sensing images of a scene according to
claim 52, further comprising the steps of:
locating an area of interest within said scene with said zoom lens set at
a first power of magnification; and
magnifying said area of interest by setting said zoom lens at a second
power of magnification greater than said first power of magnification.
54. An omnidirectional imaging apparatus for sensing an image of a scene
from a single viewpoint, comprising:
(a) a truncated, substantially paraboloid-shaped reflector
positioned to orthographically reflect principal rays of electromagnetic
radiation
radiating from said scene, said paraboloid-shaped reflector having a focus
coincident
with said single viewpoint of said omnidirectional imaging apparatus,
including said
paraboloid-shaped reflector;
(b) telecentric means, optically coupled to said paraboloid-shaped
reflector, for substantially filtering out principal rays of electromagnetic
radiation
which are not orthographically reflected by said paraboloid-shaped reflector;
(c) a plurality of beam splitters for splitting said orthographically

48
reflected principal rays of electromagnetic radiation into a plurality of ray
bundles,
each ray bundle comprising a portion of said orthographically reflected
principal rays
of electromagnetic radiation from said paraboloid-shaped reflector; and
(d) a plurality of image sensors, each image sensor positioned to
receive at least one of said plurality of ray bundles, each image sensor
thereby sensing
a portion of said image of said scene.
55. An omnidirectional imaging apparatus for sensing an image of a scene
from a single viewpoint, comprising:
(a) a truncated, substantially paraboloid-shaped reflector
positioned to orthographically reflect principal rays of electromagnetic
radiation
radiating from said scene, said paraboloid-shaped reflector having a focus
coincident
with said single viewpoint of said omnidirectional imaging apparatus,
including said
paraboloid-shaped reflector;
(b) telecentric means, optically coupled to said paraboloid-shaped
reflector, for substantially filtering out principal rays of electromagnetic
radiation
which are not orthographically reflected by said paraboloid-shaped reflector;
(c) a plurality of dichroic beam splitters for splitting said
orthographically reflected principal rays of electromagnetic radiation into a
plurality
of monochromatic principal rays of electromagnetic radiation; and
(d) a plurality of image sensors, each image sensor positioned to
receive at least one of said plurality of monochomatic principal rays of
electromagnetic radiation, thereby sensing at least one monochromatic image of
said
scene.

Description

CA 02312970 2000-06-OS
WO 99/30197 PCT/US98/25689
SPECIFICATION
BACKGROUND OF THE INVENTION
1. Field of the Invention
This invention relates to omnidirectional image sensing with reference to a
single viewpoint, and, more particularly to such image sensing using a
truncated,
substantially paraboloid-shaped reflector.
2. Discussion of the State of the Art
For many applications such as surveillance, teleconferencing, remote sensing,
10 photogrammetry, model acquisition, virtual reality, computer graphics,
machine
vision and robotics, it is desirable that an imaging system have a large field
of view so
as to be able to take in as much information as possible about the world
around it.
Traditional imaging systems include a camera with a lens that provides a
perspective projection of an image. However, a camera with even a very wide
angle
15 lens has only a limited field of view (i.e., covering less than a full
hemisphere). This
limited field of view may be expanded by tilting and panning the entire
imaging
system about its center of projection. One such system is described in S.E.
Chen,
"Quicklime VR - An Image-Based Approach to Virtual Environment Navigation",
Proc. of SIGGRAPH 95, (8):29-38, August (1995). The article by L. McMillan and
20 G. Bishop, "Plenoptic Modeling: An Image-Based Rendering System", Computer
Graphics: Proc. of SIGGRAPH, August 1995, pp. 39-46, also describes a
traditional
pan-and-tilt system. This type of system has two serious drawbacks, however,
one
being the obvious disadvantages associated with a device having critical
moving
parts, and the second being the significant amount of time required to make a
full
25 rotation in order to view the surrounding world. This time limitation makes
such a

CA 02312970 2000-06-OS
WO 99/30197 PCT/US98/25689
2
device unsuitable for real-time applications.
Another approach to increasing the field of view in an imaging system is by
employing a so called "fish eye" lens as is disclosed in E.L. Hall et al.,
"Omnidirectional Viewing Using a Fish Eye Lens", SPIE Vol. 728 Optics,
Illumination, and Image Sensing for Machine Vision (1986), p. 250. Since the
fish
eye lens has a very short focal length, the field of view may be as large as a
hemisphere. The use of such lenses in an imaging system is problematic,
however, in
that they are significantly larger and more complex than conventional lenses.
Moreover, it has been difficult to develop a fish eye lens with a fixed
viewpoint for all
points of the relevant scene. U.S. Patent No. 5,187,667 to Zimmerman, and U.S.
Patent No. 5,359,363 to Kuban et al. are also directed to the use of fish eye
lenses to
replace conventional pan and tilt mechanisms, and accordingly suffer from the
same
disadvantages.
Other prior art devices have used reflecting surfaces to increase the field of
view. One such prior art device is disclosed in V.S. Nalwa, "A True Omni-
Directional Viewer", AT&T Bell Laboratories Technical Memorandum, BLO115500-
960115-O1, Jan. 1996. Nalwa discloses the use of multiple planar reflecting
surfaces
in conjunction with multiple charge coupled device ("CCD") cameras to obtain a
360
degree panoramic image of a 50 degree band of a hemispherical scene.
Specifically,
in Nalwa, four planar mirrors are arranged in the shape of a pyramid, with one
camera
being positioned above each of the four planar reflecting sides, and with each
camera
viewing slightly more than 90 degrees by 50 degrees of the hemispherical
scene. This
system suffers from the serious drawback of requiring multiple sensors to
capture a
360-degree image. In addition, this system suffers from the inherent problems
associated with distortion at the "seams" when the separate images are
combined to
provide a full 360 degree view.
Curved reflective surfaces have also been used in conjunction with image
sensors. Both Yagi et al., "Evaluating Effectivity of Map Generation by
Tracking
Vertical Edges in Omnidirectional Image Sequence", IEEE International
Conference
on Robotics and Automation, June 1995, p. 2334, and Yagi et al., "Map-Based

CA 02312970 2000-06-OS
WO 99/30197 PCT/US98IZ5689
Navigation for a Mobile Robot With Omnidirectional Image Sensor COPIS", IEEE
Transactions on Robotics and Automation, Vol. II, No. 5, Oct. 1995, disclose a
conical projection image sensor (COPIS) which uses a conical reflecting
surface to
gather images from the surrounding environment, and processes the information
to
guide the navigation of a mobile robot. Although the COPIS is able to attain
viewing
in 360 degrees, it is not a true omnidirectional image sensor because the
field of view
is limited by the vertex angle of the conical mirror and by the viewing angle
of the
camera lens. Furthermore, the COPIS does not have a single viewpoint, but
instead
has a locus of viewpoints lying on a circle. This locus of multiple viewpoints
causes
distortion in the captured images, which cannot be eliminated to obtain pure
perspective images.
Yamazawa et al., "Obstacle Detection With Omnidirectional Image Sensor
HyperOmni Vision", IEEE International Conference on Robotics and Automation,
Oct. 1995, p. 1062, discloses a purported improvement in the COPIS system
which
involves the use of a hyperboloidal reflecting surface in place of a conical
surface. As
discussed therein, the rays of light which are reflected off the hyperboloidal
surface,
no matter where the point of origin, will all converge at a single point, thus
enabling
perspective viewing.
Although the use of a hyperboloidal mirror is advantageous in that it enables
full perspective image sensing, because the rays of light which make up the
reflected
image converge at the focal point of the reflector, positioning of the sensor
relative to
the reflecting surface is critical, and any disturbance will impair the image
quality.
Further, the use of a perspective-projection model inherently requires that as
the
distance between the sensor and the mirror increases, the cross-section of the
mirror
must increase. Therefore, practical considerations dictate that in order to
keep the
mirror at a reasonable size, the mirror must be placed close to the sensor.
This in turn
causes complications to arise with respect to the design of the image sensor
optics. In
addition, mapping sensed image to usable coordinates requires complex
calibration
due to the nature of the converging image. A further drawback is that the
relative
positions of the mirror and the optics cannot be changed while maintaining a
single

CA 02312970 2000-06-OS
WO 99/30197 PCTNS98I25689
4
viewpoint. Thus, a hyperboloidal mirror system cannot take advantage of the
relative
movement of the mirror and optics to adjust the field of view of the system,
while
maintaining a single viewpoint.
Prior to Yamazawa et al., U.S. Patent No. 3,505,465 to Donald Rees also
disclosed the use of a hyperboidal reflecting surface. Accordingly, the Rees
disclosure also suffers from the same drawbacks as that of Yamazawa et al.
The above-described prior art devices fail in one of two ways. They either
fail
to provide a truly omnidirectional imaging apparatus that is capable of
sensing a scene
from a single viewpoint, making it impossible to provide distortion-free
images with
the apparatus, or they provide apparatus that require complex calibration and
implementation.
SUMMARY OF THE INVENTION
The drawbacks of the prior art, as discussed above, are substantially solved
by
the present invention, which in one aspect is an omnidirectional imaging
apparatus for
sensing an image of a scene from a single viewpoint that includes a truncated,
substantially paraboloid-shaped reflector positioned to orthographically
reflect
principal rays of electromagnetic radiation radiating from the scene. The
paraboloid-
shaped reflector has a focus coincident with the single viewpoint of the
omnidirectional imaging apparatus. The omnidirectional imaging apparatus also
includes telecentric means, optically coupled to the paraboloid-shaped
reflector, for
substantially filtering out principal rays of electromagnetic radiation which
are not
orthographically reflected by the paraboloid-shaped reflector. The
omnidirectional
imaging apparatus further includes one or more image sensors positioned to
receive
the orthographically reflected principal rays of electromagnetic radiation
from the
paraboloid-shaped reflector, thereby sensing the image of the scene.
The paraboloid-shaped reflector of the present invention may be either convex
or concave. The telecentric means may include a telecentric lens, a
telecentric
aperture, or a collimating lens.
Preferably, the paraboloid-shaped reflector comprises a substantially

CA 02312970 2000-06-OS
WO 99130197 PCT/US98n5689
paraboloidal mirror having a surface which substantially obeys the equation
expressed
in cylindrical coordinates:
z =
2h
h2_rx
z being an axis of rotation of the surface, r being a radial coordinate, and h
being a
5 constant. As the equation represents a symmetrical surface of rotation, the
shape of
the surface is not a function of the angular coordinate ~.
In a preferred embodiment of the invention, the one or more image sensors
comprise one or more video cameras. These video cameras may employ one or more
charge-coupled devices or one or more charge injection devices. Alternatively,
the
10 one or more image sensors may comprise photographic film. In another
preferred
embodiment, at least one image sensor has a non-uniform resolution to
compensate
for the non-uniform resolution of the image reflected from the paraboloid-
shaped
reflector.
Preferably, the paraboloid-shaped reflector comprises a mirror truncated at a
plane which includes the focus of the paraboloid-shaped reflector and which is
perpendicular to the axis passing through the focus and the vertex of the
paraboloid-
shaped reflector.
In an exemplary embodiment, the paraboloid-shaped reflector is mounted on a
fixed base and the one or more image sensors are mounted on a movable base,
20 whereby movement of the one or more image sensors produces a changing field
of
view. Alternatively, the paraboloid-shaped reflector may be mounted on a
movable
base and the one or more image sensors may be mounted on a fixed base, whereby
movement of the paraboloid-shaped reflector produces a changing field of view.
In
each of these embodiments, it is also preferred that a zoom lens be provided
for
optically coupling the one or more image sensors and the paraboloid-shaped
reflector.
In a further exemplary embodiment, the one or more image sensors provide an

CA 02312970 2000-06-OS
WO 99I30I9T PCT/US98/25689
6
image signal representative of the image of the scene. An image signal
processing
apparatus is coupled to the one or more image sensors, which converts the
image
signal from the image sensors into image signal data. The image signal
processing
apparatus then maps the image signal data into a Cartesian-coordinate system
to
produce a perspective image or into a cylindrical-coordinate system to produce
a
panoramic image. The image signal processing may include interpolation means
for
providing interpolated image data, whereby the interpolated image data and the
image
signal data are combined to form a digital image. Advantageously, the image
processing apparatus may further include means for zooming in on a preselected
portion of the digital image to thereby provide an enlarged image of the
preselected
portion from a predetermined focal distance.
In a preferred arrangement, the omnidirectional imaging apparatus comprises
at least one lens optically coupling the one or more image sensors and the
paraboloid-
shaped reflector. This coupling lens may be a zoom lens, a microscope
objective, or a
field-flattening lens. Advantageously, the field-flattening lens has a field
curvature
approximately opposite to the field curvature of the paraboloid-shaped
reflector.
Preferably, the field-flattening lens is either a piano-concave lens or an
aplanatic,
meniscus lens.
In yet another preferred arrangement, the omnidirectional imaging apparatus is
used to image a substantially spherical scene by using two truncated,
substantially
paraboloid-shaped reflectors positioned to orthographically reflect principal
rays of
electromagnetic radiation radiating from two complementary hemispherical
scenes.
The two paraboloid-shaped mirrors are positioned to share a common
paraboloidal
axis. In addition, when the two paraboloid-shaped reflectors are convex, they
are
positioned back-to-back along their planes of truncation, such that they share
a
common focus point. When the two paraboloid-shaped reflectors are concave,
they
are positioned such that their vertexes coincide.
In a further exemplary embodiment of the present invention, a plurality of
beam splitters are provided for splitting the orthographically reflected
principal rays
of electromagnetic radiation from the paraboloid-shaped reflector into a
plurality of

CA 02312970 2000-06-OS
WO 99/30197 PCTNS98n5689
7
ray bundles. In this embodiment, a plurality of image sensors is required,
with each
image sensor positioned to receive at least one of the plurality of ray
bundles, and
thereby sensing a portion of the image of the scene.
In yet a further exemplary embodiment, a plurality of dichroic beam splitters
is provided for splitting the orthographically reflected principal rays of
electromagnetic radiation from the paraboloid-shaped reflector into a
plurality of
monochromatic principal rays of electromagnetic radiation. As in the previous
embodiment, a plurality of image sensors is required, with each image sensor
positioned to receive at least one of the plurality of monochomatic principal
rays of
electromagnetic radiation, and thereby sensing at least one monochromatic
image of
the scene.
In accordance with the pioneering nature of the present invention, a method
for sensing an image of a scene from a single viewpoint is also provided. In
an
exemplary embodiment, the method includes the steps of:
(a) orthographically reflecting principal rays of electromagnetic
radiation radiating from the scene on a truncated, substantially paraboloid-
shaped
reflector such that the single viewpoint of the omnidirectional imaging method
coincides with a focus point of the paraboioid-shaped reflector;
(b) telecentrically filtering out a substantial portion of any principal
rays of electromagnetic radiation which are not orthographically reflected by
the
paraboloid-shaped reflector; and
(c) sensing the image of the scene by sensing the orthographically
reflected principal rays of electromagnetic radiation from the paraboloid-
shaped
reflector with one or more image sensors.
In a further exemplary embodiment, a method for omnidirectionally sensing
images of a scene from a single viewpoint is provided, which includes:
(a) mounting a truncated, substantially paraboloid-shaped reflector
on a fixed base;
{b) mounting one or more image sensors on a movable base;
(c) orthographically reflecting principal rays of electromagnetic

CA 02312970 2000-06-OS
WO 99130197 PCTNS98J25689
8
radiation radiating from the scene on the substantially paraboloid-shaped
reflector
such that the single viewpoint of the omnidirectional imaging method coincides
with
a focus point of the paraboloid-shaped reflector;
(d) telecentrically filtering out a s~~bstantial portion of any principal
rays of electromagnetic radiation which are not orthographically reflected
from the
paraboloidal-shaped reflector;
(e) moving the movable base to a first position;
(f) sensing a first image of the scene having a first field of view by
sensing the orthographically reflected principal rays of electromagnetic
radiation from
the paraboloidal-shaped reflector with the one or more image sensors;
(g) moving the movable base to a second position different from
the first position; and
(h) sensing a second image of the scene having a second field of
view by sensing the orthographically reflected principal rays of
electromagnetic
radiation from the paraboloidal-shaped reflector with the one or more image
sensors.
Alternatively, instead of mounting the paraboloid-shaped reflector on a fixed
base and mounting the image sensors on a movable base, the paraboloid-shaped
reflector may be mounted on a movable base and the image sensors may be
mounted
on a fixed base. Preferably, the above-described method also includes the step
of
optically coupling the paraboloid-shaped reflector and the image sensors with
a zoom
lens, which may be used to magnify an area of interest in the scene.
BRIEF DESCRIPTION OF THE DRAWINGS
Exemplary embodiments of the present invention will now be described in
detail with reference in the accompanying drawings in which:
Fig. 1 A is a side view of an exemplary embodiment of an omnidirectional
imaging apparatus;
Fig. 1 B is a side view of an alternate embodiment in which a paraboloid-
shaped reflector is connected to an image sensor by a transparent support;

CA 02312970 2000-06-OS
WO 99/3p197 PCT/US98/25689
9
Fig. 2 is an isometric view of a paraboloid-shaped reflector mounted on a base
plate;
Fig. 3 is a partially isometric view of a paraboloid-shaped reflector mapped
into a cylindrical coordinate system;
Fig. 4 is a geometric representation of an orthographic reflection from a
curved reflecting surface;
Fig. 5 is an illustration of orthographic reflection from a substantially
paraboloid-shaped reflector to an image sensor;
Fig. 6 illustrates how any selected portion of a hemispherical scene can be
viewed from the single viewpoint using a paraboloid-shaped reflector;
Fig. 7 is a side view of an omnidirectional imaging apparatus with two back-
to-back substantially paraboloid-shaped reflectors and two image sensors;
Fig. 8 is a cross-sectional view of two substantially paraboloid-shaped
reflectors positioned back-to-back and having a common paraboloidal axis and a
common focus;
Fig. 9 illustrates the mapping of image data to cylindrical coordinates to
enable production of a panoramic view;
Fig. 10 is a flowchart of an exemplary embodiment of a method for sensing
and processing an image of a substantially hemispherical scene from a single
viewpoint;
Fig. 11 is a side view of an embodiment of an omnidirectional imaging
apparatus according to the present invention, which includes an extended
paraboloid-
shaped reflector;
Fig. 12 is a side view of an embodiment of an omnidirectional imaging
apparatus according to the present invention, which includes a paraboloid-
shaped
reflector truncated at a plane that is tilted with respect to the paraboloidal
axis of the
reflector;
Fig. 13 is a side view of an embodiment of an omnidirectional imaging
apparatus according to the present invention, which includes a paraboloid-
shaped
reflector that is larger than the imaging area of the image sensor;

CA 02312970 2000-06-OS
WO 99/30197 PCTNS98I25689
Fig. 14 is a side view of an embodiment of an omnidirectional imaging
apparatus according to the present invention, which includes a concave
paraboloid-
shaped reflector;
Fig. 15 is a side view of an embodiment of an omnidirectional imaging
5 apparatus according to the present invention, which includes a zoom lens
optically
coupling a paraboloid-shaped reflector and an image sensor;
Fig. 16 is a partially isometric view of an embodiment of an omnidirectional
imaging apparatus according to the present invention, which includes a
paraboloid-
shaped reflector mounted on a movable base;
10 Fig. 17A is a side view of an embodiment of an omnidirectional imaging
apparatus according to the present invention, which includes an image sensor
mounted on a movable base;
Fig. 17B is a side view of an embodiment of an omnidirectional imaging
apparatus according to the present invention, which includes a movable camera;
Fig. 17C is a side view of an embodiment of an omnidirectional imaging
apparatus according to the present invention, which includes a movable camera
and
optics;
Fig. 18 is a partially isometric view of an embodiment of an omnidirectional
imaging apparatus according to the present invention, which includes an image
sensor
comprising four charge coupled devices positioned side-by-side;
Fig. 19 is a side view of an embodiment of an omnidirectional imaging
apparatus according to the present invention, which includes multiple image
sensors
and beam splitters;
Fig. 20 is a top view of an image sensor according to an embodiment of the
present invention whose sensing elements are non-uniformly distributed and
sized;
Fig. 21 is a side view of an embodiment of an omnidirectional imaging
apparatus according to the present invention, which includes a planar mirror
that
optically couples a paraboloid-shaped reflector and an image sensor;
Fig. 22 is a side view of an embodiment of an omnidirectional imaging
apparatus according to the present invention, which includes a microscope
objective

CA 02312970 2000-06-OS
WO 99/30197 PCT/US98/25689
11
that optically couples a paraboloid-shaped reflector and an image sensor;
Fig. 23 is a side view of an embodiment of an omnidirectional imaging
apparatus according to the present invention, which includes a collimator lens
that
optically couples a paraboloid-shaped reflector and an imaging lens;
Fig. 24A is a side view of an embodiment of an omnidirectional imaging
apparatus according to the present invention, which includes a field-
flattening plano-
concave lens;
Fig. 24B is a side view of an embodiment of an omnidirectional imaging
apparatus according to the present invention, which includes a field-
flattening
meniscus Lens with aplanatic sides; and
Fig. 25 is a side view of an embodiment of an omnidirectional imaging
apparatus according to the present invention, which includes two concave
paraboloid-
shaped mirrors used to image a substantially spherical view.
DETAILED DESCRIPTION
1 S Fig. 1 A illustrates an omnidirectional imaging apparatus 100 according to
an
exemplary embodiment of the present invention. A convex paraboloid shaped
reflector 135, which is mounted on a base plate 140, is positioned to
orthographically
reflect an image of a substantially hemispherical scene 130. An image sensor
110,
such as a commercially available Sony 3CCD color video camera device 111
having a
telecentric lens or a magnifying lens 112 and a telecentric aperture 113, is
positioned
to receive the orthographic reflection of the image. The telecentric Iens or
aperture
functions to filter out all rays of light which are not perpendicular to the
plane of the
lens or aperture, i.e., background light which does not form part of the
orthographic
reflection of the hemispherical scene.
Although the description herein is with regard to visible light, the present
invention has equal application to other forms of electromagnetic radiation
such as
ultraviolet light or infrared light.
In an alternate exemplary embodiment of the imaging apparatus 100 according
to the invention shown in Fig. 1 B, the paraboloid-shaped reflector may be
coupled to

CA 02312970 2000-06-OS
WO 9913019 PCTNS98125689
12
the image sensor by a transparent support 136, such as a length of clear
tubing.
Referring again to Fig. 1 A, the video camera 110 generates an analog video
signal representative of the orthographically reflected image which is sent
through
cable 150. The video signal is converted to a digital signal by digitizer 120,
which is
a commercially available NTSC video signal analog-to-digital converter.
The digital signal is then sent through a cable 155 to a general purpose
computer 125, such as a DEC Alpha 3000/600 workstation. As will be explained
in
further detail, the computer 125 is programmed to allow the user to view any
desired
portion of the hemispherical scene, to zoom in on a selected portion of the
scene, or to
pan the scene in any desired manner.
The image sensor 110 may simply be a still or motion picture photographic
camera using conventional photographic film. The image sensor 110 may also be
a
camcorder or video camera 116 which provides a digital video signal output,
which
can be provided directly to the computer 125 without the need for the analog-
to-
digital converter 120.
Fig. 2 shows an isometric view of the paraboloid-shaped reflector 135, which
extends from base 140 from which it is formed. The reflector 135 may comprise
a
paraboloid-shaped plastic body coated with a thin layer I45 of highly
reflective metal,
such as aluminum or silver. Alternatively, the reflector 135 may comprise a
paraboloid-shaped, polished metal body. For this latter embodiment, a metal
such as
stainless steel may be u~d.
Fig. 3 illustrates in greater detail, the preferred geometry of the paraboloid-
shaped reflector 135, as well as the orthographic reflection of the image of
the
substantially hemispherical scene 130 onto the image sensor 110. The reflector
135
of Fig. 3 is defined in cylindrical coordinates, r, ~ and z, as substantially
obeying the
equation
h z_r2
2h '

CA 02312970 2000-06-OS
WO 99/30197 PCTNS98r15689
13
where z is the axis of rotation, r is the radial coordinate, and h is a
constant. The z
axis coincides with the optical axis of the imaging arrangement, and a focus
paint 315
of the paraboloid defined by equation ( 1 ) coincides with the origin of the
coordinate
system. The reflector 135 of Fig. 3 is truncated at a plane p which is
substantially
perpendicular to the z axis 310 and which includes the focus point 315 of its
paraboloidal surface.
All incoming rays 305 that would otherwise pass through the focus point 315,
are orthographically reflected towards the image sensor 110 by the reflecting
paraboloidal surface. Thus, the focus point 315 is coincident with the single
viewpoint from which the substantially hemispherical scene 130 is viewed. The
image sensor 110 is positioned along the optical axis 310 of the imaging
system and
the photosensitive surface thereof is perpendicular to the optical axis. The
use of
orthographic reflection to enable viewing of a substantially hemispherical
scene from
a single viewpoint is an advantageous feature of the present invention.
That orthographic reflection enables viewing from a single viewpoint can be
demonstrated by reference to Fig. 4. In Fig. 4, z and r are perpendicular
cylindrical
coordinates for a given value of ~, the angular coordinate. The angle of an
incoming
ray 405 relative to the r axis is 8. The incoming ray 405 is orthographically
reflected
by the reflecting surface 415 as an outgoing ray 410.
To have a single viewpoint 420, any incoming ray must satisfy
tan (8) = z/r,
and for orthographic reflection, all rays must be reflected at an angle
a=n/2,
where a is the angle bet:veen the outgoing ray 410 and the axis. For these two
constraints to be satisfied, and for the angle of incidence to equal the angle
of
reflection, it is clear that the angle, (3, between the reflected ray 410 and
the normal
direction of the surface at the point of reflection, ri, must equal

CA 02312970 2000-06-OS
WO 99130197 PCTIUS98/25689
14
_ a-8 n-28
2 °r (3 4 ' - (4)
which can also be expressed as
tang-tan8 2tan(3
tan 2 (3 = - ,
1 +tana tan6 1-tan2(3
Finally, the slope of the reflecting surface 415 in the z-i plane at the point
of reflection
is
dz
z ~ _ - -tan(3.
dr
Substituting (6) and (4) into (5) yields
dz
-2 -
dr r
dz 2 z
1-
dr) dz
The quadratic expressi n of equation (7) can be solved to obtain two solutions
for - ,
dr
but to avoid self occlusion by the reflecting surface, ;he slope of the curve
in the right
quadrant is made negative (i.e., the surface is convex). The result is
dz z z 2
+ . (8)
dr r r
If a =z/r, the above expression reduces to
n
a+ 1+a2_'. (9)
r
where h is a constant of integration. Substituting z = ra into equation (9)
yields
equation ( 1 ).
Thus, there exists a curve, which when rotated about the z axis, generates a

CA 02312970 2000-06-OS
WO 99130197 PCTNS9g125689
surface that will allow orthographic reflection of a substantially
hemispherical scene
from a single viewpoint. This curve is the parabola defined by equation (1),
which
has a single viewpoint that is coincident with the focus 420 of the parabola.
In addition to providing viewing of a substantially hemispherical scene from a
S single viewpoint, the omnidirectional imaging apparatus in accordance with
the
present invention enables viewing of any portion of the scene, enables zooming
in on
a selected portion, and enables panning of the scene, all with respect to the
single
viewpoint and without requiring image reconstruction or complex frame
transformation.
10 Fig. 5 illustrates how a portion of the substantially hemispherical scene
is
viewed by the image sensor from a single viewpoint. Fig. 5 also illustrates
how a
truncated convex substantially paraboloid-shaped reflector 135 is mapped into
a
Cartesian coordinate system. The optical axis 502 of the imaging arrangement
is
coincident with the z axis, and the focus 501 of the substantially paraboloid-
shaped
15 reflector 135 is located at the origin. Incoming rays 505, 510 from a
portion of the
scene 300 being viewed intersects the reflecting surface at points 515 and
520, which
can be defined by their respective x and y coordinates. Point 515 and 520 lie
along
imaginary radial lines 516 and 521, respectively, which originate at the
viewpoint of
the scene, i.e., the focus 501 of the paraboloid-shaped reflector. Since these
rays are
20 orthographically reflected toward the image sensor 110, which has a planar
light-
sensitive surface perpendicular to the z-axis, the projected rays will
intersect the light-
sensitive surface at the same respective x and y coordinates. Only the z
coordinate
will change. Accordingly, there is a one-to-one correspondence between the x-y
coordinate of the point of intersection with the reflector 135 of the
orthographically
25 projected ray, and the x-y coordinate of the point at which that
orthographically
projected ray intersects the planar light-sensitive surface of the image
sensor L10.
In a preferred arrangement, the image sensor 110 includes a planar charge-
coupled device ("CCD") image sensor having an array of light sensing cells.
Each
cell senses the intensity of light at its particular location in the array.
Therefore, with
30 a one-to-one correspondence, the image signals produced by the CCD cells
which

CA 02312970 2000-06-OS
WO 99/30197 PCTNS98/25689
16
cover a particular range of x-y coordinates in the grid is representative of
the rays
which are orthographically reflected from the reflecting surface 135 at points
within
the same range of x-y coordinates. Thus, mapping of the image into a Cartesian-
coordinate system is a simple task for persons skilled in the art.
With the one-to-one correspondence explained above in mind, Fig. 6
illustrates a technique for zooming in on any selected portion of the
substantially
hemispherical scene. The reflector 135 is positioned relative to orthogonal x,
y and z
axes in the same manner as in Fig. 5. In order to zoom in at a focal distance
f on a
selected portion of the scene centered around a point 550, with a specified
size, only
the image signals of the CCD cells located with the same range of x-y
coordinates as
the region of the reflecting surface projecting the selected portion of the
scene are
selected for magnification and viewing.
More particularly, to determine the appropriate light intensity for point 570
in
the selected portion of the scene, the light intensity signal generated by the
CCD cell
which lies at 580 is chosen. As shown in Fig. 6, a line segment drawn between
point
570 and focus point S51 intersects the reflector 135 at point 552. The light
intensity
at point 570 is set equal to that represented by the image signal generated by
the CCD
cell at 580 which is located at the x-y coordinate on the grid nearest to the
x-y
coordinate of point 552. The same is repeated for each CCD cell within the
same
range of x-y coordinates as the region of the reflecting surface projecting
the selected
portion of the scene. As a result of the orthographic reflection and the one-
to-one
correspondence described above, no image reconstruction or complex frame
transformation is required.
A general purpose computer 125 can be readily programmed by one skilled in
the art to perform the above steps to enable viewing of any portion of the
hemispherical scene from a single viewpoint, and to also enable zooming in on
any
particular portion to provide an enlarged image of that portion. Furthermore,
by
designating successive points along the reflector, the hemispherical scene can
be
panned as if one were viewing the scene from a single viewpoint.

CA 02312970 2000-06-OS
WO 99/30197 PCTNS9$/25689
17
In the embodiment discussed above, it is readily apparent that as one zooms in
on smaller portions of the scene, the number of CCD cells providing
information to
the computer 125 is reduced, and hence the granularity of the viewed image is
increased. In a preferred embodiment, information about points in the scene
which do
not exactly correspond to CCD cells are more closely approximated by
interpolation.
A suitable interpolation program which may be executed on computer 125 is
included
in Appendix I to this specification. The program attached as Appendix I will
map the
sensed omnidirectional image to an ordinary perspective image that is suitable
for
display on computer 125. The program requires the user to input the name,
center
location, and radius of the omnidirectional image to be converted. The program
also
requires the user to input a name for the generated perspective image, as well
as a
focal length and size for the perspective image.
Thus, instead of simply choosing the image signal generated by the nearest
CCD cell to represent portions of the image which do not precisely correspond
to a
1 S CCD cell, the image for such scene portions is estimated by the appended
program
based on a suitable average of image signals generated by CCD cells which
correspond to neighboring portions of the scene. Of course, more sophisticated
interpolation programs known to those skilled in the art, such as those that
are based
on polynomial or temporal matching, may be used without departing from the
scope
of the invention, as defined by the claims herein.
In addition to the Cartesian-coordinate mapping that has been described,
which produces a perspective image, a cylindrical-coordinate mapping may also
be
performed to achieve a panoramic image of the scene being viewed. Cylindrical-
coordinate mapping will be described with reference to Fig. 9. In Fig. 9, a
principal
ray 950 from a point 945 in a scene strikes a paraboloid-shaped reflector 935
and is
orthographically reflected toward an image sensor 910. The orthographically
reflected ray 960 impinges the image sensor at sensing element 965. To map the
point represented by sensing element 965 into cylindrical coordinates, a
truncated
cylinder 970 is imagined to surround the paraboloid-shaped reflector 935 and
image
sensor 910. The point represented by sensing element 965 is then traced back
through

CA 02312970 2000-06-OS
WO 99/30197 PCTIUS98/25689
18
rays 960 and 950, and the point of intersection 955 of the ray 950 with the
truncated
cylinder 970 is determined. Point 955 is then assigned the light intensity of
sensing
element 965. This same calculation is performed for each sensing element of
the
image sensor 910. The resultant collection of points (with appropriately
assigned
light intensities) located on the truncated cylinder 970 produces a panoramic
image of
the scene being viewed. This panoramic image can be viewed on a display by
further
mapping the truncated cylinder to a planar surface. This mapping is easily
performed
by those skilled in the art and can be visualized by imagining that the
cylinder is cut
length-wise and flattened out. Moreover, as those skilled in the art will
readily
appreciate, interpolation of image data as discussed above in relation to
Cartesian-
coordinate mapping may also be used with cylindrical-coordinate mapping.
In a preferred embodiment of the present invention, a ~/a inch CCD is used
with a 0.4 inch focal length paraboloid-shaped mirror truncated through its
focus and
having a 1.6 inch diameter. A collimating lens, such as Model No. P32921 from
EDMUND SCIENTIFIC of Barrington, New Jersey, is used with an 8.5 inch focal-
length imaging lens to optically couple the mirror to the CCD.
In a further exemplary embodiment of the invention, the omnidirectional
imaging apparatus includes an additional substantially paraboloid-shaped
reflector
735 as shown in Fig. 7. The additional reflector is positioned to
orthographically
project an image of an additional hemispherical scene 730 which is
complementary to
the hemispherical scene 130 so that together they constitute a spherical
scene. An
additional image sensor 710 is positioned to receive the image
orthographically
projected by the additional reflector 735.
An image signal representative of the orthographic reflection of the
additional
reflector 735 is converted to a digital signal by converter 720 in the same
manner as
described above, and is sent to the same general purpose computer 125 via line
725.
As shown in Fig. 8, the reflectors 135 and 735 are positioned back-to-back,
share a common axis of rotation 810, which is also the optical axis of the
imaging
apparatus, and a common focus 805, and are each truncated at a plane p which
is

CA 02312970 2000-06-OS
WO 99/30197 PCT/US98I25689
19
substantially perpendicular to the axis of rotation 8 i 0 and which includes
the focus
805.
Referring to Fig. 10, there is shown a flow chart 1000 illustrating a method
for
sensing an image of a substantially hemispherical or spherical scene from a
single
viewpoint according to an exemplary embodiment of the present invention.
Flowchart 1000 shows the necessary steps for sensing the hemispherical scene
from a
single viewpoint. The method requires orthographically reflecting the
substantially
hemispherical scene 1010, and sensing the orthographically reflected image
1020.
The method may further include the steps of converting the image signal into
image signal data 1030, mapping the image data into an appropriate coordinate
system 1040, interpolating the image data 1060 to derive approximate values
for
missing image data, and forming a digital image 1070 from the mapped image
data
and from the interpolated image data. Advantageously the steps of specifying a
viewing direction, a focal length, and an image size 1045 and zooming in 1050
on a
selected portion of the image data may be performed before the interpolation
step.
Thus far, the exemplary embodiments described have all utilized a "normal"
paraboloid-shaped reflector. As used in this specification and the appended
claims,
the term "normal" in association with a paraboloid-shaped reflector refers to
a
paraboloid-shaped reflector that is truncated at a plane that passes through
the focal
point of the paraboloid-shaped reflector and that is substantially
perpendicular to the
paraboloidal axis of the paraboloid-shaped reflector. As used in this
specification and
the appended claims, the paraboloidal axis of a paraboloid-shaped reflector is
the axis
passing through the vertex and focal point of the paraboloid-shaped reflector.
As
described above, using a normal paraboloid-shaped reflector, one can image an
entire
hemisphere (n steradians), or by placing two such reflectors back-to-back, a
complete
sphere (2~ steradians). Figs. 11 to 1 S show further exemplary embodiments of
the
omnidirectional imaging apparatus in which the paraboloid-shaped reflector may
also
take the form of various non-normal paraboloids.
Fig. 11 shows an omnidirectional imaging apparatus that is able to image a
field of view ("FOV") greater than a hemisphere using only one camera 1111 and
one

CA 02312970 2000-06-OS
WO 99/3019 PCTNS98/Z5689
paraboloid-shaped reflector 1 I35. In the embodiment of Fig. 11, the
paraboloid-
shaped reflector 1135 is an extended paraboloid that is obtained by cutting a
suitable
reflector with a plane that is normal to the axis of the paraboloid (z) but
passes below
the focal point 1130 of the paraboloid. Advantageously, because the paraboioid
5 extends below its focal point, the paraboloid-shaped reflector is able to
orthographically reflect rays from the hemisphere below its focal point. In
the
embodiment illustrated in Fig. 1 l, for example, the FOV covered by the
paraboloid-
shaped reflector is 240 degrees, or 75% of an entire sphere. Preferably, as
shown in
Fig. 11, the camera 1111 and the paraboloid-shaped reflector 1135 are coupled
by
10 optics 1112.
Fig. 12 shows an omnidirectional imaging apparatus that may be used to
image a FOV that is tilted with respect to the paraboloidal axis of the
paraboloid-
shaped reflector. The embodiment of Fig. I2 includes a camera 1211, optics
1212,
and a paraboloid-shaped reflector 1235. The paraboloid-shaped reflector 1235
is
15 truncated at a plane passing through the focus of the paraboloid-shaped
reflector 1235
and tilted with respect to its paraboloidal axis (z). The FOV of this
reflector is thus a
tilted hemisphere, as shown by the dotted lines in Fig. 12. Although the
embodiment
in Fig. 12 shows the truncation plane passing through focus of the paraboloid,
the
invention is not limited to this embodiment. The truncation plane may also
pass
20 above the focus 1230 of the paraboloid (thereby resulting in a FOV smaller
than a
hemisphere), or the truncation plane may pass below the focus 1230 (thereby
resulting
in a FOV greater than a hemisphere).
Fig. 13 shows an omnidirectional imaging apparatus that may be used to
image a FOV smaller than a hemisphere. The embodiment of Fig. I 3 includes a
camera 1311 coupled to a paraboloid-shaped reflector 1335 by optics 1312. In
this
embodiment, the paraboloid-shaped reflector 1335 is formed such that it is
"larger"
than the imaging area of the camera 1311. In this context, a paraboloid-shaped
reflector is "larger" than the imaging area of a camera if the base of a
normal
paraboloid having the same shape as the reflector (i.e., having the same
paraboloidal
constant h as defined in equation (1)) is larger than the smallest dimension
of the

CA 02312970 2000-06-OS
WO 99/30197 PCTNS98I25689
21
imaging area of the camera. Taking the case of a normal paraboloid for
illustrative
purposes, it is clear that when such a paraboloid is larger than the
imagingarea of a
camera, only a FOV smaller than a full hemisphere is capable of being captured
in the
imaging area of the camera because the orthographically reflected rays at the
outer
edges of the paraboloid will not impinge on the imaging area. Advantageously,
however, the image captured using such a paraboloid-shaped mirror has a higher
resolution than a corresponding image captured using a smaller paraboloid. As
shown
in Fig. 13, the paraboloidal axis of the paraboloid-shaped reflector 1335 (z')
may be
shifted with respect to the optical axis (z) to obtain fields of view towards
the horizon.
In addition, the paraboloid-shaped reflector 1335 need not be a normal
paraboloid, but
may be truncated in accordance with the FOV to be imaged.
Thus far, all of the embodiments discussed have comprised a convex
paraboloid-shaped reflector. In Fig. 14, an embodiment of an omnidirectional
imaging apparatus according to the present invention is shown that includes a
camera
1411, optics 1412, and a concave paraboloid-shaped reflector 1435. A concave
paraboloid-shaped reflector may be used in applications where the concealment
of the
reflector is desirable (as, for example, in outdoor applications where
protection
against weather is desirable). In the case of a concave paraboloid-shaped
reflector, the
paraboloidal image of the scene is "flipped," but the image continues to
satisfy the
single viewpoint constraint disclosed previously. Therefore, pure perspective
images
can be generated from the concave paraboloidal image, just as with the convex
paraboloidal image. In the case of the concave paraboloid, however, at most a
hemispherical field of view can be obtained with a single reflector. This
hemispherical FOV is obtained by truncating the paraboloid with a plane that
passes
through the focal point 1435 of the paraboloid (the plane being either normal
or tilted
with respect to the axis of the paraboloid (z)). Although a concave paraboloid
that is
truncated above its focal point may also be used, such a paraboloid is not
desirable
because it causes self obstruction of the image.
As shown in Fig. 25, a FOV greater than a hemisphere may be obtained by
using multiple concave paraboloid-shaped reflectors. In Fig. 25, two
paraboloid-

CA 02312970 2000-06-OS
WO 99/30197 PCTNS98r15689
22
shaped reflectors, 2535a and 2535b, are positioned such that they share a
common
paraboloidal axis (z) and their vertexes 2545 coincide. Together with image
sensors
251 la and 2511b, the two paraboloid-shaped reflectors 2535a and 2535b are
able to
image two hemispheres 2530a and 2530b, respectively. This system may be used
advantageously when the reflectors are required to be recessed for concealment
or
protection. A disadvantage to using concave mirrors in this arrangement,
instead of
using convex mirrors in the arrangement of Fig. 7, is that a small blindspot,
comprising the area between the truncation planes of the two reflectors, is
inevitable.
Fig. 1 S shows an embodiment of an omnidirectional imaging system
according to the present invention with zoom capabilities. The omnidirectional
imaging system of Fig. 15 includes a paraboloid-shaped reflector 1535, a
camera
1511, a zoom lens 1512, and relay optics 1513. (As used in this specification
and the
appended claims, relay optics and collimating optics are synonymous.) With the
zoom lens 1512 set to its lowest power, the omnidirectional imaging system
provides
an image of the entire hemisphere (or greater or less than a hemisphere if the
embodiments of Fig. 11 or Fig. 13 are used). When zoomed in, the zoom lens 1 S
I2
provides a higher magnification (and, therefore, a higher resolution) of a
smaller
FOV. While zooming in, the effective center of projection of the zoom lens
1512
must remain approximately fixed to ensure that the imaging system remains
telecentric. Preferably, the relay optics 1513 is used to ensure that the zoom
lens 1512
remains telecentric over its entire settings.
In the embodiment of Fig. 15, the zoom lens 1512 may be either fixed or
mobile with respect to the paraboloid-shaped reflector 1535. If the zoom lens
1512 is
fixed, only regions around the paraboloidal axis (z) can be observed under
magnification. Preferably, therefore, the zoom lens 1512 is equipped with some
movement means, allowing the zoom lens to be positioned over and image regions
along the outer edges of the paraboloid-shaped reflector 1535. Of course, such
movement means must ensure that the optical axis of the zoom lens 1 S 12
remains
parallel to the paraboloidal axis of the paraboloid-shaped reflector 1535 at
all times.
Fig. 16 shows an omnidirectional imaging system that may be used to produce

CA 02312970 2000-06-OS
WO 99/30197 PCTIUS98/15689
23
dynamically changing fields of view of a scene. A paraboloid-shaped reflector
1635
is mounted on a movable base 1640, which allows translation of the paraboloid-
shaped reflector 1635 along the x, y and z axes. T'he movable base 1640 may be
controlled either manually or with a computer. Using the movable base 1640, a
dynamically changing field of view of a scene could be produced, for example,
by a
circular motion of the movable base 1640 about the optical axis (z).
Preferably, the
images are image processed as described previously to obtain perspective or
panoramic views.
Fig. 16 further shows the use of a zoom lens 1612 in combination with the
movable base I 640. A zoom lens 1612 adds the capability to zoom into sections
of
the paraboloid-shaped reflector 1635 brought under the view of the imaging
system
by the movement of the movable base 1640. Preferably, a relay lens 1613 is
used to
couple the zoom lens and the paraboloid-shaped reflector 1635. In addition,
the zoom
lens 1612 preferably includes a manual or automatic focus control to ensure
that the
sharpness of images are maintained over all sections of the paraboloid-shaped
reflector 1635. Alternatively, translation of the reflector along the z axis
may also be
used to adjust the focus of an image.
Instead of moving the paraboloid-shaped reflector as in the embodiment of
Fig. 16, one or more parts of the camera or optics of the imaging system may
alternatively be moved to achieve the same effect as the Fig. 16 embodiment.
Figs.
17A, 17B, and 17C show various exemplary embodiments of such omnidirectional
imaging systems. In Fig. 17A, an image sensor 1710 (such as a CCD) is provided
with movable means; in Fig. 17B, a camera 1711 is provided with movable means;
and in Fig. 17C, both a camera 171 l and optics 1712 are provided with movable
means, for moving together simultaneously. As shown in the figures, each of
these
components may be moved along any of the x, y, or z axes to change the field
of view
being imaged. As in the embodiment of Fig. 16, a zoom lens may be used to
magnify
areas of interest. Advantageously, by moving the camera or optics instead of
moving
the paraboloid-shaped reflector, the viewpoint of the omnidirectional imaging
system
remains fixed in space at the focal point of the paraboloid-shaped reflector.

CA 02312970 2000-06-OS
WO 99I30I97 PCTNS98I25689
24
The embodiments of Fig. 16, 17A, 17B, or I7C may be used to great
advantage in a surveillance system. The omnidirectional imaging capability of
these
embodiments allows an operator to monitor an entire area of interest at once.
When
the operator observes a particular region of interest within the area being
monitored,
the operator may then select appropriate translational coordinates (for
movement of
the camera, optics, or paraboloid-shaped reflector), and appropriate zoom
settings, to
view the region of interest in greater detail.
Fig. 18 shows an omnidirectional imaging system that utilizes multiple image
sensors to achieve increased image resolution. The embodiment of Fig. 18
includes a
paraboloid-shaped reflector 1835, video electronics 1809, four CCD elements
1810a -
1810d, and imaging optics 1812. In this embodiment, the four CCD elements
1810a -
1810d are placed side-by-side in a non-overlapping arrangement. The embodiment
of
Fig. 18 takes advantage of the fact that commercial CCD elements are typically
manufactured in standard resolutions regardless of their size. Therefore, by
using
four, commercial '/4-inch CCD elements instead of a single, commercial %2-inch
CCD
element, the resolution of an image may advantageously be quadrupled. Although
Fig. 18 shows the use of CCD elements placed in a non-overlapping arrangement,
the
invention described here is not limited to such an arrangement. Thus, an
arrangement
where multiple CCD elements partially overlap can likewise be used. Moreover,
multiple image sensors may be fabricated into a single integrated circuit,
with each
image sensor connected to its own video circuitry.
Fig. 19 shows another embodiment that utilizes multiple image sensors to
increase image resolution. In this instance, the multiple image sensors are
provided
by multiple cameras 1911. Beam splitters 1916 are used to direct separate
sections of
a paraboloidal image to different cameras. Advantageously, therefore, each
portion of
the paraboloidal image is imaged with a higher resolution than if the entire
image
were imaged by one camera alone.
In another exemplary embodiment of the present invention, dichroic beam
splitters (not shown) may be used to split an image into a plurality of
monochromatic
images, which may be sensed by a plurality of monochromatic image detectors.

CA 02312970 2000-06-OS
WO 99/30197 PCT/US98125689
These monochromatic ir.~.zages may later be suitably combined into a full-
color image
by image processing means well-known in the art.
Fig. 20 shows a planar image sensor 2010, as, for example, a CCD element.
Using a typical planar image sensor with a paraboloid-shaped mirror, the
effective
5 resolution of a captured paraboloidal image is increasingly greater towards
the outer
edge of an image than at its center. For example, when a planar image sensor
is used
to capture an image reflected by a normal paraboloid-shaped reflector, the
resolution
of the captured image increases by a factor of four from the center of the
image to its
fringe. To compensate for this effect, an image sensor has sensing elements
2008
10 whose sizes and placements are varied to result in a uniform resolution
over the entire
image. This same approach may also be used to increase resolution in selected
parts
of the FOV. When specific resolution variations are difficult to incorporate,
standard
resolution variations, such as those provided by log-polar sensors, may also
be used.
One or more planar mirrors may be included in an omnidirectional imaging
15 apparatus according to the present invention for flexibility of placement
of the optics
and reflector. Fig. 21 shows a preferred embodiment in which an
omnidirectional
imaging system includes a paraboloid-shaped reflector 2135, a planar mirror
2116, a
relay lens 2113, an imaging lens 2112, and a camera 2111. In the embodiment
shown, the paraboloid-shaped reflector 2135 is positioned above a surface
2140, and
20 the planar mirror 2116, relay lens 2113, imaging lens 2112, and camera 2111
are
concealed below the surface 2140. The planar mirror 2116 is positioned beneath
an
opening 2145 in the surface 2140 and folds the image from the paraboloid-
shaped
reflector 90 degrees, thereby redirecting the image to the relay lens, imaging
lens, and
camera. Although the planar mirror is shown between the paraboloid-shaped
reflector
25 and the relay lens, the planar mirror may also be placed between the relay
lens and the
imaging lens or between the imaging lens and the camera, as those skilled in
the art
will appreciate.
Fig. 22 shows an embodiment of an omnidirectional imaging system in which
the optics between a paraboloid-shaped mirror 2235 and an image sensor 2210
comprise a low-power, inverted microscope objective 2212. In this embodiment,
the

CA 02312970 2000-06-OS
WO 99/30197 PCT/US98IZ5689
26
reflector 2235 is at the position normally occupied by the eyepiece of the
microscope
and the image sensor 2210 is at the position normally occupied by a slide. The
use of
an inverted microscope objective is advantageous for imaging since commercial
microscope objectives are well corrected for aberrations.
Fig. 23 shows an embodiment of an omnidirectional imaging system in which
a collimator lens 2313 is placed between a paraboloid-shaped mirror 2335 and
imaging optics 2312. It is desirable to use commercially available imaging
lens in
many cases to save the cost and time of designing special lenses. Most
commercial
imaging lenses, however, are intended to image scenes that are very far from
the lens.
Indeed, they are normally designed for objects that are infinitely distant
from lens.
Therefore, when used to image objects that are close to the lens, the image
suffers
from various types of aberrations which degrade the effective resolution of
the lens.
The result is a "fuzzy" or smeared out image. In this embodiment, this problem
is
solved by the use of a collimating lens 2313, which produces a virtual object
at
infinity for the imaging optics 2312. Advantageously, therefore, the use of a
collimating lens 2313 allows the use of commercially available imaging lenses.
The embodiments of Figs. 24A and 24B illustrate the use of field-flattening
lenses between an image sensor 2410 and an imaging lens 2413. Field-flattening
means are desirable because the paraboloid-shaped reflector of the present
invention,
having a typically small focal length of a few millimeters, is afflicted with
very strong
field curvature. One method of eliminating this imaging defect is to use an
image
sensor with a curved surface that matches the field curvature. More
preferably,
however, a special lens, called a field-flattening lens, may be introduced
which has a
curvature of opposite sign to that of the reflector. Therefore, the two field
curvatures
cancel, and the resultant image surface is flat, allowing the entire image to
be in sharp
focus on a planar image sensor.
Two types of preferred field-flattening lenses are illustrated in Figs. 24A
and
24B. In Fig. 24A, a piano-concave lens 2412a is shown. The piano-concave lens
2412a is placed as close as possible to the image sensor 2410. Preferably, the
plano-
concave lens 2412a is placed in contact with the image sensor window 2417. In
this

CA 02312970 2000-06-OS
WO 99130197 PCTNS98/25689
27
position, the plano-concave lens 2412a compensates for the field curvature of
the
reflector while introducing only small amounts of undesirable aberrations. '
A second type of preferred field-flattening lens, a meniscus lens 2412b, is
shown in Fig. 24B. Both of the surfaces of the meniscus lens 2412b are
aplanatic to
the incoming light. If a surface is aplanatic, it introduces no spherical
aberration,
coma or astigmatism into the beam of light; it only introduces field
curvature. The
meniscus lens 2412b has a marked field flattening effect which is determined
by the
thickness of the lens: the thicker the lens, the greater the field flattening
effect. In
contrast to the plano-concave lens 2412a of Fig. 24A, the meniscus lens 2412b
of Fig.
24B is not used in contact with the image sensor 2410.
The theory of the field-flattening lenses will now be explained. Ideally, the
surface of best focus of an optical system is a plane. With a planar surface,
a CCD or
other type of flat image sensor can match the surface of best focus over its
entire area,
thereby providing maximum resolution for the image. Unfortunately, an optical
system has a tendency to form its best images on a curved surface.
Accordingly, the
curved focal surface and the flat CCD surface cannot be matched up over their
entire
area, and some or all of the image will not be in best focus.
The field curvature of an optical system is called its Petzval curvature.
Every
optical element in an optical system contributes to the Petzval curvature for
the
system. If a surface of an optical element is refracting, its Petzval
contribution to the
curvature of the system is:
1-n
nR
where n is the refractive index of the optical element and R is the radius of
curvature
of the surface of the optical element. Clearly, the Petzval contribution of a
surface
depends on the sign of the radius. If the surface is a mirror instead of a
refracting
surface, its Petzval contribution is:

CA 02312970 2000-06-OS
WO 99/30197 PCT/US98/25689
28
2
p=__
R
The field curvature of an image is calculated by taking the sum of the
contributions of
all of the reflecting and refracting surfaces and multiplying the sum by a
simple
constant. If this value is not zero, then the field of the image is curved and
the
problem discussed above will be encountered (i.e., the surface of the image
and the
surface of the image sensor will not be completely matched).
Unfortunately, the curvatures of optical surfaces cannot be eliminated
because they are necessary for other purposes, such as for controlling
spherical
aberration, coma and astigmatism. Because the control of these aberrations
depends
on the curvatures of optical elements, if the curvature of these elements is
changed,
these aberrations may be adversely effected. There are two ways, however, in
which
the Petzval curvature of an optical system may be changed without changing the
other
aberrations of a system. These two methods form the basis for the two types of
field-
flattening lenses described above.
The first method for changing the Petzval curvature depends on the
optical characteristics of an optical surface located at the surface of an
image. If an
optical surface is located at the surface of an image (either an intermediate
image or
the final image of the optical system), then this surface will not change the
spherical
aberration, coma or astigmatism of the image. The only change will be to the
Petzval
curvature. Thus, the Petzval curvature of a system can be corrected by
inserting a
surface with an appropriate radius of curvature at the final focus of the
system. This
is the basis for the plano-concave field-flattening lens described above.
The second method for changing the Petzval curvature depends on the
optical characteristics of aplanatic surfaces. Assume there is an aplanatic
surface,
which is defined as follows: Let s be the object distance for the surface and
s' be the
image distance. Also, let n and n' be the refractive indices of the materials
before and
after the surface, respectively (where n=1 for air and n>1 for glass). If s
and s' are
related by

CA 02312970 2000-06-OS
WO 99/30197 PCT/US98125689
29
R(n~+n) ns
s = i -i.
n n
then the surface will introduce no spherical aberration or coma and only very
small
amounts of astigmatism. If now a thick lens is introduced, both of whose
surfaces
satisfy this condition, then the difference in their radii will depend on the
thickness of
the lens. This fact can main be used to control the Petzval curvature of the
system by
adjusting the thickness of the aplanatic lens. This is the basis of the thick,
meniscus
field-flattening lens discussed above.
In a preferred embodiment of the piano-concave lens 2412a of Fig. 24A, the
piano-concave lens is composed of BK7 and has a refi~active index (n) of
1.517. The
radius of the curved (concave) surface r, is 6.2 mm. The surface opposite the
curved
surface r, is flat and is placed in contact with the image detector window
2417. The
axial thickness of the lens is 1.5 mm, and the optical diameter is 3 mm.
In a preferred embodiment of the aplanatic lens 2412b of Fig. 24B, the
aplariatic lens is composed of acrylic plastic and has a refractive index (n)
of 1.494.
The radius of the curved (convex) surface r2 is 4.78 mm, and the radius of the
curved
(concave) surface r3 is 2.16 mm. The axial thickness of the lens is 6.7 mm.
The
optical diameter of the curved surface rz is 7 mm, and the optical diameter of
the
curved surface r3 is 2.7 mm.
Although the present invention has been described with reference to certain
preferred embodiments, various modifications, alterations, and substitutions
will be
known or obvious to those skilled in the art without departing from the spirit
and
scope of the invention, as defined by the appended claims.

CA 02312970 2000-06-OS
WO 99/30197 PCT/US98125689
APPENDIX I
compute image.c
#include "stdlib.h"
#include "imageutil.h"
5 #include "stdio.h"
#include "math.h"
/* int main(int argc, char **argv) */
main(argc,argv)
int argc;
10 char *argv~];
double sqrt{), atanp, sing, cos(), acosp;
unsigned char *r, *g, *b;
unsigned char *red;
15 unsigned char *green;
unsigned char *blue;
int xsize, ysize;
int xosize, yosize;
int i, j, x0, y0, x 1, y 1;
20 double theta, phi;
double ox, oy, oz;
double px, py, pz;
double qx, qy, qz;
double tempx, tempy, tempz;
25 double sx, sy, sz;
double rad, mag;

CA 02312970 2000-06-OS
WO 99/30197 PCTIUS98I25689
31
double xs, ys, zs;
double dispx, dispy;
int xcent, ycent, anew, ynew, xpix, ypix, xpoint, ypoint;
int xpixel, ypixel, indexx, indexy, xcenter, ycenter;
float radius, focal;
I* printf("completed initializationsln~n"); *I
if(argc != 4) {
printf("arguments: xcenter, ycenter, radiusln");
exit(0);
}
printf("1n");
xcent = atoi(argv[1 ]);
ycent = atoi(argv[2]);
radius = atof(argv[3]);
printf("omni-image: xcenter = %d ycenter = %d radius = %fln~n",
xcent, ycent, (float)radius);
printf("input view pixel [xnovel ynovel]: ");
scarf("%d %d", &xnew, &ynew);
printf("fin");
printf("selected view pixel: xnew = %d ynew = %dln~n", xnew, ynew);
printf("input new image parameters [xpixels ypixels focal]: ");
scarf("%d %d %f', &xpix, &ypix, &focal);

CA 02312970 2000-06-OS
WO 99130197 PCTNS98I25689
32
prin~~~~«).
printf("output image: xpixels = %d ypixels = %d focal = %flnln",
xpix, ypix, (float)focal);
loadPPM{"test.ppm", &r, &g, 8cb, &xsize, &ysize); .
printf("loaded omni-image filelnln");
xosize = xpix;
yosize = ypix;
/* printf("set new img size, xsize = %d, ysize = %d lnln", xosize, yosize); */
red = (unsigned char*)malloc(xosize * yosize * sizeof(unsigned char));
green = (unsigned char*)malloc(xosize * yosize * sizeof(unsigned char));
blue = (unsigned char*)malloc(xosize * yosize * sizeof(unsigned char));
printf("allocated memory for new image filelnln");
xcenter = xcent;
ycenter = ycent;
xpoint = ynew - ycent;
ypoint = xnew - xcent;
tempx = (double)xpoint;
tempy = {double)ypoint;
tempz = (radius*radius - (tempx*tempx + tempy*tempy))/(2*radius);

CA 02312970 2000-06-OS
WO 99130197 PCTNS98I25689
33
ox = tempx/sqrt(tempx*tempx + tempy*tempy + tempz*tempz);
oy = tempy/sqrt(tempx*tempx + tempy*tempy + tempz*tempz);
oz = tempz/sqrt(tempxy~tempx + tempy*tempy + tempz*tempz);
I* computed optical (z) axis *I
tempx = -oy;
tempy = ox;
tempz = 0;
px = tempx/sqrt(tempx*tempx + tempy*tempy + tempz*tempz);
py = tempy/sqrt(tempx*tempx + tempy*tempy + tempz*tempz);
pz = tempz/sqrt(tempx''tempx + tempy*tempy + tempz*tempz);
I* computed horizontal axis *I
tempx = py*oz - pz*oy;
tempy = pz*ox - px*oz;
tempz = px*oy - py*ox;
qx = tempx/sqrt(tempx*tempx + tempy*tempy + tempz*tempz);
qy = tempy/sqrt(tempx*tempx + tempy*tempy + tempz*tempz);
qz = tempzlsqrt(tempx''tempx + tempy*tempy + tempz*tempz);
/* computed vertical axis */
printf("computed perspective image framelnln");
/* raster scan perspective image plane */

CA 02312970 2000-06-OS
WO 99/30197 PCTIUS98/Z5689
34
for(i=O;i<ypix;i++) {
dispy = (double)i - (double)ypix/2;
for(j=O;j <xpix; j++) {
dispx = (double)xpix/2 - (double)j;
sx = ox * focal + px * dispx + qx * dispy;
sy = oy * focal + py * dispx + qy * dispy;
sz = oz * focal + pz * dispx + qz * dispy;
mag = sqrt(sx*sx + sy*sy + sz*sz);
sx = sx/mag;
sy = sy/mag;
sz = sz/mag;
/* computed vector in direction of current pixel */
phi = atan2(sy,sx);
theta = acos(s~Jsqrt(sx*sx + sy*sy + sz*sz));
/* converted vector to polar coordinates */
rad = 2*radius*(1-cos(theta))/(1-cos(2*theta));
/* found radius of intersection on parabola */
xs = rad*sin(theta)*cos(phi);
ys = rad*sin(theta)*sin(phi);
zs = rad*cos(theta);

CA 02312970 2000-06-OS
WO 99/30197 PCT/US98/Z5689
/* found x, y, z coordinates on paraboloid */
I* printf("xs = %f ys = %f zs = %flnln", (float)xs, (float)ys, (float)zs); *I
/* use xs,ys to read from input image and save in output image */
/* check if image point lies outside parabolic image */
S
if(sqrt(xs*xs + ys*ys) > radius){
red[i * xpix + j] = 255;
green[i * xpix + j~ = 255;
10 blue[i * xpix + j] = 255;
)
else{
indexx = (int)ys + xcenter;
indexy = (int)xs + ycenter;
IS
/* printf("one pixe11n1n"); */
/* write closest color value into pixel */
red[i * xpix + j] = r[indexy * xsize + indexx];
20 green[i * xpix + j] = g[indexy * xsize + indexx];
blue[i * xpix + j] = b[indexy * xsize + indexx];

CA 02312970 2000-06-OS
WO 99/30197 PC"fIUS98I25689
36
}
}
printf("computed perspective imagelnln");
savePPM("out.ppm", red, green, blue, xpix, ypix);
printf("saved new image filelnln");
system("xv out.ppm &");
free(r);
free(g);
free(b);
free(red);
free(green);
free(blue);
printf("freed allocated memorylnln");
return 0;
}

Dessin représentatif