Note: Descriptions are shown in the official language in which they were submitted.
CA 02647956 2013-10-09
SPORTS SIMULATION SYSTEM
Related Applications
[0001] This application is related to U.S. Patent Application Serial
No.
10/629,945 filed on July 30, 2003 for an invention entitled "Sports Simulation
System" and to U.S. Patent Application Serial No. 11/195,017 filed on August
2,
2005 for an invention entitled "Sports Simulation System".
Field of the Invention
[0002] The present invention relates generally to entertainment
systems and in
particular to a sports simulation system.
Background of the Invention
[0003] Sports simulation systems designed to simulate sports
experiences are
well known in the art. In many conventional sports simulation systems, a
player
propels a sports projectile such as a ball, puck, arrow, dart, etc. at a
target image
presented on a display screen. The motion of the sports projectile is detected
and
imaged and an extrapolation of the trajectory of the sports projectile is
made. The
extrapolated trajectory is then used to determine a sports result. The
displayed image
is in turn updated to reflect the sports result thereby to provide the player
with visual
feedback and simulate a sports experience.
100041 The goal of all sports simulation systems is to provide the player
with a
realistic sports experience. As a result, many variations of sports simulation
systems
have been considered in attempts to simulate accurately "real-life" sports
experiences.
For example, U.S. Patent No. 5,333,874 to Arnold et al. discloses a sports
simulator
having a housing and two arrays of infrared (IR) receivers and emitters
positioned in
the housing. A launch area is established near one end of the housing. A user
can
launch an object such as a golf ball located in the launch area and drive the
golf ball
into the housing through the planes defined by the arrays of IR emitters and
against a
screen positioned at one end of the housing. A computer is connected to the IR
receivers, which detect the passage of the object through the respective
planes. Based
upon the signals from the IR receivers, the computer uses triangulation
techniques to
determine the horizontal and vertical position, as well as the velocity of the
golf ball.
CA 02647956 2008-09-30
WO 2007/112560 PCT/CA2007/000517
-2-
The computer can also determine the spin of the golf ball and cause an image
of the
golf ball as it would have appeared traveling away from the golfer had it not
encountered the screen to be displayed on the screen.
[0005] U.S. Patent No. 5,443,260 to Stewart et al. discloses a
baseball training
and amusement apparatus that detects the speed and projected flight of a
batted
baseball. The apparatus includes a ball delivery device, a pair of detection
planes, a
computer and a video and simulation monitor. The detection planes are parallel
to
one another and are spaced apart by a distance such that a batted ball passing
through
the detection planes would be a fair ball in a real baseball game. Each
detection plane
includes a rigid frame that supports a pair of optical scanners and a pair of
light
sources. The optical scanners and light sources are positioned at opposite top
corners
of the rigid frame and are aimed downwardly into the region encompassed by the
frame.
[0006] During use, the ball delivery apparatus delivers a baseball
towards a
player positioned in front of the detection planes. When the player strikes
the
baseball with a bat and the baseball travels through the detection planes, the
optical
scanners capture images of the baseball. The images are processed to determine
the
coordinates of the baseball as it passes through each of the detection planes
as well as
the velocity of the baseball. A simulated trajectory of the baseball is then
calculated
using the determined coordinate and velocity information. The simulated
trajectory
information is used to update the graphical images presented on the monitor so
that
the simulated flight of the batted baseball is displayed to the player thereby
to
simulate a batting experience.
[0007] U.S. Patent No. 5,649,706 to Treat, Jr. et al. discloses a
hunting
simulator for in-flight detection of a launched missile such as an arrow. The
hunting
simulator includes a screen and a projector for projecting a moving target on
the
screen. Electromagnetic radiation emitters are positioned in front of the
screen
adjacent its opposite top corners and illuminate a plane in front of the
screen. Sensors
are also positioned adjacent the opposite top corners of the screen and are
responsive
to the electromagnetic radiation emitters. Retroreflective tape extends along
opposite
sides of the plane.
CA 02647956 2013-10-09
3
[0008] During use, when an arrow is launched at the screen and passes
through the plane, the sensors detect the presence of the arrow and generate
output.
The output of the sensors is used to determine the coordinates of the arrow as
well as
the velocity of the arrow. A simulated trajectory of the arrow is then
calculated and
the graphical images presented on the screen are updated accordingly to
reflect the
flight of the launched arrow. In this manner, a hunting experience is
simulated.
[0009] U.S. Patent No. 5,768,151 to Lowy et al. discloses a system
for
determining the trajectory of an object in a sports simulator. The system
includes a
baseball throwing device to deliver a baseball towards a player area. A
projector
adjacent the player area presents images on a display screen that is
positioned near the
ball throwing device and in front of a batter. Video cameras are positioned in
front of
and on opposite sides of the anticipated trajectory of a hit baseball.
[0010] During use when a baseball delivered by the ball throwing
device is hit
by the batter and passes through the fields of the view of the video cameras,
images of
the baseball are captured and a streak showing the path of the baseball
through the
fields of view is determined. The streak is used to simulate the flight of the
baseball
and to update the image presented on the display screen thereby to simulate a
batting
experience.
[0011] Although the above references show sports simulation systems
that
capture images of launched projectiles and use the image data to simulate the
flights
of the launched projectiles, these sports simulation systems fail to provide
"true to
life" sports experiences as a result of the mechanisms used to track the path
of the
launched projectiles. As will be appreciated, improved sports simulation
systems that
provide better and more realistic sports experiences are desired.
[0012] It is therefore an object of the present invention to provide a
novel
sports simulation system and a novel projectile tracking apparatus.
Summary of the Invention
[0013] Accordingly in one aspect there is provided a sports
simulation system
comprising:
a projectile tracking apparatus comprising a display surface on which a
sports scene is presented, and at least one pair of camera devices capturing
images of
a projectile tracking region disposed in front of said display surface to
detect a
CA 02647956 2013-10-09
4
launched projectile traveling through said projectile tracking region towards
said
display surface, said projectile tracking region extending from said display
surface to
a launch region spaced forward of said display surface in which contact with
said
projectile is made using a sports implement to launch said projectile from
said launch
region towards said display surface;
an overhead launch area sensing unit positioned above and aimed
generally straight down onto said launch region and capturing images of said
launch
region, said overhead launch area sensing unit comprising at least one camera
capturing images of said launch region and at least one illuminator adjacent
said at
least one camera, said at least one illuminator illuminating said launch
region from
above, the illumination provided by said at least one illuminator being
sufficient for
image capture without negatively affecting visibility of the sports scene
presented on
said display surface; and
at least one processing stage receiving image data from the camera
devices and from said overhead launch area sensing unit and determining the
three-
dimensional positions, velocity, acceleration and spin of a detected launched
projectile traveling from said launch region and through said projectile
tracking
region, the three-dimensional positions, velocity, acceleration and spin being
used by
said at least one processing stage to calculate a trajectory of said launched
projectile
into said sports scene, wherein during determination of the spin, said at
least one
processing stage processes the images captured by said overhead launch area
sensing
unit to determine a swing path of the sports implement before, during and
after
contact with said projectile and the launch angle of the projectile after
contact with
said sports implement and wherein said launch region is devoid of projectile
and/or
sports implement sensing components proximate the point of impact between the
sports implement and the projectile.
100141 In one embodiment, the at least one processing stage uses the
calculated trajectory to generate updated image data including a simulation of
the
launched projectile into the sports scene following the calculated trajectory.
A
projection device is coupled to the at least one processing stage. The
projection
device receives image data from the at least one processing stage and presents
the
sports scene including the simulation on the display surface.
CA 02647956 2013-10-09
[00151 In one embodiment, the projectile tracking apparatus includes
a frame
and at least one pair of camera devices mounted on the frame adjacent opposite
top
corners thereof. The camera devices have overlapping fields of view looking
downwardly, across and in front of the display surface and capture images of
the
5 projectile tracking region. Each camera device examines captured images
to detect
pixel clusters resembling a projectile characteristic signature thereby to
detect the
projectile in the captured images.
[0016] In one embodiment, the launch area sensing unit comprises at
least one
area-scan digital camera capturing images of the region in which contact with
the
projectile is made. The at least one area-scan camera is disposed above and
looks
down onto the region. One or more illuminators may be provided to provide
suitable
light for image capture. Images acquired by the at least one area-scan digital
camera
are processed by the at least one processing stage to detect the angle at
which impact
is made with the projectile allowing the spin of the projectile after impact
and the
projectile launch angle to be accurately determined. The images acquired by
the at
least one area-scan camera are processed on-board to determine if one or more
moving objects are within the images and if so whether the one or more moving
objects satisfy specified motion detection parameters. In this manner, only
images
containing information of interest are sent to the at least one processing
stage.
[0017] According to another aspect there is provided a sports simulation
system comprising:
a projectile tracking apparatus comprising a frame supporting a display
surface on which a video sequence portraying a sports scene is presented; and
at least
two digital camera devices having fields of view looking across and in front
of said
display surface that overlap and encompass a projectile tracking region
extending
from a projectile launch region spaced forward of said display surface to said
display
surface, each of said digital camera devices including a first processor for
processing
image data and generating two-dimensional projectile coordinates when a
projectile
that has been launched as a result of an impact with a sports implement
travels
through said projectile tracking region and is captured in images acquired by
said
digital camera devices;
an overhead launch area sensing unit positioned above and aimed
generally straight down onto said projectile launch region and capturing
images of
CA 02647956 2013-10-09
6
said projectile launch region, said overhead launch area sensing unit
comprising at
least one camera capturing images of said launch region and at least one
illuminator
adjacent said at least one camera, said at least one illuminator illuminating
said launch
region from above, the illumination provided by said at least one illuminator
being
-- sufficient for image capture without negatively affecting visibility of the
video
sequence presented on said display surface;
a host processor communicating with said digital camera devices and
said overhead launch area sensing unit, said host processor calculating a
three-
dimensional trajectory of said projectile taking into account projectile spin
using the
-- two-dimensional projectile coordinates received from each first processor
and the
image data output of said overhead launch area sensing unit and outputting
image data
including said calculated three-dimensional trajectory, wherein during
determination
of the spin, said at least one processing stage processes the images captured
by said
overhead launch area sensing unit to determine a swing path of the sports
implement
-- before, during and after contact with said projectile and the launch angle
of the
projectile after contact with said sports implement and wherein said launch
region is
devoid of projectile and/or sports implement sensing components proximate the
point
of impact between the sports implement and the projectile; and
a display unit receiving said image data and presenting said video
-- sequence on said display surface, said video sequence representing a
simulation of the
flight path of said sports projectile beginning from a contact location of
said sports
projectile with said display surface so that the simulation represents a
realistic
continuance of the travel of the sports projectile beyond the display surface.
100181 According to yet another aspect there is provided a golf
simulation
-- system comprising:
at least two digital camera devices having overlapping fields of view
looking across and in front of a display surface;
an overhead launch area sensing unit positioned above and aimed
generally straight down onto a launch region positioned a distance in front of
said
-- display surface in which contact with a golf ball is made using a golf club
and
capturing images of said launch region, said overhead launch area sensing unit
comprising at least one camera capturing images of said launch region and at
least one
CA 02647956 2013-10-09
6a
illuminator adjacent said at least one camera, said at least one illuminator
illuminating
said launch region from above;
at least one processing stage processing image data received from the
camera devices and said overhead launch area sensing unit and determining the
three-
dimensional positions, velocity, acceleration and spin of a detected launched
golf ball
traveling from its launch point to said display surface, the three-dimensional
positions, velocity, acceleration and spin being used by said at least one
processing
stage to calculate a trajectory of said launched golf ball into a golf scene
projected
onto said display surface, wherein during determination of the spin, said at
least one
processing stage processes the images captured by said overhead launch area
sensing
unit to determine the swing path of the golf club before, during and after
contact with
said golf ball, the launch angle of the golf ball after contact with said golf
club, and
the degree by which the head of the golf club is open or closed at the point
of impact
with the golf ball and wherein said launch region is devoid of golf ball
and/or golf
club sensing components proximate the point of impact between the golf club
and the
golf ball; and
a projection unit presenting said golf scene on said display surface
including a simulation of said golf ball following said calculated trajectory,
wherein
the illumination provided by said at least one illuminator is sufficient for
image
capture without negatively affecting visibility of the golf scene presented on
the
display surface.
[0018a] According to still yet another aspect there is provided a golf
simulator
comprising:
a display surface on which a golf scene is presented;
a launch region spaced a distance from said display surface to define a
golf ball travel region between said launch region and said display surface
and
configured to permit a user to launch a golf ball therefrom that travels
through said
golf ball travel region and impacts said display surface;
a launch area sensing unit comprising:
a housing configured to be mounted generally directly above
said launch region;
at least one camera mounted on said housing and capturing
images of said launch region from generally directly above; and
CA 02647956 2013-10-09
6b
at least one illuminator mounted on said housing adjacent said
at least one camera, said at least one illuminator illuminating said launch
region from
above; and
at least one processor communicating with the launch area sensing unit
and modifying the sports scene presented on said display surface in accordance
with
the trajectory of the golf ball launched at said display surface, wherein said
launch
region is devoid of golf ball and/or golf club sensing components proximate
the point
of impact between the golf club and the golf ball.
[0019] The sports simulation system provides advantages in that since
the
positions, velocity, acceleration, spin and launch angle of the launched
projectile are
determined, the flight of the launched projectile can be realistically
simulated. This of
course results in a more enjoyable sports experience. The sports simulation
system
also provides advantages in that virtually any type of projectile may be
tracked
avoiding the need for a specialized projectile. In addition, the projectile
may be
launched at the projectile tracking apparatus from a variety of positions
providing
greater flexibility and enhancing the sports experience.
Brief Description of the Drawings
[0020] An embodiment will now be described more fully with reference
to the
accompanying drawings in which:
Figure 1 is a perspective of a sports simulation system in accordance
with the present invention;
Figure 2 is a side elevation view of the sports simulation system of
Figure 1;
Figure 3 is a top plan view of the sports simulation system of Figure 1;
Figure 4 is a front elevation view of a projectile tracking apparatus
forming part of the sports simulation system of Figure 1;
CA 02647956 2008-09-30
WO 2007/112560
PCT/CA2007/000517
-7-
Figure 5 is an enlarged front elevation view, partly in section, of a
portion of the projectile tracking apparatus of Figure 4 showing a digital
camera;
Figure 6 is a side schematic view of a projectile launch area sensing
unit forming part of the sports simulation system of Figure 1;
Figures 7 and 8 are flowcharts showing steps performed during player
interaction with the sports simulation system of Figure 1; and
Figure 9 is an overhead view of a golf club making an impact with a
golf ball within a projectile launch area of the sports simulation system of
Figure 1.
Detailed Description of the Preferred Embodiment
[0021] Turning now to Figure 1, a sports simulation system is shown
and is
generally identified by reference numeral 100. As can be seen, sports
simulation
system 100 includes a projectile tracking apparatus 102 disposed in front of a
projectile launch area A in which a player P stands. A projectile launch area
sensing
unit 103 is disposed above the launch area A. A host computer 104 is coupled
to the
projectile tracking apparatus 102 and to the projectile launch area sensing
unit 103 via
a high-speed serial data link and to a ceiling mounted front video projector
106 that is
aimed at the projectile tracking apparatus 102. The host computer 104 outputs
video
image data to the projector 106, which in turn projects a video sequence on
the
projectile tracking apparatus 102. The video sequence portrays a visually
apparent
three-dimensional sports scene including a target T at which a projectile is
to be
launched. In this embodiment, the sports simulation system 100 simulates golf
and
thus, the three-dimensional sports scene is golf related and includes an image
of a golf
course hole, practice range etc. The projectile to be launched at the
projectile tracking
apparatus of course is a golf ball GB.
[0022] The projectile tracking apparatus 102 outputs two-dimensional
projectile position data to the host computer 104 when the launched golf ball
GB
travels through a projectile tracking region monitored by the projectile
tracking
apparatus. The projectile launch area sensing unit 103 outputs image data
representing the motion of the golf club through the launch area A before,
during and
after impact with the golf ball to host computer 104. The host computer 104 in
turn
processes the two-dimensional projectile position data and the projectile
launch area
CA 02647956 2008-09-30
WO 2007/112560
PCT/CA2007/000517
-8-
sensing unit image data to determine the three-dimensional positions, launch
velocity,
acceleration, spin and launch angle of the golf ball so that the trajectory of
the golf
ball can be accurately calculated. The calculated trajectory is then used to
determine
a sports result and to update the image data conveyed to the projector 106 so
that the
presented video sequence shows a simulation of the golf ball travel into the
visually
apparent three-dimensional scene as well as the determined sports result. As a
result,
the projectile tracking apparatus 102, projectile launch area sensing unit
103, the host
computer 104 and the projector 106 form a closed loop.
[0023] Figures 2 to 5 better illustrate the projectile tracking
apparatus 102. As
can be seen, the projectile tracking apparatus 102 includes an upright,
inverted U-
shaped frame 110 having a pair of side posts 112 and a crossbar 114 extending
between the upper ends of the posts 112. A screen 122 is supported by the
frame 110.
In this embodiment, the screen 122 has a 4:3 aspect ratio making it
particularly suited
for displaying conventional television images. Those of skill in the art will
however,
appreciate that other image formats can be used. The screen 122 is loosely
fastened to
the back of the frame 110 at spaced locations.
[0024] The screen 122 includes multiple layers and is designed to
reduce
projectile bounce as well as enhance protection behind the screen. The first
or front
layer of the screen 122 is formed of highly reflective nylon having some
elasticity to
resist permanent stretching/pocketing and abrasion. As a result, the front
layer
provides an excellent display surface 124 on which images projected by the
projector
106 are presented. The second or intermediate layer of the screen 122 is
formed of
soft and thick material and is designed to absorb projectile energy with
reduced elastic
effect thereby to inhibit stretching and or damage to the front layer. The
third or back
layer of the screen 122 is formed of a tough heavy canvas to which the
intermediate
layer can transfer energy. The back layer also inhibits excess deformation of
the
intermediate layer when contacted by a launched projectile. As a result, if
the
projectile tracking apparatus 102 is placed adjacent a wall surface or the
like, the back
layer protects the surface behind the screen 122 from projectile strike
thereby to
inhibit damage to the surface and/or significant projectile rebound. If a
space is
provided behind the projectile tracking apparatus 102, the back layer provides
ample
protection for the space.
CA 02647956 2008-09-30
WO 2007/112560
PCT/CA2007/000517
-9-
[0025] A pair of high speed digital cameras 128 is accommodated
within the
frame 110 with each camera being positioned adjacent a different top corner of
the
frame. Thus, the digital cameras 128 are positioned in front of the player and
to the
left side and right side of the anticipated projectile path. The digital
cameras 128 are
also angled to point downwardly and towards the player position so that the
fields of
view of the cameras are generally perpendicular and overlap in a region
extending
from the projectile launch point to the screen 122. In this manner, the path
of the
projectile can be tracked from its launch point until it impacts the screen
and then as it
rebounds from the screen 122.
[0026] In this embodiment, each digital camera 128 has at least a 640 by
480
pixel array and includes built-in processing capabilities comprising field
programmable gate arrays, a high performance 32-bit microprocessor and high
speed
memory. The distributed processing capabilities achieved by using the digital
cameras 128 and the host computer 104 allow the digital cameras to be operated
at
very high frame rates thereby allowing multiple images of a fast moving
projectile to
be captured as it travels through the projectile tracking region 120. This is
due to the
fact that the digital cameras 128 need only send data to the host computer 104
relating
to images in which projectile motion has been detected allowing high speed
projectiles to be tracked without excessive bandwidth between the host
computer 104
and the digital cameras 128 being needed. For example, in the case of a
projectile
travelling through the projectile tracking region 120 at a speed of 200 miles
per hour,
the frame rates of the digital cameras 128 are selected such that at least
four images of
the projectile are captured by each digital camera 128. The viewing angles of
the
digital cameras 128 and the dimensions of the frame 110 are selected to
provide the
digital cameras 128 with a resolving accuracy of approximately lmm per pixel.
As a
result, a small projectile such as a golf ball will activate approximately 12
pixels per
image. This resolving accuracy enables even small, very fast moving launched
projectiles to be readily determined in captured images and as a result,
reduces false
projectile detection.
[0027] The on-board processors of the digital cameras 128 execute a motion
detection routine to determine if a projectile exists in the captured images
and if so,
whether the projectile satisfies specified motion detection parameters
defining a
CA 02647956 2008-09-30
WO 2007/112560 PCT/CA2007/000517
-10-
projectile characteristic signature. The projectile characteristic signature
is used to
ensure the detected projectile has characteristics matching the projectile in
question,
in this case, a struck golf ball. The projectile can therefore be
distinguished from
other objects captured in the images such as for example, the golf club head.
In this
example, the projectile characteristic signature specifies allowable
projectile size,
shape, reflectivity and speed.
100281 Infrared (IR) light emitting diode (LED) arrays (not shown)
are also
positioned within the posts 112 beside the digital cameras 128. The
illumination axes
of the IR LED arrays are generally coincident with the optical axes OA of the
digital
cameras. Each IR LED array emits IR radiation that is directed into the
projectile
tracking region 120. As the digital cameras 128 are responsive to both visible
and
infrared light, providing the background IR illumination allows the projectile
tracking
apparatus 102 to work well in a variety of ambient lighting conditions. In
situations
where a small fast moving projectile is launched, the IR illumination allows
for
detection of the projectile without interfering with the visual quality of the
displayed
image presented on the screen 122.
100291 Audio speakers 140 are provided on the posts 112 and are aimed
forwardly toward the launch area A. The audio speakers 140 are driven by an
audio
amplifier (not shown) accommodated within the frame 110. The audio amplifier
receives audio input from the host computer 104 during play that is conveyed
to the
audio speakers 140 for broadcast thereby to enhance the sports experience.
100301 The projectile launch area sensing unit 103 is disposed
directly over
the launch area A and comprises an area-scan digital camera 160, a forty-five
(45)
degree mirror 162, a plurality of illuminators 164 in the form of halogen
spotlights
and a power supply (not shown) for the spotlights 164. The spotlights 164 are
aimed
to provide sufficient illumination in the launch area A to permit image
capture
without adversely affecting visibility of the image projected on the screen
122. The
area-scan digital camera 160 is ceiling mounted horizontally approximately ten
(10)
feet above the launch area A. The optical axis of the digital camera 160 is in
line with
the center of the mirror 162 so that the field of view of the area-scan
digital camera is
re-directed downwardly over the center of the launch area A. In this
embodiment, the
CA 02647956 2008-09-30
WO 2007/112560
PCT/CA2007/000517
-11-
field of view of the area-scan digital camera 160 encompasses a three (3) foot
by three
(3) foot region.
[0031] Similar to the digital cameras 128 in the projectile tracking
apparatus
102, the area-scan digital camera 160 includes an on-board processor that
executes a
motion detection routine. During execution of the motion detection routine, as
images
are captured by the area-scan digital camera 160, the images are examined to
determine if one or more moving objects exist therein that satisfy specified
motion
parameters. In this example, the motion parameters are selected to allow the
on-board
processor to detect when either a moving golf club and golf ball or both is in
captured
images. Captured images including one or more moving objects satisfying the
specified motion parameters are sent to the host computer 104 for further
processing.
[0032] The projector 106 preferably has a resolution of at least
800x600, at
least 1200 ANSI Lumens brightness, a short throw lens, vertical 'keystone'
correction, and the capacity to accept digital RGB computer video signals, and
NTSC/PAL baseband television video signals. Projectors having this set of
features
include the Epson Powerlite 820P, the Toshiba TDP-DI-US, the InFocus LP650 and
the Sanyo XP30 for example.
[0033] The host computer 104 is preferably an IBM compatible personal
computer including an Intel Pentium processor, at least 128 MB SDRAM, a high-
speed hard drive, and a DVD player. The host computer 104 also includes a
display
adapter assembly including a reconfigurable 32-bit video memory buffer
partitioned
into three separate buffers. One of the buffers is used to store primary
foreground
image data representing one or more independent foreground action elements if
appropriate for the sports scene being displayed. A second of the buffers is
used to
store background image data and the third buffer is used to store projectile
trajectory
image data. The display adapter assembly treats the foreground action,
background
and projectile trajectory image data as overlay image planes that are combined
seamlessly to generate the video image data that is output to the projector
106. The
overlay image planes are non-destructive so that when a foreground action
element
and/or projectile moves over an underlying image plane it is not necessary to
redraw
the underlying image plane. To reduce peak processing requirements, the host
computer 104 updates the background image data less frequently than the
foreground
CA 02647956 2008-09-30
WO 2007/112560 PCT/CA2007/000517
-12-
image data. The host computer 104 provides the output video image data to the
projector 106 on a video output channel. The host computer 104 receives
external
video feeds on a television/satellite/cable input channel, a video game input
channel
and an Internet input channel.
[0034] The host computer 104 is mounted within a protective enclosure (not
shown) having external connectors to enable the computer to be coupled to the
projector 106, the projectile tracking apparatus 102 and the projectile launch
area
sensing unit 103. The enclosure also includes external connectors to allow the
host
computer 104 to receive the television/satellite/cable, external video game
and
Internet feeds. Input controls are also provided on the enclosure to allow a
player to
interact with the host computer 104.
[0035] A high speed digital serial interface such as 1EEE1394 is used
between
the host computer 104, the projectile tracking apparatus 102 and the
projectile launch
area sensing unit 103. Using this standard interface provides a low cost, high
performance solution while avoiding use of expensive analog frame grabbers.
The
interface also simplifies wiring as the cameras 128 can be daisy-chained
without loss
of signal integrity.
[0036] The host computer 104 executes sports simulation software
stored in
the SDRAM. In this example, the sports simulation software includes a golf
simulation module that requires a player to hit the golf ball GB at the
projectile
tracking apparatus 102 in response to the video sequence displayed on the
screen 122.
[0037] To provide a realistic playing experience, a high resolution
elevation
map of the golf course terrain is used. The course terrain elevation map is
constructed
from a combination of two-dimensional images that include overhead satellite
and/or
aerial photographs used in conjunction with digital photographs taken from
ground
level. Using photogrammetry techniques, these orthogonal views are combined
together. Using common points in the images i.e. edges of sand hazards, trees
etc., a
three-dimensional model is synthesized without requiring reference targets to
be
applied to the terrain of interest.
[0038] During training, practice or game play, the host computer 104
outputs
video image data to the projector 106 causing the projector 106 to project a
video
sequence portraying a visually apparent three-dimensional sports scene on the
display
CA 02647956 2008-09-30
WO 2007/112560 PCT/CA2007/000517
-13-
surface 124 that includes a target at which the projectile is to be launched
(see step
500 in Figure 7). The host computer 104 also conditions the digital cameras
128 to
capture a background image of the projectile tracking region 120 devoid of a
projectile (step 502) and then scan the projectile tracking region to look for
the
presence of a launched projectile at a very high frame rate (step 504). The
player is
then prompted to launch the golf ball GB at the screen 122 (step 506).
[0039] At this stage, the digital cameras 128 continually capture and
process
images to detect the existence of a projectile. When the player launches the
projectile
at the projectile tracking apparatus 102 by striking the golf ball with a golf
club and
the projectile enters the projectile tracking region 120, the projectile
appears in the
images captured by the digital cameras 128. Thus, the digital cameras 128
synchronously capture a series of images of the projectile as it travels
through the
projectile tracking region 120 (step 508). The captured images are in turn
processed
by the on-board processors of the digital cameras 128 to determine if the
captured
images include a detected projectile satisfying the projectile characteristic
signature.
[0040] If the detected projectile satisfies the projectile
characteristic signature,
the images are further processed to determine the center of mass of the
projectile in
each image and its position in rectangular coordinates. As a result, a series
of two-
dimensional rectangular coordinates representing the two-dimensional positions
of the
projectile as it travels through the projectile tracking region 120 relative
to each
digital camera 128 is generated. The two-dimensional rectangular coordinates
generated by the digital cameras 128 are in turn conveyed to the host computer
104.
[0041] At the same time, the area-scan digital camera 160 of the
projectile
launch area sensing unit 103 continually captures and processes images to look
for the
existence of a swinging golf club passing through the launch area A and the
launched
golf ball exiting the launch area A. When a swinging golf club and launched
golf ball
are detected, the area-scan digital camera 160 outputs the captured images to
the host
computer 104.
[0042] Upon receipt of the projectile coordinates from the projectile
tracking
apparatus 102, the host computer 104 calculates the positions of the
projectile's center
of mass in three-dimensional space throughout its travel through the
projectile
tracking region 120 including its collision and rebound with the screen 122
using
CA 02647956 2008-09-30
WO 2007/112560 PCT/CA2007/000517
-14-
triangulation techniques (see step 520 in Figure 8). With the position of the
projectile
in three-dimensional space known during its travel through the projectile
tracking
region 120 and knowing the frame rates of the digital cameras 128, the host
computer
104 calculates the launch velocity of the projectile and the velocity of the
projectile
over each image frame (step 522). The host computer 104 then compares each
calculated velocity with the previously calculated velocity to determine the
acceleration of the projectile (step 524).
[0043] Upon receipt of the image data from the projectile launch area
sensing
unit 103, the host computer 104 analyzes the club head swing path 200 (see
Figure 9)
to determine where the club head hits the golf ball GB and to determine the
initial golf
ball trajectory or launch angle after being hit. The host computer 104 also
defines a
club head motion vector 202 as the tangent line along the club head swing path
200.
By estimating the initial golf ball trajectory, a golf ball motion vector 206
is
measured. Using this vector, a club face vector 208 can be determined as the
line
perpendicular to the tangent 210 of the club face at the impact point of the
golf ball
and the club face. By comparing the club head motion vector 202 and the club
face
vector 208, a determination can be made as to whether the club face is open or
closed
upon impact with the golf ball. The degree to which the club head motion
vector 202
is not parallel to the club face vector 208 at the point of impact determines
the amount
of side spin that the golf ball will have. This enables the host computer 104
to
calculate the spin of the golf ball based on the angle of the club face at the
point of
contact with the golf ball as well as on the impact and rebound angles of the
projectile
with and from the screen 122 (also step 524).
[0044] With the three-dimensional positions, launch velocity,
acceleration,
spin and launch angle of the projectile known, the host computer 104
extrapolates an
accurate trajectory for the projectile allowing a realistic simulation of
curved and/or
arcing projectiles to be generated (step 526). The computed projectile
trajectory is
then used to determine a sports result by computing the intersection of the
calculated
projectile trajectory with the displayed video image (step 528). With the
projectile
trajectory computed and the sports result determined, the host computer 104
updates
the image data that is conveyed to the projector 106 so that the video
sequence
CA 02647956 2013-10-09
displayed on the display surface 124 of the screen 122 shows the simulated
flight of
the projectile and the sports result (step 530).
[0045] During video sequence display, when a simulation of the
projectile
flight is shown a graphical duplicate of the projectile is projected onto the
display
5 surface 124 of the screen 122 that begins its flight from the impact
point of the
projectile with the screen 122. In this manner, the projectile appears to
continue its
trajectory into the video scene thereby to achieve a realistic video effect.
The three-
dimensional scene is then updated in accordance with the sports result,
allowing game
play or practice to continue.
10 [0046] Although the sports simulation system 100 has been
described as
including a ceiling mounted front projector 106 in combination with a screen
122,
those of skill in the art will appreciate that alternative projection devices
may be used.
For example, a rear video projector may be used to project images onto the
rear
surface of the display screen 122.
15 [0047] Those of skill in the art will appreciate that the
projectile tracking
apparatus 102 may include additional cameras at different locations to view
the
projectile tracking region and detect the existence of a launched projectile.
Those of
skill in the art will also appreciate that the number of processing stages may
be
increased or decreased as desired to handle processing of the digital camera
image
data effectively in real-time and provide a realistic projectile simulation.
[0048] If desired, the projectile launch area sensing unit 103 may
include
additional cameras. The projectile launch area sensing unit may include any
number
of illuminators or none at all if the ambient light conditions are sufficient
to provide
for adequate image capture.
[0049] While the sports simulation system is described as simulating golf,
it
will be appreciated that the sports simulation system may be used to simulate
other
sports where a projectile is launched. In such cases, the projectile
characteristic
signatures are updated to enable launched projectiles to be accurately
tracked.
[0050] Although a preferred embodiment of the present invention has
been
described, those of skill in the art will appreciate that variations and
modifications
may be made without departing from the scope thereof as defined by the
appended
claims.
