Note: Descriptions are shown in the official language in which they were submitted.
CA 02690987 2015-02-12
1 =
METHOD AND APPARATUS FOR CHROMA KEY PRODUCTION
This application claims priority to international Patent Application Serial
No.
PCT/US2007/015254 entitled "Method and Apparatus for Chroma Key Production",
filed
on June 29, 2007.
The present principles relate to chrome keying. More particularly, it relates
to the
production of an adaptive chroma key.
A chroma key generally places foreground objects in a background scene. Since
chroma key parameters are calculated for every field based on the foreground
scene
only, the foreground object cannot adapt to the background scene lighting.
Thus, the
use of ambient or artificial light changes.
In fact, chroma key parameters are based on homogenized studio-lighting
conditions rather than the lighting in the background scene. This type of
lighting
mismatch fails to produce a natural chroma key.
It is an aspect of the present principles to provide an adapative chroma key
that
overcomes the shortfalls of the prior art.
This and other aspects are achieved by the method for producing a chroma key
which includes determining an area of interest (A01) for a composite scene
using a
foreground key, and generating a chroma key taking into consideration physical
properties of pixels in the determined A01.
In accordance with another aspect of the present principles the apparatus for
producing a chroma key includes a source selection device configured to
determine an
area of interest (A01) for a composite scene using a foreground key and
generate a
chrome key taking into consideration physical properties of pixels in the
determined
A01. The source selection device could include a plurality of
WSLega11061775\00050111484402v1
CA 02690987 2009-12-17
WO 2009/005511
PCT/US2007/020192
2
input sources, a processor, switching logic in communication with the
processor,
and an adaptive chrome key sub-system connected to the processor and
switching logic, said adaptive chroma key sub-system operating under the
control
of the processor to selectively combine two or more of the input sources for
the
composite scene.
Other aspects and features of the present principles will become apparent from
the following detailed description considered in conjunction with the
accompanying
drawings. It is to be Understood, however, that the *drawings are designed
solely for
purposes of illustration and not as a definition of the limits of the present
principles, for
lb which reference should be made to the appended claims: It should be further
understood that the drawings are not necessarily drawn to scale and that,
unless
otherwise indicated, they are merely intended to conceptually illustrate the
structures
and procedures described herein.
In the drawings wherein like reference numerals denote similar components
throughout the views: =
Figure 1 is a graphical representation of a foreground scene in a green-
colored
virtual studio set;
Figure 2 is a graphical representation of the foreground objects in the scene;
Figure 3 is a graphical representation of the background scene intended for
=
use with the foregoing scene;
Figure 4 is a graphical representation of the foreground key;
Figure 5 is graphical representation of the composite output of both the
foreground and background scenes;
CA 02690987 2009-12-17
WO 2009/005511
PCT/US2007/020192
3
Figure 6 is a graphical representation of the Area of interest (A01) in the .
. composite scene;
Figure 7 is a graphical representation of the, composite scene with both
brighter background and foreground objects;
Figure 8 is a graphical representation of the composite scene with both darker
background and foreground objects;
Figures 9a-9c are graphical representations showing how *different hues in a
background scene can create different but matching foreground objects; =
Figure 10a is a flow diagram of the luminance determination for the display of
the composite image according to an implementation of the, present principles;
Figure 10b is a block diagram of the determination of the A01 for the.chroma
key according to an implementation of the present principles;
Figure 11 is flow diagram of the chrominance determination and application-
for
the display of the composite image according to an implementation of the
present
principles;
Figure. 1.2 is a block diagram of a switcher apparatus.configured to implement
the adaptive chroma keying of the present principles; and
Figure 13 is a block diagram of an adaptive chroma . key subs system
according to an implementation of the present principles.
= 20 = Generally speaking, a chroma key allows the placement of
foreground objects in
a background scene.. Since the chroma key parameters. are calculated for every
field
based on the foreground scene only, the foreground object cannot adapt to the
= background scene lighting (e.g., ambient and artificial) changes. In
fact, the chroma
CA 02690987 2015-02-12
4
key parameters are often homogenized for studio-lighting conditions rather
than the
lighting in the background scene.
The present principles provides an alternative to this chroma keying scheme by
allowing an adaptive creation of the chroma key using luminance and
chrominance
information from the background scene.
FIGS. 1-5 show some background information for understanding the concepts of
the present principles. FIG. 1 shows a foreground scene 10 (represented by the
people
in the picture) and a green-colored virtual studio set 12. FIG. 2 shows the
foreground
objects 10 without the virtual studio set 12. FIG. 3 shows a background scene
14 to be
displayed on the virtual studio set. FIG. 4 shows a foreground key 16
representing the
images of the foregoing scene to be superimposed over the background scene.
FIG. 5
is a representation of the composite output image resulting from combination
of the
background scene 14 and the foreground scene 10.
Initially, and in accordance with the present principles, a determination as
to the
area of interest (A01) between the foreground scene and the background scene
is made
(step 102 FIG. 10a). Referring to FIG. 10b, in order to do this, a foreground
key is
overlayed onto the background scene (110), and an identification of all the
pixels of the
background scene that falls inside the foreground key is made (112). This
identification
constitutes the A01. FIG. 6 shows a graphical representation of the A01 18 as
obtained
from this process.
In accordance with one implementation, the information from the background
scene is used by the chroma key logic to adaptively create the chroma key. In
doing
this, the method includes considering one or more physical properties of the
pixels in
W5Lega1\081775100050\11484402v1
CA 02690987 2009-12-17
WO 2009/005511
PCT/US2007/020192
the determined A01 in order to create the chroma key. In the present example,
these
.
physical properties include the luminance and the chrominance of the pixels.
Figure 10a shows the method 100 for considering the luminance of the A01, and
Figure 11 shows the method 150 for considering the chrominance of the A01.
5 Referring to Figure 10a; initially the determination (102) as to the A01
for the
composite scene is made.. The average luminance of pixels in the A01 is then
calculated (104), 'and the average luminance of. a sampled area in the
foreground.
scene is also calculated (106). Once these luminanoe calculations have been
made,
the !Lima in the foreground scene is linked to the luma in the A01 (108). In
other _words,
we are applying the difference (delta) in luma A01 to luma in the foreground
scene for.
eveiy field. If the background has dramatic lighting changes, such as a video
clip show
through bright and dark streets of Manahattan. The foreground object (e.g., a
new
reader or report) will adapt to the background scene and' change its lighting
=
accordingly. Figure 7 shows an example of a brighter background having
brighter
foreground objects, and Figure 8 shows an example of a darker background
having
darker foreground objects.
= Figure 11 shows an example of the method 150 where the chrominance signal
is considered. After the determination 102 of the A01; it is determined (120)
whether a
constant vector is applied to all pixels in the A01. If not, the foreground
Scene remains
unchanged (124). If there is a constant vector applied to all pixels in the
AOI, a small.
percentage of the sarrie constant vector is applied to the resulting
foreground scene.
= For example, if the background scene is a disco club with rotating multi-
colored light
beams, the foreground object adapts to hue changes in the background scene
(i.e., if a
red beam of light falls on the AOI, a slight tinge of red will appear on the
foreground
CA 02690987 2009-12-17
WO 2009/005511
PCT/US2007/020192
6
object as well. Thus, different hues in the background scene.can create
different, but
=
matching objects in the foreground. This concept is shown in the exemplary
images of
Figures 9a-9c. In each figure, the hue is different, resulting in a change in
the
foreground object color and thereby an overall change of the entire composite
image
displayed. Figure 9b shows the effect of a redish hue (represented by an array
of very
small dots covering the entire Figure 9b) added to the background lighting.
and the
overall effect of the same on the foreground objects (i.e., the matching of
the same with
the background), while Figure 9c shows the effect of .a greenish hue
(represented by
an array of very small dashes covering the entire Figure 9c) in the background
scene.
Figure 12 shows a block diagram of a switcher system 200. programmed to
operate in accordance with the present principles. The switcher 202 includes a
plurality of inputs 208, a processor 204 and switching logic 206 in
communication with
the processor. The processor 208 can include an onboard memory 210, or may be
linked to an external storage medium, such as a hard disk drive, a compact
disc drive,
a flash memory or other solid state memory device, or any other memory storage
means. The adaptive chroma key sub-system 207 is communication with the
processor 204 and switching logic 206 and is configured to perform the method
of the
present principle and take one of said inputs 208 having a background scene
and
selectively combine it with another input having .a foreground scene to
provide a
desired composite scene at its output 212.
Figure 13 shows a block diagram of the adaptive chroma key sub-system 207
according to an implementation of the present principles. The foreground video
250
and foreground key 252 are interpolated by .interpolator 258. After hue
selection (via
primary hue selector 262 and secondary hue selector 266) and suppression (via
=
CA 02690987 2009-12-17
WO 2009/005511
PCT/US2007/020192
7
primary suppression 264 and secondary suppression 268, the video 270 is passed
to
the next logical subsystem in the switcher 202 (e.g., switching logic 206).
The
secondary hue selector 266 outputs the foreground information which processed
(clip &
gain 278) before the background changes are applied 280. The background video
'254
and background key 256 are interpolated by. interpolator 260, and the A01 is
then
determined 272. As described=above, once the A01 has been determined; the Luma
change 274 and Chrorna change 276 of the Aoi is determined and are applied 280
to
the foreground. At this. stage, the offset 282 is applied to the foreground
key signal and
the foreground key 212 is output.
. The various aspects, implementations, and features may be implemented
in one or more of a variety of manners, even if described above without
reference
to a particular manner or using only one manner. For example, .the various
=
aspects, implementations, and features may be implemented using, for. example,
one or more of a method, an apparatus, an apparatus or processing device for
.15 performing a method, a program or other set of instructions, an
apparatus that -
includes a program or a set of instructions, and a computer readable medium.
= = An apparatus may include, for example, 'discrete or integrated
hardware, =
firmware, and software. As an example, an apparatus may include, for example,
a processor, which refers to. processing devices in general, including, for
example;
a microprocessor, an integrated circuit, or a programmable logic device. As
another example, an apparatus may include one or more computer readable
media having instructions for carrying out one or more processes.
A computer readable medium may include, for example, a 'software carrier
or other storage device such as, for example, a hard disk, a compact diskette,
a
CA 02690987 2009-12-17
WO 2009/005511 PCT/US2007/020192
8
random access memory ("RAM"), or a read-only memory ("ROM"): A computer
readable medium also may include, for example, formatted electromagnetic
waves encoding or transmitting instructions.. Instructions may be, for
example, in.
hardware, firmware, software, or in an electromagnetic wave. Instructions may
be
found in, for example, an operating system, a separate application, Or a
combination of the two. -A processor may be characterized; therefore, as, for
example, both a device configured to carry out a process and a device that =
includes a computer readable medium having instructions for tarrying = out a
process.
A number of implementations, have been described. .Nevertheless, it will be
understood that various modifications may be made. For example, elements of
different implementations may be combined, supplemented, modified, or removed
to produce other implementations. Accordingly, other implementations are
within.
the scope of the following claims:
