Language selection

Search

Patent 2840548 Summary

Third-party information liability

Some of the information on this Web page has been provided by external sources. The Government of Canada is not responsible for the accuracy, reliability or currency of the information supplied by external sources. Users wishing to rely upon this information should consult directly with the source of the information. Content provided by external sources is not subject to official languages, privacy and accessibility requirements.

Claims and Abstract availability

Any discrepancies in the text and image of the Claims and Abstract are due to differing posting times. Text of the Claims and Abstract are posted:

  • At the time the application is open to public inspection;
  • At the time of issue of the patent (grant).
(12) Patent: (11) CA 2840548
(54) English Title: RAPID IMAGE RENDERING ON DUAL-MODULATOR DISPLAYS
(54) French Title: RENDU D'IMAGE RAPIDE SUR DES ECRANS D'AFFICHAGE A DOUBLE MODULATEUR
Status: Granted and Issued
Bibliographic Data
(51) International Patent Classification (IPC):
  • G09G 3/20 (2006.01)
  • G09G 3/36 (2006.01)
(72) Inventors :
  • WARD, GREGORY JOHN (United States of America)
  • WHITEHEAD, LORNE A. (Canada)
  • SEETZEN, HELGE (Canada)
  • HEIDRICH, WOLFGANG (Canada)
(73) Owners :
  • DOLBY LABORATORIES LICENSING CORPORATION
(71) Applicants :
  • DOLBY LABORATORIES LICENSING CORPORATION (United States of America)
(74) Agent: OYEN WIGGS GREEN & MUTALA LLP
(74) Associate agent:
(45) Issued: 2018-03-13
(22) Filed Date: 2005-05-27
(41) Open to Public Inspection: 2006-02-02
Examination requested: 2014-01-21
Availability of licence: N/A
Dedicated to the Public: N/A
(25) Language of filing: English

Patent Cooperation Treaty (PCT): No

(30) Application Priority Data:
Application No. Country/Territory Date
60/591,829 (United States of America) 2004-07-27

Abstracts

English Abstract


Apparatus and methods are provided that employ one or more of a
variety of techniques for reducing the time required to display high
resolution images on a high dynamic range display having a light source
layer and a display layer. In one technique, the image resolution is
reduced, an effective luminance pattern is determined for the reduced
resolution image, and the resolution of the effective luminance pattern is
then increased to the resolution of the display layer. In another
technique, the light source layer's point spread function is decomposed
into a plurality of components, and an effective luminance pattern is
determined for each component. The effective luminance patterns are
then combined to produce a total effective luminance pattern.
Additional image display time reduction techniques are provided.


French Abstract

Un appareil et des procédés mettant en uvre une ou plusieurs techniques visant à réduire le temps nécessaire à laffichage dimages à haute résolution sur un écran à grande plage dynamique présentant une couche source lumineuse et une couche daffichage. Selon une technique, la résolution de limage est réduite, un motif de luminance effective est déterminé pour la résolution dimage réduite et la résolution du motif de luminance effective est ensuite augmentée jusquà la résolution de la couche daffichage. Selon une autre technique, la fonction détalement des points de la couche source lumineuse est décomposée en une pluralité de composantes et un motif de luminance effective est déterminé pour chaque composante. Les motifs de luminance effective sont ensuite combinés pour produire un motif de luminance effective total. Des techniques de réduction de temps daffichage dimage supplémentaires sont également décrites.

Claims

Note: Claims are shown in the official language in which they were submitted.


-22-
What is claimed is:
1. A method for displaying an image on a display comprising a light source
layer and
a display layer, the method comprising:
determining based at least in part on image data driving values for light
sources of the light source layer;
determining an effective luminance pattern of the light source layer by a
method including:
determining a contribution to the effective luminance pattern for
each of a plurality of components of a point spread function for light
sources of the light source layer; and,
combining the contributions to the effective luminance pattern of
each of the components to yield effective luminance pattern data; and,
determining driving values for the display layer based at least in part on the
effective luminance pattern data and the image data.
2. A method according to claim 1 wherein determining the contribution to
the
effective luminance pattern for each of the plurality of components of the
point
spread function is performed at a first spatial resolution lower than that of
the
image data.
3. A method according to claim 2 wherein the first spatial resolution is a
factor of 4
or more in each dimension lower than the resolution of the image data.
4. A method according to claim 1 or 2 wherein determining the contribution
to the
effective luminance pattern for each of the plurality of components of the
point
spread function is performed at first spatial resolution for one of the
components
and at a second spatial resolution different from the first spatial resolution
for a
second one of the components.
5. A method according to claim 4 wherein the first one of the components is
narrower
than the second one of the components and the first spatial resolution is
higher than
the second spatial resolution.

-23-
6. A method according to claim 4 wherein the second one of the components
is more
slowly varying than the first one of the components and the first spatial
resolution
is higher than the second spatial resolution.
7. A method according to any one of claims 1 to 6 wherein the components
comprise
Gaussian components.
8. A method according to any one of claims 1 to 7 wherein determining the
contributions to the effective luminance pattern comprises, for each of the
components of the point spread function, for each of the light sources,
multiplying
values that define the component of the point spread function by a value
representing the intensity of the light source.
9. A method according to any one of claims 1 to 7 comprising upsampling the
effective luminance pattern data.
10. A method according to claim 9 wherein the upsampling comprises
interpolation.
11. A method according to claim 10 wherein the interpolation comprises
linear
interpolation.
12. A method according to claim 10 wherein the interpolation comprises
Gaussian
interpolation.
13. A method according to claim 10 wherein the interpolation comprises
convolving
an iteratively-derived interpolation function with the effective luminance
pattern
values.
14. A method according to any one of claims 1 to 13 wherein determining the
effective
luminance pattern of the light source layer is performed in the 8-bit domain.

-24-
15. A method according to claim 14 wherein the point spread function is
characterized
by 16-bit data words.
16. A method according to claim 15 comprising separately processing low
byte and
high byte parts of the 16-bit data words.
17. A method according to claim 15 comprising separately processing first,
second and
third segments of the point spread function, the first segment comprising high
byte
parts of the 16-bit data words, the second segment comprising low byte parts
of the
16-bit data words for points outside a radius, and the third segment low byte
parts
of the 16-bit data words for points within the radius.
18. A method according to any one of claims 1 to 17 wherein determining the
driving
values for the display layer comprises dividing the image data by the
effective
luminance pattern data.
19. A method according to any one of claims 1 to 18 comprising applying the
driving
values for light sources of the light source layer to drive the light sources
of the
display layer and applying the driving values for the display layer to drive
the
display layer.
20. Display apparatus comprising:
a light source layer
a display layer,
a controller configured to:
determine, based at least in part on image data, first driving values
for light sources of the light source layer;
determine an effective luminance pattern of the light source layer by
a method including:
determining a contribution to the effective luminance pattern
for each of a plurality of components of a point spread function for
light sources of the light source layer; and,

-25-
combining the contributions to the effective luminance
pattern of each of the components to yield effective luminance
pattern data; and,
determine second driving values for the display layer based at least
in part on the effective luminance pattern data and the image data;
a first interface connected to the light source layer to apply the first
driving
values to the light source layer; and,
a second interface connected to the display layer to apply the second
driving values to the display layer.
21. Display apparatus according to claim 20 wherein the controller is
configured to
determine the contribution to the effective luminance pattern for each of the
plurality of components of the point spread function at a spatial resolution
lower
than that of the image data.
22. Display apparatus according to claim 20 or 21 wherein the controller is
configured
to determine the contribution to the effective luminance pattern for each of
the
plurality of components of the point spread function at first spatial
resolution for
one of the components and at a second spatial resolution different from the
first
spatial resolution for a second one of the components.
23. Display apparatus according to claim 22 wherein the first one of the
components is
narrower than the second one of the components and the first spatial
resolution is
higher than the second spatial resolution.
24. Display apparatus according to claim 22 wherein the second one of the
components is more slowly varying than the first one of the components and the
first spatial resolution is higher than the second spatial resolution.
25. Display apparatus according any one of claims 20 to 24 wherein the
components
comprise Gaussian components.

-26-
26. Display apparatus according to any one of claims 20 to 25 wherein the
controller is
configured to upsample the effective luminance pattern data.
27. Display apparatus according to claim 26 wherein the controller is
configured to
perform the upsampling by interpolation.
28. Display apparatus according to claim 27 wherein the interpolation
comprises linear
interpolation.
29. Display apparatus according to claim 27 wherein the interpolation
comprises
Gaussian interpolation.
30. Display apparatus according to claim 27 wherein the controller is
configured to
perform the interpolation by convolving an iteratively-derived interpolation
function with the effective luminance pattern values.
31. Display apparatus according to claim 20 comprising a lookup table
comprising
values for the point spread function.
32. Display apparatus according to claim 31 wherein the point spread
function is
characterized by 16-bit data words.
33. Display apparatus according to claim 20 wherein the controller
comprises an 8-bit
graphics processor.
34. Display apparatus according to claim 33 wherein the controller is
configured to
determine the effective luminance pattern of the light source layer in the 8-
bit
domain using the 8-bit graphics processor.
35. Display apparatus according to claim 32 wherein the controller is
configured to
separately process low byte and high byte parts of the 16-bit data words.

-27-
36. Display apparatus according to claim 32 wherein the controller is
configured to
separately process first, second and third segments of the point spread
function, the
first segment comprising high byte parts of the 16-bit data words, the second
segment comprising low byte parts of the 16-bit data words for points outside
a
radius, and the third segment low byte parts of the 16-bit data words for
points
within the radius.
37. Display apparatus according to any one of claims 20 to 36 wherein the
controller is
configured to determine the driving values for the display layer by dividing
the
image data by the effective luminance pattern data.
38. A method for displaying an image on a display comprising a light source
layer and
a display layer, the method comprising:
determining based at least in part on image data driving values for light
sources of the light source layer;
determining an effective luminance pattern of the light source layer by a
method including:
for each of a plurality of light sources of the light source layer:
separately determining contributions to the effective
luminance pattern of higher-order and lower-order parts of a set of
point spread function values; and,
combining the contributions to the effective luminance
pattern of the higher-order and lower-order parts of the point spread
function values to yield effective luminance pattern data; and,
determining driving values for the display layer based at least in part on the
effective luminance pattern data and the image data.
39. A method according to claim 38 comprising separately processing first,
second and
third segments of the point spread function, the first segment comprising the
higher
order part of the point spread function, the second segment comprising the
lower
order part of the point spread function for points outside a radius, and the
third
segment comprising the lower order parts of the point spread function for
points
within the radius.

-28-
40. A method according to claim 38 wherein the point spread function values
comprise
16-bit values.
41. A method according to claim 40 comprising separately processing first,
second and
third segments of the point spread function, the first segment comprising high
byte
parts of the 16-bit data words, the second segment comprising low byte parts
of the
16-bit data words for points outside a radius, and the third segment
comprising low
byte parts of the 16-bit data words for points within the radius.
42. A method according to claim 40 wherein the higher-order parts comprise
high byte
parts of the 16-bit data words and the high byte parts of the 16-bit data
words are
zero outside of the radius.
43. A method according to any one of claims 38 to 41 wherein the higher-
order parts
of the set of point spread function values have a narrower support than the
lower-
order parts of the set of point spread function values.
44. Display apparatus comprising:
a light source layer
a display layer,
a controller configured to:
determine, based at least in part on image data, first driving values
for light sources of the light source layer;
determine an effective luminance pattern of the light source layer by
a method including:
for each of a plurality of light sources of the light source
layer:
separately determining contributions to the effective
luminance pattern of higher-order and lower-order parts of a
set of point spread function values; and,
combining the contributions to the effective
luminance pattern of the higher-order and lower-order point

-29-
spread function values to yield effective luminance pattern
data; and,
determine second driving values for the display layer based at least
in part on the effective luminance pattern data and the image data;
a first interface connected to the light source layer to apply the first
driving
values to the light source layer; and,
a second interface connected to the display layer to apply the second
driving values to the display layer.
45. Display apparatus according to claim 44 comprising a lookup table
comprising
values for the point spread function.
46. Display apparatus according to claim 45 wherein the point spread
function is
characterized by 16-bit data words.
47. Display apparatus according to claim 44 wherein the controller
comprises an 8-bit
graphics processor.
48. Display apparatus according to claim 47 wherein the controller is
configured to
determine the effective luminance pattern of the light source layer in the 8-
bit
domain using the 8-bit graphics processor.
49. Display apparatus according to claim 44 or 45 wherein the controller is
configured
to separately process first, second and third segments of the point spread
function,
the first segment comprising the higher order parts of the point spread
function, the
second segment comprising the lower order parts of the point spread function
for
points outside a radius, and the third segment comprising the lower order
parts of
the point spread function for points within the radius.
50. Display apparatus according to claim 49 wherein the higher order parts
of the point
spread function are zero outside of the radius.

-30-
51. Display apparatus according to claim 49 wherein the wherein the point
spread
function values comprise 16-bit values, the higher order parts of the point
spread
function comprise high byte parts of the 16-bit data words and the lower order
parts of the point spread function comprise low byte parts of the 16-bit data
words.
52. Display apparatus according to any one of claims 44 to 51 wherein the
higher-
order parts of the set of point spread function values have a narrower support
than
the lower-order parts of the set of point spread function values.
53. Display apparatus according to any one of claims 44 to 48 wherein the
light
sources of the light source layer comprise light-emitting diodes.
54. Display apparatus according to any one of claims 44 to 53 wherein the
display
layer comprises a light-transmissive panel
55. Display apparatus according to claim 54 wherein the display layer
comprises an
LCD panel.
56. Display apparatus according to any one of claims 44 to 55 wherein the
controller is
configured to determine the effective luminance pattern of the light source
layer
initially at a resolution lower than a resolution of the display layer and the
controller is configured to subsequently increase the resolution of the
effective
luminance pattern.
57. Display apparatus according to claim 56 wherein the controller is
configured to
increase the resolution of the effective luminance pattern to a resolution
matching
the resolution of the display layer.
58. Display apparatus according to claim 56 or 57 wherein the controller is
configured
to increase the resolution of the effective luminance pattern by upsampling
using
an iteratively-derived interpolation function.

Description

Note: Descriptions are shown in the official language in which they were submitted.


CA 02840548 2014-01-21
RAPID IMAGE RENDERING ON DUAL-MODULATOR
DISPLAYS
Technical Field
[0001] This invention pertains to systems and methods for
displaying images on displays of the type that have two modulators. A
first modulator produces a light pattern and a second modulator
modulates the light pattern produced by the first modulator to yield an
image.
Background
[0002] International patent publication WO 02/069030 published 6
September 2002 and international patent publication WO 03/077013
published 18 September 2003 disclose displays which have a modulated
light source layer and a modulated display layer. The modulated light
source layer is driven to produce a comparatively low-resolution
representation of an image. The low-resolution representation is
modulated by the display layer to provide a higher resolution image
which can be viewed by an observer. The light source layer may
comprise a matrix of actively modulated light sources, such as light
emitting diodes (LEDs). The display layer, which is positioned and
aligned in front of the light source layer, may be a liquid crystal display
(LCD).
[0003] If the two layers have different spatial resolutions (e.g. the
light source layer's resolution may be about 0.1% that of the display
layer) then both software correction methods and psychological effects
(such as veiling luminance) prevent the viewer from noticing the
resolution mismatch.
[0004] Electronic systems for driving light modulators such as
arrays of LEDs or LCD panels are well understood to those skilled in
the art. For example, LCD computer displays and televisions are
commercially available. Such displays and televisions include circuitry

CA 02840548 2014-01-21
- 2 -
for controlling the amount of light transmitted by individual pixels in an
LCD panel. The task of deriving driving from image data signals to
control a light source layer and display layer can be computationally
expensive. Deriving such signals can be executed by a processor of a
computer's video/graphics card, or by some other appropriate processor
integral to a computer, to the display itself or to a secondary device.
[0005] The task of deriving from image data signals to control a
light source layer and display layer can be computationally expensive.
Deriving such signals can be executed by a processor of a computer's
video/graphics card, or by some other appropriate processor integral to
a computer, to the display itself or to a secondary device. Performance
limitations of the processor can undesirably limit the rate at which
successive image frames can be displayed. For example, if the
processor is not powerful enough to process incoming video data at the
frame rate of the video data then an observer may detect small pauses
between successive frames of a video image such as a movie. This can
distract the observer and negatively affecting the observer's image
viewing experience.
[0006] There is a need for practical, cost effective and efficient
systems for displaying images on displays of the general type described
above.
Brief Description of Drawings
[0007] The appended drawings illustrate non-limiting embodiments
of the invention.
[0008] Figure 1 graphically depicts segmentation of a point spread
function (PSF) into narrow and wide base Gaussian segments.

CA 02840548 2014-01-21
- 3 -
[0009] Figures 2A, 2B and 2C graphically depict the splitting of a
16-bit point spread function (PSF) into two 8-bit (high and low byte)
segments.
[0010] Figure 3 graphically depicts the transitional behaviour of
8-bit high and low byte point spread function values relative to a 16-bit
range.
[0011] Figure 4 graphically depicts high and low byte point spread
functions corresponding to the point spread function depicted in
Figure 1.
[0012] Figure 5 graphically depicts application of an iteratively-
derived interpolation function to derive an interpolated effective
luminance pattern (ELP) closely approximating an actual effective
luminance pattern (ELP).
[0013] Figure 6 is a schematic diagram of a display.
[0014] Figure 7 is a flowchart illustrating a method for displaying
an image on a display having a controllable light source layer and a
controllable display layer.
[0015] Figure 8 is a flowchart illustrating a method for
determining an effective luminance pattern.
[0016] Figure 9 is a flowchart illustrating a method for
determining an effective luminance pattern or a component of an
effective luminance pattern.

CA 02840548 2014-01-21
- 4 -
Description
[0017] Throughout the following description, specific details are
set forth in order to provide a more thorough understanding of the
invention. However, the invention may be practiced without these
particulars. In other instances, well known elements have not been
shown or described in detail to avoid unnecessarily obscuring the
invention. Accordingly, the specification and drawings are to be
regarded in an illustrative, rather than a restrictive, sense.
[0018] The invention may be applied in a wide range of
applications wherein an image is displayed by producing a light pattern
that is determined at least in part by image data, and modulating the
light pattern to yield an image. The light pattern may be produced by
any suitable apparatus. Some examples include:
= A plurality of light sources driven by driver circuits that permit
brightnesses of the light sources to be varied.
= A fixed or variable light source combined with a reflection type
or transmission type modulator that modulates light from the light
source.
The following description relates to non-limiting example embodiments
in which the light pattern is produced on one side of an LCD panel by
an array of light-emitting diodes and the LCD panel is controlled to
modulate the light of the light pattern to produce a viewable image. In
this example, the array of LEDs can be considered to constitute a first
modulator and the LCD panel constitutes a second modulator.
[0019] In general, rendering image frames or a frame set for
display on an LED/LCD layer display entails the following
computational steps:
1. Obtaining image data (which may be full screen or partial screen
image data).

CA 02840548 2014-01-21
-5-
2. Deriving from the image data appropriate driving values for each
LED of the first modulator, using suitable techniques well known
to persons skilled in the art (e.g. nearest neighbour interpolation
which may be based on factors such as intensity and colour).
3. The derived LED driving values and the point spread functions of
LEDs on the LED layer as well as the characteristics of any
layers between the LED layer and the LCD layer are used to
determine the effective luminance pattern which will result on the
LCD layer when the LED driving values are applied to the LED
layer.
4. The image defined by the image data is then divided by the
effective luminance pattern to obtain raw modulation data for the
LCD layer.
5. In some cases, the raw modulation data is modified to address
issues such as non-linearities or other artifacts arising in either of
the LED or LCD layers. These issues can be dealt with using
suitable techniques well known to persons skilled in the art (e.g.
scaling, gamma correction, value replacement operations, etc.).
For example, creating the modified modulation data may involve
altering the raw modulation data to match a gamma correction
curve or other specific characteristics of the LCD layer.
6. Final modulation data for the LCD (which may be the raw
modulation data or the modified modulation data) and the driving
data for the LEDs are applied to drive the LCD and LED layers
to produce the desired image.
[0020] Various ways to reduce the computational cost of (i.e. to
speed up) generating the final modulation data for use in displaying
images are described herein. These include:
= Performing at least some parts of the computation in a lower
precision domain (for example, by performing computations in
the 8-bit domain instead of in the 16-bit domain); and,

CA 02840548 2014-01-21
- 6 -
= Implementing one or more of the options for efficiently
establishing an effective luminance pattern that are described
herein.
While these techniques may be implemented individually, any suitable
combinations of the techniques described herein may be used.
Effective Luminance Pattern Determination
[0021] The point spread function of each LED in an LED layer is
determined by the geometry of the LED. A simple technique for
determining an LED layer's total effective luminance pattern is to
initially multiply each LED's point spread function (specifically, the
point spread function of the light which is emitted by the LED and
passes through all optical structures between the LED and LCD layers)
by a selected LED driving value and by an appropriate scaling
parameter to obtain the LED's effective luminance contribution, for that
driving value, to each pixel on the LCD layer.
[0022] In this way, the luminance contributions of every LED in
the LED layer can be determined and summed to obtain the total
effective luminance pattern, on the LCD layer, that will be produced
when the selected driving values are applied to the LED layer.
However, these multiplication and addition operations are very
computationally expensive (i.e. time consuming), because the effective
luminance pattern must be determined to the same spatial resolution as
the LCD layer in order to facilitate the division operation of step 4
above.
[0023] The computational expense is especially great if the LED
point spread function has a very wide "support." The "support" of an
LED point spread function is the number of LCD pixels that are
illuminated in a non-negligible amount by an LED. The support can be
specified in terms of a radius, measured in LCD layer pixels, at which

CA 02840548 2014-01-21
- 7 -
the LED point spread function becomes so small that is perceptually
irrelevant to an observer. The support corresponds to a number of LCD
pixels that are illuminated in a significant amount by each LED.
[0024] For example, consider a hexagonal LED array in which the
centre of each LED is spaced from the immediately adjacent LEDs by a
distance equal to 50 of the LCD layer's pixels. If each LED has a point
spread function having a support of 150 LCD pixels then each pixel in
the center portion of the LCD layer will be illuminated by light from
approximately 35 of the LEDs. Calculation of the effective luminance
pattern for this example accordingly requires 35 operations for each
pixel of the LCD layer, in order to account for the light contributed to
each pixel by each relevant LED. Where the LCD layer has a high
spatial resolution, this is very computationally expensive (i.e. time
consuming).
Resolution Reduction
[0025] The time required to determine the effective luminance
pattern produced on the LCD can be reduced by computing the effective
luminance pattern at a reduced spatial resolution that is lower than that
of the high resolution image which is to appear on the LCD layer. This
is feasible because the point spread functions of individual light sources
are generally smoothly varying. Therefore, the effective luminance
pattern will be relatively slowly varying at the resolution of the LCD.
It is accordingly possible to compute the effective luminance pattern at a
lower resolution and then to scale the effective luminance pattern up to
a desired higher resolution, without introducing significant artifacts.
[0026] The scaling may be done using suitable linear, Gaussian or
other interpolation techniques. Such spatial resolution reduction yields
an approximately linear decrease in the computational cost of
establishing the effective luminance pattern. Many available

CA 02840548 2014-01-21
- 8 -
interpolation methods that can be used to scale up an effective
luminance pattern computed at a lower resolution are computationally
inexpensive as compared to the computational cost of computing the
effective luminance pattern at the resolution of the LCD or other second
light modulator.
[0027] Using the foregoing example, a 10-times resolution
reduction in both the width and height directions yields an approximate
100-times reduction in computational cost. This is because the total
number of pixels in the reduced resolution image is 100-times fewer
than the total number of pixels in the high resolution image which is to
appear on the LCD layer. Each pixel in the reduced resolution image
still receives light from 35 LEDs, necessitating 35 computational
operations per pixel¨but those operations are applied to 100-times
fewer pixels in comparison to a case in which the computations are
performed separately for every pixel in the actual high resolution image
which is to appear on the LCD layer.
Point Spread Function Decomposition
[0028] The computational cost of image rendering can also be
reduced by decomposing the point spread function of each light source
(e.g. each LED) into several components (e.g. by performing a
Gaussian decomposition) in such a way that the recombination of all of
the components yields the original point spread function. An effective
luminance pattern can then be determined separately for each
component. Once an effective luminance pattern has been determined
for each component, those effective luminance patterns can be
combined to produce a total effective luminance pattern. The
combination may be made by summing, for example.

CA 02840548 2014-01-21
- 9 -
[0029] Computing the effective luminance patterns contributed by
the components may be performed at the resolution of the LCD layer or
at a reduced resolution, as described above.
[0030] A speed benefit is attained even if the effective luminance
pattern for each component is computed at the resolution of the LCD
layer since hardware components specially adapted to perform rapid
computations based upon standard point spread functions (e.g. Gaussian
point spread functions) are commercially available. Such hardware
components are not normally commercially available for the typically
non-standard point spread function of the actual LEDs in the display's
LED layer¨necessitating resort to considerably slower computational
techniques using general purpose processors.
[0031] A greater speed benefit is attained if the resolution
reduction technique described above is used to determine an effective
luminance pattern for each component. Moreover, different spatial
resolutions can be applied to different components of the point spread
functions to yield even greater speed benefits. For example, Figure 1
depicts (solid line) an example LED point spread function having a
steep central portion 10 and a wide tail portion 12. In this situation, the
actual point spread function can be decomposed into a narrow base
Gaussian component 14A and a wide base Gaussian component 14B, as
depicted.
[0032] The wide base Gaussian component 14B (dotted line)
contributes relatively little image intensity, in comparison to narrow
base Gaussian segment 14A (dashed line). Further, wide base Gaussian
component 14B is more slowly varying than narrow base Gaussian
component 14A. Accordingly, an effective luminance pattern for
narrow base Gaussian component 14A may be determined at a
moderately high spatial resolution while an effective luminance pattern

CA 02840548 2014-01-21
- 10 -
for the wide base Gaussian component 14B can be computed at a
significantly lower spatial resolution. This preserves a substantial
portion of the image intensity information contained in narrow base
Gaussian component 14A and is still relatively fast since the effective
support of the narrow base Gaussian segment is small and thus few
LCD pixels are covered by that component. By contrast, since wide
base Gaussian component 14B contains relatively little image intensity
information, that component can be processed relatively quickly at low
resolution without substantially degrading the resolution of the total
effective luminance pattern produced by combining the patterns derived
for each component.
8-bit Segmentation
[0033] Image data is typically provided in 16-bit word form.
High-end (i.e. more expensive) graphic processors typically perform
computations in the 16-bit domain. Such processors may have dedicated
16-bit or floating point arithmetic units that can perform 16-bit
operations quickly. The need for a high-end processor capable of
performing 16-bit operations quickly can be alleviated by computing the
effective luminance pattern in the 8-bit domain. Such computations can
be performed reasonably quickly by less expensive processors.
[0034] Each LED's point spread function is a two dimensional
function of intensity versus distance relative to the center of the LED.
Such a point spread function may be characterized by a plurality of
16-bit data words. Where the point spread function is represented by a
look up table, many 16-bit values are required to define the point spread
function; for example, one value may be provided for every LCD pixel
lying on or within a circle centered on the LED and having a radius
corresponding to the support of the point spread function.

CA 02840548 2014-01-21
- 11 -
[0035] Each one of those 16-bit data words has an 8-bit high byte
component and an 8-bit low byte component (any 16-bit value A can be
divided into two 8-bit values B and C such that A = B *28 + C, where
B is the "high byte" and C is the "low byte"). The 8-bit values are
preferably extracted only after all necessary scaling and manipulation
operations have been applied to the input 16-bit data. Figure 2A depicts
a 16-bit point spread function; Figures 2B and 2C respectively depict
the 8-bit high and low byte components of the Figure 2A 16-bit point
spread function.
[0036] A 16-bit data word is capable of representing integer values
from 2 -1 to 216-1 (i.e. from 0 to 65535). An 8-bit byte is capable of
representing integer values from 2 -1 to 28-1 (i.e. from 0 to 255). The
"support" (as previously defined) of a point spread function
characterized by an 8-bit high byte component is much smaller
(narrower) than the support of the point spread function as a whole.
This is because the 8-bit high byte component reaches the lowest value
(zero) of its 255 possible values, when the 16-bit data word
characterizing the point spread function as a whole reaches the value
255 out of its range of 65535 possible values. The remaining 255
values are provided by the low byte component with the high byte
component's value equal to zero. The effective luminance pattern
corresponding to the narrow base 8-bit high byte component can
accordingly be rapidly determined, without substantial loss of image
intensity information. The resolution reduction and/or other techniques
described above may be used to further speed up the determination of
the effective luminance pattern for the 8-bit high byte component.
[0037] The support of a point spread function characterized by an
8-bit low byte component is comparatively wide. Specifically, although
the 8-bit low byte component has only 255 possible values, those values
decrease from 255 to 0 (out of 65535 values for the point spread

CA 02840548 2014-01-21
- 12 -
function as a whole) and those 255 values correspond to the 255 lowest
intensity levels (i.e. levels at which the value of the high byte
component is equal to zero). Those 255 levels represent the values of
the point spread function in its peripheral parts.
[0038] The low byte component can be separated into two regions.
A central region, lying within the boundary on which the point spread
function characterized by the high byte component reaches zero. In the
central region the low-byte component typically varies in an irregular
saw-tooth pattern (as depicted in Figure 3) if the original 16-bit point
spread function is reasonably smooth. This is because, in the central
region, the portion of the point spread function characterized by the low
byte component augments the portion of the point spread function
characterized by the high byte component.
[0039] For example, consider a transition from the 16-bit value
10239 to the 16-bit value 9728. The 16-bit value 10239 has a high byte
component value of 39 and a low byte component value of 255 (i.e.
39*256 +255 =10239). Consequently, the low byte component's
contribution to the point spread function is initially 255 and the high
byte component's contribution is initially 39. The value of the high
byte component's contribution remains at 39, while the value of the low
byte component's contribution smoothly decreases from 255 to 0¨the
point at which the original 16-bit point spread function has the value
9984 (i.e. 39*256 + 0). The value of the high byte component's
contribution to the point spread function then changes smoothly from 39
to 38, but that change is accompanied by an abrupt change (from 0 to
255) in the value of the low byte component's contribution to the point
spread function.
[0040] As seen in Figure 4, inside a radius R of the original point
spread function (and where the value of the high byte component's

CA 02840548 2014-01-21
- 13 -
contribution to the point spread function is non-zero) the resulting
saw-tooth pattern of the low byte component's contribution to the point
spread function is characteristic of the original point spread function.
Outside the radius R, the value of the high byte component's
contribution to the point spread function is zero, and the value of the
low byte component's contribution changes smoothly.
[0041] The contributions from the low-byte component of the point
spread function can be processed differently in these two regions (i.e.
the regions inside and outside the radius R) to avoid unwanted artifacts.
For example, to preserve a substantial portion of the image intensity
information contained in the region inside the radius R, the effective
luminance pattern for that region is preferably determined using the
same relatively high resolution used to determine the effective
luminance pattern for the high byte component's contribution to the
point spread function, as previously described. By contrast, the
effective luminance pattern for the region outside the radius R can be
determined using a much lower resolution, without substantial loss of
image intensity information.
[0042] After the three point spread function segments (i.e. the high
byte component, the region of the low byte component inside the radius
R, and the region of the low byte component outside the radius R) have
been processed as aforesaid, the results are individually up-sampled to
match the resolution of the LCD layer, then recombined with
appropriate scaling factors being applied. Recombination typically
involves summation of the values for the two low byte component
regions and the value for the high byte component, after the value for
the high byte component has been multiplied by 256.

CA 02840548 2014-01-21
- 14 -
Interpolation
[0043] If an effective luminance pattern value is determined using
a resolution lower than the resolution of the LCD layer, it is necessary
to up-sample that value to match the resolution of the LCD layer.
Interpolation techniques for up-sampling low resolution images into high
resolution images are well known, with both linear and Gaussian based
techniques being common. Although such prior art techniques can be
used in conjunction with the above described techniques, accuracy, or
speed, or both may be improved by utilizing an interpolation technique
which is optimized for a particular display configuration. Optimization
facilitates higher resolution image compression, minimizes introduction
of unwanted interpolation artifacts, and reduces the image rendering
time. In extreme cases, an interpolation technique can be used to
reduce the resolution of the effective luminance pattern resolution to
match the resolution of the LED layer.
[0044] Prior art interpolation techniques are often restricted to use
with specific pre-interpolation data, or to use with specific interpolation
functions. The interpolation techniques used to match the resolution of
the effective luminance pattern to that of the LCD display do not need
to satisfy such restrictions, provided convolution of the pre-interpolation
data with the selected interpolation function will yield an effective
luminance pattern having adequate similarity to the actual effective
luminance pattern.
[0045] The required degree of similarity depends on the display
application. Different applications require different degrees of
similarity¨in some applications relatively small deviations may
unacceptably distract an observer, whereas larger deviations may be
tolerable in other applications (such as applications involving television
or computer game images in which relatively large deviations
nonetheless yield images of quality acceptable to most observers).

CA 02840548 2014-01-21
- 15 -
Consequently, it is not necessary to apply the interpolation technique
directly to the actual LED driving values or to the actual LED point
spread function.
[0046] For example, Figure 5 depicts the result obtained by using
an iteratively-derived interpolation technique to reduce the resolution of
the effective luminance pattern to match the resolution of the LED
layer. The pixel values at the LED layer's resolution are not the LED
driving values¨they are the luminance values of the effective luminance
pattern before interpolation. The interpolation function can be
determined using standard iteration methods and a random starting
condition. As seen in Figure 5, convolution of the iteratively-derived
interpolation function with the effective luminance pattern values yields
results which are reasonably close to the actual effective luminance
pattern.
[0047] Many different interpolation techniques can be used. There
need not be any correlation between the interpolation function and the
LEDs' point spread function, the LED driving values, or any other
characteristic of the display, provided the selected interpolation function
and the input parameters selected for use with that function yields a
result reasonably close to the actual effective luminance pattern.
Example Embodiments
[0048] Figures 6 to show some example embodiments of the
invention. Figure 6 shows a display 30 comprising a modulated light
source layer 32 and a display layer 34. Light source layer 32 may
comprise, for example:
= an array of controllable light sources such as LEDs;
= a fixed-intensity light source and a light modulator disposed to
spatially modulate the intensity of light from the light source;
= some combination of these.

CA 02840548 2014-01-21
- 16 -
In the illustrated embodiment, light source layer 32 comprises an array
of LEDs 33.
[0049] Display layer 34 comprises a light modulator that further
spatially modulates the intensity of light incident on display layer 34
from light source layer 32. Display layer 34 may comprise an LCD
panel or other transmission-type light modulator, for example. Display
layer 34 typically has a resolution higher than a resolution of light
source layer 32. Optical structures 36 suitable for carrying light from
light source layer 32 to display layer 34 may be provided between light
source layer 32 and display layer 34. Optical structures 36 may
comprise elements such as open space, light diffusers, collimators, and
the like.
[0050] In the illustrated embodiment a controller 40 comprising a
data processor 42 and suitable interface electronics 44A for controlling
light source layer 32 and 44B for controlling display layer 34 receives
image data 48 specifying images to be displayed on display 30.
Controller 40 drives the light emitters (e.g. LEDs 33) of light source
layer 34 and the pixels 35 of display layer 34 to produce the desired
image for viewing by a person or persons. A program store 46
accessible to processor 42 contains software instructions that, when
executed by processor 42 cause processor 42 to execute a method as
described herein.
[0051] Controller 40 may comprise a suitably programmed
computer having appropriate software/hardware interfaces for
controlling light source layer 32 and display layer 34 to display an
image specified by image data 48.
[0052] Figure 7 shows a method 50 for displaying image data on a
display of the general type shown in Figure 6. Method 50 begins by

CA 02840548 2014-01-21
- 17 -
receiving image data 48 at block 52. In block 54 first driving signals for
light source layer 32 are derived from image data 48. Suitable known
methods may be applied to obtain the first driving signals in block 54.
[0053] In block 56 method 50 computes an effective luminance
pattern. The effective luminance pattern may be computed from the first
driving signals and known point spread functions for the light sources
of light source layer 32. Block 56 computes the effective luminance
pattern at a resolution that is lower than a resolution of display layer 34.
For example, block 56 may compute the effective luminance pattern at
a resolution that is a factor of 4 or more smaller in each dimension (in
some embodiments a factor in the range of 4 to 16 smaller in each
dimension) than the resolution of display layer 34.
[0054] In block 60 the effective luminance pattern computed in
block 56 is upsampled to the resolution of display layer 34. This may be
done through the use of any suitable interpolation technique for
example. In block 62 second driving signals for the display layer are
determined from the upsampled effective luminance pattern and the
image data. The second driving signals may also take into account
known characteristics of the display layer and any desired image
corrections, colour corrections or the like.
[0055] In block 64 the first driving signal obtained in block 54 is
applied to the light source layer and the second driving signals of block
62 are applied to the display layer to display an image for viewing.
[0056] Figure 8 shows a method 70 for computing an effective
luminance pattern. Method 70 may be applied within block 56 of
method 50 or may be used in other contexts. Method 70 begins by
computing an ELP for each component of the point spread function for
the light sources of light source layer 32 (blocks 72A, 72B and 72C -

CA 02840548 2014-01-21
- 18 -
collectively blocks 72). Blocks 72 may be performed in any sequence or
may be performed in parallel with one another. Figure 8 shows three
PSF components 73A, 73B and 73C and three corresponding blocks 72.
The method could be practised with two or more PSF components 73.
[0057] The components of the point spread function (PSF) will
typically have been predetermined. A representation of each component
is stored in a location accessible to processor 42. Each of blocks 72 may
comprise, for each light source of light source layer 32, multiplying
values that define a component of the point spread function by a value
representing the intensity of the light source. In block 74 the effective
luminance patterns determined in blocks 72 are combined, for example
by summing, to yield an overall estimate of the effective luminance
pattern that would be produced by applying the first driving signals to
light source layer 32.
[0058] Figure 9 illustrates a method 80 that may be applied for
computing effective luminance patterns. Method 80 may be applied to:
= computing the effective luminance pattern in block 56 of method
50; or
= computing the effective luminance patterns for individual
components of a point spread function in blocks 72 of method 70;
or
= applied in other contexts.
[0059] Method 80 begins in block 82 with data characterizing a
point spread function (or a PSF component) for a light source of light
source layer 32 and data indicative of how intensely the light source will
operate under the control of the first driving signals. Method 80
combines these values (e.g. by multiplying them together) to obtain a
set of values characterizing the contribution of the light source to the
effective luminance pattern at various spatial locations.

CA 02840548 2014-01-21
- 19 -
[0060] Block 84 obtains high-order and low-order components of
the resulting values. In some embodiments, the resulting values are 16-
bit words, the high-order component is an 8-bit byte and the low-order
component is an 8-bit byte.
[0061] Contributions to the ELP are determined separately for the
high-order and low-order components in blocks 86 and 88. For each
light source, the area of support for which values are included in the
high-order contribution of 86 is typically significantly smaller than the
area of support for which values are included in the low-order
contribution of block 88.
[0062] Block 88 typically computes the low-order contribution for
points located within the area of support of the high-order contribution
(block 90) separately than for points located outside of the area of
support of the high-order contribution (block 92). Blocks 86, 90 and 92
may be performed in any order or simultaneously.
[0063] In block 94 the contributions from blocks 86, 90 and 92 are
combined to yield an overall ELP. The computations in blocks 86 90
and 92 may be performed primarily or entirely in the 8-bit domain (i.e.
using 8-bit operations on 8-bit operands) in the case that the high-order
and low-order components are 8-bit bytes or smaller.
[0064] Certain implementations of the invention comprise
computer processors which execute software instructions which cause
the processors to perform a method of the invention. For example, one
or more processors in a computer or other display controller may
implement the methods of Figures 7, 8 or 9 by executing software
instructions in a program memory accessible to the processors. The
invention may also be provided in the form of a program product. The

CA 02840548 2014-01-21
- 20 -
program product may comprise any medium which carries a set of
computer-readable signals comprising instructions which, when
executed by a data processor, cause the data processor to execute a
method of the invention. Program products according to the invention
may be in any of a wide variety of forms. The program product may
comprise, for example, physical media such as magnetic data storage
media including floppy diskettes, hard disk drives, optical data storage
media including CD ROMs, DVDs, electronic data storage media
including ROMs, flash RAM, or the like or transmission-type media
such as digital or analog communication links. The computer-readable
signals on the program product may optionally be compressed or
encrypted.
[0065] Where a component (e.g. a member, part, assembly,
device, processor, controller, collimator, circuit, etc.) is referred to
above, unless otherwise indicated, reference to that component
(including a reference to a "means") should be interpreted as including
as equivalents of that component any component which performs the
function of the described component (i.e., that is functionally
equivalent), including components which are not structurally equivalent
to the disclosed structure which performs the function in the illustrated
exemplary embodiments of the invention.
[0066] As will be apparent to those skilled in the art in the light
of
the foregoing disclosure, many alterations and modifications are
possible in the practice of this invention. For example,
= The light source layer may comprise a number of different types
of light source that have point spread functions different from one
another;
= The display may comprise a colour display and the computations
described above may be performed separately for each of a
number of colours.

CA 02840548 2014-01-21
- 21 -
[0067] While a number of example aspects and embodiments have
been discussed above, those of skill in the art will recognize certain
modifications, permutations, additions and sub-combinations thereof. It
is therefore intended that the following appended claims and claims
hereafter introduced are interpreted to include all such modifications,
permutations, additions and sub-combinations as are within their true
scope.

Representative Drawing
A single figure which represents the drawing illustrating the invention.
Administrative Status

2024-08-01:As part of the Next Generation Patents (NGP) transition, the Canadian Patents Database (CPD) now contains a more detailed Event History, which replicates the Event Log of our new back-office solution.

Please note that "Inactive:" events refers to events no longer in use in our new back-office solution.

For a clearer understanding of the status of the application/patent presented on this page, the site Disclaimer , as well as the definitions for Patent , Event History , Maintenance Fee  and Payment History  should be consulted.

Event History

Description Date
Common Representative Appointed 2019-10-30
Common Representative Appointed 2019-10-30
Grant by Issuance 2018-03-13
Inactive: Cover page published 2018-03-13
Pre-grant 2018-01-24
Inactive: Final fee received 2018-01-24
Letter Sent 2017-08-15
Notice of Allowance is Issued 2017-08-15
Notice of Allowance is Issued 2017-08-15
Inactive: Approved for allowance (AFA) 2017-08-10
Inactive: Q2 passed 2017-08-10
Amendment Received - Voluntary Amendment 2017-02-24
Inactive: Report - No QC 2016-09-01
Inactive: S.30(2) Rules - Examiner requisition 2016-09-01
Change of Address or Method of Correspondence Request Received 2016-05-30
Amendment Received - Voluntary Amendment 2016-02-10
Inactive: Report - No QC 2015-08-11
Inactive: S.30(2) Rules - Examiner requisition 2015-08-11
Letter sent 2014-02-26
Inactive: Cover page published 2014-02-17
Inactive: First IPC assigned 2014-02-07
Inactive: Divisional record deleted 2014-02-07
Inactive: Divisional record deleted 2014-02-07
Divisional Requirements Determined Compliant 2014-02-07
Inactive: IPC assigned 2014-02-07
Inactive: IPC assigned 2014-02-07
Application Received - Regular National 2014-02-04
Letter Sent 2014-02-04
Letter sent 2014-02-04
Inactive: Office letter 2014-02-04
Letter Sent 2014-02-04
Letter Sent 2014-02-04
Letter Sent 2014-02-04
Letter Sent 2014-02-04
Letter Sent 2014-02-04
Application Received - Divisional 2014-01-24
Inactive: Pre-classification 2014-01-21
Request for Examination Requirements Determined Compliant 2014-01-21
Application Received - Divisional 2014-01-21
All Requirements for Examination Determined Compliant 2014-01-21
Application Received - Divisional 2014-01-21
Application Published (Open to Public Inspection) 2006-02-02

Abandonment History

There is no abandonment history.

Maintenance Fee

The last payment was received on 2017-05-01

Note : If the full payment has not been received on or before the date indicated, a further fee may be required which may be one of the following

  • the reinstatement fee;
  • the late payment fee; or
  • additional fee to reverse deemed expiry.

Please refer to the CIPO Patent Fees web page to see all current fee amounts.

Owners on Record

Note: Records showing the ownership history in alphabetical order.

Current Owners on Record
DOLBY LABORATORIES LICENSING CORPORATION
Past Owners on Record
GREGORY JOHN WARD
HELGE SEETZEN
LORNE A. WHITEHEAD
WOLFGANG HEIDRICH
Past Owners that do not appear in the "Owners on Record" listing will appear in other documentation within the application.
Documents

To view selected files, please enter reCAPTCHA code :



To view images, click a link in the Document Description column. To download the documents, select one or more checkboxes in the first column and then click the "Download Selected in PDF format (Zip Archive)" or the "Download Selected as Single PDF" button.

List of published and non-published patent-specific documents on the CPD .

If you have any difficulty accessing content, you can call the Client Service Centre at 1-866-997-1936 or send them an e-mail at CIPO Client Service Centre.


Document
Description 
Date
(yyyy-mm-dd) 
Number of pages   Size of Image (KB) 
Abstract 2018-02-02 1 24
Description 2014-01-21 21 981
Abstract 2014-01-21 1 24
Claims 2014-01-21 11 402
Drawings 2014-01-21 9 104
Representative drawing 2014-02-14 1 8
Cover Page 2014-02-17 1 43
Claims 2016-02-10 9 340
Cover Page 2018-02-16 2 44
Maintenance fee payment 2024-04-18 49 2,035
Acknowledgement of Request for Examination 2014-02-04 1 175
Courtesy - Certificate of registration (related document(s)) 2014-02-04 1 103
Courtesy - Certificate of registration (related document(s)) 2014-02-04 1 103
Courtesy - Certificate of registration (related document(s)) 2014-02-04 1 103
Courtesy - Certificate of registration (related document(s)) 2014-02-04 1 103
Courtesy - Certificate of registration (related document(s)) 2014-02-04 1 103
Commissioner's Notice - Application Found Allowable 2017-08-15 1 163
Correspondence 2014-02-04 1 16
Correspondence 2014-02-04 1 38
Correspondence 2014-02-26 1 39
Examiner Requisition 2015-08-11 5 263
Amendment / response to report 2016-02-10 13 509
Correspondence 2016-05-30 38 3,505
Examiner Requisition 2016-09-01 4 209
Amendment / response to report 2017-02-24 6 235
Final fee 2018-01-24 2 59