SCALEABLE RESOLUTION MOTION IMAGE RECORDING AND STORAGE SYSTEM

SYSTEME D'ENREGISTREMENT ET DE STOCKAGE D'IMAGES ANIMEES A ECHELLE DE RESOLUTION REGLABLE

English Abstract

A system for scaling resolution of an image input stream to a desired
resolution is provided. The system comprises a processor for pre-processing
the image input stream; an IO system for receiving the image input stream; an
input analog converter for converting the image input stream, a frame buffer
for storing image frames, a router for decorrelating the image input stream
and dividing the stream among one or more processors, a temporal processor for
performing temporal transforms of the image input stream, a DMA channel for
delivering the encoded data to a storage device, and a local storage device
for storing the processed image input stream. The method uses the described
system components to create a scalable resolution motion image recording that
can be stored and playback on any number of different playback devices.

French Abstract

L'invention concerne un système permettant de régler la résolution d'un flux d'images d'entrée à une échelle de résolution désirée. Ce système comprend un processeur permettant de prétraiter le flux d'images d'entrée, un système d'E/S qui reçoit le flux d'images d'entrée, un convertisseur analogique d'entrée qui convertit le flux d'images d'entrée, un topogramme binaire qui stocke les images individuelles, un routeur qui effectue une décorrélation du flux d'images d'entrée et répartit ce flux entre un ou plusieurs processeurs, un processeur temporel qui effectue les transformations temporelles du flux d'images d'entrée, un canal DMA qui envoie les données codées dans la mémoire, et une mémoire locale qui stocke le flux d'images d'entrée traité. L'invention concerne également un procédé qui permet de réaliser, au moyen des composantes du système décrit, un enregistrement d'images animées à échelle de résolution réglable, qui peut être stocké et visionné dans n'importe quel dispositif lecture.

Claims

I claim:
1. An system for scaling resolution of an image input stream, the system
comprising:
a processor for pre-processing the image input stream;
coupled to the processor, an IO system for receiving the image input stream;
coupled to the IO system, an input analog converter;
coupled to the input analog converter, a frame buffer for storing image
frames;
coupled to the input analog converter, a router for decorrelating the image
input
stream and dividing the stream among one or more processors;
coupled to the muter, a temporal processor for performing temporal transforms
of the image input stream;
coupled to the temporal processor, a DMA channel for delivering the encoded
data to a
storage device; and
coupled to the DMA channel, a local storage device for storing the processed
image input stream.
2. The system of claim 1, wherein the input analog converter further comprises
an
analog bandsplitter for splitting the image input stream into multiple
streams.
3. The system of claim 1, wherein the IO system further comprises a color
space
converter for decorrelating the color components.
4. The system of claim 1, wherein system comprises four DMA channels.
5. The system of claim 1, wherein the system comprises more than one temporal
processor.
6. The system of claim 1, wherein the system further comprises, coupled to the
muter and the spatial transform processor, a temporal transform module.
7. A method for processing an image input stream, the method comprising the
steps of:
receiving a desired image input stream conversion rate;
converting the image input stream into one or more representational streams;
calculating a frame conversion rate;
responsive to the frame conversion rate being lower than the desired
conversion rate,
responsive to the image input stream components being correlated, converting
the color space of the image input stream; and
splitting the image input stream into one or more representational streams;
7

routing the image streams to one or more signal processors;
performing temporal transform processing on the representational streams; and
storing the representation streams post-temporal processing.
8. The method of claim 7, wherein the step of splitting the image input stream
into
two representational streams comprises splitting the image stream into four
quad component
representations.
9. The method of claim 7, wherein the step of splitting the image input stream
into
two representational streams comprises applying an analog bandsplit to the
stream.
10. The method of claim 7, wherein the step of performing temporal transform
processing on the representational streams comprises using a separate temporal
processor to
process each representational stream.
11. The method of claim 7, wherein the step of performing temporal transform
processing on the representational streams includes performing a spatial
multiband transform
of the stream.
12. The method of claim 11, wherein the step of performing temporal transform
processing on the representational streams includes quantifying the data in
the stream.
13. The method of claim 12, wherein the step of performing temporal transform
processing on the representational streams includes entropy encoding the
stream.
14. The method of claim 7, wherein the step of storing the streams comprises:
calculating the data budget of a first storage device used to store the data;
responsive to the stream requiring a data budget higher than is supported by
the first
storage device, queuing the data to a second storage device until the first
storage device is able
to accept the stream.
8

Description

CA 02361474 2001-08-02
WO 00/46978 PCT/US00/03103
SCALEABLE RESOLUTION MOTION IMAGE RECORDING
AND STORAGE SYSTEM
Inventor: Kenbe D. Goertzen
Related Application
The subject matter of this application is related to the subject matter of the
following
commonly owned applications: Serial Number 09/112,668, attorney docket number
3486,
titled "Apparatus And Method For Entropy Coding", filed on July 9, 1998, also
by Kenbe
Goertzen; Serial Number , attorney docket number 4753, titled "A System And
Method For Improving Compressed Image Appearance Using Stochastic Resonance
And
Energy Replacement", filed concurrently, also by Kenbe Goertzen; Serial Number
,
attorney docket number 4755, titled "Optimized Signal Quantification", filed
concurrently,
also by Kenbe Goertzen; and, Serial Number , attorney docket number 4756,
titled
"Quality Priority Image Storage and Communication", filed concurrently, also
by Kenbe
Goertzen; the contents of which are incorporated by reference as if fully
disclosed herein.
Technical Field
This invention pertains to the field of digital signal compression and
quantification. More specifically, the present invention related to a system
and method that
provides scaleable resolution image recording and storage.
Shortcomings of Prior Art
Electronic motion image recorders have traditionally been designed for one or
at most
only a few motion image formats. With the advent of various high definition
video, medical,
scientific, and industrial formats, there is a requirement for single systems
which can support a
broad range of motion image formats. As motion image frame rates, image sizes,
and pixel
resolutions increase, there is a tremendous increase in the amount of data
that must be
maintained to represent the motion image stream in the sample domain. This
places a large
burden on the processing, storage, and communications costs to support these
data sets at the
desired resolution.
Summary of Invention
The present invention provides an efficient and cost effective system which
can be
configured to effectively support all motion image formats. The system can be
modularly
expanded to increase throughput, and can trade off system throughput between
the number of
image streams, their frame rate, frame resolution, and pixel resolution. The
system uses
subband methods throughout to support variable image size and frame rate
recording. This

CA 02361474 2001-08-02
WO 00/46978 PCT/ZJS00/03103
allows a single image stream to be divided into multiple lower rate streams to
allow more
reasonable processing rates. The present invention also enables the
application of a variable
sized array of standardized image processing components.
More specifically, optimized storage and efficient communication is achieved
by
storing image streams in the information domain, rather than the sample
domain. This
typically provides a dramatic reduction in storage and communication
requirements, while still
providing a guaranteed recording quality. Unlike conventional video tape
recorders, this
system can also place recordings onto the same removable storage medium at any
desired
image resolution, frame rate, and quality.
Brief Description of the Drawings
Detailed Description
The following description describes the method of the present invention as
performed
by the system of the present invention. The method described, however, could
also be
performed by an alternate image processing device.
The following provides a list of components that may be included in the system
of the
present invention. The function accomplished by each component is also
provided. As will be
appreciated by one skilled in the art, this list is neither exhaustive nor
inclusive and other
components that provide similar functionality, such as other band limiters,
band sputters or
color space conversion modules, may also be added.
Component Function
Input Bandsplit Input converters and optional analog bandsplit
Deframer digital data stream in
Color space optional RGB -> CbYCr
Bandlimit 1D or 2D Bandlimit and arbitrary rate conversion of components
Bandsplit Full band 2 pixel color to half~and chroma dual pixel "quad"
CbYCr -> CbLCrH
Full band 4 pixel color to visual space "quad"
2

CA 02361474 2001-08-02
WO 00/46978 PCT/US00/03103
CbYCr -> CbLCrD
Monochrome to bandsplit "quad"
MM -> --L--H
Monochrome to 2D bandsplit "quad"
MMMM -> LHVD
Inverse Pulldown Sample only a portion of the input fields
or frames
Framebuffer Buffer
and optional time
multiplex frames
Interlace ProcessingInterlaced image processing, to allow
interlaced image to be
processed as progressive
Router Routes
N streams to N
IP channels
Using time multiplexed frames
Using simple (Harry temporal bandsplit
of frames
Temporal IP Temporal transform and optional time
multiplex of frames
Interlace ProcessingInterlaced image processing, to allow
interlaced image to be
processed as progressive
Spatial IP Spatial
transform
Quantification Component and temporal quantification
as required
Entropy coding Entropy coding, potentially for each
temporatial and spatial
component
DMA channels Provides channels for data movement
RAM Queues Peak buffering
Disk Queues Peak buffering
Disk,Tape, Final output
or Channel
The IO subsystems accept motion image input streams in one of numerous pixel
formats.
RGBA Red, Green, Blue, and Alpha
YCbCrA Luminance, Blue color difference, Red Color difference, and Alpha
YC Luminance and alternating color difference
M Monochrome
MM Two sequential monochrome samples
MMMM Four sequential monochrome samples
3

CA 02361474 2001-08-02
WO 00/46978 PCT/US00/03103
The input analog converter can operate at up to a fixed rate of N conversions
per
second, where N is determined by the particular implementation. If a
conversion rate higher
than N is desired, the input system can use an analog bandsplit of the input
signal into two or
four streams at one half or one fourth rate to allow multiple channel
conversion at or above N.
In one embodiment, the digital IO system converts these input pixel streams
into quad
component representational streams. In each case, it is assumes that the
components have
been decorrelated and can be processed separately. If the components have not
been
decorrelated, however, color space conversion may be used to decorrelate the
color
components. This could also involve subband splitting certain color components
in order to
further decorrelate the components and spread the bandwidth evenly among the
available
channels. While splitting the pixel stream into quad component representations
is described as
one embodiment, the image processing resources can also process the pixel
stream by splitting
it into a number of different component representations. For example, the
present system may
1 S be used to process dual channel image streams. In these cases, the first
and third components
are assigned to the first channel, and the second and fourth are assigned to
the second.
CbYCrA Single color pixel, four component full band
CbLCrH Dual color pixel, two subband luma component (L and H), half band
chroma
Luma is bandsplit using either a Harr transform, an Odd near orthagonal 7 tap
filter, or an Odd near orthagonal 9 tap filter.
CbLCrD Quad color pixel, two halfband luma components (L and D), quarter band
chroma. Luma is limited to a diagonal square providing 1/2 the original
bandwidth, but full horizontal and vertical response. This is bandsplit using
a 2D
filter of 3x3 or 9x9 taps into a low half and a diagonal half. The color is
bandlimited to half
band in both dimensions.
LMMH Quad monochrome pixel, 1D four subband components
LHVD Quad monochrome pixel, 2D four subband components
The frame buffers can store 1 to N frames of the input stream and distribute 1
to N
frames among 1 to N signal processors. This allows time multiplexing image
processing
resources, or the support of temporal processing in the frame buffer or muter.
N is
determined by the required quad throughput divided by the quad throughput of
the individual
4

CA 02361474 2001-08-02
WO 00/46978 PCT/LJS00/03103
signal processors. This also allows for support of temporal processing in the
frame buffer or
muter.
The muter can accomplish simple spatial or temporal bandsplits. This
accomplishes
additional decorrelation and allows subdivision of the signal stream among
computing
resources. It can also be used to support image resolutions or throughputs
that exceed the
physical buffers or individual throughput supported by the image processing
modules.
This system can also perforni temporal transform processing. In the preferred
embodiment, a temporal processor is required for each quad stream which will
require
processing. The input to a temporal processor is a stream of image frames. The
output is a
stream of shuffled temporal transformed frames. For example, if one temporal
transform is
selected, an alternate stream of low and high temporal component frames is
produced. If two
temporal transforms are selected, a stream of low, midlow, midhigh, and high
temporal
component frames is produced. Temporal transforms can be inserted as desired
between the
muter and the spatial transform processor.
From I to N spatial signal processors accept the quad streams of frames. They
perform
a spatial multiband transform, quantify the data to the specified recording
signal quality and
entropy encode it. From 4 to 4N DMA channels deliver the encoded data to a RAM
buffer
which handles peak transfer loads.
The data can then be transferred to queuing or output files on disks, tapes,
or other
peripherals that are capable of storing digital information. If the data rate
is higher than the
specified peripheral can support, the information is queued to a faster
storage device until the
desired peripheral can accept the data. In the case of peak tolerant recording
at a fixed rate, a
phase delay is specified and a hierarchy of processes related to the hierarchy
of buffer devices
monitor the rate and potentially control the quality level. This allows the
process to be as
tolerant of peaks as possible. Each process in charge of buffering determines
whether the data
budget is being exceeded over the time scale of interest to this process. If
the local average
rate exceeds a set point, the quality level of the images being received by
the output or storage
device is reduced until the local average rate no longer exceeds the set
point. If the average is
5

CA 02361474 2001-08-02
WO 00/46978 PCT/US00/03103
below the set point, and the quality level is below the target, the quality
level of the images is
increased.
Example:
Here is an example of this unique storage process. The target data rate is 12
MB/S and a
quality level may be 66 dB. This would allow peaks rates as high as 80
megabyte per second
to occur for one frame time, peaks as high as 40 to occur for up to about a
second, and peaks
up to 20 megabyte per second for a few seconds. The steps up or down are all
added together
and to the target to obtain the next quality level. The quality level is re-
computed for every
frame.
Process Timescale Setpoint Peak Rate dB Steps
To RAM 4 Frames 40 MB/S 80 MB/S 3 dB
To Disk 4 second 20 MB/S 40 MB/S 1 dB
To Tape/Channel 30 second 10 MB/S 12 MB/S 1/2 dB
The playback process is a reversal of the recording process, with the
exception that playing
from a slower peripheral than required induces a preroll time while a faster
peripheral is used
to queue the required amount of data. A preroll time is the time needed to
transfer data from a
slower peripheral to a faster storage device. For example, if the data was
stored on a compact
disc, the data could be prerolled to RAM in order to provide playback at the
appropriate frame
rate and quality level.
Although the description above contains many detailed descriptions, these
descriptions
should not be construed as limiting the scope of the invention but merely as
providing
illustrations of some of the presently preferred implementations of this
invention. For
example, although this method was described with reference to standard motion
and still
images, this method can be used to optimize quantification of any signal
stream. Thus the
scope of the invention should be determined by the appended claims and their
legal
equivalents, rather than by examples given.
6