Note: Descriptions are shown in the official language in which they were submitted.
CA 02268238 1999-04-06
DIGITAL VIDEO LAN SYSTEM
The present invention relates to a video monitoring and display systems for
use in a LAN
environment, and more particularly to a digital video mixer and compressor
system.
S
BACKGROUND OF THE INVENTION
It is well known to provide video security systems in which video cameras are
used to
generate video signals representative of locations for which security
surveillance is desired. In a
typical system, some or all of the video signals are displayed on separate
video screens for
monitoring by security personnel. It is also known to record some or all of
the video signals on
video tape, either to provide evidentiary support for the observations by
security personnel or in
cases where "real-time" human monitoring of the signals is impractical or is
not desired.
However, video tape suffers from serious drawbacks as a storage medium,
particularly in
view of the large quantity of video information generated by video security
systems. A major
concern is the sheer quantity of tapes to be stored, especially when it is
desired to record signals
generated by a large number of surveillance cameras. lvloreover, in a large
system many video
tape recorders may be required, resulting in a large capital expenditure, and
also the need for
allocate space for the recorders. Another problem is the need to frequently
change tape cassettes.
In United States Patent 4,943,854 a Video surveillance system for selectively
selecting
processing and displaying the outputs of a plurality of TV cameras is
described. The multi-video
recorder incorporates a multiplexing means permitting selection of the video
signal from each of
a plurality of TV cameras according to a predetermined importance or priority,
and a priority
changed due to occurrence of a new event and transmission of the video image
thus selected over
a single line and recording it by a single video tape recorder by time shared
multiplexing the
video signal at every frame, field or unit of time. It also can reproduce or
demultiplex the video
signal recorded as multiplexed, with each designated TV camera or by
sequentially selecting the
video signal from each TV camera. The system also has a means for inserting in
the time shared
multiplexed video signal and control signal such as select information, event
information, time
information or the like, so it can selectively demultiple~x the video signals
recorded as
multiplexed, as desired.
CA 02268238 1999-04-06
In order to display full motion video, a frame rate of 30 frames per second
(fps) is
normally required. A limitation of the time-shared multiplexed systems is the
reduced frame rate
and hence choppy picture. For example, for four video inputs, a multiplexed
system will have a
reduced frame rate of approximately 7.5 frames per second. Furthermore, in
current systems if
groups of cameras are located a relatively remote area, the video signal has
to be fed back to a
central recording or monitoring station. This requires multiple coaxial cables
for video
transmission, or even if a multiplexing technique is used, a single long
length of coaxial cable is
required. Long cable runs require line amplifiers. In t>oth cases, the costs
are substantial.
A further disadvantage of current systems is that the images are recorded as
analog
signals on VHS or similar tapes, making archiving cumbersome.
Thus there is a need for a system that reduces the reliance on coaxial cable
for video
transmission, allows a distributed system architecture iinstead of linking all
cameras individually,
hence saving in cabling and line amplifier costs. Furthermore such a system
should provide
digital video or image recording on a computer mass storage device or the like
, including being
able to run under software control for providing flexibility of controlling
image routing or
viewing, recording, playback and archiving.
Current systems do not provide live speed and quality across multiple channels
nor do
they provide flexibility of a large number of cameras v~ithout losing frames.
SUMMARY OF THE INVENTION
The present invention seeks to provide a solution to the problem of combining
multiple
video inputs into a single display without a significant loss of frames.
In accordance with this invention there is provided a digital video system
comprising:
a) a video mixer for spatially mixing a plurality of analog video input
signals in real
time and compressing said mixed signals for transmission; and
b) a receiver for receiving and decompressing said compressed signals for
display in
near real time.
In accordance with a further embodiment of thc~ invention there is provided a
plurality of
video mixers coupled to a network and a transmitter for receiving the
compressed images from
the plurality of transmitters.
2
CA 02268238 1999-04-06
In accordance with a still further embodiment there is provided a video system
comprising:
a) a video digitizer for receiving and digitizing a plurality of analog video
inputs;
b) a mixer for receiving said plurality of digitized video signals and
spatially mixing
said signals in real time into a single frame of a predetermined n by m pixel
image;
c) a video compressor for compressing each said single frame images;
d) a transmitter for transmitting said images; and
e) a receiver for receiving and decompressing said transmitted images.
BRIEF DESCRIPTION OF THE DRAWINGS
A better understanding of the invention will be obtained by reference to the
detailed
description below in conjunction with the following drawings in which:
Figure 1 is a block diagram of a video transmitter subsystem according to an
embodiment of the invention;
Figure 2 is a schematic diagram of a singe frame;
Figure 3 is a software flow diagram for the video transmitter system;
Figure 4 is a schematic diagram of digital video system according to the
present
invention; and
Figures 5(a) - (t) illustrate various screen shots of a user interface.
DETAILED DESCRIPTION OF A PREFERRED EMBODIMENT
Referring to Figure 1 a video transmitter subsystem according to an embodiment
of the
invention is shown generally by numeral 1. The video subsysteml is a
standalone unit, which is
capable of spatially mixing four analog NTSC or PAL video inputs in real-time
into a single
frame of 640 x 480 pixels image. Each input image occupies a quadrant of the
frame 60 as
shown schematically in figure 2. The video subsysteml compresses each image
frame 60 using
wavelet compression based hardware for a programmable compression ration of
4:1 to 350:1.
3
CA 02268238 1999-04-06
An advantage of wavelet compression is the blocky artifacts, commonly
appearing in JPEG type
compression, are totally eliminated. The resulting wavelet compressed image
data is then
transmitted via a (100MB or better) Ethernet network in which plurality of
video transmitters
may be joined together.
In general, the video mixer is considered to have two main sections. First, is
a computer
(running QNX operating system) and second, is a PCI PC plug-in board which
processes the
incoming video images and performs wavelet compression. A number of video
mixers may be
installed in a LAN hosted for example by an NT '"'server. The server is
responsible for
retrieving and storing images obtained from the mixers. In this way software,
updates to the
video mixers may be performed from the central server or from the first mixer,
thereby
minimizing on-site maintenance of each mixer. The computer may be a PC
motherboard of a
baby AT type. The computer should include sufficient memory, hard disk space
and power
source to run the operating system and video transmitter application program
(discussed below).
This type of computer including a PCI interface and network adapter is well
known in the art and
will not be discussed further.
The video mixer 1 comprises an analog front end 4 having four independent
video inputs
A, B, C and D coupled to respective anti-alias passive circuits for lightly
low-pass filtering the
video inputs which are then fed into respective text overlay chips (not shown)
before being
coupled to respective video decoders 8 labeled Decode;rA, DecoderB, DecoderC
and DecoderD
respectively. In a preferred embodiment, the decoders are a Samsung KS0122
multistandard
video decoder and the text overlay chip is a uPD6465. Details of these devices
including
application notes are available in their respective data sheets incorporated
herein by reference.
Based on the above decoder device, the composite video signals are digitized
into 4:2:2 8-bit
YcrCb data format at a digitization frequency of 12.27 MHz by the decoders 8.
The digitization
is selected for a square pixel type and therefore the spatial resolution is
640x480 pixels per field.
The digitized outputs are coupled to a time base corrected spatial mixer 20,
which
comprises a plurality of FIFO registers 24 for storage of pixel data from the
analog front end; a
time base corrector 26 and a quadruple retrieval controller (QRC) 28. The
spatial mixer 20
assembles a 640x480 pixel frame which is comprised of four horizontally
decimated image fields
(of size 320x240) from each of the digitized outputs.
4
CA 02268238 1999-04-06
Each of the four digitized channels is coupled 1:o a dedicated bank of the
FIFO's such that
each bank stores at least 320x240 pixels. The FIFO banks in each of the
channels are coupled to
a common output video bus and are individually addressable such that any of
the 320x240 fields
from any of the channels may be retrieved. In a preferred embodiment, each
FIFO register bank
S consists of two uPD42280 FIFO chips. Each chip is capable of holding one
320x240 field of
pixel data. Details of these devices including application notes are available
in their data sheets
incorporated herein by reference.
The pixel data retrieval is asynchronous to any of the input video timing,
thus the time
base corrector 26 is coupled to the FIFO's and the decoders to control the
retl-ieval process when
the input video timing and the retrieved data timing drift with respect to
each other.
The retrieved data from the FIFO's are coupled to a wavelet compression engine
40,
which is implemented in an Analog Devices multiforniat video Codec chip, part
number
ADV601. The ADV601 accepts YcrCb pixel data from its video interface and
performs wavelet
compression thereon. The compressed data is output to a host bus interface 44.
Conversely the
chip may also perform decompression of data received. on its host bus
interface. The ADV601's
host interface is a 32-bit wide interface and consists of four 32-bit
registers. These registers are
I/O mapped onto a PCI host bus. Compressed data can thus be transferred using
PCI burst I/O
when an interrupt is generated by data being available in the ADV601 internal
FIFO.
Since the wavelet compression engine can take video on a field by field (i.e.
640x240)
basis, the quadruple image frame must be fed into the engine in two halves --
upper and lower
half. The retrieval algorithm for implementing this is implemented in the QRC.
It may be noted
that the spatial mixer may also be bypassed in order to provide full 640x480
fields to the wavelet
engine for compression. As described above, the quadruple image mixer is made
up of four
banks of field FIFOs (uPD42280). Each decoder channel has a dedicated FIFO
bank which is
capable of storing two quarter size fields of video. Thc; incoming digitized
video is written into
the FIFO bank using decoder timing. Since the stored video is of quarter size,
the video data
from the decoder is decimated first (by the decoder). Each bank of stored
video is read in
sequence to construct a quadruple image frame of sixty 640x480. The tope two
images are
retrieved as field-1 input to the wavelet engine while the bottom two images
are retl-ieved as
field-2. The reason that two banks of FIFO are needed for each decoder channel
is that TBC
function is implemented to avoid read and write collision caused by
asynchronous timing
CA 02268238 1999-04-06
between the input and ADV601 video timing. The TBC function is implemented in
the
firmware.
For single source compression, this quadruple mixer is bypassed. The video
data and
video sync signals are fed directly into the wavelet engine which is placed in
slave mode. Write
control signals are generated by an Altera chip, preferably 6016which is
capable of taking four
asynchronous clocks from each decoder.
Retrieval logic signals, ADV601 syncs and read control signals are generated
by another
Altera chip.
In quadruple mode,ADV601 is placed in master mode where all the video sync
signals
are generated. In single source mode, the sync signals are provided by the
video decoders.
During compression the ADV601 requires external numeric processing on the
statistics
of the current field data. This means that an embedded computer will evaluate
the bin-width
values based on the statistics of the field data. The bin-width statistics are
transferred via the
ADV601 host interface and then the PCI interface to or from the embedded
computer.
The PCI interface consists of a device, which is capable of bridging the on-
board devices
onto the PCI bus. It manages the I/O and memory mapping, read/write cycles and
DMA cycles.
It should be capable of bursting data at a rate of 20-30 MB per second.
Typically the PCI
interface is implemented by a PLX PCI9080 PCI accelerator chip.
A generals purpose 8-bit I/O port is provided to the PCI controller and is
configured as an
4 input 4 output bits. It is mapped on the PCI bus i/o, and is thus under the
control of the
embedded computer. This port may be used to control alarm equipment.
In a preferred embodiment, a video encoder 50 is provided for generating
composite and
S-video output signals for monitoring or for playback of the decompressed
video frame. The
encoder is a Brooktree BT866 device that is configured in a slave mode and is
coupled to the
timing from the ADV601 chip. Data for the encoder is tapped off the data
stream to the
aADV601 video bus. This data is non-interlaced, therefore a field FIFO
register 52 is needed to
convert the data back to interlaced video before it is fed to the encoder. The
FIFO registers 52
are implemented in four uPD42280 devices used to hold two fields of pixel
data.
Refernng now to figure 3, a software flow diagram for the video mixer 1 is
shown
generally by numeral 100. The software in general is centered on the
transmission of
compressed image data to the system host via the Ethernet LAN. The software
comprises an
6
CA 02268238 1999-04-06
application program 116 for executing high level commands from a remote host.
This includes
video channel switching, alarm ports reporting, transmission of wavelet data
to the remote host
when requested, Bin-width calculation, track burn-in time/date, general
maintenance of the
connection to the network and self diagnostics when requested by the host. The
software also
includes a number of low-level drivers and libraries.
A device driver module 102 is provided for the PCI 9080 chip for communication
between the video mixer and the computer. The driver 102 includes low-level
services for
handling PCI bus i/o, DMA and IRQ. It contains functions for translation
between physical and
linear memory, interrupt dispatcher and bus master control. Detailed
implementation of these
functions is well known in the art and will be not discussed further. The
software also includes a
I2C bus communication module 104 for obtaining data packets from the
application and
translating data into signal streams. A video control module 106 is provided
for controlling the
video hardware including decoder, encoder and field FIFO. The functions
implemented by this
module include selection of video capture mode, control of contrast and
brightness and
initialization of video chips, NTSC/PAL selection and such like. A TCP/IP
manager108 is
implemented as a socket for handling command and compressed data transmission
to the host. It
calls a QNX LAN driverl 10 to perform the data transmission. A wavelet control
module 112
implements functions to control the data transfer betwf;en the ADV601 and the
host memory,
compression ratio, ADV601 initialization and such like. A Time/Date port
control module 114 is
provided to load a desired burn-in time/date on each selected video channel.
Referring to figure 4, a schematic diagram of a digital video system is shown
generally
by numeral 400. The system comprises a plurality of video mixers 1 including
their respective
computer coupled to a host computer 404 via an Ethernet LAN 406. The host
computer 404
includes a software-based receiver, which communicates with the multiple
digital video
transmitters 1 on the Ethernet network to receive compressed digital images.
The received
images are decompressed in software and displayed 410 in near real time (12-15
frames per
second) and can be recorded to a hard disk 412 in real time (30 fps) in their
native compressed
form. In a preferred embodiment, a database is used for image storage,
retrieval, playback and
manipulation. If multiple receiver stations are coupled. to the network then
any station may have
access to any of the other receivers live or recorded data.
7
CA 02268238 1999-04-06
The receiver includes modules for setting alarrr~ points and outputs at each
video receiver.
In a preferred embodiment, the receiver software is implemented in Visual
Basic and the
underlying database is a Btrieve database engine with option to use other
proprietary types of
database engines. The receiver also includes image processing software for
displaying multiple
views of multiple cameras and a user input interface shown in figures 5 for
setting various
parameters such as security, camera scheduling, reports and such like.
Thus, it may be seen that the present invention provides a fully integrated
near real time
video surveillance, monitoring and capture system that is cost effective and
highly flexible.
Further, the mixer does not lose any frames during the actual mixing, i.e. at
30 fps. Other
configurations of the mixer may also be implemented, such as having one or
more mixer boards
plug directly into the receiver computer without using a LAN.
Although the invention has been described with reference to certain specific
embodiments, various modifications thereof will be apparent to those skilled
in the art without
departing from the spirit and scope of the invention as outlined in the claims
appended hereto.
8
