Note: Descriptions are shown in the official language in which they were submitted.
CA 02410748 2002-11-O1
1
AudioNideo-Over-IP Method, System and Apparatus
Field of the Invention
The present invention relates generally to audio/video transmission
techniques and applications, and more particularly to "Video-over-IP"
transmission methods and devices.
Background of the Invention
Internet audio/video streaming methods are generally poor in quality,
and lack stability and flexibility. While methods are known that deliver video
streaming over IP networks, they do not provide automated protocol
switching, require large amounts of bandwidth, and tend to be easily
interrupted due to inherent network architectures.
Existing methods typically require from about 3 - 45 Mbps of
transmission bandwidth channel for the same signal quality. They also require
separate stand-alone devices to pertorm standards conversion for both the
input and the output, which usually results in degradation of quality and
delays. In existing devices, the protocol switching is not embedded, and
therefore has to be manually selected by an operator, which sometimes can
be incorrect for the given scenario or a certain network infrastructure,
causing
interruptions in video/audio transmission.
What is needed is a way to provide audio/video capture, compression,
transmission and decompression of a video and audio signal all in the same
apparatus in an automated manner.
For the foregoing reasons, there is a need for an improved method and
apparatus for "Video over IP".
CA 02410748 2002-11-O1
2
Summary of the Invention
The present invention is directed to an audio/video-over-IP method,
system and apparatus. The method includes the steps of receiving and/or
transmitting an audio and/or video signal, and automatically switching
communication protocols to ensure signal viability.
In an aspect of the invention, the method further includes the step of
automatically applying at least one signal maintenance technique to maintain
signal viability/quality.
The invention requires a mere 1 Mbps while others need to have at
least 3 - 45 Mbps for the same signal quality. The invention provides
standards converting on the input and output together in the same device.
Automated standards conversion, automated protocol switching, and mirror
checking result in low bandwidth requirements for high quality video. Both the
encoder and decoder are assembled using relatively inexpensive "off the
shelf' components to provide a high quality video transmission device
compatible within any IP network, in addition to virtually all audio/video
standards.
Other aspects and features of the present invention will become
apparent to those ordinarily skilled in the art upon review of the following
description of specific embodiments of the invention in conjunction with the
accompanying figures.
Brief Description of the Drawings
These and other features, aspects, and advantages of the present
invention will become better understood with regard to the following
description, appended claims, and accompanying drawings where:
Figure 1 is an overview of an audio/video-over-IP method in
accordance with the present invention;
CA 02410748 2002-11-O1
3
Figure 2 is an overview of an audio/video-over-IP apparatus in
accordance with the present invention;
Figure 3 illustrates audio/video capture and compression schematics
(units 100 -170);
Figure 4 illustrates a schematic drawing of an embodiment of the
present invention;
Figure 5 illustrates a signal transmission routine (units 300 - 570); and
Figure 6 illustrates a signal receiving and output schematics (units 600
- 770).
Detailed Description of the Presently Preferred Embodiment
The present invention is directed to an audio/video-over-IP method,
system and apparatus. As illustrated in Figure 1, the method includes the
steps of receiving and/or transmitting an audio and/or video signal 102, and
automatically switching communication protocols to ensure signal viability
104. In an embodiment of the present invention, the method further includes
the step of automatically applying at least one signal maintenance technique
to maintain signal viability/quality.
As illustrated in Figure 2, the apparatus includes means for receiving
and/or transmitting an audio and/or video signal 12, and means for
automatically switching communication protocols to ensure signal viability 14.
The invention optimizes connection between audio/video source and
receivers) by means of converting original audio/video signal into a digital
compressed audio/video data stream and transmitting this data stream from
source location to receivers) locations) using an IP network, such as the
Internet, to achieve a quality of audio/video reception that is close to or
equal
to that of a satellite link.
The invention captures an analog audio/video signal from the source,
converts the signal into a digital format and transmitting that signal over an
IP
(Internet Protocol) network to a receiver. Audio/video compression techniques
CA 02410748 2002-11-O1
4
are used to reduce the storage space and bandwidth required to operate with
digital audio/video data. The optimization and switching of transmission
protocols ensures that the connection between a client and an audio/video
source will stay on as long as required, and provide the necessary connection
quality sufficient for an acceptable level of audio/video reception.
An encoding apparatus to capture of analog audio/video signal and
scramble it into a coded bit stream. The apparatus performs the following
procedures upon software execution. It authenticates a client by means of a
username and password, checks for a valid IP address and the unique MAC
(Media Access Control) address of the client, then a load balancing procedure
provides continuous uninterrupted audio/video streaming at or above the
average signal quality, withstanding network lags, increased amount of hops
(connecting IP network routers between server and client). In instances when
a default server detects a lag (delay) between itself and a client, the
database
is prompted to perform a mirror search for the closest mirror server. This is
provided by instructing server computers that are listed as mirrors to send an
echo signal (PING) to a client, calculating the lowest echo response time, and
based on the lowest calculated response time, redirecting a client to request
an audio/video data stream from the closest and/or fastest mirror server.
A protocol switching procedure is executed upon the server and client
establishing a successful connection. The server is instructed to send a
specific data packet that will predetermine if a multicast (UDP) packet
reached
the clients machine and inferring that a multicast UDP protocol can be used.
In the event that no UDP packet reached the client computer, an appropriate
response protocol is instructed to initialize a Unicast (UDP) method of a
packet delivery. In the unlikely event of a UDP "lock" by a firewall, proxy
server or other network device, when the client cannot receive any UDP
packets, the server will switch to the less efficient but more stable HTTP
protocol.
If a network slowdown or quality degradation is observed, the server
can be instructed to perform various efficiency techniques. These techniques
CA 02410748 2002-11-O1
can include resizing the video to a smaller format, reducing the frame rate to
sustain video quality, decreasing the audio sampling rate, and increasing
buffering times, therefore managing the client's connection bandwidth, all
geared towards optimizing and increasing the chances of non-stop
5 audio/video reception by the client.
Video format detection is performed upon the completion of the
aforementioned steps. The video data stream is then passed to an encoding
engine, which analyzes and encodes the video in its original format, such as
PAL SECAM or NTSC.
According to another aspect of the present invention, there is provided
a decoding apparatus, configured to only listen for incoming data streams by
means of a network interface card (NIC). The decoding device is designed to
receive scrambled audio/video bit streams, and can convert a stream into a
viewable audio/video signal in addition to the option of storing it in digital
format on a storage device, and/or storing it in analog device, such as a VCR
type device.
Figure 3 illustrates an analog or digital audio/video input device [100]
that provides encoding engine with a raw digital audio/video data. Data can be
taken from a local storage device, or be streamed live into the capture
device,
from camera, microphone, a satellite live feed, and the like. Video source can
be in any format such as NTSC, PAL, or SECAM, and any resolution. Inputs
supported are Composite Video, S-Video, RGB/SCART. A method is to fully
decode the incoming video signal into separate components (RGB or YUV),
mix this with the scan converter components, and re-encode back to video.
The video signal is never decoded, so it remains at a very high bandwidth,
particularly true when using a composite video input. Signal delays (from
video input to the encoding engine [110] are minimized to approximately 20ns.
Sync pulse width and sub carrier frequency of the video input remain
unchanged. Video input sync & sub carrier SC/H timings are unaffected. In
brief, this input device provides encoding engine with a digital video stream.
CA 02410748 2002-11-O1
6
Audio source can be in any format including digital WAV (CD) format, 5.1
Dolby (6-channel audio) or a regular tape recorder or a microphone.
[110J Audio/Video compression encoding engine. A software-based
audio/video compression processes digital audio/video stream with a
audio/video compression codec, so that the quality of audio/video stream can
be preserved without significant degradation, and a reduced size of the binary
audio/video stream that can be decompressed and converted to audio/video
output by a decoder device employing the same compression codec as in
encoder engine. The audio/video compression codec ratio can vary greatly
from virtually loss-less, less than 1 % video quality loss, to a low bandwidth
digital audio/video stream with a much higher loss of video and audio quality.
[115] Optional storage of compressed and uncompressed digital
audio/video signal for later transmission, compression, playback or editing is
performed automatically by a user pre-selecting a storage option on the
encoder. Digitized and/or compressed audio/video stream is then sent to the
storage device for archiving. Archived files can be edited, played back,
compressed, and/or transmitted unchanged at a later time.
The transmission and broadcast engine [120]. A device comprised of a
network interface card (NIC) and software network engine in charge of
distribution of the digital audio/video stream over IP networks such as the
Internet. The engine can distribute an audio/video signal as a point-to-point
connection and broadcast like, point-to-multipoint distribution using UDP
(User Datagram Protocol) and/or HTTP protocols. The transmission and
broadcast engine are able to multicast a single audio/video stream so that
multiple users can receive it simultaneously. This procedure only functions
where permitted by network operators, since many networks lock out the
Multicast UDP protocol to avoid unnecessary network traffic.
/P network (1301. This is a network "cloud" that consists of 2 or more
computers. This can be the Internet with millions of clients or merely a local
area network (LAN) with just a few computers. This network should operate
CA 02410748 2002-11-O1
7
using the IP (Internet Protocol) format, and preferably support UDP multicast
for a more efficient broadcast distribution.
A network receiver f 1401. The network receiver comprises a network
interface card and software drivers that are able to communicate using IP
protocols with an encoding device and it's transmission and broadcast engine.
This device should have a connection to the same IP network as the encoder
device, and be capable of receiving a digital audio/video stream in the same
format as it is was sent without dropping packets, and maintaining the same
rate of reception that was set by the encoder device. The received
compressed digital audio/video stream is then sent to a decoder engine,
and/or optionally to a storage device for archiving.
A decoder engine f 1501 is implemented as a software algorithm that is
able to decode compressed digital audio/video stream received by the
network receiver from an encoder by means of an IP network, or from a
storage device. The compressed audio/video stream is converted into an
uncompressed audio/video signal that can be used by an output device.
[160] A storage medium that provides space for the archiving of
compressed digital audio/ video data streams. Stored data can be retrieved
and sent to the decoder engine for a decompression procedure to convert the
audio/video stream into an uncompressed playable format, or be used by
editing software to perform desired editing procedures.
[170] Audio/video output device converts uncompressed digital
audio/video stream from the decoder engine into a playable analog
audio/video format that will be sent to a playback device such as monitor or
speakers to further monitor the output.
Figure 4 illustrates [300] Initialization of algorithm functions and
routines. [310] Program intertace loading, Graphic User Interface including
login and password input fields. This is in form of a web page that can be
CA 02410748 2002-11-O1
viewed by any computer with Internet browser capabilities and an IP network
connection.
[320] Request for login and password, as well as probing of an IP
address and sending this information to an encoding unit that checks the
validity of a connected user.
[330] Authentication procedure detects if the username and password
are authenticated. If authentication fails, then an error message is displayed
and a log entry added with a timestamp and the IP information of a possible
unauthorized user [360].
[340] An IP Check procedure verifies an authorized decoder device by
comparing the client's MAC (Media Access Control) address and subnet mask
with an existing record in an access database. MAC address is a unique
identifier that is proprietary to every network device and no two are the
same.
This function is particularly important for ensuring a secure connection in
point-to-point sessions where audio/video information is only intended for one
specific subscriber.
[350] By compiling results from authentication and IP check
procedures, the program has two options: grant access to a user or redirect to
an error message and a log entry creation procedure [330] if authentication
fails.
[355] An error message function displays an event to a user depending
on the result received from the authentication procedure or an IP check, and
logs the client's IP address and timestamps it.
[360] If authorization is granted, the decoder is checked for a '1 on 1'
priority tag meaning it will receive the audio/video data stream alone and no
other device will be authorized to view the same data channel, unless it has
the same priority tag. The tag can be configured to be granted only to mirror
servers and/or a connection that requires higher security and session
stability.
CA 02410748 2002-11-O1
9
This can be used to increase security for copyrighted material, or sensitive
audio/video transmissions such as audio\video conferences, as well as to
increase the quality of the connection.
[365] If the connection is tagged as 1-1, then the source is set to
"direct", meaning that the client will get the audio/video stream directly
from
the encoder. If there is no 1-1 tag present then the session will proceed to a
mirror checking procedure, which will appoint an appropriate source for the
audio/video data stream.
[370] Mirror Check procedure. A mirror lookup routine is initiated after a
user has been checked for a 1-1 priority tag, and barring any such detection,
can then access mirror servers if any are present.
[375] If no mirror server computers are present, the client will attempt
to connect to the encoder server directly. However, it can only be available
if
no 1-1 connections are established between the encoder and other clients.
[380 -385] Check if 1-1 connection has been established with a server,
if it is true that it is not a '1-1' session, it will be refused a connection
and will
proceed to an error message and termination signal, since all the resources
and bandwidth available to the encoding server should be allocated to 1-1
sessions. This in turn indicates that it is a closed session, or mirror
servers
have established connections with the encoder and will provide other users
with retransmissions of the audio/video stream.
[390] If mirror servers) are present then redirect the session to °Find
Fastest mirror" [390], if not then redirect to "Direct" source [400]. Users iP
address is determined, after which a host lookup is performed. From the
acquired client's host information, perform a WHOIS function to an Internic
database to determine the registration country of a primary host name, and an
entry is added to a log file. A database is then contacted to lookup a mirror
site in the given country or region to ensure a faster connection and better
network quality. If no results are found using the WHOIS function, redirect
the
CA 02410748 2002-11-O1
user to the fastest mirror server as determined by a PING command, or a
default streaming server.
[400] Set server source to "Direct" if bandwidth and CPU load permit,
5 and no next fastest mirror can be set as source.
[420] Protocol Check is initiated after a successful username/password
and IP authentication. Data packets are sent in Multicast and Unicast (UDP)
protocols, if the response is received within a permitted time frame,
Multicast
10 or Unicast is adopted as the default streaming protocol for the given
user/group. If the UDP protocol is locked for the user, then the HTTP protocol
will be used as the transport protocol for the audio/video stream.
[430] Check bandwidth procedure. A bandwidth check is initiated after
the Protocol Check has been accomplished by sending data packets at
different buffer size values and determining the mean value of the response
time. If the result confirms an appropriate time for network performance, the
user is passed directly to a Mirror lookup function with default buffer time
settings. If degraded network performance has been detected reapply a
Bandwidth check function with lower buffer values. After the results have been
analyzed, techniques can be applied such as setting the buffering time to a
value higher than the default value, decreasing the audio sampling rate,
decreasing the bandwidth settings, resizing the video, and/or reducing the
frame rate; all to achieve sustainable audio/video quality with lower
bandwidth
settings.
[440] Adjust buffering, resize video and adjust audio bit rate to match
the available bandwidth.
[450] Display a warning if the bandwidth is lower than is necessary for
high quality audio/video reception, such as "The bandwidth speed is Low".
[460] Increase the buffering value, resize the video, reduce the frame
rate, and change the sampling rate of the audio stream.
CA 02410748 2002-11-O1
11
[470] Initialize video streaming procedure.
[480] Receive a termination signal after the streaming session is
complete or interrupted.
[490] When fastest mirror server is found, the decoder is redirected to
the fastest "Mirror" source to receive an audio/video stream from that server.
[500] A protocol check is initiated after successful username/password
and IP authentications. Data packets are sent in Multicast and Unicast
protocols, if the response is received within a permitted time frame,
multicast
or unicast is adopted as the default streaming protocol for the given
user/group.
[510] Check bandwidth procedure. A Bandwidth Check is initiated after
the Protocol Check has been accomplished by sending data packets at
different buffer size values and determining the mean value of the response
time. If the result confirms an appropriate time for network performance, the
user is passed directly to a Mirror lookup function with default buffer time
settings. If degraded network performance has been detected, reapply a
Bandwidth check function with lower buffer values. After the results have been
analyzed, set the buffering time to a value higher than the default value,
decrease the bandwidth settings, resize the video, reduce the frame rate; all
geared towards achieving better video quality with lower bandwidth settings.
[520] Display a warning if the bandwidth is lower than is necessary for
high quality audio video reception, such as "The bandwidth speed is Low".
[530] Increase the buffering value, resize the video and reduce the
frame rate.
[540] Set the buffering and resize video to match the available
bandwidth.
CA 02410748 2002-11-O1
12
[550] Initialize video streaming procedure.
[560] Receive termination signal after the streaming is complete or
interrupted.
[570] End of program.
Figure 5 Illustrates [600] Initialization of algorithm functions and
routines. [610] Program intertace loading, Graphic User Interface displaying
login and password input fields.
[620] Ask user for Server IP address, username and password
[630] Establish connection to server using a TCP/IP connection
[640] Check authorization of username and password, IP and MAC
addresses
[650] Error function, gives an error message if the client was unable to
establish a successful connection to the server if username and password did
not match
(660] Establish a session with the Server with given credentials
[670] Send an Echo signal, pertorm a handshake, and exchange
headers to validate all connection properties and settings
[680] Open a listening port on the Client side for incoming video bit
streams.
[690] Start receiving streaming packets.
CA 02410748 2002-11-O1
13
[700] Prompt user to select an output type, such as video out and/or
storage.
[710] Storage selected instead of direct to video, therefore monitor
output
[720] Analyze video
[730] Check for the existence of a video output port, such as
Composite or S-Video. The lower the graphics resolution and refresh rate, the
better the image quality. All scan converters store the computer image to be
converted to video in their own internal memory, and in order to do so the
computer image has to be "sampled" multiple times during each scan line.
Each sample stores one pixel of information in memory. The number of
samples taken is proportional to the image quality, such as the more samples
the better. Higher graphics resolutions take less time to display each scan-
line than lower ones do, therefore there will be more samples per line for
lower resolution modes, since there is more time for more samples to be
taken, and hence this will provide a better image quality.
[740] Error message "No video output port detected", and video signal
is directed to default VGA port.
[750] Initialize and select video output port (S-VHS, Analog composite
or SCART). Absolute maximum resolution 1600x1200, Maximum 1024x768
with no line dropping in NTSC, 1280x1024 in PAL. 24 bit compatible - 23 bits
stored. 24 kHz to 100 kHz horizontal scan rate. Virtually any vertical scan
rate
accepted, therefore the horizontal scan rate is more important. Separate TTL-
level HSync & VSync positive or negative are performed.
[760] Selecting output format. The lower the graphics resolution, the
better the 'vertical' image quality. Video monitors have a fixed number of
lines
available for displaying pictures; for PAL it is 576 and for NTSC 480.
Although
some of these are typically off the top and bottom edges of the screen.
CA 02410748 2002-11-O1
14
Therefore, the more scan-lines a graphics resolution has, such as an 800x600
resolution has 600 scan-lines), the more difficult is it to squeeze all these
lines
into the limited number available on a TV monitor. Thus, lowering the graphics
resolution helps to improve image quality. Software resizes the video signal
accordingly to fit all lines with the aspect ratio of the original image
preserved.
[770] End.
The invention creates a video/audio link that enables a video signal of
a broadcast quality NTSC 525 Lines/60Hz, PAL 625 lines/50Hz be sent and
received over any IP network with bandwidth not exceeding 1 Mbps for the
above resolutions. The system acts as a standard converter that encodes a
video signal in any format be it NTSC, SECAM, or PAL, and decodes it in any
desired format NTSC or SECAM in real time. The software provides
uninterrupted video broadcasting by using mirror technology, which finds the
optimal connection between client and server such as by maintaining fewer
hops and avoiding congested zones. By employing a protocol switching
technology, the invention enables a video/audio signal to penetrate any
virtually any IP network to deliver a stable audio/video stream between client
and server. For example, if a multicast protocol is blocked by a router or
firewall settings, the software will switch to a slightly less efficient, yet
more
circumstance-appropriate unicast protocol.
As well, if the connection between "encoder" and "decoder" is found to
be slower than between "mirror" and "decoder" it switches to the optimal
streaming server for more reliable stream acquisition.
Video transmission method and apparatus, captures audio/video
signals) from an analog source, digitizes, compresses and transmits digitized
video and audio signals over IP networks. It is designed to provide video and
audio links) for Point-To-Point or Point-To-Multipoint transmissions. It
ensures accurate transmission of TV quality video and CD quality audio
signals over IP networks.
CA 02410748 2002-11-O1
In embodiments of the present invention, uses include content delivery
networks; telecommunications; live event streaming; corporate meetings;
distance education; and telemedicine.
5 The invention requires a mere 1 Mbps while others need to have at
least 3 - 45 Mbps for the same signal quality. The invention provides
standards converting on the input and output together in the same device.
Automated standards conversion, automated protocol switching, and mirror
checking result in low bandwidth requirements for high quality video. Both the
10 encoder and decoder are assembled using relatively inexpensive "off the
shelf" components to provide a high quality video transmission device
compatible within any IP network, in addition to virtually all audio/video
standards.
15 Although the present invention has been described in considerable
detail with reference to certain preferred embodiments thereof, other versions
are possible. Therefore, the spirit and scope of the appended claims should
not be limited to the description of the preferred embodiments contained
herein.
