Note: Descriptions are shown in the official language in which they were submitted.
CA 02541880 2006-04-06
WO 2005/036881 PCT/IB2004/003304
-1-
AUGMENTED VIDEO SURVEILLANCE SYSTEM
BACKGROUND TO THE INVENTION
THIS invention relates to an augmented video surveillance system.
Video surveillance or CCTV (closed circuit television) surveillance systems
usually make use of a number of video cameras and communication links
to transfer camera-captured analogue images to remote video monitors
where they can be observed by human operators or to video image
recordal equipment which records them for subsequent viewing by human
operators. Such systems are entirely reliant upon the operator's ability to
discern, from the images viewed, whether a particular event has taken
place and whether any particular action is required.
Recognised problems with known video surveillance systems, whether for
security or other purposes, are boredom and inefficient or improper reaction
to specified events by surveillance operators. A further problem is that
there are only limited methods for measuring or monitoring the pertormance
of surveillance operators.
The present invention seeks to address these problems.
SUMMARY OF THE INVENTION
According to the invention there is provided an augmented video
surveillance system in which a surveillance video is augmented by
insertion, into live video images seen by an operator, of synthetic or
realistic objects and/or by visual enhancement of predetermined events
taking place in the live video image seen by the operator, such insertion
requiring specific action by the operator, and monitoring the action taken by
the operator in response to such insertion.
CA 02541880 2006-04-06
WO 2005/036881 PCT/IB2004/003304
-2-
The objective of the invention is to improve the performance and
effectiveness of the operator and to measure his performance online. In
accordance with the invention this object is addressed by engaging the
operator's attention by interaction with the live video with the result that
the
operator's levels of vigilance and interest are elevated. The system
proposed by the invention may be used to provide on-line operator training.
The invention also provides apparatus for use in the system summarized
above.
BRIEF DESCRIPTION OF THE DRAWING
The invention will now be described in more detail, by way of example only,
with reference to the accompanying block diagram which illustrates the
principles of the invention.
SPECIFIC DESCRIPTION
Referring to the diagram, the numeral 10 indicates an event controller
which controls and implements video augmentation according to the
invention. The event controller 10 can be programmed to augment a live
video seen by an operator in a number of different ways which may be
implemented alone or in combination with one another.
In one version live video events are brought to the attention of the operator.
These are actual live events taking place in the scene observed by the
operator. Examples bf typical events include entry of a person into the
observed scene, addition or removal of inanimate objects to the scene. This
is indicated by the numeral 11 in the diagram.
As described in a simultaneously filed, co-pending patent application filed
by the present applicant and entitled "Enhanced Video Based Surveillance
CA 02541880 2006-04-06
WO 2005/036881 PCT/IB2004/003304
-3-
System" a background image or model is generated and, during the course
of live video monitoring, the overall image is segmented into respective
background and foreground segments and video enhancement techniques
are employed visually to enhance all or selected foreground features or
events, thereby drawing the operator's attention to such features and
events.
The system described above may be interfaced with a video motion
detection system also described in the aforementioned co-pending patent
application, in which case the operator's attention is drawn, by camera
switching and/or image enhancement to specified motion events in the
observed image. This is indicated by the numeral 13 in the diagram.
The operator is trained to respond to specified foreground features or
events, in each case in a predetermined manner. The system may be used
to measure the operator's efficiency in responding in the appropriate
manner to specified events.
The invention proposes the insertion of certain synthetic or realistic objects
or events into the image viewed by the operator. In one example the event
controller 10 acts in response to a rules-based event generation module 12
to initiate the generation, by a synthetic object generator 14, of a synthetic
event or object and augments the raw, live video image 16 by inserting
such event or object into the image, thereby forming an augmented video
image 18 which is viewed by the operator. In each case the controller 10
controls all parameters relating to the insertion, e.g. the frequency of
insertion, the duration of the insertion and so on.
Synthetic objects are objects which are inserted into the viewed image and
are classified as synthetic because they do not blend into the imaged scene
and can be clearly identified by the operator as artificially inserted
objects.
Typical examples are fanciful shapes such as stars, squares, circles, still
pictures of persons or objects, or animated sequences of person and/or
objects. In each case the insertion may require specific action by the
CA 02541880 2006-04-06
WO 2005/036881 PCT/IB2004/003304
-4-
operator, as indicated by the numeral 20. The operator may, for example,
be required to operate a mouse pointer and to click on the insertion in order
to remove it. The operator's reaction time and ability to carry out the
required action can be monitored to provide an indication of the operator's
efficiency and training and can, if necessary, be logged as part of the
operator's performance record.
Associated with the insertion of synthetic objects can be certain specific
tasks that need to be performed by the operator, such tasks requiring
higher levels of cognitive engagement than a simple response of
acknowledging the presence of the synthetic object by clicking on it. For
example, this could be a discrimination task whereby the operator is
required to distinguish between different categories of inserted synthetic
objects. For example, still pictures or images of persons may show such
persons displaying normal, unsuspicious behaviour and may require a first
type of response by the operator. On the other hand, pictures of persons
displaying suspicious behaviour may require a totally different type of
response by the operator. Once again, the operator's reaction to the
inserted objects) can be used to provide a measure of ability and
efficiency, with the accuracy of completion of the associated task providing
an additional level of information on operator performance.
The object insertion technique described in the preceding paragraph also
serves a valuable function in on-line training of the operator in that it can
raise the operator's awareness of certain types of personal behaviour which
may be considered undesirable from a security point of view, for example a
person assuming a certain bodily posture in the vicinity of a high-risk area
such as a cash-counting or gem sorting station. Through these types of
synthetic objects, operators are sensitised to the types of
behaviours/postures . that they should detect, thereby increasing the
likelihood of detecting incidents associated with such behaviours/postures
when they actually do occur. Often such incidents have a fairly love
frequency of occurrence.
CA 02541880 2006-04-06
WO 2005/036881 PCT/IB2004/003304
-5-
The invention also envisages the augmentation of the live video by
insertion of realistic objects or events, under the control of the event
controller 10 and in accordance with the rules-based event generatio n
module 12. In this case, realistic video objects, such as animated
sequences extracted from previously recorded images of the scene under
surveillance, as indicated by the block 22 in the diagram, are inserted back
into the live video image 16 to produce the augmented image 18 in such a
manner that the 'operator is unable to distinguish between artificially
inserted objects and real objects of the scene. So, for instance, live video
footage of a person moving in the surveilled scene can be inserted to
produce the augmented image.
The operator will once again be required to react in a certain way to the
augmented video which he sees. Where, for instance, augmentation
involves insertion of video footage showing an act of theft, the technique
can be used to monitor the operator's ability to recognize the act and to
react accordingly. Thus, where performance on inserted synthetic objects)
provides a proxy on operator performance, realistic insertion provide a
direct measure of operator performance on completely realistic incidents.
Combining both approaches in operator performance monitoring also
prevents unrealistic estimates of performance of operator who simply
concentrate on elevating their performance on inserted synthetic objects at
the expense of detecting real incidents in the surveillance video.
In each of the examples described above, the operator will be required to
interact, in accordance with his training, via an interface generator 24.
Where a synthetic object has been inserted, the operator may for example
be required to click once, with a mouse pointer, on an image such as a
fanciful image or a realistic image of non-threatening type or to click twice
on an image representing a security threat. The event controller may be
programmed to give feedback to the operator, via the interface 24, to
indicate that the correct action has been taken. The inserted object may for
instance be caused to flash andlor to disappear from the image, thereby
acknowledging the correctness of the operator's action.
