Introduction | Demonstrations | Download ACE | Feedback | My ACE
Surveillance is a new word in the security industry. Based on the
advanced video-recognition technology, it enables the
deployment of video surveillance systems capable of automatically
generating and managing the information about objects and actions in
What it stands for:
ACE stands for Annotated
(or Automatically extracted) Critical Evidence.
ACE stands for Automated
ACE stands on guard for
safety and security.
Introductory Demo Video:
Output of the
ACE: The entire activity captured by the surveillance system
over several hours (17:00 till 24:00 observed from the office window)
is summarized into 2 minutes (600Kb) of annotated video comprised of
Critical Evidence Snapshots (CES)
state-of-the-art video surveillance systems
Most of present video
surveillance manufactures are concerned with the quality and the
quantity of surveillance video data one can acquire with their systems
(quoting their own commercials: they bring "the highest picture quality
and video performance", "most advance digital video compression
technologies", "complete control of Pan, Tilt and the powerful 44X
Zoom", "total remoteness", "wireless internet connection", "greater
detail and clarity").
Few of them realize
however that, regardless of how good or how much video you captured, it
all will become useless unless you have time and opportunity
to watch it (either live or recorded) in order to recognize the events
or the objects of interest there. Even a simple one-camera one-day
recording may result in such amount of data that a single person may
not be able to handle!
Simple motion detection
(or more exactly video-frame differencing) employed in many
off-the-shelf video recording equipment does not resolve the problem.
More complex background modeling technique is also not sufficient for
The two big
problems - from the end-user standpoint:
space problem: The first one deals with the excessive amount
of video data which usually saved somewhere for be analyzed when
needed. This is the way presently commercially available DVRs (Digital
Video Recorders) work. -- They digitize 24 (or 48 or more) hours of
video on hard-drive, which can then be viewed and analyzed by a human
when needed. The need to review the recorded surveillance data usually
arises post-factum - after a criminal act has been committed.
For example, after the London bombing, millions of hours of digitized
video data from thousands of cameras were browsed by the Scotland Yard
officers searching for the data which could lead to identifying the
bombers and their accomplices.
- Data management problem: The
above problem is not only about not having a big hard-drive,
but also a problem of not having time to go though all recorded data
searching for what you need.
Having too much stored data is just as bad as not having any data at
all, since, if the amount of data is so large that it cannot be managed
within reasonable amount of time and efforts, it is useless.
Therefore, it is critical for the video surveillance to be operational
to store only that video data which is useful, i.e. the data containing
The two big
problems - from the video recognition research standpoint:
criteria for the A.C.E surveillance system:
To resolve the data
management and recording space problems, the surveillance system has to
satisfy the following criteria. It should:
- provide data, such as
evidence, that would be both useful and easily managed.
- be affordable, easily installed and operated - i.e.
run on my desktop computer with off-the-shelf cameras:
web-cams, CCTV cameras or hand-held, which can be possibly wireless for
viewing remote areas,
- run in real-time, 24/7, non-stop everyday, and, at
the same time,
- be merciful to my hard-drive space nor my
time, or in other words,
- be as much automated as possible - i.e. take as
much load from me as possible in recognizing and archiving the captured
pieces of evidence.
Current video surveillance
technology does not meet these criteria. What has been developed as a
result of our research is a new type of the video surveillance
technology that meets.
A new concept:
Critical Evidence Snapshot (C.E.S.)
Critical Evidence Snapshot is defined as a video snapshot
that provides to a viewer a piece of information that is both
useful and new.
CES client captures video
from one or more video sources, performs on-line video recognition of
all captured video data and then sends video-frames and all acquired
CES to the CES server.
For each video frame of each video source, in real-time (online) the
CES client performs:
- Detection of
object(s) in video based on colour, motion and background information.
- Computation of the attributes of the detected
object(s)., such as location, shape, velocity, colour,
texture, and their gradients.
- Recognition of object(s) as either new or already
seen, based on its attributes.
- Classifying frame as either CES (i.e providing new
information) or not.
- Extracting and creating CES
annotations: timestamps, augmentations, counters, contours.
- When face is close, face memorization / recognition
tasks permissible by the quality of data.
- a) If a video frame is CES, then it is sent to the
CES server along with the annotations;
b) It it is not, then resolution-reduced version of it is sent to the
CES server collects
video-frames and CES-es from all CES clients (using either a
TCP-IP protocol or secure ftp) and prepares them for viewing on a
security desk monitor using a web-scripting code.
At any point of time, a
security officer has an option of switching between
live video (shown as a flow of resolution-reduced video
frames) - which a normal and most common mode of operation,
- viewing Critical Evidence summarized
video (by clicking a replay button). As CES-es are played
back as a resolution-reduced video, an officer has an option of seeing
the actual resolution snapshots.
In addition, for each
video-camera, the last acquired time-stamped CES and the activity log
plotted on a time-line are also made visible to the officer so that
s/he always has a clear picture on what is and was happening in the
camera field of view.
Surveillance technology has been the key inspiration for developing the
Video Analytics Platform (VAP) by CBSA, which is described in these
- Dmitry O. Gorodnichy, Tony Mungham, Automated video surveillance: challenges and solutions. ACE Surveillance (Annotated Critical Evidence) case study NATO SET-125 Symposium "Sensor and Technology for Defence against Terrorism", Mainheim, April 2008.
- Dmitry O. Gorodnichy, Mohammad A. Ali, Elan Dubrofsky, Kris Woodbeck. Zoom on Evidence with the ACE Surveillance . International
Workshop on Video Processing and Recognition (VideoRec’07). May 28-30,
2007. Montreal , QC , Canada . NRC
49349 - [Pdf and Poster]
- Dmitry O. Gorodnichy, ACE Surveillance: The Next Generation Surveillance for Long-Term Monitoring and Activity Summarization. First International Workshop on Video Processing for Security (VP4S-06), June 7-9, Quebec City, Canada. [Pdf] [Poster]
Last updated: 2010-X-05