Note: Descriptions are shown in the official language in which they were submitted.
CA 03147418 2021-12-13
LIVING BODY DETECTION METHOD AND SYSTEM FOR HUMAN FACE BY
USING TWO LONG-BASELINE CAMERAS
BACKGROUND OF THE INVENTION
Technical Field
[0001] The present invention relates to the technical field of human face
recognition, and more
particularly to a method and a system for long-baseline binocular human face
liveness detection.
Description of Related Art
[0002] With the continuous technical development in the fields of human
identity recognition
and verification, and smart detection and recognition for images, the
technologies for human
face recognition have become mature every day. At the same time, various
masquerade attacks
focused on human face recognition and verification systems have increasingly
emerged and
seriously threaten reliability and security of these systems. A practical
approach known as
liveness detection for human faces has thus been introduced to confront such
masquerade attacks
and ensure system security.
[0003] Currently, liveness detection using normal cameras can be divided into
three types. The
first one is an image liveness detection method purely based on software,
which conducts
liveness detection according to texture, background, lighting, and other
features of detected
images. However, as this approach is very sensitive to the environment, its
detection
performance is relatively instable, making its applicability limited. The
second type is a video
liveness detection method based on interaction with users. It determines
whether the object under
detection is living by asking a user to make a series of movements. Its
detection result
nevertheless highly depends on how well the user follows movement standards,
leading to
inferior user experience, and this approach is still vulnerable to recorded
video clips. The third
approach to liveness detection is based on information captured by additional
hardware
equipment. This approach usually uses a short-baseline binocular camera to
capture human-face
images and realizes liveness detection based on the additional information
acquired by an
auxiliary camera. Unfortunately, in practical use, a short-baseline binocular
camera is less stable
in three-dimensional recovery and the approach involves complicated
computation, making the
1
Date recue / Date received 2021-12-13
CA 03147418 2021-12-13
overall recognition efficiency unsatisfactory.
SUMMARY OF THE INVENTION
[0004] In order to address the shortcomings of the prior art, embodiments of
the present
invention provide a method and a system for long-baseline binocular human face
liveness
detection. The technical schemes are described below.
[0005] In one aspect, the present invention provides a method for long-
baseline binocular
human face liveness detection. The method comprises:
[0006] frontally capturing a first human-face image using a primary camera at
one end of a long
baseline, and detecting whether the first human-face image has dimensions
meeting a set of
predetermined dimensional criteria;
[0007] if the dimensions of the first human-face image meet the set of
predetermined
dimensional criteria, determining whether a secondary camera located at an
opposite end of the
long baseline is able to capture a second human-face image;
[0008] if the secondary camera fails to capture the second human-face image,
determining that
the current human face is not a live human face; and if the secondary camera
is able to capture
the second human-face image, normalizing the first human-face image and the
second human-
face image to a predetermined pixel size, respectively;
[0009] training the normalized first and second human-face images by means of
a neural
network model, so as to obtain a liveness detection score; and
[0010] determining whether the liveness detection score meets a predetermined
score criterion,
and if yes, determining that the current human face is a live human face, or
if not, determining
that the current human face is not a live human face.
[0011] Further, the secondary camera includes one or more cameras, which are
settled in the
same plane as the primary camera, and located at any locations above, below,
at the left to and/or
at the right to the primary camera.
[0012] Further, the step of training the normalized first and second human-
face images by
means of a neural network model, so as to obtain a liveness detection score
comprises:
2
Date recue / Date received 2021-12-13
CA 03147418 2021-12-13
[0013] extracting image quality features of the first human-face image and
frame structure
features of the second human-face image, downwardly equalizing the dimensions
of the first and
second human-face images;
[0014] performing weighted fusion on the image quality features and the frame
structure
features, so as to obtain fusion features; and
[0015] obtaining the liveness detection score according to the fusion
features.
[0016] Further, the image quality feature comprises: definition of the human
face, noise, light
performance, and spectral features; and the frame structure features include:
line structure
features and texture features of the image.
[0017] Further, the step of performing weighted fusion on the image quality
features and the
frame structure features, so as to obtain fusion features comprises:
[0018] multiplying the image quality features and the frame structure features
by their
respective learnable parameters, in which the learnable parameters are
acquired from samples of
live human faces through the neural network model by means of training.
[0019] Further, the neural network model is a twin depth neural network model,
which
comprises two feature extractors and one fully connected classifier.
[0020] In a second aspect, the present invention provides a system long-
baseline binocular
human face liveness detection. The system comprises an image-acquiring
apparatus and a
detection system;
[0021] in which the image-acquiring apparatus comprises:
[0022] a primary camera, located at one of the two ends of the long baseline
to be right opposite
to the human face to be detected, and configured to capture the first human-
face image;
[0023] a secondary camera, located at the other end of the long-baseline, and
configured to for
capture the second human-face image;
[0024] the detection system comprises:
[0025] a human face detecting module, for detecting whether the primary camera
has captured
the first human-face image, and whether the secondary camera has captured the
second human-
face image, and determining whether the dimensions of the first human-face
image meet the
predetermined dimensional criteria;
3
Date recue / Date received 2021-12-13
CA 03147418 2021-12-13
[0026] a human-face image processing module, for normalizing the first human-
face image and
the second human-face image to the predetermined pixel size, respectively;
[0027] a human face liveness recognizing module, comprising a neural network
model, for
training the normalized first and second human-face images, so as to obtain
the liveness
detection score, determining whether the liveness detection score meets a
predetermined score
criterion, and if yes, determining that the current human face is a live human
face, or if not,
determining that the current human face is not a live human face.
[0028] Further, the secondary camera includes one or more cameras, which are
settled in the
same plane as the primary camera, and located at any locations above, below,
at the left to and/or
at the right to the primary camera.
[0029] Further, the primary camera comprises a camera and a filter for
filtering off non-visible
light; the secondary camera is any one or more of an infrared camera, a wide-
angle camera, and
a visible light camera.
[0030] Further, the human-face image processing module comprises: twin depth
neural network
model, which comprises two feature extractors and one fully connected
classifier.
[0031] The technical schemes of the embodiment of the present invention
provide the following
beneficial effects:
1.The present invention uses the long-baseline binocular cameras to capture
human-
face images and uses the twin neural network model to extract image features,
thereby
obtaining the liveness detection score, so as to accurately and efficiently
detect and
recognize liveness human-face images, and overcome defects of the existing
human
face recognition technologies about inconsistent recognition effect, demanding
hardware requirements and high computation workloads for processing images;
2.The long-baseline binocular camera of the present invention may comprise
primary
and secondary image-capturing apparatus that capture images at the same time,
thereby being able to fast identify normal objects faking living human faces
without
delay. As to advanced fakes that are difficult to recognize, the present
inventio can
also recognize fast in virtue of the neural network model, thus having good
recognition efficiency;
4
Date recue / Date received 2021-12-13
CA 03147418 2021-12-13
3.The present invention uses the primary and secondary cameras to extracting
image
quality features that are sensitive to imaging materials and high distinctive
and image
frame structure features that are insensitive to external factors like noise
and ambient
lighting as feature factors for recognizing fake images, thereby enjoying the
high
accuracy supported by the image quality features, but also benefiting from the
high
robustness provided by the frame structure features.
BRIEF DESCRIPTION OF THE DRAWINGS
[0032] To better illustrate the technical schemes as disclosed in the
embodiments of the present
invention, accompanying drawings referred in the description of the
embodiments below are
introduced briefly. It is apparent that the accompanying drawings as recited
in the following
description merely provide a part of possible embodiments of the present
invention, and people
of ordinary skill in the art would be able to obtain more drawings according
to those provided
herein without paying creative efforts, wherein:
[0033] FIG. 1 is a flowchart of a method for long-baseline binocular human
face liveness
detection according to one embodiment of the present invention;
[0034] FIG. 2 is a schematic drawing showing arrangement of primary and
secondary cameras
in one embodiment of the present invention; and
[0035] FIG. 3 is a modular diagram of a system for long-baseline binocular
human face liveness
detection according to one embodiment of the present invention.
DETAILED DESCRIPTION OF THE INVENTION
[0036] To make the foregoing objectives, features, and advantages of the
present invention
clearer and more understandable, the following description will be directed to
some
embodiments as depicted in the accompanying drawings to detail the technical
schemes
disclosed in these embodiments. It is, however, to be understood that the
embodiments referred
herein are only a part of all possible embodiments and thus not exhaustive.
Based on the
embodiments of the present invention, all the other embodiments can be
conceived without
creative labor by people of ordinary skill in the art, and all these and other
embodiments shall
Date recue / Date received 2021-12-13
CA 03147418 2021-12-13
be embraced in the scope of the present invention.
[0037] In view that the existing liveness detection technologies for
recognizing human faces
have problems such as inconsistent detection effects and complicated
computation,
embodiments of the present invention disclose a method and a system for long-
baseline
binocular human face liveness detection, which implement the technical schemes
as described
below.
[0038] As shown in FIG. 1, a method for long-baseline binocular human face
liveness detection
comprises:
[0039] frontally capturing a first human-face image using a primary camera at
one end of a long
baseline, and detecting whether the first human-face image has dimensions
meeting a set of
predetermined dimensional criteria;
[0040] if the dimensions of the first human-face image meet the set of
predetermined
dimensional criteria, determining whether a secondary camera located at an
opposite end of the
long baseline is able to capture a second human-face image;
[0041] if the secondary camera fails to capture the second human-face image,
determining that
the current human face is not a live human face; and if the secondary camera
is able to capture
the second human-face image, normalizing the first human-face image and the
second human-
face image to a predetermined pixel size, respectively;
[0042] training the normalized first and second human-face images by means of
a neural
network model, so as to obtain a liveness detection score; and
[0043] determining whether the liveness detection score meets a predetermined
score criterion,
and if yes, determining that the current human face is a live human face, or
if not, determining
that the current human face is not a live human face.
[0044] It is to be noted that in the foregoing method, the term baseline
refers to the linear
distance between the two cameras, and the phrase long baseline indicates a
baseline that is longer
than a short baseline. The primary camera is mainly used to frontally capture
the first human-
face image, so the first human-face image is actually the front image of the
detected human face.
The predetermined set of dimensional criteria is associated with the distance
between the human
face and the primary camera. To set the dimensional criteria, the distance
between the human
6
Date recue / Date received 2021-12-13
CA 03147418 2021-12-13
face and the primary camera may be determined at first, and then the
dimensional criteria can
be set. When the primary camera captures the first human-face image, it may
prompt a user to
stand at a designated position, or put his/her face into the prompt frame in
the display screen, so
as to measure the size of the first human-face image. If the first human-face
image has its
dimensions smaller than the predetermined set of dimensional criteria, it is
determined that the
current human face is not a living human face. If it meets the predetermined
dimensional criteria,
the method proceeds with subsequent determination. Generally, if the human
face currently
under detection is a living human face, the secondary camera located at the
other end of the long
baseline is enabled to capture an image of a part of the human face. If the
human face currently
under detection is not a living human face, since the image facing a human
face is a plane, and
the linear distance between the secondary camera and the primary camera is
relatively long, the
secondary camera usually cannot capture any part of the human-face image.
Based on this
principle, when the first human-face image has its dimensions meeting the
predetermined
dimensional criteria, it is to be determined whether the secondary camera can
capture the second
human-face image. If the secondary camera fails to capture the second human-
face image, it is
determined that the human face under detection is not a living human face.
Therein, the second
human-face image is usually an image of part of the human face.
[0045] If the secondary camera has captured the second human-face image, the
next step is to
combine the image features of the first and second human-face images to
determine whether the
detected human face is from a living person. In the foregoing method,
normalization refers to
the process where images are processed and transformed into a standard format
according to a
series of criteria. In the embodiment of the present invention, the first
human-face image is
preferably normalized to 128*128 pixels, and the second human-face image is
preferably
normalized to 64*64 pixels. The normalized first and second human-face images
are then put
into the neural network model for feature extraction and training, so as to
obtain a liveness
detection score. The liveness detection score is dependent on the image
quality of the first
human-face image and the frame structure features of the second human-face
image. At last, the
liveness detection score is compared to a predetermined score criterion.
Therein, the
predetermined score criterion is a value determined by using a large number of
liveness human-
7
Date recue / Date received 2021-12-13
CA 03147418 2021-12-13
face images as training samples to be trained by the neural network model. The
criterion is
usually a threshold value. If the liveness detection score falls with the
threshold, it is determined
that the currently detected human face is a living human face. Otherwise, it
is determined that
the currently detected human face is not a living human face.
[0046] FIG. 2 shows a possible arrangement of the primary camera and the
secondary camera.
The primary camera 1 and the secondary camera 2 are located at in the same
plane, so as to
ensure that the primary camera and the secondary camera have their vertical
distances from the
human face are equal. The connection line between the primary camera and the
secondary
camera is the long baseline 3. The secondary camera may be one camera located
at any location
above, below, at the left or at the right to the primary camera.
Alternatively, the secondary camera
may include plural cameras located at any locations above, below, at the left
or at the right to the
primary camera.
[0047] It is to be noted that if the secondary camera is plural secondary
camera, the images
captured thereby each present a part of the human-face image taken from
different angles. If the
currently detected human face is a living human face, each of the secondary
cameras shall be
able to capture a part of the second human-face image. Therefore, if there is
one or more of the
secondary cameras failing to capture a second human-face image, it is
determined that the
currently detected human face is not a living human face.
[0048] Specifically, in the foregoing method, the step of training the
normalized first and second
human-face images by means of a neural network model, so as to obtain a
liveness detection
score comprises:
[0049] extracting image quality features of the first human-face image and
frame structure
features of the second human-face image, downwardly equalizing the dimensions
of the first and
second human-face images;
[0050] performing weighted fusion on the image quality features and the frame
structure
features, so as to obtain fusion features; and
[0051] obtaining the liveness detection score according to the fusion
features.
[0052] It is to be noted that the image quality features are image features
extracted from the
first human-face image. Since the first human-face image is the front image of
the human face,
8
Date recue / Date received 2021-12-13
CA 03147418 2021-12-13
the first human-face image has to be measured in terms of image quality. The
image quality
features include: the human face definition of the image, the noise level of
the image, the light
performance, the spectral features, and optionally small-wave features. If the
currently detected
human face is not a living human face, since the face human face must be
different from a living
human face in terms of skin texture, the captured image(s) will certainly
differ from an image
captured from a living human face particularly in features that presenting the
material of the
capture object such as the skin texture definition, the noise level, the light
performance, and the
spectral performance. Additionally, the fake human face is usually in the form
of an electronic
or printed picture, and it is known that both of these have their imaging
definition poorer than a
physical human face, having higher noise levels, and tend to show reflection
and moire patterns.
Hence, by investigating into the foregoing image quality features, it is
possible to determine
whether the detected human face is a faker picture or a living human face.
[0053] For further enhancing accuracy of determination, the method disclosed
in the
embodiment of the present invention combines the first human-face image and
the second
human-face image to determine whether the detected human face is a living. As
to the frame
structure features of the second human-face image, if the currently detected
human face is
determined to be from a faker picture, there may be a frame in the captured
image or there may
be unnaturalness existing at the border between the human face in the image
and the background
opposite to the naturalness as shown in the living human-face image. The frame
structure
features thus define how well the image of the object fused with the
background and may include:
the texture grains in the image, the line structure features at the border of
the object and the
texture structure features.
[0054] After the image quality features and the frame structure features are
acquired, the two
are subject to weighted fusion to generate fusion features. Specifically, the
two multiply their
respective learnable parameters. The learnable parameters are weight values of
the two features
obtained from liveness human face samples through training at the neural
network model.
[0055] In the foregoing method, the neural network model is a twin depth
neural network model.
The twin depth neural network model comprises two feature extractors and one
fully connected
classifier. Therein, the feature extractor may be realized using a feature
extractor of an existing
9
Date recue / Date received 2021-12-13
CA 03147418 2021-12-13
neural network model. For example, the feature extractor of the present
invention may adopt the
structures of the input layer and the feature extracting layer of the ResNet-
50 model with the
fully connected classifier installed downward the feature extractor, and
include successively an
average pooling layer, an FC fully connected layer and a Softmax layer.
[0056] It is to be noted that ResNet-50 is a deep training neural network
model, which
implements a "shortcut connection." Such a connection solution helps
processing efficiency. The
feature extracting structure of a ResNet-50 model is composed of a 7x7
convolution layer, a 3x3
max-pool layer, and 16 residual blocks. Each of the residual blocks is
composed of 3 convolution
layers, namely a front and a back lx1 convolution layer sandwiching a 3x3
intermediate
convolution layer. The whole feature extracting structure thus has 49
convolution layers. Data
input to the feature extractor is processed by the 7x7 convolution layer, the
3x3 max-pool layer,
and the 16 residual blocks successively before eventually become extracted
feature map. The
twin depth neural network model of the embodiment of the present invention is
structurally
optimized for the ResNet-50 model, and is suitable for feature extracting
works for both of the
first and second human-face images as required by the technical scheme of the
present invention.
[0057] On the other hand, as shown in FIG. 3, the present invention further
discloses a system
for long-baseline binocular human face liveness detection on the basis of the
foregoing method.
The system comprises an image-acquiring apparatus and a detection system.
[0058] The image-acquiring apparatus comprises: a primary camera, located at
one of the two
ends of the long baseline to be right opposite to the human face to be
detected, and configured
to capture the first human-face image; a secondary camera, located at the
other end of the long-
baseline, and configured to for capture the second human-face image.
[0059] The detection system comprises: a human face detecting module, for
detecting whether
the primary camera has captured the first human-face image, and whether the
secondary camera
has captured the second human-face image, and determining whether the
dimensions of the first
human-face image meet the predetermined dimensional criteria; a human-face
image processing
module, for normalizing the first human-face image and the second human-face
image to the
predetermined pixel size, respectively; a human face liveness recognizing
module, comprising
a neural network model, for training the normalized first and second human-
face images, so as
Date recue / Date received 2021-12-13
CA 03147418 2021-12-13
to obtain the liveness detection score, determining whether the liveness
detection score meets a
predetermined score criterion, and if yes, determining that the current human
face is a live human
face, or if not, determining that the current human face is not a live human
face.
[0060] In the foregoing image-acquiring apparatus, the secondary camera may be
one or more,
which are settled in the same plane as the primary camera, and located at any
locations above,
below, at the left to and/or at the right to the primary camera. primary
camera comprises a camera
and a filter for filtering off non-visible light; secondary camera is any one
or more of an infrared
camera, a wide-angle camera, and a visible light camera.
[0061] In the foregoing detection system, the human face liveness recognizing
module is
specifically used to perform extracting image quality features of the first
human-face image and
frame structure features of the second human-face image at the neural network
model,
downwardly equalizing the dimensions of the first and second human-face
images; performing
weighted fusion on the image quality features and the frame structure
features, so as to obtain
fusion features; and obtaining the liveness detection score according to the
fusion features.
Therein, the image quality features include: definition of the human face,
noise levels, light
performance, and spectral features; frame structure feature comprises: line
structure features and
texture features of the image.
[0062] The foregoing neural network model is a twin depth neural network
model. The twin
depth neural network model comprises two feature extractors and one fully
connected classifier.
For example, the feature extractor adopts the structures of the input layer
and the feature
extracting layer of the ResNet-50 model with the fully connected classifier
installed downward
the feature extractor, and include successively an average pooling layer, an
FC fully connected
layer and a Softmax layer of the ResNet-50 model. Therein, the average pooling
layer is for
lowering the size of the fusion features. The FC fully connected layer and the
Softmax layer are
used to acquire the human face liveness detection score.
[0063] The technical schemes of the embodiments of the present invention
provides provide
the following beneficial effects:
1.The present invention uses the long-baseline binocular cameras to capture
human-
face images and uses the twin neural network model to extract image features,
thereby
11
Date recue / Date received 2021-12-13
CA 03147418 2021-12-13
obtaining the liveness detection score, so as to accurately and efficiently
detect and
recognize liveness human-face images, and overcome defects of the existing
human
face recognition technologies about inconsistent recognition effect, demanding
hardware requirements and high computation workloads for processing images;
2.The long-baseline binocular camera of the present invention may comprise
primary
and secondary image-capturing apparatus that capture images at the same time,
thereby being able to fast identify normal objects faking living human faces
without
delay. As to advanced fakes that are difficult to recognize, the present
inventio can
also recognize fast in virtue of the neural network model, thus having good
recognition efficiency; and
3.The present invention uses the primary and secondary cameras to extracting
image
quality features that are sensitive to imaging materials and high distinctive
and image
frame structure features that are insensitive to external factors like noise
and ambient
lighting as feature factors for recognizing fake images, thereby enjoying the
high
accuracy supported by the image quality features, but also benefiting from the
high
robustness provided by the frame structure features.
[0064] All the foregoing optional technical schemes may be used in any
combination to form
optional embodiments of the present invention, and the possibilities are not
exhaustively
described herein. The above description merely refers to some preferred
embodiments of the
present invention and is not intended to limit the present invention. Any
modification,
replacement, or improvement following the spirit and principle of the present
invention shall be
embraced in the scope protected by the present invention.
12
Date recue / Date received 2021-12-13
