Face Recognition 人脸辨识 Python 教学【python】

人脸辨识在 Computer Vision 中一直是很火热的话题，也是目前广为人知的一项技术。本质上分为 Face Verification、Face Recognition：前者为验证两张人脸是否为同一个人，属于一对一的过程；后者则是从数据库里辨识出相同的人脸，属于一对多的过程。

本文将要使用 Python 来进行人脸辨识的实作，过程分为几个阶段:

Face Detection
Face Align
Feature extraction
Create Database
Face Recognition

首先安装相关 library

$ pip install scikit-learn
$ pip install onnxruntime

Face Detection

这部分要进行人脸侦测，可以使用 Python API MTCNN、RetinaFace，这边示范使用 RetinaFace 来进行侦测。

安裝 RetinaFace

$ pip install retinaface

侦测
接着就可以来侦测人脸啦~输出会有预测框左上角跟右下角、两个眼睛、鼻子、嘴巴两边的座标值

import cv2
from retinaface import RetinaFace

detector = RetinaFace(quality="normal")
img_path = '001.jpg'
img_bgr = cv2.imread(img_path, cv2.IMREAD_COLOR)
img_rgb = cv2.cvtColor(img_bgr, cv2.COLOR_BGR2RGB)
detections = detector.predict(img_rgb)
print(detections)

img_result = detector.draw(img_rgb, detections)
img = cv2.cvtColor(img_result, cv2.COLOR_RGB2BGR)
cv2.imshow("windows", img)
key = cv2.waitKey()
if key == ord("q"):
   print("exit")

cv2.destroyWindow("windows")

# output[{‘x1’: 243, ‘y1’: 142, ‘x2’: 557, ‘y2’: 586, ‘left_eye’: (303, 305), ‘right_eye’: (431, 346), ‘nose’: (305, 403), ‘left_lip’: (272, 468), ‘right_lip’: (364, 505)}]

❗ 若在使用 RetinaFace 的时候，出现以下错误

有可能是因为无法导入 shapely.geometry 模块的关系，因此要先去下载 Shapely package，下载网址 → 

下载完后再执行以下指令

$ pip install

测试是否安装成功

$ python
>>> from shapely.geometry 
import Polygon

Face Align

这部分要来将人脸特征点进行对齐，需要先定义对齐的座标，在 onnx arcface_inference.ipynb 里的 Preprocess images 中可以看到。

接着就用 skimage 套件 transform.SimilarityTransform() 得到要变换的矩阵，然后进行对齐。

import cv2
from retinaface import RetinaFace
import numpy as np
from skimage import transform as trans

src = np.array([
   [30.2946, 51.6963],
   [65.5318, 51.5014],
   [48.0252, 71.7366],
   [33.5493, 92.3655],
   [62.7299, 92.2041]], dtype=np.float32)
   
 for i, face_info in enumerate(detections):
    face_position = [face_info['x1'], face_info['y1'], face_info['x2'], face_info['y2']]
    face_landmarks = [face_info['left_eye'], face_info['right_eye'], face_info['nose'], face_info['left_lip'],
                      face_info['right_lip']]

    dst = np.array(face_landmarks, dtype=np.float32).reshape(5, 2)
    tform = trans.SimilarityTransform()
    tform.estimate(dst, src)
    M = tform.params[0:2, :]
    aligned = cv2.warpAffine(imgRGB, M, (112, 112), borderValue=0)

对齐特征点后的人脸会呈现以下样子

Feature extraction

这部分要提取刚刚对齐后的人脸特征，这边示范使用 onnx ArcFace model。

InsightFace-REST 模型 arcface_r100_v1 下载 → 
onnx 官方模型下载 → 

如果是下载 onnx 官方模型需要先进行更新，因为该模型的 BatchNorm 节点中 spatial 为 0，参考: https://github.com/onnx/models/issues/156。不过转换过后的模型准确度较差，因此资料集需要放两张比较能够侦测出来。

import onnx

model = onnx.load("model/arcfaceresnet100-8.onnx")

for node in model.graph.node:
    if (node.op_type == "BatchNormalization"):
        for attr in node.attribute:
            if (attr.name == "spatial"):
                attr.i = 1

onnx.save(model, "model/arcfaceresnet100.onnx")

接着使用模型进行提取~ 将对齐后的人脸做转置，再转换 dtype 为 float32，最后进行 inference

import numpy as np
import onnxruntime as rt
from sklearn.preprocessing import normalize

onnx_path = "model/arcface_r100_v1.onnx"
extractor = rt.InferenceSession(onnx_path)

t_aligned = np.transpose(aligned, (2, 0, 1))
inputs = t_aligned.astype(np.float32)
input_blob = np.expand_dims(inputs, axis=0)

first_input_name = extractor.get_inputs()[0].name
first_output_name = extractor.get_outputs()[0].name

predict = extractor.run([first_output_name], {first_input_name: input_blob})[0]
final_embedding = normalize(predict).flatten()

Create Database

这部分要将辨识的人脸资料写进资料库里，这边资料库是使用 sqlite。

首先，准备要辨识的人脸资料

接着把上面的 Face Detection、Face Align、Feature extraction 写成函数，调用比较方便。然后将资料夹的图片分别进行侦测、对齐、提取特征后，再写入资料库里。

import sqlite3
import io
import os

def adapt_array(arr):
   out = io.BytesIO()
   np.save(out, arr)
   out.seek(0)
   return sqlite3.Binary(out.read())

def convert_array(text):
   out = io.BytesIO(text)
   out.seek(0)
   return np.load(out)
   
def load_file(file_path):
   file_data = {}
   for person_name in os.listdir(file_path):
      person_file = os.path.join(file_path, person_name)

      total_pictures = []
      for picture in os.listdir(person_file):
         picture_path = os.path.join(person_file, picture)
         total_pictures.append(picture_path)

      file_data[person_name] = total_pictures

   return file_data

sqlite3.register_adapter(np.ndarray, adapt_array)
sqlite3.register_converter("ARRAY", convert_array)
conn_db = sqlite3.connect('database.db')
conn_db.execute("CREATE TABLE face_info 
            (id INT PRIMARY KEY NOT NULL, 
            name TEXT NOT NULL, 
            embedding ARRAY NOT NULL)")

file_path = 'database'
if os.path.exists(file_path):
   file_data = load_file(file_path)

   for i, person_name in enumerate(file_data.keys()):
      picture_path = file_data[person_name]
      sum_embeddings = np.zeros([1, 512])
      for j, picture in enumerate(picture_path):
         img_rgb, detections = face_detect(picture)
         position, landmarks, embeddings = get_embeddings(img_rgb, detections)
         sum_embeddings += embeddings

      final_embedding = sum_embeddings / len(picture_path)
      adapt_embedding = adapt_array(final_embedding)
      
      conn_db.execute("INSERT INTO face_info (id, name, embedding) VALUES (?, ?, ?)", (i, person_name, adapt_embedding))
   conn_db.commit()
   conn_db.close()

确认是否写入数据里

import sqlite3
import numpy as np
import io

def adapt_array(arr):
   out = io.BytesIO()
   np.save(out, arr)
   out.seek(0)
   return sqlite3.Binary(out.read())

def convert_array(text):
   out = io.BytesIO(text)
   out.seek(0)
   return np.load(out)

sqlite3.register_adapter(np.ndarray, adapt_array)
sqlite3.register_converter("array", convert_array)
conn_db = sqlite3.connect('database.db')

cursor = conn_db.execute("SELECt * FROM face_info")
db_data = cursor.fetchall()
for data in db_data:
   print(data)
   
conn_db.close()

Face Recognition

这部分是要将资料库里的人脸特征跟输入照片进行比对，这边使用 L2-Norm 来计算之间的距离。最后再设定 threshold，若 L2-Norm 距离大于 threshold 表示输入照片不为资料库里的任何一个人；反之，L2-Norm 距离最小的人脸与输入照片为同一个人。

import numpy as np
import sqlite3
import io
import os

conn_db = sqlite3.connect(db_path)
cursor = conn_db.execute("SELECt * FROM face_info")
db_data = cursor.fetchall()

total_distances = []
total_names = []
for data in db_data:
    total_names.append(data[1])
    db_embeddings = convert_array(data[2])
    distance = round(np.linalg.norm(db_embeddings - embeddings), 2)
    total_distances.append(distance)
total_result = dict(zip(total_names, total_distances))
idx_min = np.argmin(total_distances)

distance, name = total_distances[idx_min], total_names[idx_min]
conn_db.close()

if distance < threshold:
    return name, distance, total_result
else:
    name = "Unknown Person"
    return name, distance, total_result

接下来就来测试看看吧！由以下测试结果可以看出，在资料库的人脸都有正确的识别到，而不在的则会显示 Unknown Person。

Face Recognition 人脸辨识 Python 教学【python】

Python相关栏目本月热门文章