PeekingDuckling

1. Description

This is an implementation of facial identification algorithm to detect and identify the faces of the 3 team members Clarence, Eric Lee and Eric Kwok from other detected faces (Others).

We will be using the PeekingDuck framework for this mini project.

1.1 Example

2. Usage

2.1 Running the PeekingDuck nodes directly

python -m src.runner

usage: runner.py [-h] [--type {live_video,recorded_video,live_video_and_save}] [--input_filepath INPUT_FILEPATH] [--input_source INPUT_SOURCE] [--save_video_path SAVE_VIDEO_PATH] [--fps FPS]

Facial Recoginition algorithm

optional arguments:
  -h, --help            show this help message and exit
  --type {live_video,recorded_video,live_video_and_save}
                        Whether to use live webcam video or from a recorded video, or from a live webcam video and saving the recorded frames as a video file.
  --input_filepath INPUT_FILEPATH
                        The path to your video files if --type is 'recorded_video'
  --input_source INPUT_SOURCE
                        Input source integer value. Refer to cv2 VideoCapture class. Applicable for --type ['live_video' | 'live_video_and_save']
  --save_video_path SAVE_VIDEO_PATH
                        Path for video to be saved. Applicable for --type 'live_video_and_save'
  --fps FPS             Frames per second for video to be saved. Applicable for --type 'live_video_and_save'

2.2 Using the PeekingDuck from the web interface

python -m src.camera

2.3 Face recognition using only 1 photo

python -m src.app

On a separate terminal, issue the following command

python -m src.python_client <path_to_your_image>

3. Model

3.1 Face Detection

In this repository, we will be using the the library from PeekingDuck to perform facial detection.

For the face detection, the MTCNN pretrained model from the PeekingDuck's framework was being implemented.

3.2 Face Identification

For face identification, cropped images (224 x 224) obtained from Face detection stage is passed to the pretrained RESNET50 model (trained on VGGFace2 dataset) with a global average pooling layer to obtain the Face Embedding. The face embedding is then used to compare to the database of face embeddings obtained from the members to verify if the detected face belongs to one of the 3 members.
Comparison of the face embedding is done using a 1-NN model, and a threshold is set using cosine similarity, below which the image will be classified as 'others'

The face embeddings were built using 651 images from Clarence, 644 images from Eric Kwok and 939 images from Eric Lee.

A low dimensional representation of the face embedding database of the 3 members using the first 2 principal components from the PCA of the face embeddings can be found in the image below.

Augmentation to have the 4 extra images per image using random rotations of (+/-) 20 degrees and random contrasting were used in building the database so that it can be more robust. The PCA of the augmented database can be seen in the image below

4. Performance

The facial classification algorithm was able to achieve an overall accuracy of 99.4% and a weighted F1 score of 99.4% with 183 test images from Clarence, 179 from Eric Kwok, 130 from Eric Lee and 13,100 images from non-members obtained from this database.

Below shows the confusion matrix from the test result.
.

The test was conducted with the tuned threshold on the validation dataset, and the performance of the model with various thresholds can be seen in the graph below. The threshold that yields the best performance is around 0.342.

5. Authors and Acknowledgements

The authors would like to thank the mentor Lee Ping for providing us with the technical suggestions as well as the inputs on the implementation of this project.

Authors:

• Eric Kwok (Backend Face Identification)
• Eric Lee (Implementation of PeekingDuck Framework)
• Clarence Lam (Frontend Web Interface)

References (Non exhausive)

• PeekingDuck
• How to Perform Face Detection with Deep Learning