This code finds bounding box of a single human mouth.

Last update: Nov 27, 2022

Related tags

Overview

Swab AI

This code finds bounding box of a single human mouth. In comparison to other face segmentation methods, it is relatively insusceptible to open mouth conditions, e.g., yawning, surgical robots, etc. The mouth coordinates are found in a more certified way using two independent algorithms. Therefore, the algorithm can be used in more sensitive applications.

Sample Output

Selected images from YawDD and CelebAMask dataset.

Example Code

only for debugging purposes cv2.imwrite("output.jpg", cv2.rectangle(img, points[0], points[1], (0, 0, 255), 2)) print("output.jpg is written.") ">

import swab_ai
import cv2

img = cv2.imread("input.jpg")  # img is numpy array

points = swab_ai.Get_BoundingBox(img)  # returns bounding box coordinates of img

if points: # bounding box is found
    # save output file -> only for debugging purposes
    cv2.imwrite("output.jpg", cv2.rectangle(img, points[0], points[1], (0, 0, 255), 2))
    print("output.jpg is written.")

Python API

`swab_ai.py`

It loads required libraries and models. This module contains three functions:

GetFace(input_image, show=False, save=False, live=False): calls FaceInference.py and returns coordinates of face bounding box and two landmarks of mouth.
GetMouth(face_image, show=False, save=False, live=False): return mouth mask.
Get_BoundingBox(input_image, show=False, save=False, live=False): calls GetFace and GetMouth functions and checks if the bounding box could be certified by checking for sufficient overlap of two independent algorithms. This function returns the mouth bounding box and also prints the FPS in the output. In case mouth bounding box could not be found Frame Droppe is printed in the console.

`live_test.py`

For debugging purposes: The algorithm is run on live webcam video. The result of face detection and segmentation algorithms is visualized in real-time.

Dockerfile

We have used the latest version of pytorch image pytorch:1.9.0-cuda11.1-cudnn8-runtime as of now.

To build the docker image and run the container in one step:

 docker build -t swabai .
 docker run --runtime=nvidia swabai

Note that --runtime=nvidia is necessary to enable GPU.

If you see the message output.jpg is written, it means that everything is working correctly. To see the output image use this command to mount local working directory:

 docker run -v $PWD:/usr/src/app/ --runtime=nvidia swabai

Model

The first time that the code is run, model is downloaded automatically. Every time that the code restarts, it checks for model updates.

You might also like...

Pytorch based library to rank predicted bounding boxes using text/image user's prompts.

pytorch_clip_bbox: Implementation of the CLIP guided bbox ranking for Object Detection. Pytorch based library to rank predicted bounding boxes using t

50 Nov 27, 2022

Pytorch ImageNet1k Loader with Bounding Boxes.

ImageNet 1K Bounding Boxes For some experiments, you might wanna pass only the background of imagenet images vs passing only the foreground. Here, I'v

11 Oct 15, 2022

[CVPR2021] UAV-Human: A Large Benchmark for Human Behavior Understanding with Unmanned Aerial Vehicles

UAV-Human Official repository for CVPR2021: UAV-Human: A Large Benchmark for Human Behavior Understanding with Unmanned Aerial Vehicle Paper arXiv Res

129 Jan 4, 2023

Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors, CVPR 2021

Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors Human POSEitioning System (H

66 Dec 21, 2022

Human Action Controller - A human action controller running on different platforms.

Releases(v0.2)

v0.2(Dec 3, 2021)

Stage 3 (final stage) of the project
Source code(tar.gz)
Source code(zip)
v0.1(Sep 30, 2021)

Stage 2 of the project
Source code(tar.gz)
Source code(zip)

This code finds bounding box of a single human mouth.

Related tags

Overview

Swab AI

Sample Output

Example Code

Python API

swab_ai.py

live_test.py

Dockerfile

Model

You might also like...

Pytorch based library to rank predicted bounding boxes using text/image user's prompts.

Pytorch ImageNet1k Loader with Bounding Boxes.

[CVPR2021] UAV-Human: A Large Benchmark for Human Behavior Understanding with Unmanned Aerial Vehicles

Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors, CVPR 2021

Human Action Controller - A human action controller running on different platforms.

Python scripts for performing 3D human pose estimation using the Mobile Human Pose model in ONNX.

StyleGAN-Human: A Data-Centric Odyssey of Human Generation

7th place solution of Human Protein Atlas - Single Cell Classification on Kaggle

"3D Human Texture Estimation from a Single Image with Transformers", ICCV 2021

Releases(v0.2)

v0.2(Dec 3, 2021)

v0.1(Sep 30, 2021)

Owner

iThermAI

An all-in-one application to visualize multiple different local path planning algorithms

A simple program for training and testing vit

This repository contains the implementations related to the experiments of a set of publicly available datasets that are used in the time series forecasting research space.

Council-GAN - Implementation for our paper Breaking the Cycle - Colleagues are all you need (CVPR 2020)

AdvStyle - Official PyTorch Implementation

A library to inspect itermediate layers of PyTorch models.

Evaluating saliency methods on artificial data with different background types

Implementation of the "PSTNet: Point Spatio-Temporal Convolution on Point Cloud Sequences" paper.

Instant-Teaching: An End-to-End Semi-Supervised Object Detection Framework

AMTML-KD: Adaptive Multi-teacher Multi-level Knowledge Distillation

Code repo for "Towards Interpretable Deep Networks for Monocular Depth Estimation" paper.

Implementation of momentum^2 teacher

It's A ML based Web Site build with python and Django to find the breed of the dog

PyTorch implementation of NeurIPS 2021 paper: "CoFiNet: Reliable Coarse-to-fine Correspondences for Robust Point Cloud Registration"

Code and data form the paper BERT Got a Date: Introducing Transformers to Temporal Tagging

Official Implementation of DE-DETR and DELA-DETR in "Towards Data-Efficient Detection Transformers"

Blender Add-on that sets a Material's Base Color to one of Pantone's Colors of the Year

The lightweight PyTorch wrapper for high-performance AI research. Scale your models, not the boilerplate.

An implementation of the 1. Parallel, 2. Streaming, 3. Randomized SVD using MPI4Py

My coursework for Machine Learning (2021 Spring) at National Taiwan University (NTU)

`swab_ai.py`

`live_test.py`