This code finds bounding box of a single human mouth.

Last update: Nov 27, 2022

Related tags

Overview

Swab AI

This code finds bounding box of a single human mouth. In comparison to other face segmentation methods, it is relatively insusceptible to open mouth conditions, e.g., yawning, surgical robots, etc. The mouth coordinates are found in a more certified way using two independent algorithms. Therefore, the algorithm can be used in more sensitive applications.

Sample Output

Selected images from YawDD and CelebAMask dataset.

Example Code

only for debugging purposes cv2.imwrite("output.jpg", cv2.rectangle(img, points[0], points[1], (0, 0, 255), 2)) print("output.jpg is written.") ">

import swab_ai
import cv2

img = cv2.imread("input.jpg")  # img is numpy array

points = swab_ai.Get_BoundingBox(img)  # returns bounding box coordinates of img

if points: # bounding box is found
    # save output file -> only for debugging purposes
    cv2.imwrite("output.jpg", cv2.rectangle(img, points[0], points[1], (0, 0, 255), 2))
    print("output.jpg is written.")

Python API

`swab_ai.py`

It loads required libraries and models. This module contains three functions:

GetFace(input_image, show=False, save=False, live=False): calls FaceInference.py and returns coordinates of face bounding box and two landmarks of mouth.
GetMouth(face_image, show=False, save=False, live=False): return mouth mask.
Get_BoundingBox(input_image, show=False, save=False, live=False): calls GetFace and GetMouth functions and checks if the bounding box could be certified by checking for sufficient overlap of two independent algorithms. This function returns the mouth bounding box and also prints the FPS in the output. In case mouth bounding box could not be found Frame Droppe is printed in the console.

`live_test.py`

For debugging purposes: The algorithm is run on live webcam video. The result of face detection and segmentation algorithms is visualized in real-time.

Dockerfile

We have used the latest version of pytorch image pytorch:1.9.0-cuda11.1-cudnn8-runtime as of now.

To build the docker image and run the container in one step:

 docker build -t swabai .
 docker run --runtime=nvidia swabai

Note that --runtime=nvidia is necessary to enable GPU.

If you see the message output.jpg is written, it means that everything is working correctly. To see the output image use this command to mount local working directory:

 docker run -v $PWD:/usr/src/app/ --runtime=nvidia swabai

Model

The first time that the code is run, model is downloaded automatically. Every time that the code restarts, it checks for model updates.

You might also like...

Pytorch based library to rank predicted bounding boxes using text/image user's prompts.

pytorch_clip_bbox: Implementation of the CLIP guided bbox ranking for Object Detection. Pytorch based library to rank predicted bounding boxes using t

50 Nov 27, 2022

Pytorch ImageNet1k Loader with Bounding Boxes.

ImageNet 1K Bounding Boxes For some experiments, you might wanna pass only the background of imagenet images vs passing only the foreground. Here, I'v

11 Oct 15, 2022

[CVPR2021] UAV-Human: A Large Benchmark for Human Behavior Understanding with Unmanned Aerial Vehicles

UAV-Human Official repository for CVPR2021: UAV-Human: A Large Benchmark for Human Behavior Understanding with Unmanned Aerial Vehicle Paper arXiv Res

129 Jan 4, 2023

Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors, CVPR 2021

Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors Human POSEitioning System (H

66 Dec 21, 2022

Human Action Controller - A human action controller running on different platforms.

Releases(v0.2)

v0.2(Dec 3, 2021)

Stage 3 (final stage) of the project
Source code(tar.gz)
Source code(zip)
v0.1(Sep 30, 2021)

Stage 2 of the project
Source code(tar.gz)
Source code(zip)

This code finds bounding box of a single human mouth.

Related tags

Overview

Swab AI

Sample Output

Example Code

Python API

swab_ai.py

live_test.py

Dockerfile

Model

You might also like...

Pytorch based library to rank predicted bounding boxes using text/image user's prompts.

Pytorch ImageNet1k Loader with Bounding Boxes.

[CVPR2021] UAV-Human: A Large Benchmark for Human Behavior Understanding with Unmanned Aerial Vehicles

Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors, CVPR 2021

Human Action Controller - A human action controller running on different platforms.

Python scripts for performing 3D human pose estimation using the Mobile Human Pose model in ONNX.

StyleGAN-Human: A Data-Centric Odyssey of Human Generation

7th place solution of Human Protein Atlas - Single Cell Classification on Kaggle

"3D Human Texture Estimation from a Single Image with Transformers", ICCV 2021

Releases(v0.2)

v0.2(Dec 3, 2021)

v0.1(Sep 30, 2021)

Owner

iThermAI

One-line your code easily but still with the fun of doing so!

PoolFormer: MetaFormer is Actually What You Need for Vision

[ACM MM 2021] Multiview Detection with Shadow Transformer (and View-Coherent Data Augmentation)

Single Image Random Dot Stereogram for Tensorflow

Implementation for "Domain-Specific Bias Filtering for Single Labeled Domain Generalization"

Fast, modular reference implementation and easy training of Semantic Segmentation algorithms in PyTorch.

Implementation of CVAE. Trained CVAE on faces from UTKFace Dataset to produce synthetic faces with a given degree of happiness/smileyness.

Official implementation for "Image Quality Assessment using Contrastive Learning"

Text Summarization - WCN — Weighted Contextual N-gram method for evaluation of Text Summarization

Sequential GCN for Active Learning

SMD-Nets: Stereo Mixture Density Networks

Pytorch Implementation of the paper "Cross-domain Correspondence Learning for Exemplar-based Image Translation"

render sprites into your desktop environment as shaped windows using GTK

Official Implementation of Few-shot Visual Relationship Co-localization

object recognition with machine learning on Respberry pi

Build a medical knowledge graph based on Unified Language Medical System (UMLS)

Pytorch implementation for the paper: Contrastive Learning for Cold-start Recommendation

El-Gamal on Elliptic Curve (Python)

Pytorch implementation of NeurIPS 2021 paper: Geometry Processing with Neural Fields.

Official PyTorch(Geometric) implementation of DPGNN(DPGCN) in "Distance-wise Prototypical Graph Neural Network for Node Imbalance Classification"

`swab_ai.py`

`live_test.py`