This code finds bounding box of a single human mouth.

Last update: Nov 27, 2022

Related tags

Overview

Swab AI

This code finds bounding box of a single human mouth. In comparison to other face segmentation methods, it is relatively insusceptible to open mouth conditions, e.g., yawning, surgical robots, etc. The mouth coordinates are found in a more certified way using two independent algorithms. Therefore, the algorithm can be used in more sensitive applications.

Sample Output

Selected images from YawDD and CelebAMask dataset.

Example Code

only for debugging purposes cv2.imwrite("output.jpg", cv2.rectangle(img, points[0], points[1], (0, 0, 255), 2)) print("output.jpg is written.") ">

import swab_ai
import cv2

img = cv2.imread("input.jpg")  # img is numpy array

points = swab_ai.Get_BoundingBox(img)  # returns bounding box coordinates of img

if points: # bounding box is found
    # save output file -> only for debugging purposes
    cv2.imwrite("output.jpg", cv2.rectangle(img, points[0], points[1], (0, 0, 255), 2))
    print("output.jpg is written.")

Python API

`swab_ai.py`

It loads required libraries and models. This module contains three functions:

GetFace(input_image, show=False, save=False, live=False): calls FaceInference.py and returns coordinates of face bounding box and two landmarks of mouth.
GetMouth(face_image, show=False, save=False, live=False): return mouth mask.
Get_BoundingBox(input_image, show=False, save=False, live=False): calls GetFace and GetMouth functions and checks if the bounding box could be certified by checking for sufficient overlap of two independent algorithms. This function returns the mouth bounding box and also prints the FPS in the output. In case mouth bounding box could not be found Frame Droppe is printed in the console.

`live_test.py`

For debugging purposes: The algorithm is run on live webcam video. The result of face detection and segmentation algorithms is visualized in real-time.

Dockerfile

We have used the latest version of pytorch image pytorch:1.9.0-cuda11.1-cudnn8-runtime as of now.

To build the docker image and run the container in one step:

 docker build -t swabai .
 docker run --runtime=nvidia swabai

Note that --runtime=nvidia is necessary to enable GPU.

If you see the message output.jpg is written, it means that everything is working correctly. To see the output image use this command to mount local working directory:

 docker run -v $PWD:/usr/src/app/ --runtime=nvidia swabai

Model

The first time that the code is run, model is downloaded automatically. Every time that the code restarts, it checks for model updates.

You might also like...

Pytorch based library to rank predicted bounding boxes using text/image user's prompts.

pytorch_clip_bbox: Implementation of the CLIP guided bbox ranking for Object Detection. Pytorch based library to rank predicted bounding boxes using t

50 Nov 27, 2022

Pytorch ImageNet1k Loader with Bounding Boxes.

ImageNet 1K Bounding Boxes For some experiments, you might wanna pass only the background of imagenet images vs passing only the foreground. Here, I'v

11 Oct 15, 2022

[CVPR2021] UAV-Human: A Large Benchmark for Human Behavior Understanding with Unmanned Aerial Vehicles

UAV-Human Official repository for CVPR2021: UAV-Human: A Large Benchmark for Human Behavior Understanding with Unmanned Aerial Vehicle Paper arXiv Res

129 Jan 4, 2023

Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors, CVPR 2021

Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors Human POSEitioning System (H

66 Dec 21, 2022

Human Action Controller - A human action controller running on different platforms.

Releases(v0.2)

v0.2(Dec 3, 2021)

Stage 3 (final stage) of the project
Source code(tar.gz)
Source code(zip)
v0.1(Sep 30, 2021)

Stage 2 of the project
Source code(tar.gz)
Source code(zip)

This code finds bounding box of a single human mouth.

Related tags

Overview

Swab AI

Sample Output

Example Code

Python API

swab_ai.py

live_test.py

Dockerfile

Model

You might also like...

Pytorch based library to rank predicted bounding boxes using text/image user's prompts.

Pytorch ImageNet1k Loader with Bounding Boxes.

[CVPR2021] UAV-Human: A Large Benchmark for Human Behavior Understanding with Unmanned Aerial Vehicles

Human POSEitioning System (HPS): 3D Human Pose Estimation and Self-localization in Large Scenes from Body-Mounted Sensors, CVPR 2021

Human Action Controller - A human action controller running on different platforms.

Python scripts for performing 3D human pose estimation using the Mobile Human Pose model in ONNX.

StyleGAN-Human: A Data-Centric Odyssey of Human Generation

7th place solution of Human Protein Atlas - Single Cell Classification on Kaggle

"3D Human Texture Estimation from a Single Image with Transformers", ICCV 2021

Releases(v0.2)

v0.2(Dec 3, 2021)

v0.1(Sep 30, 2021)

Owner

iThermAI

Theano is a Python library that allows you to define, optimize, and evaluate mathematical expressions involving multi-dimensional arrays efficiently. It can use GPUs and perform efficient symbolic differentiation.

RL and distillation in CARLA using a factorized world model

This repository contains part of the code used to make the images visible in the article "How does an AI Imagine the Universe?" published on Towards Data Science.

FedScale: Benchmarking Model and System Performance of Federated Learning

[NeurIPS 2021] Garment4D: Garment Reconstruction from Point Cloud Sequences

Official Repsoitory for "Mish: A Self Regularized Non-Monotonic Neural Activation Function" [BMVC 2020]

TipToiDog - Tip Toi Dog With Python

Caffe implementation for Hu et al. Segmentation for Natural Language Expressions

Open-source python package for the extraction of Radiomics features from 2D and 3D images and binary masks.

ivadomed is an integrated framework for medical image analysis with deep learning.

Repository for the COLING 2020 paper "Explainable Automated Fact-Checking: A Survey."

Publication describing 3 ML examples at NSLS-II and interfacing into Bluesky

Revisiting Oxford and Paris: Large-Scale Image Retrieval Benchmarking

This dlib-based facial login system

Uncertainty Estimation via Response Scaling for Pseudo-mask Noise Mitigation in Weakly-supervised Semantic Segmentation

PyTorch implementation of SimCLR: A Simple Framework for Contrastive Learning of Visual Representations

Code for One-shot Talking Face Generation from Single-speaker Audio-Visual Correlation Learning (AAAI 2022)

Subgraph Based Learning of Contextual Embedding

Breaking the Dilemma of Medical Image-to-image Translation

Code for our paper at ECCV 2020: Post-Training Piecewise Linear Quantization for Deep Neural Networks

`swab_ai.py`

`live_test.py`