手部21个关键点检测,二维手势姿态,手势识别,pytorch,handpose

Overview

HandPose X

手势 21 个关键点检测 , handpose

重要更新

添加 onnx 模块,预训练模型中有转好的resnet50-onnx模型,注意:目前不支持rexnetv1

"dpcas" 项目地址:https://codechina.csdn.net/EricLee/dpcas

项目 - 首发布地址

https://codechina.csdn.net/EricLee/handpose_x

项目Wiki

注意:项目的相关资料信息或是更新状态会同步到项目Wiki,以便更好的对该项目进行维护和不断发展,谢谢大家对该项目的关注!

项目介绍

注意:该项目不包括手部检测部分,手部检测项目地址:https://codechina.csdn.net/EricLee/yolo_v3
该项目是对手的21个关键点进行检测,示例如下 :

  • 图片示例:
    image
  • 视频示例:
    video

Demo小样

  • 示例1 - 按键操作
    因为考虑到目前没有三维姿态不好识别按键按下三维动作,所以目前采用二维方式。
    该示例的原理:通过简单的IOU跟踪,对二维目标如手的边界框或是特定手指的较长时间位置稳定性判断确定触发按键动作的时刻,用特定指尖的二维坐标确定触发位置。
    (注意:目前示例并未添加到工程,后期整理后会进行发布,只是一个样例,同时希望同学们自己尝试写自己基于该项目的小应用。)
    keyboard

  • 示例2 - 手势交互:指定区域物体识别
    该示例的出发点是希望通过手势指定用户想要识别的物体。那么就要选中物体的准确边界框才能达到理想识别效果。如果待识别目标边界框太大会引入背景干扰,太小又会时目标特征不完全。所以希望通过手势指定较准确的目标边界框。因为边界框涉及左上、右下两个二维坐标,所以通过两只手的特定指尖来确定。且触发逻辑与示例1相同。
    该示例的原理:通过简单的IOU跟踪,对二维目标如手的边界框或是特定手指的较长时间位置稳定性判断确定触发按键动作的时刻,用特定指尖的二维坐标确定触发位置。
    (注意:目前示例并未添加到工程,后期整理后会进行发布,只是一个样例,同时希望同学们自己尝试写自己基于该项目的小应用。)
    该示例依赖于另外一个物体识别分类项目。

keyboard

  • 以下是对书上狗的图片进行分类识别的样例,同学们可以根据自己对应的物体识别分类需求替换对应的分类识别模型即可。

recoobj_book

该物体识别分类项目的地址为: https://codechina.csdn.net/EricLee/classification

  • 示例3 - 静态手势
    通过手关键点的二维角度约束关系定义静态手势。
    示例中手势包括:fist five gun love one six three thumbup yeah
    目前该示例由于静态手势数据集的限制,目前用手骨骼的二维角度约束定义静态手势,原理如下图,计算向量AC和DE的角度,它们之间的角度大于某一个角度阈值(经验值)定义为弯曲,小于摸一个阈值(经验值)为伸直。
    注:这种静态手势识别的方法具有局限性,有条件还是通过模型训练的方法进行静态手势识别。

gs

视频示例如下图:
gesture

  • 示例4 - 静态手势交互(识别)
    通过手关键点的二维角度约束关系定义静态手势。
    该项目通过手势操作选择分类识别区域或是ocr识别区域,送入分类识别网络或是第三方web识别服务,亦或是检索数据库等应用。

    原理:通过二维约束获得静态手势,该示例是通过 食指伸直(one) 和 握拳(fist)分别代表范围选择和清空选择区域。
    建议最好还是通过分类模型做静态手势识别鲁棒和准确高,目前局限于静态手势训练集的问题用二维约束关系定义静态手势替代。

ocrreco

项目配置

1、软件

  • 作者开发环境:
  • Python 3.7
  • PyTorch >= 1.5.1
  • opencv-python

2、硬件

  • 普通USB彩色(RGB)网络摄像头

数据集

该数据集包括网络图片及数据集< >筛选动作重复度低的部分图片,进行制作(如有侵权请联系删除),共49062个样本。
< >数据集,其官网地址 http://www.rovit.ua.es/dataset/mhpdataset/
感谢《Large-scale Multiview 3D Hand Pose Dataset》数据集贡献者:Francisco Gomez-Donoso, Sergio Orts-Escolano, and Miguel Cazorla. "Large-scale Multiview 3D Hand Pose Dataset". ArXiv e-prints 1707.03742, July 2017.

模型

1、目前支持的模型 (backbone)

  • resnet18 & resnet34 & resnet50 & resnet101
  • squeezenet1_0 & squeezenet1_1
  • ShuffleNet & ShuffleNetV2
  • MobileNetV2
  • rexnetv1
  • shufflenet_v2_x1_5 ,shufflenet_v2_x1_0 , shufflenet_v2_x2_0 (torchvision 版本)

2、预训练模型

项目使用方法

模型训练

  • 根目录下运行命令: python train.py (注意脚本内相关参数配置 )

模型推理

  • 根目录下运行命令: python inference.py (注意脚本内相关参数配置 )

onnx使用

  • step1: 设定相关配置包括模型类型和模型参数路径,根目录下运行命令: python model2onnx.py (注意脚本内相关参数配置 )
  • step2: 设定onnx模型路径,根目录下运行命令: python onnx_inference.py (注意脚本内相关参数配置 )
  • 建议

检测手bbox后,进行以下的预处理,crop手图片送入手关键点模型进行推理,   
可以参考 hand_data_iter/datasets.py,数据增强的样本预处理代码部分,   
关键代码如下:     
  img 为原图  ,np为numpy  
  x_min,y_min,x_max,y_max,score = bbox  
  w_ = max(abs(x_max-x_min),abs(y_max-y_min))  

  w_ = w_*1.1  

  x_mid = (x_max+x_min)/2  
  y_mid = (y_max+y_min)/2  

  x1,y1,x2,y2 = int(x_mid-w_/2),int(y_mid-w_/2),int(x_mid+w_/2),int(y_mid+w_/2)  

  x1 = np.clip(x1,0,img.shape[1]-1)  
  x2 = np.clip(x2,0,img.shape[1]-1)  

  y1 = np.clip(y1,0,img.shape[0]-1)  
  y2 = np.clip(y2,0,img.shape[0]-1)  

联系方式 (Contact)

特别感谢

Owner
Eric.Lee
Eric.Lee
GFPGAN is a blind face restoration algorithm towards real-world face images.

GFPGAN is a blind face restoration algorithm towards real-world face images.

Applied Research Center (ARC), Tencent PCG 25.6k Jan 04, 2023
Allow foreign key attributes in list_display with '__'

django-related-admin Allow foreign key attributes in Django admin change list list_display with '__' This is based on DjangoSnippet 2996 which was mad

Petr Dlouhý 62 Nov 18, 2022
Legacy django jet rebooted , supports only Django 3

Django JET Reboot Rebooting the original project : django-jet. Django Jet is modern template for Django admin interface with improved functionality. W

215 Dec 31, 2022
Extends the Django Admin to include a extensible dashboard and navigation menu

django-admin-tools django-admin-tools is a collection of extensions/tools for the default django administration interface, it includes: a full feature

Django Admin Tools 731 Dec 28, 2022
Passhunt is a simple tool for searching of default credentials for network devices, web applications and more. Search through 523 vendors and their 2084 default passwords.

Passhunt is a simple tool for searching of default credentials for network devices, web applications and more. Search through 523 vendors and their 2084 default passwords.

Viral Maniar 1.1k Dec 31, 2022
PyMMO is a Python-based MMO game framework using sockets and PyGame.

PyMMO is a Python framework/template of a MMO game built using PyGame on top of Python's built-in socket module.

Luis Souto Maior 61 Dec 18, 2022
Jet Bridge (Universal) for Jet Admin – API-based Admin Panel Framework for your application

Jet Bridge for Jet Admin – Admin panel framework for your application Description About Jet Admin: https://about.jetadmin.io Live Demo: https://app.je

Jet Admin 1.3k Dec 27, 2022
Real-time monitor and web admin for Celery distributed task queue

Flower Flower is a web based tool for monitoring and administrating Celery clusters. Features Real-time monitoring using Celery Events Task progress a

Mher Movsisyan 5.5k Dec 28, 2022
Disable dark mode in Django admin user interface in Django 3.2.x.

Django Non Dark Admin Disable or enable dark mode user interface in Django admin panel (Django==3.2). Installation For install this app run in termina

Artem Galichkin 6 Nov 23, 2022
A high-level app and dashboarding solution for Python

Panel provides tools for easily composing widgets, plots, tables, and other viewable objects and controls into custom analysis tools, apps, and dashboards.

HoloViz 2.5k Jan 03, 2023
A flat theme for Django admin interface. Modern, fresh, simple.

Django Flat Theme django-flat-theme is included as part of Django from version 1.9! 🎉 Please use this app if your project is powered by an older Djan

elky 416 Sep 22, 2022
A user-friendly JSON editing form for Django admin

A user-friendly JSON editing form for Django admin

Bharat Chauhan 141 Dec 30, 2022
With Django Hijack, admins can log in and work on behalf of other users without having to know their credentials.

Django Hijack With Django Hijack, admins can log in and work on behalf of other users without having to know their credentials. Docs 3.x docs are avai

1.2k Jan 02, 2023
Extendable, adaptable rewrite of django.contrib.admin

django-admin2 One of the most useful parts of django.contrib.admin is the ability to configure various views that touch and alter data. django-admin2

Jazzband 1.2k Dec 29, 2022
Lazymux is a tool installer that is specially made for termux user which provides a lot of tool mainly used tools in termux and its easy to use

Lazymux is a tool installer that is specially made for termux user which provides a lot of tool mainly used tools in termux and its easy to use, Lazymux install any of the given tools provided by it

DedSecTL 1.8k Jan 09, 2023
Code to reproduce experiments in the paper "Task-Oriented Dialogue as Dataflow Synthesis" (TACL 2020).

Code to reproduce experiments in the paper "Task-Oriented Dialogue as Dataflow Synthesis" (TACL 2020).

Microsoft 274 Dec 28, 2022
Video Visual Relation Detection (VidVRD) tracklets generation. also for ACM MM Visual Relation Understanding Grand Challenge

VidVRD-tracklets This repository contains codes for Video Visual Relation Detection (VidVRD) tracklets generation based on MEGA and deepSORT. These tr

25 Dec 21, 2022
EOD (Easy and Efficient Object Detection) is a general object detection model production framework.

EOD (Easy and Efficient Object Detection) is a general object detection model production framework.

383 Jan 07, 2023
WebVirtCloud is virtualization web interface for admins and users

WebVirtCloud is a virtualization web interface for admins and users. It can delegate Virtual Machine's to users. A noVNC viewer presents a full graphical console to the guest domain. KVM is currently

Anatoliy Guskov 1.3k Dec 29, 2022
Drop-in replacement of Django admin comes with lots of goodies, fully extensible with plugin support, pretty UI based on Twitter Bootstrap.

Xadmin Drop-in replacement of Django admin comes with lots of goodies, fully extensible with plugin support, pretty UI based on Twitter Bootstrap. Liv

差沙 4.7k Dec 31, 2022