Multi-task yolov5 with detection and segmentation based on yolov5

Last update: Dec 30, 2022

Related tags

Deep Learning yolov5ds

Overview

YOLOv5DS

Multi-task yolov5 with detection and segmentation based on yolov5(branch v6.0)

decoupled head
anchor free
segmentation head

README中文

Ablation experiment

All experiments is trained on a small dataset with 47 classes ,2.6k+ images for training and 1.5k+ images for validation:

model	P	R	[email protected]	[email protected]:95
yolov5s	0.536	0.368	0.374	0.206
yolov5s+train scrach	0.452	0.314	0.306	0.152
yolov5s+decoupled head	0.555	0.375	0.387	0.214
yolov5s + decoupled head+class balance weights	0.541	0.392	0.396	0.217
yolov5s + decoupled head+class balance weights	0.574	0.386	0.403	0.22
yolov5s + decoupled head+seghead	0.533	0.383	0.396	0.212

The baseline model is yolov5s. and decoupled head, add class balance weights all helps to improve MAP.

Adding a segmentation head can still get equivalent MAP as single detection model.

Training Method

python trainds.py

As VOC dataset do not offer the box labels and mask labels, so we forward this model with a detection batch and a segmention batch , and accumulate the gradient , than update the whole model parameters.

MAP

To compare with the SSD512, we use VOC07+12 training set as the detection training set, VOC07 test data as detection test data, for segmentation ,we use VOC12 segmentation datset as training and test set.

the input size is 512(letter box).

model	VOC2007 test
SSD512	79.8
yolov5s+seghead(512)	79.2

The above results only trained less than 200 epoch, weights

demo

see detectds.py.

Train custom data

Use labelme to label box and mask on your dataset;

the box label format is voc, you can use voc2yolo.py to convert to yolo format,

the mask label is json files , you should convert to mask .png image labels,like VOC2012 segmentation labels.

see how to arrange your detection dataset with yolov5 , then arrange your segmentation dataset same as yolo files , see data/voc.yaml:


# Train/val/test sets as 1) dir: path/to/imgs, 2) file: path/to/imgs.txt, or 3) list: [path/to/imgs1, path/to/imgs2, ..]
path: .  # dataset root dir
train: VOC/det/images/train  # train images (relative to 'path') 118287 images
val: VOC/det/images/test  # train images (relative to 'path') 5000 images
road_seg_train: VOC/seg/images/train   # road segmentation data
road_seg_val: VOC/seg/images/val

# Classes
nc: 20  # number of classes
segnc: 20

names: ['aeroplane', 'bicycle', 'bird', 'boat',
           'bottle', 'bus', 'car', 'cat', 'chair',
           'cow', 'diningtable', 'dog', 'horse',
           'motorbike', 'person', 'pottedplant',
           'sheep', 'sofa', 'train', 'tvmonitor']  # class names

segnames: ['aeroplane', 'bicycle', 'bird', 'boat',
           'bottle', 'bus', 'car', 'cat', 'chair',
           'cow', 'diningtable', 'dog', 'horse',
           'motorbike', 'person', 'pottedplant',
           'sheep', 'sofa', 'train', 'tvmonitor']

change the config in trainds.py and :

python trainds.py

test image folder with :
```
python detectds.py
```

Comments

请问我在对训好的模型运行val.py时出现这个错误可能是什么问题

im = cv2.resize(im, new_unpad, interpolation=cv2.INTER_LINEAR) cv2.error: OpenCV(4.1.2) C:\projects\opencv-python\opencv\modules\imgproc\src\resize.cpp:3723: error: (-215:Assertion failed) inv_scale_x > 0 in function 'cv::resize'

opened by zhangfx123 0

Multi-task yolov5 with detection and segmentation based on yolov5

Related tags

Overview

YOLOv5DS

Ablation experiment

Training Method

MAP

demo

Train custom data

Reference

You might also like...

a basic code repository for basic task in CV(classification,detection,segmentation)

A novel Engagement Detection with Multi-Task Training (ED-MTT) system

YOLOv5 Series Multi-backbone, Pruning and quantization Compression Tool Box.

A Python training and inference implementation of Yolov5 helmet detection in Jetson Xavier nx and Jetson nano

YOLOv5 🚀 is a family of object detection architectures and models pretrained on the COCO dataset

Implementation of PyTorch-based multi-task pre-trained models

Drone detection using YOLOv5

YOLOv5 detection interface - PyQt5 implementation

YOLOv5 + ROS2 object detection package

Comments

请问我在对训好的模型运行val.py时出现这个错误可能是什么问题

Releases(v6.0)

v6.0(Dec 16, 2021)

Owner

Project code for weakly supervised 3D object detectors using wide-baseline multi-view traffic camera data: WIBAM.

Tensorflow implementation of MIRNet for Low-light image enhancement

FaceAnon - Anonymize people in images and videos using yolov5-crowdhuman

Code for Low-Cost Algorithmic Recourse for Users With Uncertain Cost Functions

CFC-Net: A Critical Feature Capturing Network for Arbitrary-Oriented Object Detection in Remote Sensing Images

SporeAgent: Reinforced Scene-level Plausibility for Object Pose Refinement

This is an open source library implementing hyperbox-based machine learning algorithms

Code for "Primitive Representation Learning for Scene Text Recognition" (CVPR 2021)

Lightweight, Portable, Flexible Distributed/Mobile Deep Learning with Dynamic, Mutation-aware Dataflow Dep Scheduler; for Python, R, Julia, Scala, Go, Javascript and more

POCO: Point Convolution for Surface Reconstruction

Implementation of ResMLP, an all MLP solution to image classification, in Pytorch

Spatial Transformer Nets in TensorFlow/ TensorLayer

DI-smartcross - Decision Intelligence Platform for Traffic Crossing Signal Control

Official implementation of "Motif-based Graph Self-Supervised Learning forMolecular Property Prediction"

Alternatives to Deep Neural Networks for Function Approximations in Finance

LibFewShot: A Comprehensive Library for Few-shot Learning.

ICNet and PSPNet-50 in Tensorflow for real-time semantic segmentation

Code accompanying the paper "Knowledge Base Completion Meets Transfer Learning"

This repository is an open-source implementation of the ICRA 2021 paper: Locus: LiDAR-based Place Recognition using Spatiotemporal Higher-Order Pooling.

Repository for Multimodal AutoML Benchmark