Multi-task yolov5 with detection and segmentation based on yolov5

Last update: Dec 30, 2022

Related tags

Deep Learning yolov5ds

Overview

YOLOv5DS

Multi-task yolov5 with detection and segmentation based on yolov5(branch v6.0)

decoupled head
anchor free
segmentation head

README中文

Ablation experiment

All experiments is trained on a small dataset with 47 classes ,2.6k+ images for training and 1.5k+ images for validation:

model	P	R	[email protected]	[email protected]:95
yolov5s	0.536	0.368	0.374	0.206
yolov5s+train scrach	0.452	0.314	0.306	0.152
yolov5s+decoupled head	0.555	0.375	0.387	0.214
yolov5s + decoupled head+class balance weights	0.541	0.392	0.396	0.217
yolov5s + decoupled head+class balance weights	0.574	0.386	0.403	0.22
yolov5s + decoupled head+seghead	0.533	0.383	0.396	0.212

The baseline model is yolov5s. and decoupled head, add class balance weights all helps to improve MAP.

Adding a segmentation head can still get equivalent MAP as single detection model.

Training Method

python trainds.py

As VOC dataset do not offer the box labels and mask labels, so we forward this model with a detection batch and a segmention batch , and accumulate the gradient , than update the whole model parameters.

MAP

To compare with the SSD512, we use VOC07+12 training set as the detection training set, VOC07 test data as detection test data, for segmentation ,we use VOC12 segmentation datset as training and test set.

the input size is 512(letter box).

model	VOC2007 test
SSD512	79.8
yolov5s+seghead(512)	79.2

The above results only trained less than 200 epoch, weights

demo

see detectds.py.

Train custom data

Use labelme to label box and mask on your dataset;

the box label format is voc, you can use voc2yolo.py to convert to yolo format,

the mask label is json files , you should convert to mask .png image labels,like VOC2012 segmentation labels.

see how to arrange your detection dataset with yolov5 , then arrange your segmentation dataset same as yolo files , see data/voc.yaml:


# Train/val/test sets as 1) dir: path/to/imgs, 2) file: path/to/imgs.txt, or 3) list: [path/to/imgs1, path/to/imgs2, ..]
path: .  # dataset root dir
train: VOC/det/images/train  # train images (relative to 'path') 118287 images
val: VOC/det/images/test  # train images (relative to 'path') 5000 images
road_seg_train: VOC/seg/images/train   # road segmentation data
road_seg_val: VOC/seg/images/val

# Classes
nc: 20  # number of classes
segnc: 20

names: ['aeroplane', 'bicycle', 'bird', 'boat',
           'bottle', 'bus', 'car', 'cat', 'chair',
           'cow', 'diningtable', 'dog', 'horse',
           'motorbike', 'person', 'pottedplant',
           'sheep', 'sofa', 'train', 'tvmonitor']  # class names

segnames: ['aeroplane', 'bicycle', 'bird', 'boat',
           'bottle', 'bus', 'car', 'cat', 'chair',
           'cow', 'diningtable', 'dog', 'horse',
           'motorbike', 'person', 'pottedplant',
           'sheep', 'sofa', 'train', 'tvmonitor']

change the config in trainds.py and :

python trainds.py

test image folder with :
```
python detectds.py
```

Comments

请问我在对训好的模型运行val.py时出现这个错误可能是什么问题

im = cv2.resize(im, new_unpad, interpolation=cv2.INTER_LINEAR) cv2.error: OpenCV(4.1.2) C:\projects\opencv-python\opencv\modules\imgproc\src\resize.cpp:3723: error: (-215:Assertion failed) inv_scale_x > 0 in function 'cv::resize'

opened by zhangfx123 0

Multi-task yolov5 with detection and segmentation based on yolov5

Related tags

Overview

YOLOv5DS

Ablation experiment

Training Method

MAP

demo

Train custom data

Reference

You might also like...

a basic code repository for basic task in CV(classification,detection,segmentation)

A novel Engagement Detection with Multi-Task Training (ED-MTT) system

YOLOv5 Series Multi-backbone, Pruning and quantization Compression Tool Box.

A Python training and inference implementation of Yolov5 helmet detection in Jetson Xavier nx and Jetson nano

YOLOv5 🚀 is a family of object detection architectures and models pretrained on the COCO dataset

Implementation of PyTorch-based multi-task pre-trained models

Drone detection using YOLOv5

YOLOv5 detection interface - PyQt5 implementation

YOLOv5 + ROS2 object detection package

Comments

请问我在对训好的模型运行val.py时出现这个错误可能是什么问题

Releases(v6.0)

v6.0(Dec 16, 2021)

Owner

Provably Rare Gem Miner.

Pytorch implementation for "Implicit Semantic Response Alignment for Partial Domain Adaptation"

Implements Stacked-RNN in numpy and torch with manual forward and backward functions

The Official Repository for "Generalized OOD Detection: A Survey"

SAS output to EXCEL converter for Cornell/MIT Language and acquisition lab

Code for reproducing key results in the paper "InfoGAN: Interpretable Representation Learning by Information Maximizing Generative Adversarial Nets"

Implementing SYNTHESIZER: Rethinking Self-Attention in Transformer Models using Pytorch

Object detection GUI based on PaddleDetection

Self-Supervised Learning for Domain Adaptation on Point-Clouds

This is a JAX implementation of Neural Radiance Fields for learning purposes.

AdamW optimizer and cosine learning rate annealing with restarts

Low-dose Digital Mammography with Deep Learning

Learning to Adapt Structured Output Space for Semantic Segmentation, CVPR 2018 (spotlight)

Code for Multiple Instance Active Learning for Object Detection, CVPR 2021

🚩🚩🚩

Video Representation Learning by Recognizing Temporal Transformations. In ECCV, 2020.

Simple improvement of VQVAE that allow to generate x2 sized images compared to baseline

NER for Indian languages

[ICCV21] Official implementation of the "Social NCE: Contrastive Learning of Socially-aware Motion Representations" in PyTorch.

Code Repo for the ACL21 paper "Common Sense Beyond English: Evaluating and Improving Multilingual LMs for Commonsense Reasoning"