Deep Learning Datasets Maker is a QGIS plugin to make datasets creation easier for raster and vector data.

Overview

Deep Learning Dataset Maker

Python 3.8 QGIS 3.16.13

Deep Learning Datasets Maker is a QGIS plugin to make datasets creation easier for raster and vector data.

image

How to use

  1. Download and install QGIS and clone the repo :
git clone [email protected]:deepbands/deep-learning-datasets-maker.git
  1. Copy folder named deep-learning-datasets-maker in QGIS configuration folder and choose the plugin from plugin manager in QGIS (If not appeared restart QGIS).

    • You can know this folder from QGIS Setting Menu at the top-left of QGIS UI Settings > User Profiles > Open Active Profile Folder .
    • Go to python/plugins then paste the deep-learning-datasets-maker folder.
    • Full path should be like : C:\Users\$USER\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins\deep-learning-datasets-maker.
  2. Open QGIS, load your raster and vector data then select the output paths for rasterized, images and labels then click ok.

TODO

  • Fix: If vector layer saved in memory not in file, rasterize can't work.
  • Fix: Splitiing Image Size.
Comments
  • Error running Quick Queries with QuickOSM recently started on Mac

    Error running Quick Queries with QuickOSM recently started on Mac

    I have been trying to run a query that I have previously had no problem running at all and have run into this error every time. Here are my system specs and whatnot:

    QGIS version | 3.22.4-Białowieża QGIS code revision 3f4577ce6e Qt version | 5.14.2 Python version | 3.8.7 GDAL/OGR version | 3.2.1 PROJ version | 6.3.2 EPSG Registry database version | v9.8.6 (2020-01-22) GEOS version | 3.9.1-CAPI-1.14.2 SQLite version | 3.31.1 PostgreSQL client version | 12.3 SpatiaLite version | 4.3.0a QWT version | 6.1.4 QScintilla2 version | 2.11.4 OS version | macOS 11.0

    Active Python plugins DigitizingTools | 1.5.1 QuickOSM | 2.0.1 quick_map_services | 0.19.29 usgs_stream_mapper | 0.2 sagaprovider | 2.12.99 grassprovider | 2.12.99 db_manager | 0.1.20 MetaSearch | 0.3.5

    And this is the dialogue from the QuickOSM Panel

    2022-03-19T18:47:06     INFO    All OSM objects with the key 'natural'='water' in the canvas or layer extent are going to be downloaded.
    2022-03-19T18:47:06     INFO    Query: natural_water
    2022-03-19T18:47:06     INFO    Encoded URL: https://lz4.overpass-api.de/api/interpreter?data=[out:xml] [timeout:25];%0A(%0A node[%22natural%22%3D%22water%22]( 27.49988,-82.45185,27.55754,-82.35723);%0A way[%22natural%22%3D%22water%22]( 27.49988,-82.45185,27.55754,-82.35723);%0A relation[%22natural%22%3D%22water%22]( 27.49988,-82.45185,27.55754,-82.35723);%0A);%0A(._;%3E;);%0Aout body;&info=QgisQuickOSMPlugin
    2022-03-19T18:47:09     INFO    Request completed
    2022-03-19T18:47:09     INFO    Checking OSM file content /private/var/folders/8s/cldnh_0n2wx39hw880qwh0lh0000gp/T/request-tNoEkc.osm
    2022-03-19T18:47:09     INFO    The OSM file is: /private/var/folders/8s/cldnh_0n2wx39hw880qwh0lh0000gp/T/request-tNoEkc.osm
    2022-03-19T18:47:09     CRITICAL   A critical error occurred, this is the traceback:
    2022-03-19T18:47:09     CRITICAL    
    2022-03-19T18:47:09     CRITICAL    base_processing_panel.py
    2022-03-19T18:47:09     CRITICAL    Error: Algorithm qgis:checkvalidity not found
                 
    2022-03-19T18:47:09     CRITICAL     File "/Users/Cecilia1/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/QuickOSM/ui/base_processing_panel.py", line 47, in run
                  self._run()
                 
                  File "/Users/Cecilia1/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/QuickOSM/ui/quick_query_panel.py", line 423, in _run
                  num_layers = process_quick_query(
                 
                  File "/Users/Cecilia1/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/QuickOSM/core/process.py", line 318, in process_quick_query
                  return process_query(
                 
                  File "/Users/Cecilia1/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/QuickOSM/core/process.py", line 257, in process_query
                  return open_file(
                 
                  File "/Users/Cecilia1/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/QuickOSM/core/process.py", line 106, in open_file
                  layers = osm_parser.processing_parse()
                 
                  File "/Users/Cecilia1/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/QuickOSM/core/parser/osm_parser.py", line 133, in processing_parse
                  validity = processing.run(
                 
                  File "/Applications/QGIS-LTR.app/Contents/MacOS/../Resources/python/plugins/processing/tools/general.py", line 108, in run
                  return Processing.runAlgorithm(algOrName, parameters, onFinish, feedback, context)
                 
                  File "/Applications/QGIS-LTR.app/Contents/MacOS/../Resources/python/plugins/processing/core/Processing.py", line 171, in runAlgorithm
                  raise QgsProcessingException(msg)
    

    Finally: The Processing panel shows this dialogue each time I try a query.

    Error: Algorithm qgis:checkvalidity not found

    I have made sure my language is american english and that the startup style of the app is Macintosh. If anyone could please please help me I would be so appreciative. This function is so important to the research I am doing for my senior thesis. Thank you so much for taking time to read this.

    opened by cecilia-hampton 1
  • Init create COCO

    Init create COCO

    @Youssef-Harby The function is implemented, please check and use it. If you can, it can be merged into develop. But at present, the cutting image is repeated and can be modified 😃

    opened by geoyee 1
  • Error running in QGIS

    Error running in QGIS

    Using on a tiff image with 4 bands RGB and alpha QGIS 3.22.4

    2022-03-10T15:24:22 WARNING Traceback (most recent call last): File "C:\Users/Chris/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\split_rs_data.py", line 315, in run rasterize(ras_path, vec_path, output) File "C:\Users/Chris/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\utils\rasterize.py", line 11, in rasterize lyr = vec_ds.GetLayer() AttributeError: 'NoneType' object has no attribute 'GetLayer'

    opened by Chris-Has 1
  • Couldn't load plugin 'deep-learning-datasets-maker' due to an error when calling its classFactory() method

    Couldn't load plugin 'deep-learning-datasets-maker' due to an error when calling its classFactory() method

    Couldn't load plugin 'deep-learning-datasets-maker' due to an error when calling its classFactory() method

    subprocess.CalledProcessError: Command '['python3', '-m', 'pip', 'install', 'Cython', 'scikit-image', 'Pillow', 'pycocotools']' returned non-zero exit status 1. Traceback (most recent call last): File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\utils\COCO\pycococreatortools\pycococreatortools.py", line 10, in from pycocotools import mask File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 685, in _import mod = _builtin_import(name, globals, locals, fromlist, level) ModuleNotFoundError: No module named 'pycocotools'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last): File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 335, in startPlugin plugins[packageName] = package.classFactory(iface) File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker_init_.py", line 35, in classFactory from .split_rs_data import SplitRSData File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 685, in _import mod = _builtin_import(name, globals, locals, fromlist, level) File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\split_rs_data.py", line 46, in from .utils.COCO import clip_from_file, slice, from_mask_to_coco File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 685, in _import mod = builtin_import(name, globals, locals, fromlist, level) File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\utils\COCO_init.py", line 1, in from .shape_to_coco import clip_from_file, slice, from_mask_to_coco File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 685, in _import mod = _builtin_import(name, globals, locals, fromlist, level) File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\utils\COCO\shape_to_coco.py", line 16, in from .pycococreatortools import * File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 685, in _import mod = builtin_import(name, globals, locals, fromlist, level) File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\utils\COCO\pycococreatortools_init.py", line 1, in from .pycococreatortools import * File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 685, in _import mod = _builtin_import(name, globals, locals, fromlist, level) File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\utils\COCO\pycococreatortools\pycococreatortools.py", line 16, in ["python3", '-m', 'pip', 'install', 'Cython', 'scikit-image', 'Pillow', 'pycocotools']) File "C:\PROGRA~1\QGIS3~1.4\apps\Python37\lib\subprocess.py", line 328, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['python3', '-m', 'pip', 'install', 'Cython', 'scikit-image', 'Pillow', 'pycocotools']' returned non-zero exit status 1.

    Python version: 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit (AMD64)] QGIS version: 3.4.13-Madeira Madeira, 567300ccf1

    Python Path: C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python/plugins C:\Users\pprt\matterport\models\research C:\Users\pprt\matterport\models\research\slim C:\Program Files\QGIS 3.4\bin\python37.zip C:\PROGRA~1\QGIS3~1.4\apps\Python37\DLLs C:\PROGRA~1\QGIS3~1.4\apps\Python37\lib C:\Program Files\QGIS 3.4\bin C:\Users\pprt\AppData\Roaming\Python\Python37\site-packages C:\Users\pprt\AppData\Roaming\Python\Python37\site-packages\tf_unet-0.1.2-py3.7.egg C:\PROGRA~1\QGIS3~1.4\apps\Python37 C:\PROGRA~1\QGIS3~1.4\apps\Python37\lib\site-packages C:\PROGRA~1\QGIS3~1.4\apps\Python37\lib\site-packages\win32 C:\PROGRA~1\QGIS3~1.4\apps\Python37\lib\site-packages\win32\lib C:\PROGRA~1\QGIS3~1.4\apps\Python37\lib\site-packages\Pythonwin C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python

    enhancement 
    opened by aldinorizaldy 3
  • An error occurred during execution of following code

    An error occurred during execution of following code

    An error occurred during execution of following code: pyplugin_installer.instance().installPlugin('deep-learning-datasets-maker', stable=False)

    Traceback (most recent call last): File "", line 1, in File "/usr/share/qgis/python/pyplugin_installer/installer.py", line 333, in installPlugin self.processDependencies(plugin["id"]) File "/usr/share/qgis/python/pyplugin_installer/installer.py", line 682, in processDependencies dlg = QgsPluginDependenciesDialog(plugin_id, to_install, to_upgrade, not_found) File "/usr/share/qgis/python/pyplugin_installer/qgsplugindependenciesdialog.py", line 92, in init _make_row(data, i, name) File "/usr/share/qgis/python/pyplugin_installer/qgsplugindependenciesdialog.py", line 63, in _make_row widget.use_stable_version = data['use_stable_version'] KeyError: 'use_stable_version'

    Python version: 3.8.10 (default, Nov 26 2021, 20:14:08) [GCC 9.3.0]

    QGIS version: 3.22.3-Białowieża 'Białowieża', 1628765ec7

    Python path: ['/usr/share/qgis/python', '/home/robotics/.local/share/QGIS/QGIS3/profiles/default/python', '/home/robotics/.local/share/QGIS/QGIS3/profiles/default/python/plugins', '/usr/share/qgis/python/plugins', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages', '/home/robotics/.local/share/QGIS/QGIS3/profiles/default/python', '/home/robotics/.local/share/QGIS/QGIS3/profiles/default/python/plugins/DeepLearningTools']

    bug 
    opened by makamkkumar 1
Releases(0.2.1)
Owner
deepbands
deepbands
This is an official implementation for "Video Swin Transformers".

Video Swin Transformer By Ze Liu*, Jia Ning*, Yue Cao, Yixuan Wei, Zheng Zhang, Stephen Lin and Han Hu. This repo is the official implementation of "V

Swin Transformer 981 Jan 03, 2023
The code for the CVPR 2021 paper Neural Deformation Graphs, a novel approach for globally-consistent deformation tracking and 3D reconstruction of non-rigid objects.

Neural Deformation Graphs Project Page | Paper | Video Neural Deformation Graphs for Globally-consistent Non-rigid Reconstruction Aljaž Božič, Pablo P

Aljaz Bozic 134 Dec 16, 2022
PyTorch implementation of the Transformer in Post-LN (Post-LayerNorm) and Pre-LN (Pre-LayerNorm).

Transformer-PyTorch A PyTorch implementation of the Transformer from the paper Attention is All You Need in both Post-LN (Post-LayerNorm) and Pre-LN (

Jared Wang 22 Feb 27, 2022
A crash course in six episodes for software developers who want to become machine learning practitioners.

Featured code sample tensorflow-planespotting Code from the Google Cloud NEXT 2018 session "Tensorflow, deep learning and modern convnets, without a P

Google Cloud Platform 2.6k Jan 08, 2023
A PyTorch toolkit for 2D Human Pose Estimation.

PyTorch-Pose PyTorch-Pose is a PyTorch implementation of the general pipeline for 2D single human pose estimation. The aim is to provide the interface

Wei Yang 1.1k Dec 30, 2022
Objective of the repository is to learn and build machine learning models using Pytorch. 30DaysofML Using Pytorch

30 Days Of Machine Learning Using Pytorch Objective of the repository is to learn and build machine learning models using Pytorch. List of Algorithms

Mayur 119 Nov 24, 2022
This code is a toolbox that uses Torch library for training and evaluating the ERFNet architecture for semantic segmentation.

ERFNet This code is a toolbox that uses Torch library for training and evaluating the ERFNet architecture for semantic segmentation. NEW!! New PyTorch

Edu 104 Jan 05, 2023
Code accompanying the NeurIPS 2021 paper "Generating High-Quality Explanations for Navigation in Partially-Revealed Environments"

Generating High-Quality Explanations for Navigation in Partially-Revealed Environments This work presents an approach to explainable navigation under

RAIL Group @ George Mason University 1 Oct 28, 2022
DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision

The Official PyTorch Implementation of DiscoBox: Weakly Supervised Instance Segmentation and Semantic Correspondence from Box Supervision

Shiyi Lan 3 Oct 15, 2021
Outlier Exposure with Confidence Control for Out-of-Distribution Detection

OOD-detection-using-OECC This repository contains the essential code for the paper Outlier Exposure with Confidence Control for Out-of-Distribution De

Nazim Shaikh 64 Nov 02, 2022
ByteTrack: Multi-Object Tracking by Associating Every Detection Box

ByteTrack ByteTrack is a simple, fast and strong multi-object tracker. ByteTrack: Multi-Object Tracking by Associating Every Detection Box Yifu Zhang,

Yifu Zhang 2.9k Jan 04, 2023
The official code repository for examples in the O'Reilly book 'Generative Deep Learning'

Generative Deep Learning Teaching Machines to paint, write, compose and play The official code repository for examples in the O'Reilly book 'Generativ

David Foster 1.3k Dec 29, 2022
SimplEx - Explaining Latent Representations with a Corpus of Examples

SimplEx - Explaining Latent Representations with a Corpus of Examples Code Author: Jonathan Crabbé ( Jonathan Crabbé 14 Dec 15, 2022

This repository contains the code for our paper VDA (public in EMNLP2021 main conference)

Virtual Data Augmentation: A Robust and General Framework for Fine-tuning Pre-trained Models This repository contains the code for our paper VDA (publ

RUCAIBox 13 Aug 06, 2022
Resco: A simple python package that report the effect of deep residual learning

resco Description resco is a simple python package that report the effect of dee

Pierre-Arthur Claudé 1 Jun 28, 2022
Vision Deep-Learning using Tensorflow, Keras.

Welcome! I am a computer vision deep learning developer working in Korea. This is my blog, and you can see everything I've studied here. https://www.n

kimminjun 6 Dec 14, 2022
PyTorch code for our paper "Image Super-Resolution with Non-Local Sparse Attention" (CVPR2021).

Image Super-Resolution with Non-Local Sparse Attention This repository is for NLSN introduced in the following paper "Image Super-Resolution with Non-

143 Dec 28, 2022
GRaNDPapA: Generator of Rad Names from Decent Paper Acronyms

GRaNDPapA: Generator of Rad Names from Decent Paper Acronyms Trying to publish a new machine learning model and can't write a decent title for your pa

264 Nov 08, 2022
[PAMI 2020] Show, Match and Segment: Joint Weakly Supervised Learning of Semantic Matching and Object Co-segmentation

Show, Match and Segment: Joint Weakly Supervised Learning of Semantic Matching and Object Co-segmentation This repository contains the source code for

Yun-Chun Chen 60 Nov 25, 2022
Attendance Monitoring with Face Recognition using Python

Attendance Monitoring with Face Recognition using Python A python GUI integrated attendance system using face recognition to take attendance. In this

Vaibhav Rajput 2 Jun 21, 2022