Deep Learning Datasets Maker is a QGIS plugin to make datasets creation easier for raster and vector data.

Overview

Deep Learning Dataset Maker

Python 3.8 QGIS 3.16.13

Deep Learning Datasets Maker is a QGIS plugin to make datasets creation easier for raster and vector data.

image

How to use

  1. Download and install QGIS and clone the repo :
git clone [email protected]:deepbands/deep-learning-datasets-maker.git
  1. Copy folder named deep-learning-datasets-maker in QGIS configuration folder and choose the plugin from plugin manager in QGIS (If not appeared restart QGIS).

    • You can know this folder from QGIS Setting Menu at the top-left of QGIS UI Settings > User Profiles > Open Active Profile Folder .
    • Go to python/plugins then paste the deep-learning-datasets-maker folder.
    • Full path should be like : C:\Users\$USER\AppData\Roaming\QGIS\QGIS3\profiles\default\python\plugins\deep-learning-datasets-maker.
  2. Open QGIS, load your raster and vector data then select the output paths for rasterized, images and labels then click ok.

TODO

  • Fix: If vector layer saved in memory not in file, rasterize can't work.
  • Fix: Splitiing Image Size.
Comments
  • Error running Quick Queries with QuickOSM recently started on Mac

    Error running Quick Queries with QuickOSM recently started on Mac

    I have been trying to run a query that I have previously had no problem running at all and have run into this error every time. Here are my system specs and whatnot:

    QGIS version | 3.22.4-Białowieża QGIS code revision 3f4577ce6e Qt version | 5.14.2 Python version | 3.8.7 GDAL/OGR version | 3.2.1 PROJ version | 6.3.2 EPSG Registry database version | v9.8.6 (2020-01-22) GEOS version | 3.9.1-CAPI-1.14.2 SQLite version | 3.31.1 PostgreSQL client version | 12.3 SpatiaLite version | 4.3.0a QWT version | 6.1.4 QScintilla2 version | 2.11.4 OS version | macOS 11.0

    Active Python plugins DigitizingTools | 1.5.1 QuickOSM | 2.0.1 quick_map_services | 0.19.29 usgs_stream_mapper | 0.2 sagaprovider | 2.12.99 grassprovider | 2.12.99 db_manager | 0.1.20 MetaSearch | 0.3.5

    And this is the dialogue from the QuickOSM Panel

    2022-03-19T18:47:06     INFO    All OSM objects with the key 'natural'='water' in the canvas or layer extent are going to be downloaded.
    2022-03-19T18:47:06     INFO    Query: natural_water
    2022-03-19T18:47:06     INFO    Encoded URL: https://lz4.overpass-api.de/api/interpreter?data=[out:xml] [timeout:25];%0A(%0A node[%22natural%22%3D%22water%22]( 27.49988,-82.45185,27.55754,-82.35723);%0A way[%22natural%22%3D%22water%22]( 27.49988,-82.45185,27.55754,-82.35723);%0A relation[%22natural%22%3D%22water%22]( 27.49988,-82.45185,27.55754,-82.35723);%0A);%0A(._;%3E;);%0Aout body;&info=QgisQuickOSMPlugin
    2022-03-19T18:47:09     INFO    Request completed
    2022-03-19T18:47:09     INFO    Checking OSM file content /private/var/folders/8s/cldnh_0n2wx39hw880qwh0lh0000gp/T/request-tNoEkc.osm
    2022-03-19T18:47:09     INFO    The OSM file is: /private/var/folders/8s/cldnh_0n2wx39hw880qwh0lh0000gp/T/request-tNoEkc.osm
    2022-03-19T18:47:09     CRITICAL   A critical error occurred, this is the traceback:
    2022-03-19T18:47:09     CRITICAL    
    2022-03-19T18:47:09     CRITICAL    base_processing_panel.py
    2022-03-19T18:47:09     CRITICAL    Error: Algorithm qgis:checkvalidity not found
                 
    2022-03-19T18:47:09     CRITICAL     File "/Users/Cecilia1/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/QuickOSM/ui/base_processing_panel.py", line 47, in run
                  self._run()
                 
                  File "/Users/Cecilia1/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/QuickOSM/ui/quick_query_panel.py", line 423, in _run
                  num_layers = process_quick_query(
                 
                  File "/Users/Cecilia1/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/QuickOSM/core/process.py", line 318, in process_quick_query
                  return process_query(
                 
                  File "/Users/Cecilia1/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/QuickOSM/core/process.py", line 257, in process_query
                  return open_file(
                 
                  File "/Users/Cecilia1/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/QuickOSM/core/process.py", line 106, in open_file
                  layers = osm_parser.processing_parse()
                 
                  File "/Users/Cecilia1/Library/Application Support/QGIS/QGIS3/profiles/default/python/plugins/QuickOSM/core/parser/osm_parser.py", line 133, in processing_parse
                  validity = processing.run(
                 
                  File "/Applications/QGIS-LTR.app/Contents/MacOS/../Resources/python/plugins/processing/tools/general.py", line 108, in run
                  return Processing.runAlgorithm(algOrName, parameters, onFinish, feedback, context)
                 
                  File "/Applications/QGIS-LTR.app/Contents/MacOS/../Resources/python/plugins/processing/core/Processing.py", line 171, in runAlgorithm
                  raise QgsProcessingException(msg)
    

    Finally: The Processing panel shows this dialogue each time I try a query.

    Error: Algorithm qgis:checkvalidity not found

    I have made sure my language is american english and that the startup style of the app is Macintosh. If anyone could please please help me I would be so appreciative. This function is so important to the research I am doing for my senior thesis. Thank you so much for taking time to read this.

    opened by cecilia-hampton 1
  • Init create COCO

    Init create COCO

    @Youssef-Harby The function is implemented, please check and use it. If you can, it can be merged into develop. But at present, the cutting image is repeated and can be modified 😃

    opened by geoyee 1
  • Error running in QGIS

    Error running in QGIS

    Using on a tiff image with 4 bands RGB and alpha QGIS 3.22.4

    2022-03-10T15:24:22 WARNING Traceback (most recent call last): File "C:\Users/Chris/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\split_rs_data.py", line 315, in run rasterize(ras_path, vec_path, output) File "C:\Users/Chris/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\utils\rasterize.py", line 11, in rasterize lyr = vec_ds.GetLayer() AttributeError: 'NoneType' object has no attribute 'GetLayer'

    opened by Chris-Has 1
  • Couldn't load plugin 'deep-learning-datasets-maker' due to an error when calling its classFactory() method

    Couldn't load plugin 'deep-learning-datasets-maker' due to an error when calling its classFactory() method

    Couldn't load plugin 'deep-learning-datasets-maker' due to an error when calling its classFactory() method

    subprocess.CalledProcessError: Command '['python3', '-m', 'pip', 'install', 'Cython', 'scikit-image', 'Pillow', 'pycocotools']' returned non-zero exit status 1. Traceback (most recent call last): File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\utils\COCO\pycococreatortools\pycococreatortools.py", line 10, in from pycocotools import mask File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 685, in _import mod = _builtin_import(name, globals, locals, fromlist, level) ModuleNotFoundError: No module named 'pycocotools'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last): File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 335, in startPlugin plugins[packageName] = package.classFactory(iface) File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker_init_.py", line 35, in classFactory from .split_rs_data import SplitRSData File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 685, in _import mod = _builtin_import(name, globals, locals, fromlist, level) File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\split_rs_data.py", line 46, in from .utils.COCO import clip_from_file, slice, from_mask_to_coco File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 685, in _import mod = builtin_import(name, globals, locals, fromlist, level) File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\utils\COCO_init.py", line 1, in from .shape_to_coco import clip_from_file, slice, from_mask_to_coco File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 685, in _import mod = _builtin_import(name, globals, locals, fromlist, level) File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\utils\COCO\shape_to_coco.py", line 16, in from .pycococreatortools import * File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 685, in _import mod = builtin_import(name, globals, locals, fromlist, level) File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\utils\COCO\pycococreatortools_init.py", line 1, in from .pycococreatortools import * File "C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python\qgis\utils.py", line 685, in _import mod = _builtin_import(name, globals, locals, fromlist, level) File "C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\deep-learning-datasets-maker\utils\COCO\pycococreatortools\pycococreatortools.py", line 16, in ["python3", '-m', 'pip', 'install', 'Cython', 'scikit-image', 'Pillow', 'pycocotools']) File "C:\PROGRA~1\QGIS3~1.4\apps\Python37\lib\subprocess.py", line 328, in check_call raise CalledProcessError(retcode, cmd) subprocess.CalledProcessError: Command '['python3', '-m', 'pip', 'install', 'Cython', 'scikit-image', 'Pillow', 'pycocotools']' returned non-zero exit status 1.

    Python version: 3.7.0 (v3.7.0:1bf9cc5093, Jun 27 2018, 04:59:51) [MSC v.1914 64 bit (AMD64)] QGIS version: 3.4.13-Madeira Madeira, 567300ccf1

    Python Path: C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins C:/PROGRA~1/QGIS3~1.4/apps/qgis-ltr/./python/plugins C:\Users\pprt\matterport\models\research C:\Users\pprt\matterport\models\research\slim C:\Program Files\QGIS 3.4\bin\python37.zip C:\PROGRA~1\QGIS3~1.4\apps\Python37\DLLs C:\PROGRA~1\QGIS3~1.4\apps\Python37\lib C:\Program Files\QGIS 3.4\bin C:\Users\pprt\AppData\Roaming\Python\Python37\site-packages C:\Users\pprt\AppData\Roaming\Python\Python37\site-packages\tf_unet-0.1.2-py3.7.egg C:\PROGRA~1\QGIS3~1.4\apps\Python37 C:\PROGRA~1\QGIS3~1.4\apps\Python37\lib\site-packages C:\PROGRA~1\QGIS3~1.4\apps\Python37\lib\site-packages\win32 C:\PROGRA~1\QGIS3~1.4\apps\Python37\lib\site-packages\win32\lib C:\PROGRA~1\QGIS3~1.4\apps\Python37\lib\site-packages\Pythonwin C:/Users/pprt/AppData/Roaming/QGIS/QGIS3\profiles\default/python

    enhancement 
    opened by aldinorizaldy 3
  • An error occurred during execution of following code

    An error occurred during execution of following code

    An error occurred during execution of following code: pyplugin_installer.instance().installPlugin('deep-learning-datasets-maker', stable=False)

    Traceback (most recent call last): File "", line 1, in File "/usr/share/qgis/python/pyplugin_installer/installer.py", line 333, in installPlugin self.processDependencies(plugin["id"]) File "/usr/share/qgis/python/pyplugin_installer/installer.py", line 682, in processDependencies dlg = QgsPluginDependenciesDialog(plugin_id, to_install, to_upgrade, not_found) File "/usr/share/qgis/python/pyplugin_installer/qgsplugindependenciesdialog.py", line 92, in init _make_row(data, i, name) File "/usr/share/qgis/python/pyplugin_installer/qgsplugindependenciesdialog.py", line 63, in _make_row widget.use_stable_version = data['use_stable_version'] KeyError: 'use_stable_version'

    Python version: 3.8.10 (default, Nov 26 2021, 20:14:08) [GCC 9.3.0]

    QGIS version: 3.22.3-Białowieża 'Białowieża', 1628765ec7

    Python path: ['/usr/share/qgis/python', '/home/robotics/.local/share/QGIS/QGIS3/profiles/default/python', '/home/robotics/.local/share/QGIS/QGIS3/profiles/default/python/plugins', '/usr/share/qgis/python/plugins', '/usr/lib/python38.zip', '/usr/lib/python3.8', '/usr/lib/python3.8/lib-dynload', '/usr/local/lib/python3.8/dist-packages', '/usr/lib/python3/dist-packages', '/home/robotics/.local/share/QGIS/QGIS3/profiles/default/python', '/home/robotics/.local/share/QGIS/QGIS3/profiles/default/python/plugins/DeepLearningTools']

    bug 
    opened by makamkkumar 1
Releases(0.2.1)
Owner
deepbands
deepbands
Reading list for research topics in Masked Image Modeling

awesome-MIM Reading list for research topics in Masked Image Modeling(MIM). We list the most popular methods for MIM, if I missed something, please su

ligang 231 Dec 07, 2022
A Fast Monotone Rotating Shallow Water model

pyRSW A Fast Monotone Rotating Shallow Water model How fast? As fast as a sustained 2 Gflop/s per core on a 2.5 GHz cpu (or 2048 Gflop/s with 1024 cor

Guillaume Roullet 13 Sep 28, 2022
PyTorch Code for "Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learning"

Generalization in Dexterous Manipulation via Geometry-Aware Multi-Task Learning [Project Page] [Paper] Wenlong Huang1, Igor Mordatch2, Pieter Abbeel1,

Wenlong Huang 40 Nov 22, 2022
ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphs

(Comet-) ATOMIC 2020: On Symbolic and Neural Commonsense Knowledge Graphs Paper Jena D. Hwang, Chandra Bhagavatula, Ronan Le Bras, Jeff Da, Keisuke Sa

AI2 152 Dec 27, 2022
Differentiable Annealed Importance Sampling (DAIS)

Differentiable Annealed Importance Sampling (DAIS) This repository contains the code to reproduce the DAIS results from the paper Differentiable Annea

Guodong Zhang 6 Dec 26, 2021
Code for "Solving Graph-based Public Good Games with Tree Search and Imitation Learning"

Code for "Solving Graph-based Public Good Games with Tree Search and Imitation Learning" This is the code for the paper Solving Graph-based Public Goo

Victor-Alexandru Darvariu 3 Dec 05, 2022
DeepMetaHandles: Learning Deformation Meta-Handles of 3D Meshes with Biharmonic Coordinates

DeepMetaHandles (CVPR2021 Oral) [paper] [animations] DeepMetaHandles is a shape deformation technique. It learns a set of meta-handles for each given

Liu Minghua 73 Dec 15, 2022
CBREN: Convolutional Neural Networks for Constant Bit Rate Video Quality Enhancement

CBREN This is the Pytorch implementation for our IEEE TCSVT paper : CBREN: Convolutional Neural Networks for Constant Bit Rate Video Quality Enhanceme

Zhao Hengrun 3 Nov 04, 2022
Learning to Self-Train for Semi-Supervised Few-Shot

Learning to Self-Train for Semi-Supervised Few-Shot Classification This repository contains the TensorFlow implementation for NeurIPS 2019 Paper "Lear

86 Dec 29, 2022
i3DMM: Deep Implicit 3D Morphable Model of Human Heads

i3DMM: Deep Implicit 3D Morphable Model of Human Heads CVPR 2021 (Oral) Arxiv | Poject Page This project is the official implementation our work, i3DM

Tarun Yenamandra 60 Jan 03, 2023
A computational block to solve entity alignment over textual attributes in a knowledge graph creation pipeline.

How to apply? Create your config.ini file following the example provided in config.ini Choose one of the options below to run: Run with Python3 pip in

Scientific Data Management Group 3 Jun 23, 2022
Tensorflow/Keras Plug-N-Play Deep Learning Models Compilation

DeepBay This project was created with the objective of compile Machine Learning Architectures created using Tensorflow or Keras. The architectures mus

Whitman Bohorquez 4 Sep 26, 2022
Reproduction of Vision Transformer in Tensorflow2. Train from scratch and Finetune.

Vision Transformer(ViT) in Tensorflow2 Tensorflow2 implementation of the Vision Transformer(ViT). This repository is for An image is worth 16x16 words

sungjun lee 42 Dec 27, 2022
Reinforcement Learning for finance

Reinforcement Learning for Finance We apply reinforcement learning for stock trading. Fetch Data Example import utils # fetch symbols from yahoo fina

Tomoaki Fujii 159 Jan 03, 2023
PointCloud Annotation Tools, support to label object bound box, ground, lane and kerb

PointCloud Annotation Tools, support to label object bound box, ground, lane and kerb

halo 368 Dec 06, 2022
New approach to benchmark VQA models

VQA Benchmarking This repository contains the web application & the python interface to evaluate VQA models. Documentation Please see the documentatio

4 Jul 25, 2022
Pip-package for trajectory benchmarking from "Be your own Benchmark: No-Reference Trajectory Metric on Registered Point Clouds", ECMR'21

Map Metrics for Trajectory Quality Map metrics toolkit provides a set of metrics to quantitatively evaluate trajectory quality via estimating consiste

Mobile Robotics Lab. at Skoltech 31 Oct 28, 2022
Breaking Shortcut: Exploring Fully Convolutional Cycle-Consistency for Video Correspondence Learning

Breaking Shortcut: Exploring Fully Convolutional Cycle-Consistency for Video Correspondence Learning Yansong Tang *, Zhenyu Jiang *, Zhenda Xie *, Yue

Zhenyu Jiang 12 Nov 16, 2022
This code is a near-infrared spectrum modeling method based on PCA and pls

Nirs-Pls-Corn This code is a near-infrared spectrum modeling method based on PCA and pls 近红外光谱分析技术属于交叉领域,需要化学、计算机科学、生物科学等多领域的合作。为此,在(北邮邮电大学杨辉华老师团队)指导下

Fu Pengyou 6 Dec 17, 2022
Source code for our paper "Learning to Break Deep Perceptual Hashing: The Use Case NeuralHash"

Learning to Break Deep Perceptual Hashing: The Use Case NeuralHash Abstract: Apple recently revealed its deep perceptual hashing system NeuralHash to

<a href=[email protected]"> 11 Dec 03, 2022