MINERVA: An out-of-the-box GUI tool for offline deep reinforcement learning

Overview

MINERVA: An out-of-the-box GUI tool for offline deep reinforcement learning

PyPI version test Docker Cloud Build Status Documentation Status Maintainability codecov MIT

MINERVA is an out-of-the-box GUI tool for offline deep reinforcement learning, designed for everyone including non-programmers to do reinforcement learning as a tool.

Documentation: https://minerva-ui.readthedocs.io

Chat: Gitter

key features

âš¡ All You Need Is Dataset

MINERVA only requires datasets to start offline deep reinforcement learning. Any combinations of vector observations and image observations with discrete actions and continuous actions are supported.

🔰 Stunning GUI

MINERVA provides designed with intuitive GUI to let everyone lerverage extremely powerful algorithms without barriers. The GUI is developed as a Single Page Application (SPA) to make it work in the eye-opening speed.

🚀 Powerful Algorithm

MINERVA is powered by d3rlpy, a powerful offline deep reinforcement learning library for Python, to provide extremely powerful algorithms in an out-of-the-box way. The trained policy can be exported as TorchScript and ONNX.

installation

PyPI

$ pip install minerva-ui

Docker

$ docker run -d --gpus all -p 9000:9000 --name minerva takuseno/minerva:latest

update guide

If you update MINERVA, the database schema should be also updated as follows:

$ pip install -U minerva-ui
$ minerva upgrade-db

usage

run server

At the first time, ~/.minerva will be automatically created to store database, uploaded datasets and training metrics.

$ minerva run

By default, you can access to MINERVA interface at http://localhost:9000 . You can change the host and port with --host and --port arguments respectively.

delete data

You can delete entire data (~/.minerva) as follows:

$ minerva clean

contributions

build

$ npm install
$ npm run build

coding style

This repository is fully formatted with yapf and standard. You can format the entire scripts as follows:

$ ./scripts/format

lint

This repository is fully analyzed with Pylint, ESLint and sass-lint. You can run analysis as follows:

$ ./scripts/lint

test

The unit tests are provided as much as possible. This repository is using pytest-cov instead of pytest. You can run the entire tests as follows:

$ ./scripts/test

acknowledgement

This work is supported by Information-technology Promotion Agency, Japan (IPA), Exploratory IT Human Resources Project (MITOU Program) in the fiscal year 2020.

The concept of the GUI software for deep reinforcement learning is inspired by DeepAnalyzer from Ghelia inc. I'm showing the great respect to the team here.

Comments
  • Bump y18n from 4.0.0 to 4.0.3

    Bump y18n from 4.0.0 to 4.0.3

    Bumps y18n from 4.0.0 to 4.0.3.

    Changelog

    Sourced from y18n's changelog.

    4.0.3 (2021-04-07)

    Bug Fixes

    • release: 4.x.x should not enforce Node 10 (#126) (1e21a53)

    4.0.1 (2020-11-30)

    Bug Fixes

    Commits
    Maintainer changes

    This version was pushed to npm by oss-bot, a new releaser for y18n since your current version.


    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies javascript 
    opened by dependabot[bot] 1
  • Bump pillow from 7.1.2 to 8.1.1

    Bump pillow from 7.1.2 to 8.1.1

    Bumps pillow from 7.1.2 to 8.1.1.

    Release notes

    Sourced from pillow's releases.

    8.1.1

    https://pillow.readthedocs.io/en/stable/releasenotes/8.1.1.html

    8.1.0

    https://pillow.readthedocs.io/en/stable/releasenotes/8.1.0.html

    Changes

    Dependencies

    Deprecations

    ... (truncated)

    Changelog

    Sourced from pillow's changelog.

    8.1.1 (2021-03-01)

    • Use more specific regex chars to prevent ReDoS. CVE-2021-25292 [hugovk]

    • Fix OOB Read in TiffDecode.c, and check the tile validity before reading. CVE-2021-25291 [wiredfool]

    • Fix negative size read in TiffDecode.c. CVE-2021-25290 [wiredfool]

    • Fix OOB read in SgiRleDecode.c. CVE-2021-25293 [wiredfool]

    • Incorrect error code checking in TiffDecode.c. CVE-2021-25289 [wiredfool]

    • PyModule_AddObject fix for Python 3.10 #5194 [radarhere]

    8.1.0 (2021-01-02)

    • Fix TIFF OOB Write error. CVE-2020-35654 #5175 [wiredfool]

    • Fix for Read Overflow in PCX Decoding. CVE-2020-35653 #5174 [wiredfool, radarhere]

    • Fix for SGI Decode buffer overrun. CVE-2020-35655 #5173 [wiredfool, radarhere]

    • Fix OOB Read when saving GIF of xsize=1 #5149 [wiredfool]

    • Makefile updates #5159 [wiredfool, radarhere]

    • Add support for PySide6 #5161 [hugovk]

    • Use disposal settings from previous frame in APNG #5126 [radarhere]

    • Added exception explaining that repr_png saves to PNG #5139 [radarhere]

    • Use previous disposal method in GIF load_end #5125 [radarhere]

    ... (truncated)

    Commits
    • 741d874 8.1.1 version bump
    • 179cd1c Added 8.1.1 release notes to index
    • 7d29665 Update CHANGES.rst [ci skip]
    • d25036f Credits
    • 973a4c3 Release notes for 8.1.1
    • 521dab9 Use more specific regex chars to prevent ReDoS
    • 8b8076b Fix for CVE-2021-25291
    • e25be1e Fix negative size read in TiffDecode.c
    • f891baa Fix OOB read in SgiRleDecode.c
    • cbfdde7 Incorrect error code checking in TiffDecode.c
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies python 
    opened by dependabot[bot] 1
  • Bump ini from 1.3.5 to 1.3.8

    Bump ini from 1.3.5 to 1.3.8

    Bumps ini from 1.3.5 to 1.3.8.

    Commits
    • a2c5da8 1.3.8
    • af5c6bb Do not use Object.create(null)
    • 8b648a1 don't test where our devdeps don't even work
    • c74c8af 1.3.7
    • 024b8b5 update deps, add linting
    • 032fbaf Use Object.create(null) to avoid default object property hazards
    • 2da9039 1.3.6
    • cfea636 better git push script, before publish instead of after
    • 56d2805 do not allow invalid hazardous string as section name
    • See full diff in compare view
    Maintainer changes

    This version was pushed to npm by isaacs, a new releaser for ini since your current version.


    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Bump axios from 0.19.2 to 0.21.1

    Bump axios from 0.19.2 to 0.21.1

    Bumps axios from 0.19.2 to 0.21.1.

    Release notes

    Sourced from axios's releases.

    v0.21.1

    0.21.1 (December 21, 2020)

    Fixes and Functionality:

    • Hotfix: Prevent SSRF (#3410)
    • Protocol not parsed when setting proxy config from env vars (#3070)
    • Updating axios in types to be lower case (#2797)
    • Adding a type guard for AxiosError (#2949)

    Internal and Tests:

    • Remove the skipping of the socket http test (#3364)
    • Use different socket for Win32 test (#3375)

    Huge thanks to everyone who contributed to this release via code (authors listed below) or via reviews and triaging on GitHub:

    v0.21.0

    0.21.0 (October 23, 2020)

    Fixes and Functionality:

    • Fixing requestHeaders.Authorization (#3287)
    • Fixing node types (#3237)
    • Fixing axios.delete ignores config.data (#3282)
    • Revert "Fixing overwrite Blob/File type as Content-Type in browser. (#1773)" (#3289)
    • Fixing an issue that type 'null' and 'undefined' is not assignable to validateStatus when typescript strict option is enabled (#3200)

    Internal and Tests:

    • Lock travis to not use node v15 (#3361)

    Documentation:

    • Fixing simple typo, existant -> existent (#3252)
    • Fixing typos (#3309)

    Huge thanks to everyone who contributed to this release via code (authors listed below) or via reviews and triaging on GitHub:

    ... (truncated)

    Changelog

    Sourced from axios's changelog.

    0.21.1 (December 21, 2020)

    Fixes and Functionality:

    • Hotfix: Prevent SSRF (#3410)
    • Protocol not parsed when setting proxy config from env vars (#3070)
    • Updating axios in types to be lower case (#2797)
    • Adding a type guard for AxiosError (#2949)

    Internal and Tests:

    • Remove the skipping of the socket http test (#3364)
    • Use different socket for Win32 test (#3375)

    Huge thanks to everyone who contributed to this release via code (authors listed below) or via reviews and triaging on GitHub:

    0.21.0 (October 23, 2020)

    Fixes and Functionality:

    • Fixing requestHeaders.Authorization (#3287)
    • Fixing node types (#3237)
    • Fixing axios.delete ignores config.data (#3282)
    • Revert "Fixing overwrite Blob/File type as Content-Type in browser. (#1773)" (#3289)
    • Fixing an issue that type 'null' and 'undefined' is not assignable to validateStatus when typescript strict option is enabled (#3200)

    Internal and Tests:

    • Lock travis to not use node v15 (#3361)

    Documentation:

    • Fixing simple typo, existant -> existent (#3252)
    • Fixing typos (#3309)

    Huge thanks to everyone who contributed to this release via code (authors listed below) or via reviews and triaging on GitHub:

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • What does Eposide id correspond to in a recommendation dataset

    What does Eposide id correspond to in a recommendation dataset

    I have a recommender dataset where I have userID, user_feature1,user_featureX...., ItemID,Item_feature1...Item_featureY.., reward(sales amount),datetime.. how should I restructure this data to be used/tested in Minerva? In the tutorial, https://minerva-ui.readthedocs.io/en/v0.40/getting_started.html, I am unable to correlate what will be the eposide ID in my dataset.

    opened by ayush488 0
  • ERROR: (sqlite3.OperationalError) unable to open database file.

    ERROR: (sqlite3.OperationalError) unable to open database file.

    Hi,

    I just installed the Minerva. However when I execute minerva run command I receive following error:

    sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file (Background on this error at: http://sqlalche.me/e/14/e3q8)

    Can you please tell me what is causing this error and how can I fix it? Thanks in advance.

    bug 
    opened by Zaharah 3
Releases(v0.40)
  • v0.40(May 29, 2021)

    Support d3rlpy v0.90!

    MINERVA now supports d3rlpy v0.90 and there are the highlights for MINERVA.

    • more precise CQL and BEAR implementations
    • predict_value is fixed for unscaled actions
    • CRR algorithm is available
    Source code(tar.gz)
    Source code(zip)
  • v0.32(Mar 6, 2021)

    a way to upload images is changed

    Previously, the image files are uploaded via selecting the directory that contains all the files. However this is an easy way, the browser limits the number of files due to the efficiency issue. So, from this version, you can upload zipped file that contains the image files. This is the more efficient and fast way. image

    Source code(tar.gz)
    Source code(zip)
  • v0.31(Mar 6, 2021)

  • v0.30(Mar 5, 2021)

    Support d3rlpy v0.70!

    MINERVA now supports d3rlpy v0.70, which has a lot of progress from the v0.41 that is the previous dependency.

    These are some highlights related to MINERVA.

    • the continuous action is automatically normalized based on the dataset statistics
    • discrete action option has been removed since the action-space is automatically detected
    • many other internal enhancements
    Source code(tar.gz)
    Source code(zip)
  • v0.20(Dec 21, 2020)

    Support d3rlpy v0.41!

    MINERVA now supports d3rlpy v0.41, which has a lot of progress from the v0.23 that is the previous dependency.

    These are the some highlights related to MINERVA.

    • extremely fast mini-batch creation
    • extremely fast frame stacking for image observation
    • extremely fast N-step TD calculation
    • new metrics
    • etc

    Algorithm Selection

    MINERVA now provides many many algorithms for both discrete and continuous control datasets. You can choose an algorithm at the project creation dialog.

    discrete algorithms

    • DQN
    • Double DQN
    • AWR
    • CQL
    • BCQ
    • SAC

    continuous algorithms

    • DDPG
    • TD3
    • SAC
    • BCQ
    • BEAR
    • CQL
    • AWR
    • AWAC
    • PLAS

    Of course, there is the Q function option to incorporate arbitrary algorithms with the powerful distributional Q functions.

    Q functions

    • mean
    • Quantile Regression
    • Implicit Quantile Network
    • Fully parameterized Quantile Function
    Source code(tar.gz)
    Source code(zip)
  • v0.12(Sep 9, 2020)

  • v0.11(Sep 9, 2020)

    Dataset Dashboard

    • upload datasets
      • discrete/continuous action-space
      • vector/image observations

    Project Dashboard

    • train with CQL with detailed configurations.

    Export Policy

    • Support TorchScript and ONNX.
    Source code(tar.gz)
    Source code(zip)
Owner
Takuma Seno
Machine learning engineer at Sony AI / Ph.D CS student at Keio University.
Takuma Seno
🤗 Push your spaCy pipelines to the Hugging Face Hub

spacy-huggingface-hub: Push your spaCy pipelines to the Hugging Face Hub This package provides a CLI command for uploading any trained spaCy pipeline

Explosion 30 Oct 09, 2022
Yolov5+SlowFast: Realtime Action Detection Based on PytorchVideo

Yolov5+SlowFast: Realtime Action Detection A realtime action detection frame work based on PytorchVideo. Here are some details about our modification:

WuFan 181 Dec 30, 2022
Quantile Regression DQN a Minimal Working Example, Distributional Reinforcement Learning with Quantile Regression

Quantile Regression DQN Quantile Regression DQN a Minimal Working Example, Distributional Reinforcement Learning with Quantile Regression (https://arx

Arsenii Senya Ashukha 80 Sep 17, 2022
Framework for joint representation learning, evaluation through multimodal registration and comparison with image translation based approaches

CoMIR: Contrastive Multimodal Image Representation for Registration Framework 🖼 Registration of images in different modalities with Deep Learning 🤖

Methods for Image Data Analysis - MIDA 55 Dec 09, 2022
Simply enable or disable your Nvidia dGPU

EnvyControl (WIP) Simply enable or disable your Nvidia dGPU Usage First clone this repo and install envycontrol with sudo pip install . CLI Turn off y

Victor Bayas 292 Jan 03, 2023
A new benchmark for Icon Question Answering (IconQA) and a large-scale icon dataset Icon645.

IconQA About IconQA is a new diverse abstract visual question answering dataset that highlights the importance of abstract diagram understanding and c

Pan Lu 24 Dec 30, 2022
[NeurIPS 2021] "G-PATE: Scalable Differentially Private Data Generator via Private Aggregation of Teacher Discriminators"

G-PATE This is the official code base for our NeurIPS 2021 paper: "G-PATE: Scalable Differentially Private Data Generator via Private Aggregation of T

AI Secure 14 Oct 12, 2022
Wandb-predictions - WANDB Predictions With Python

WANDB API CI/CD Below we capture the CI/CD scenarios that we would expect with o

Anish Shah 6 Oct 07, 2022
Learning to Self-Train for Semi-Supervised Few-Shot

Learning to Self-Train for Semi-Supervised Few-Shot Classification This repository contains the TensorFlow implementation for NeurIPS 2019 Paper "Lear

86 Dec 29, 2022
Learn the Deep Learning for Computer Vision in three steps: theory from base to SotA, code in PyTorch, and space-repetition with Anki

DeepCourse: Deep Learning for Computer Vision arthurdouillard.com/deepcourse/ This is a course I'm giving to the French engineering school EPITA each

Arthur Douillard 113 Nov 29, 2022
Flexible Option Learning - NeurIPS 2021

Flexible Option Learning This repository contains code for the paper Flexible Option Learning presented as a Spotlight at NeurIPS 2021. The implementa

Martin Klissarov 7 Nov 09, 2022
LoFTR:Detector-Free Local Feature Matching with Transformers CVPR 2021

LoFTR-with-train-script LoFTR:Detector-Free Local Feature Matching with Transformers CVPR 2021 (with train script --- unofficial ---). About Megadepth

Nan Xiaohu 15 Nov 04, 2022
Code accompanying our NeurIPS 2021 traffic4cast challenge

Traffic forecasting on traffic movie snippets This repo contains all code to reproduce our approach to the IARAI Traffic4cast 2021 challenge. In the c

Nina Wiedemann 2 Aug 09, 2022
Code and real data for the paper "Counterfactual Temporal Point Processes", available at arXiv.

counterfactual-tpp This is a repository containing code and real data for the paper Counterfactual Temporal Point Processes. Pre-requisites This code

Networks Learning 11 Dec 09, 2022
A Closer Look at Reference Learning for Fourier Phase Retrieval

A Closer Look at Reference Learning for Fourier Phase Retrieval This repository contains code for our NeurIPS 2021 Workshop on Deep Learning and Inver

Tobias Uelwer 1 Oct 28, 2021
PyElastica is the Python implementation of Elastica, an open-source software for the simulation of assemblies of slender, one-dimensional structures using Cosserat Rod theory.

PyElastica PyElastica is the python implementation of Elastica: an open-source project for simulating assemblies of slender, one-dimensional structure

Gazzola Lab 105 Jan 09, 2023
GB-CosFace: Rethinking Softmax-based Face Recognition from the Perspective of Open Set Classification

GB-CosFace: Rethinking Softmax-based Face Recognition from the Perspective of Open Set Classification This is the official pytorch implementation of t

Alibaba Cloud 5 Nov 14, 2022
RATE: Overcoming Noise and Sparsity of Textual Features in Real-Time Location Estimation (CIKM'17)

RATE: Overcoming Noise and Sparsity of Textual Features in Real-Time Location Estimation This is the implementation of RATE: Overcoming Noise and Spar

Yu Zhang 5 Feb 10, 2022
AdelaiDepth is an open source toolbox for monocular depth prediction.

AdelaiDepth is an open source toolbox for monocular depth prediction.

Adelaide Intelligent Machines (AIM) Group 743 Jan 01, 2023
Official repo for QHack—the quantum machine learning hackathon

Note: This repository has been frozen while we consider the submissions for the QHack Open Hackathon. We hope you enjoyed the event! Welcome to QHack,

Xanadu 118 Jan 05, 2023