MINERVA: An out-of-the-box GUI tool for offline deep reinforcement learning

Overview

MINERVA: An out-of-the-box GUI tool for offline deep reinforcement learning

PyPI version test Docker Cloud Build Status Documentation Status Maintainability codecov MIT

MINERVA is an out-of-the-box GUI tool for offline deep reinforcement learning, designed for everyone including non-programmers to do reinforcement learning as a tool.

Documentation: https://minerva-ui.readthedocs.io

Chat: Gitter

key features

All You Need Is Dataset

MINERVA only requires datasets to start offline deep reinforcement learning. Any combinations of vector observations and image observations with discrete actions and continuous actions are supported.

🔰 Stunning GUI

MINERVA provides designed with intuitive GUI to let everyone lerverage extremely powerful algorithms without barriers. The GUI is developed as a Single Page Application (SPA) to make it work in the eye-opening speed.

🚀 Powerful Algorithm

MINERVA is powered by d3rlpy, a powerful offline deep reinforcement learning library for Python, to provide extremely powerful algorithms in an out-of-the-box way. The trained policy can be exported as TorchScript and ONNX.

installation

PyPI

$ pip install minerva-ui

Docker

$ docker run -d --gpus all -p 9000:9000 --name minerva takuseno/minerva:latest

update guide

If you update MINERVA, the database schema should be also updated as follows:

$ pip install -U minerva-ui
$ minerva upgrade-db

usage

run server

At the first time, ~/.minerva will be automatically created to store database, uploaded datasets and training metrics.

$ minerva run

By default, you can access to MINERVA interface at http://localhost:9000 . You can change the host and port with --host and --port arguments respectively.

delete data

You can delete entire data (~/.minerva) as follows:

$ minerva clean

contributions

build

$ npm install
$ npm run build

coding style

This repository is fully formatted with yapf and standard. You can format the entire scripts as follows:

$ ./scripts/format

lint

This repository is fully analyzed with Pylint, ESLint and sass-lint. You can run analysis as follows:

$ ./scripts/lint

test

The unit tests are provided as much as possible. This repository is using pytest-cov instead of pytest. You can run the entire tests as follows:

$ ./scripts/test

acknowledgement

This work is supported by Information-technology Promotion Agency, Japan (IPA), Exploratory IT Human Resources Project (MITOU Program) in the fiscal year 2020.

The concept of the GUI software for deep reinforcement learning is inspired by DeepAnalyzer from Ghelia inc. I'm showing the great respect to the team here.

Comments
  • Bump y18n from 4.0.0 to 4.0.3

    Bump y18n from 4.0.0 to 4.0.3

    Bumps y18n from 4.0.0 to 4.0.3.

    Changelog

    Sourced from y18n's changelog.

    4.0.3 (2021-04-07)

    Bug Fixes

    • release: 4.x.x should not enforce Node 10 (#126) (1e21a53)

    4.0.1 (2020-11-30)

    Bug Fixes

    Commits
    Maintainer changes

    This version was pushed to npm by oss-bot, a new releaser for y18n since your current version.


    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies javascript 
    opened by dependabot[bot] 1
  • Bump pillow from 7.1.2 to 8.1.1

    Bump pillow from 7.1.2 to 8.1.1

    Bumps pillow from 7.1.2 to 8.1.1.

    Release notes

    Sourced from pillow's releases.

    8.1.1

    https://pillow.readthedocs.io/en/stable/releasenotes/8.1.1.html

    8.1.0

    https://pillow.readthedocs.io/en/stable/releasenotes/8.1.0.html

    Changes

    Dependencies

    Deprecations

    ... (truncated)

    Changelog

    Sourced from pillow's changelog.

    8.1.1 (2021-03-01)

    • Use more specific regex chars to prevent ReDoS. CVE-2021-25292 [hugovk]

    • Fix OOB Read in TiffDecode.c, and check the tile validity before reading. CVE-2021-25291 [wiredfool]

    • Fix negative size read in TiffDecode.c. CVE-2021-25290 [wiredfool]

    • Fix OOB read in SgiRleDecode.c. CVE-2021-25293 [wiredfool]

    • Incorrect error code checking in TiffDecode.c. CVE-2021-25289 [wiredfool]

    • PyModule_AddObject fix for Python 3.10 #5194 [radarhere]

    8.1.0 (2021-01-02)

    • Fix TIFF OOB Write error. CVE-2020-35654 #5175 [wiredfool]

    • Fix for Read Overflow in PCX Decoding. CVE-2020-35653 #5174 [wiredfool, radarhere]

    • Fix for SGI Decode buffer overrun. CVE-2020-35655 #5173 [wiredfool, radarhere]

    • Fix OOB Read when saving GIF of xsize=1 #5149 [wiredfool]

    • Makefile updates #5159 [wiredfool, radarhere]

    • Add support for PySide6 #5161 [hugovk]

    • Use disposal settings from previous frame in APNG #5126 [radarhere]

    • Added exception explaining that repr_png saves to PNG #5139 [radarhere]

    • Use previous disposal method in GIF load_end #5125 [radarhere]

    ... (truncated)

    Commits
    • 741d874 8.1.1 version bump
    • 179cd1c Added 8.1.1 release notes to index
    • 7d29665 Update CHANGES.rst [ci skip]
    • d25036f Credits
    • 973a4c3 Release notes for 8.1.1
    • 521dab9 Use more specific regex chars to prevent ReDoS
    • 8b8076b Fix for CVE-2021-25291
    • e25be1e Fix negative size read in TiffDecode.c
    • f891baa Fix OOB read in SgiRleDecode.c
    • cbfdde7 Incorrect error code checking in TiffDecode.c
    • Additional commits viewable in compare view

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies python 
    opened by dependabot[bot] 1
  • Bump ini from 1.3.5 to 1.3.8

    Bump ini from 1.3.5 to 1.3.8

    Bumps ini from 1.3.5 to 1.3.8.

    Commits
    • a2c5da8 1.3.8
    • af5c6bb Do not use Object.create(null)
    • 8b648a1 don't test where our devdeps don't even work
    • c74c8af 1.3.7
    • 024b8b5 update deps, add linting
    • 032fbaf Use Object.create(null) to avoid default object property hazards
    • 2da9039 1.3.6
    • cfea636 better git push script, before publish instead of after
    • 56d2805 do not allow invalid hazardous string as section name
    • See full diff in compare view
    Maintainer changes

    This version was pushed to npm by isaacs, a new releaser for ini since your current version.


    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • Bump axios from 0.19.2 to 0.21.1

    Bump axios from 0.19.2 to 0.21.1

    Bumps axios from 0.19.2 to 0.21.1.

    Release notes

    Sourced from axios's releases.

    v0.21.1

    0.21.1 (December 21, 2020)

    Fixes and Functionality:

    • Hotfix: Prevent SSRF (#3410)
    • Protocol not parsed when setting proxy config from env vars (#3070)
    • Updating axios in types to be lower case (#2797)
    • Adding a type guard for AxiosError (#2949)

    Internal and Tests:

    • Remove the skipping of the socket http test (#3364)
    • Use different socket for Win32 test (#3375)

    Huge thanks to everyone who contributed to this release via code (authors listed below) or via reviews and triaging on GitHub:

    v0.21.0

    0.21.0 (October 23, 2020)

    Fixes and Functionality:

    • Fixing requestHeaders.Authorization (#3287)
    • Fixing node types (#3237)
    • Fixing axios.delete ignores config.data (#3282)
    • Revert "Fixing overwrite Blob/File type as Content-Type in browser. (#1773)" (#3289)
    • Fixing an issue that type 'null' and 'undefined' is not assignable to validateStatus when typescript strict option is enabled (#3200)

    Internal and Tests:

    • Lock travis to not use node v15 (#3361)

    Documentation:

    • Fixing simple typo, existant -> existent (#3252)
    • Fixing typos (#3309)

    Huge thanks to everyone who contributed to this release via code (authors listed below) or via reviews and triaging on GitHub:

    ... (truncated)

    Changelog

    Sourced from axios's changelog.

    0.21.1 (December 21, 2020)

    Fixes and Functionality:

    • Hotfix: Prevent SSRF (#3410)
    • Protocol not parsed when setting proxy config from env vars (#3070)
    • Updating axios in types to be lower case (#2797)
    • Adding a type guard for AxiosError (#2949)

    Internal and Tests:

    • Remove the skipping of the socket http test (#3364)
    • Use different socket for Win32 test (#3375)

    Huge thanks to everyone who contributed to this release via code (authors listed below) or via reviews and triaging on GitHub:

    0.21.0 (October 23, 2020)

    Fixes and Functionality:

    • Fixing requestHeaders.Authorization (#3287)
    • Fixing node types (#3237)
    • Fixing axios.delete ignores config.data (#3282)
    • Revert "Fixing overwrite Blob/File type as Content-Type in browser. (#1773)" (#3289)
    • Fixing an issue that type 'null' and 'undefined' is not assignable to validateStatus when typescript strict option is enabled (#3200)

    Internal and Tests:

    • Lock travis to not use node v15 (#3361)

    Documentation:

    • Fixing simple typo, existant -> existent (#3252)
    • Fixing typos (#3309)

    Huge thanks to everyone who contributed to this release via code (authors listed below) or via reviews and triaging on GitHub:

    ... (truncated)

    Commits

    Dependabot compatibility score

    Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.


    Dependabot commands and options

    You can trigger Dependabot actions by commenting on this PR:

    • @dependabot rebase will rebase this PR
    • @dependabot recreate will recreate this PR, overwriting any edits that have been made to it
    • @dependabot merge will merge this PR after your CI passes on it
    • @dependabot squash and merge will squash and merge this PR after your CI passes on it
    • @dependabot cancel merge will cancel a previously requested merge and block automerging
    • @dependabot reopen will reopen this PR if it is closed
    • @dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
    • @dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
    • @dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)
    • @dependabot use these labels will set the current labels as the default for future PRs for this repo and language
    • @dependabot use these reviewers will set the current reviewers as the default for future PRs for this repo and language
    • @dependabot use these assignees will set the current assignees as the default for future PRs for this repo and language
    • @dependabot use this milestone will set the current milestone as the default for future PRs for this repo and language

    You can disable automated security fix PRs for this repo from the Security Alerts page.

    dependencies 
    opened by dependabot[bot] 0
  • What does Eposide id correspond to in a recommendation dataset

    What does Eposide id correspond to in a recommendation dataset

    I have a recommender dataset where I have userID, user_feature1,user_featureX...., ItemID,Item_feature1...Item_featureY.., reward(sales amount),datetime.. how should I restructure this data to be used/tested in Minerva? In the tutorial, https://minerva-ui.readthedocs.io/en/v0.40/getting_started.html, I am unable to correlate what will be the eposide ID in my dataset.

    opened by ayush488 0
  • ERROR: (sqlite3.OperationalError) unable to open database file.

    ERROR: (sqlite3.OperationalError) unable to open database file.

    Hi,

    I just installed the Minerva. However when I execute minerva run command I receive following error:

    sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) unable to open database file (Background on this error at: http://sqlalche.me/e/14/e3q8)

    Can you please tell me what is causing this error and how can I fix it? Thanks in advance.

    bug 
    opened by Zaharah 3
Releases(v0.40)
  • v0.40(May 29, 2021)

    Support d3rlpy v0.90!

    MINERVA now supports d3rlpy v0.90 and there are the highlights for MINERVA.

    • more precise CQL and BEAR implementations
    • predict_value is fixed for unscaled actions
    • CRR algorithm is available
    Source code(tar.gz)
    Source code(zip)
  • v0.32(Mar 6, 2021)

    a way to upload images is changed

    Previously, the image files are uploaded via selecting the directory that contains all the files. However this is an easy way, the browser limits the number of files due to the efficiency issue. So, from this version, you can upload zipped file that contains the image files. This is the more efficient and fast way. image

    Source code(tar.gz)
    Source code(zip)
  • v0.31(Mar 6, 2021)

  • v0.30(Mar 5, 2021)

    Support d3rlpy v0.70!

    MINERVA now supports d3rlpy v0.70, which has a lot of progress from the v0.41 that is the previous dependency.

    These are some highlights related to MINERVA.

    • the continuous action is automatically normalized based on the dataset statistics
    • discrete action option has been removed since the action-space is automatically detected
    • many other internal enhancements
    Source code(tar.gz)
    Source code(zip)
  • v0.20(Dec 21, 2020)

    Support d3rlpy v0.41!

    MINERVA now supports d3rlpy v0.41, which has a lot of progress from the v0.23 that is the previous dependency.

    These are the some highlights related to MINERVA.

    • extremely fast mini-batch creation
    • extremely fast frame stacking for image observation
    • extremely fast N-step TD calculation
    • new metrics
    • etc

    Algorithm Selection

    MINERVA now provides many many algorithms for both discrete and continuous control datasets. You can choose an algorithm at the project creation dialog.

    discrete algorithms

    • DQN
    • Double DQN
    • AWR
    • CQL
    • BCQ
    • SAC

    continuous algorithms

    • DDPG
    • TD3
    • SAC
    • BCQ
    • BEAR
    • CQL
    • AWR
    • AWAC
    • PLAS

    Of course, there is the Q function option to incorporate arbitrary algorithms with the powerful distributional Q functions.

    Q functions

    • mean
    • Quantile Regression
    • Implicit Quantile Network
    • Fully parameterized Quantile Function
    Source code(tar.gz)
    Source code(zip)
  • v0.12(Sep 9, 2020)

  • v0.11(Sep 9, 2020)

    Dataset Dashboard

    • upload datasets
      • discrete/continuous action-space
      • vector/image observations

    Project Dashboard

    • train with CQL with detailed configurations.

    Export Policy

    • Support TorchScript and ONNX.
    Source code(tar.gz)
    Source code(zip)
Owner
Takuma Seno
Machine learning engineer at Sony AI / Ph.D CS student at Keio University.
Takuma Seno
This repository accompanies our paper “Do Prompt-Based Models Really Understand the Meaning of Their Prompts?”

This repository accompanies our paper “Do Prompt-Based Models Really Understand the Meaning of Their Prompts?” Usage To replicate our results in Secti

Albert Webson 64 Dec 11, 2022
This repository contains all data used for writing a research paper Multiple Object Trackers in OpenCV: A Benchmark, presented in ISIE 2021 conference in Kyoto, Japan.

OpenCV-Multiple-Object-Tracking Python is version 3.6.7 to install opencv: pip uninstall opecv-python pip uninstall opencv-contrib-python pip install

6 Dec 19, 2021
use tensorflow 2.0 to tell a dog and cat from a specified picture

dog_or_cat use tensorflow 2.0 to tell a dog and cat from a specified picture This is one of the classic experiments for the introduction of deep learn

你这个代码我看不懂 1 Oct 22, 2021
Convolutional Neural Networks on Graphs with Fast Localized Spectral Filtering

Graph ConvNets in PyTorch October 15, 2017 Xavier Bresson http://www.ntu.edu.sg/home/xbresson https://github.com/xbresson https://twitter.com/xbresson

Xavier Bresson 287 Jan 04, 2023
PyTorch code for ICLR 2021 paper Unbiased Teacher for Semi-Supervised Object Detection

Unbiased Teacher for Semi-Supervised Object Detection This is the PyTorch implementation of our paper: Unbiased Teacher for Semi-Supervised Object Detection

Facebook Research 366 Dec 28, 2022
Differential Privacy for Heterogeneous Federated Learning : Utility & Privacy tradeoffs

Differential Privacy for Heterogeneous Federated Learning : Utility & Privacy tradeoffs In this work, we propose an algorithm DP-SCAFFOLD(-warm), whic

19 Nov 10, 2022
You are AllSet: A Multiset Function Framework for Hypergraph Neural Networks.

AllSet This is the repo for our paper: You are AllSet: A Multiset Function Framework for Hypergraph Neural Networks. We prepared all codes and a subse

Jianhao 51 Dec 24, 2022
Implementation of the federated dual coordinate descent (FedDCD) method.

FedDCD.jl Implementation of the federated dual coordinate descent (FedDCD) method. Installation To install, just call Pkg.add("https://github.com/Zhen

Zhenan Fan 6 Sep 21, 2022
Official implementation for "QS-Attn: Query-Selected Attention for Contrastive Learning in I2I Translation" (CVPR 2022)

QS-Attn: Query-Selected Attention for Contrastive Learning in I2I Translation (CVPR2022) https://arxiv.org/abs/2203.08483 Unpaired image-to-image (I2I

Xueqi Hu 50 Dec 16, 2022
Learning to Stylize Novel Views

Learning to Stylize Novel Views [Project] [Paper] Contact: Hsin-Ping Huang ([ema

34 Nov 27, 2022
Face Mask Detection System built with OpenCV, TensorFlow using Computer Vision concepts

Face mask detection Face Mask Detection System built with OpenCV, TensorFlow using Computer Vision concepts in order to detect face masks in static im

Vaibhav Shukla 1 Oct 27, 2021
Learnable Boundary Guided Adversarial Training (ICCV2021)

Learnable Boundary Guided Adversarial Training This repository contains the implementation code for the ICCV2021 paper: Learnable Boundary Guided Adve

DV Lab 27 Sep 25, 2022
😮The official implementation of "CoNeRF: Controllable Neural Radiance Fields" 😮

CoNeRF: Controllable Neural Radiance Fields This is the official implementation for "CoNeRF: Controllable Neural Radiance Fields" Project Page Paper V

Kacper Kania 61 Dec 24, 2022
A scikit-learn compatible neural network library that wraps PyTorch

A scikit-learn compatible neural network library that wraps PyTorch. Resources Documentation Source Code Examples To see more elaborate examples, look

4.9k Jan 03, 2023
Use unsupervised and supervised learning to predict stocks

AIAlpha: Multilayer neural network architecture for stock return prediction This project is meant to be an advanced implementation of stacked neural n

Vivek Palaniappan 1.5k Jan 06, 2023
Implementation of Uniformer, a simple attention and 3d convolutional net that achieved SOTA in a number of video classification tasks

Uniformer - Pytorch Implementation of Uniformer, a simple attention and 3d convolutional net that achieved SOTA in a number of video classification ta

Phil Wang 90 Nov 24, 2022
Implementation of "StrengthNet: Deep Learning-based Emotion Strength Assessment for Emotional Speech Synthesis"

StrengthNet Implementation of "StrengthNet: Deep Learning-based Emotion Strength Assessment for Emotional Speech Synthesis" https://arxiv.org/abs/2110

RuiLiu 65 Dec 20, 2022
Object classification with basic computer vision techniques

naive-image-classification Object classification with basic computer vision techniques. Final assignment for the computer vision course I took at univ

2 Jul 01, 2022
Distributed Deep learning with Keras & Spark

Elephas: Distributed Deep Learning with Keras & Spark Elephas is an extension of Keras, which allows you to run distributed deep learning models at sc

Max Pumperla 1.6k Jan 05, 2023
Build and run Docker containers leveraging NVIDIA GPUs

NVIDIA Container Toolkit Introduction The NVIDIA Container Toolkit allows users to build and run GPU accelerated Docker containers. The toolkit includ

NVIDIA Corporation 15.6k Jan 01, 2023