On Anytime Learning At Macroscale

Learning from sequential data dumps

(key) Requirements

Python 3.7
Pytorch 1.9.0
Hydra 1.1.0 (pip install hydra-core & pip install hydra-submitit-launcher)

Structure

├── crlapi           
  ├── benchmark.py    # Creates the data stream, feeds it to the model and evaluates it
  ├── core.py         # Abstract classes for 
  ├── logger.py   
  ├── sl
    ├── architectures
      ├── ...         # NN architectures used in this project
    ├── clmodels
      ├── ...         # Models (e.g. Single, gEns, ..., )
    ├── streams
      ├── ...         # CIFAR and MNIST stream implementatins

Running Experiments

To run experiments, you need to call the dataset specific run file, and you need to pass the configuration of the run. We have place the configurations in the previous directory (../configs). The config structure is as follows

    ├── configs
        ├── mnist
           ├── run.py                 # run file
           ├── test_usage_gmoe.yaml   # This is the "gMoE" model
           ├── test_finetune_mlp.yaml # This is the "Single Model"
           ... 
        ├── cifar
           ├── run.py                 # run file
           ├── test_finetune_vgg.yaml # This is the "Single Model"
           ├── test_usage_gmoe.yaml   # This is the "gMoE" model
           ...

To run an e.g. mnist gMoE run, the command is (launched from the directory just above (so cd ..)

PYTHONPATH=./ python configs/mnist/run.py -cn test_usage_gmoe n_megabatches=2 replay=1 clmodel.max_epochs=200

Important arguments

n_megabatches : controls the number of megabatches. So n_megabatches=1 is your regular full dataset training
replay : whether to use replay or not
clmodel.init_from_scratch : whether to reinitialize the model at every MB. Should only be used when replay=1
device : use cuda or cpu depending on your hardware

License

alma is released under the MIT license. See LICENSE for additional details about it. See also our Terms of Use and Privacy Policy.

Anytime Learning At Macroscale

Related tags

Overview

On Anytime Learning At Macroscale

(key) Requirements

Structure

Running Experiments

Important arguments

License

Owner

Meta Research

Backprop makes it simple to use, finetune, and deploy state-of-the-art ML models.

Implementation of deep learning models for time series in PyTorch.

A flexible CTF contest platform for coming PKU GeekGame events

Data from "Datamodels: Predicting Predictions with Training Data"

LightGBM + Optuna: no brainer

A simple python program which predicts the success of a movie based on it's type, actor, actress and director

Python library for multilinear algebra and tensor factorizations

Machine-care - A simple python script to take care of simple maintenance tasks

MLR - Machine Learning Research

AutoTabular automates machine learning tasks enabling you to easily achieve strong predictive performance in your applications.

Turns your machine learning code into microservices with web API, interactive GUI, and more.

Covid-polygraph - a set of Machine Learning-driven fact-checking tools

Class-imbalanced / Long-tailed ensemble learning in Python. Modular, flexible, and extensible

Machine learning template for projects based on sklearn library.

A GitHub action that suggests type annotations for Python using machine learning.

Neural Machine Translation (NMT) tutorial with OpenNMT-py

Dieses Projekt ermöglicht es den Smartmeter der EVN (Netz Niederösterreich) über die Kundenschnittstelle auszulesen.

Customers Segmentation with RFM Scores and K-means

This is a Machine Learning model which predicts the presence of Diabetes in Patients

Spark development environment for k8s