On Anytime Learning At Macroscale

Learning from sequential data dumps

(key) Requirements

Python 3.7
Pytorch 1.9.0
Hydra 1.1.0 (pip install hydra-core & pip install hydra-submitit-launcher)

Structure

├── crlapi           
  ├── benchmark.py    # Creates the data stream, feeds it to the model and evaluates it
  ├── core.py         # Abstract classes for 
  ├── logger.py   
  ├── sl
    ├── architectures
      ├── ...         # NN architectures used in this project
    ├── clmodels
      ├── ...         # Models (e.g. Single, gEns, ..., )
    ├── streams
      ├── ...         # CIFAR and MNIST stream implementatins

Running Experiments

To run experiments, you need to call the dataset specific run file, and you need to pass the configuration of the run. We have place the configurations in the previous directory (../configs). The config structure is as follows

    ├── configs
        ├── mnist
           ├── run.py                 # run file
           ├── test_usage_gmoe.yaml   # This is the "gMoE" model
           ├── test_finetune_mlp.yaml # This is the "Single Model"
           ... 
        ├── cifar
           ├── run.py                 # run file
           ├── test_finetune_vgg.yaml # This is the "Single Model"
           ├── test_usage_gmoe.yaml   # This is the "gMoE" model
           ...

To run an e.g. mnist gMoE run, the command is (launched from the directory just above (so cd ..)

PYTHONPATH=./ python configs/mnist/run.py -cn test_usage_gmoe n_megabatches=2 replay=1 clmodel.max_epochs=200

Important arguments

n_megabatches : controls the number of megabatches. So n_megabatches=1 is your regular full dataset training
replay : whether to use replay or not
clmodel.init_from_scratch : whether to reinitialize the model at every MB. Should only be used when replay=1
device : use cuda or cpu depending on your hardware

License

alma is released under the MIT license. See LICENSE for additional details about it. See also our Terms of Use and Privacy Policy.

Anytime Learning At Macroscale

Related tags

Overview

On Anytime Learning At Macroscale

(key) Requirements

Structure

Running Experiments

Important arguments

License

Owner

Meta Research

A webpage that utilizes machine learning to extract sentiments from tweets.

Library of Stan Models for Survival Analysis

Automated machine learning: Review of the state-of-the-art and opportunities for healthcare

Python 3.6+ toolbox for submitting jobs to Slurm

A toolkit for geo ML data processing and model evaluation (fork of solaris)

scikit-multimodallearn is a Python package implementing algorithms multimodal data.

LILLIE: Information Extraction and Database Integration Using Linguistics and Learning-Based Algorithms

A GitHub action that suggests type annotations for Python using machine learning.

Data from "Datamodels: Predicting Predictions with Training Data"

Primitives for machine learning and data science.

CrayLabs and user contibuted examples of using SmartSim for various simulation and machine learning applications.

🎛 Distributed machine learning made simple.

🤖 ⚡ scikit-learn tips

Spark development environment for k8s

Neighbourhood Retrieval (Nearest Neighbours) with Distance Correlation.

Tangram makes it easy for programmers to train, deploy, and monitor machine learning models.

Tool for producing high quality forecasts for time series data that has multiple seasonality with linear or non-linear growth.

Built various Machine Learning algorithms (Logistic Regression, Random Forest, KNN, Gradient Boosting and XGBoost. etc)

A model to predict steering torque fully end-to-end

This repository contains the code to predict house price using Linear Regression Method