ML Optimizers from scratch using JAX

Last update: Jul 29, 2022

Overview

ML Optimizers from scratch using JAX

Implementations of some popular optimizers from scratch for a simple model i.e., Linear Regression on a dataset of 5 features. The goal of this project was to understand how these optimizers work under the hood and try to do a toy implementation myself. I also use a bit of JAX magic to perform the differentiation of the loss function w.r.t to the weights and the bias without explicitly writing their derivatives as a separate function. This can help to generalize this notebook for other types of loss functions as well.

The optimizers I have implemented are -

Batch Gradient Descent
Batch Gradient Descent + Momentum
Nesterov Accelerated Momentum
Adagrad
RMSprop
Adam
Adamax
Nadam
Adabelief

References -

Owner

Shreyansh Singh

AI Engineer at @Mastercard. Ex-Intern at @Samsung. CS Grad, IIT (BHU) Varanasi. CTF player with @abs0lut3pwn4g3

GitHub Repository

Uses WiFi signals :signal_strength: and machine learning to predict where you are

Uses WiFi signals and machine learning (sklearn's RandomForest) to predict where you are. Even works for small distances like 2-10 meters.

5k Jan 09, 2023

Implementation of deep learning models for time series in PyTorch.

List of Implementations: Currently, the reimplementation of the DeepAR paper(DeepAR: Probabilistic Forecasting with Autoregressive Recurrent Networks

275 Dec 28, 2022

scikit-learn is a python module for machine learning built on top of numpy / scipy

About scikit-learn is a python module for machine learning built on top of numpy / scipy. The purpose of the scikit-learn-tutorial subproject is to le

122 Dec 12, 2022

A simple application that calculates the probability distribution of a normal distribution

probability-density-function General info An application that calculates the probability density and cumulative distribution of a normal distribution

1 Oct 25, 2022

GRaNDPapA: Generator of Rad Names from Decent Paper Acronyms

Generator of Rad Names from Decent Paper Acronyms

264 Nov 08, 2022

🔬 A curated list of awesome machine learning strategies & tools in financial market.

1.6k Dec 30, 2022

ThunderGBM: Fast GBDTs and Random Forests on GPUs

Documentations | Installation | Parameters | Python (scikit-learn) interface What's new? ThunderGBM won 2019 Best Paper Award from IEEE Transactions o

648 Dec 16, 2022

A comprehensive set of fairness metrics for datasets and machine learning models, explanations for these metrics, and algorithms to mitigate bias in datasets and models.

AI Fairness 360 (AIF360) The AI Fairness 360 toolkit is an extensible open-source library containg techniques developed by the research community to h

1.9k Jan 06, 2023

Metric learning algorithms in Python

metric-learn: Metric Learning in Python metric-learn contains efficient Python implementations of several popular supervised and weakly-supervised met

1.3k Dec 28, 2022

Library for machine learning stacking generalization.

stacked_generalization Implemented machine learning *stacking technic[1]* as handy library in Python. Feature weighted linear stacking is also availab

114 Jul 19, 2022

MLReef is an open source ML-Ops platform that helps you collaborate, reproduce and share your Machine Learning work with thousands of other users.

The collaboration platform for Machine Learning MLReef is an open source ML-Ops platform that helps you collaborate, reproduce and share your Machine

1.4k Dec 27, 2022

Price forecasting of SGB and IRFC Bonds and comparing there returns

Project_Bonds Project Title : Price forecasting of SGB and IRFC Bonds and comparing there returns. Introduction of the Project The 2008-09 global fina

1 Oct 28, 2021

Distributed Tensorflow, Keras and PyTorch on Apache Spark/Flink & Ray

A unified Data Analytics and AI platform for distributed TensorFlow, Keras and PyTorch on Apache Spark/Flink & Ray What is Analytics Zoo? Analytics Zo

2.5k Dec 28, 2022

MachineLearningStocks is designed to be an intuitive and highly extensible template project applying machine learning to making stock predictions.

Using python and scikit-learn to make stock predictions

1.3k Jan 03, 2023

ML Optimizers from scratch using JAX

Related tags

Overview

ML Optimizers from scratch using JAX

Owner

Shreyansh Singh

Uses WiFi signals :signal_strength: and machine learning to predict where you are

Implementation of deep learning models for time series in PyTorch.

scikit-learn is a python module for machine learning built on top of numpy / scipy

A simple application that calculates the probability distribution of a normal distribution

GRaNDPapA: Generator of Rad Names from Decent Paper Acronyms

🔬 A curated list of awesome machine learning strategies & tools in financial market.

ThunderGBM: Fast GBDTs and Random Forests on GPUs

A comprehensive set of fairness metrics for datasets and machine learning models, explanations for these metrics, and algorithms to mitigate bias in datasets and models.

Metric learning algorithms in Python

Library for machine learning stacking generalization.

MLReef is an open source ML-Ops platform that helps you collaborate, reproduce and share your Machine Learning work with thousands of other users.

Price forecasting of SGB and IRFC Bonds and comparing there returns

Distributed Tensorflow, Keras and PyTorch on Apache Spark/Flink & Ray

MachineLearningStocks is designed to be an intuitive and highly extensible template project applying machine learning to making stock predictions.

Temporal Alignment Prediction for Supervised Representation Learning and Few-Shot Sequence Classification

Short PhD seminar on Machine Learning Security (Adversarial Machine Learning)

Azure Cloud Advocates at Microsoft are pleased to offer a 12-week, 24-lesson curriculum all about Machine Learning

Code Repository for Machine Learning with PyTorch and Scikit-Learn

CrayLabs and user contibuted examples of using SmartSim for various simulation and machine learning applications.

customer churn prediction prevention in telecom industry using machine learning and survival analysis