NCVX (NonConVeX): A User-Friendly and Scalable Package for Nonconvex Optimization in Machine Learning.

Last update: Aug 03, 2022

Overview

NCVX

NCVX: A User-Friendly and Scalable Package for Nonconvex Optimization in Machine Learning.

Please check https://ncvx.org for detailed instructions (introduction, installation, settings, examples...).

Brief Introduction

Optimizing nonconvex (NCVX) problems, especially those nonsmooth and constrained, is an essential part of machine learning and deep learning. But it is hard to reliably solve this type of problems without optimization expertise. Existing general-purpose NCVX optimization packages are powerful, but typically cannot handle nonsmoothness. GRANSO is among the first packages targeting NCVX, nonsmooth, constrained problems. However, it has several limitations such as the lack of auto-differentiation and GPU acceleration, which preclude the potential broad deployment by non-experts. To lower the technical barrier for the machine learning community, we revamp GRANSO into a user-friendly and scalable python package named NCVX, featuring auto-differentiation, GPU acceleration, tensor input, scalable QP solver, and zero dependency on proprietary packages. As a highlight, NCVX can solve general constrained deep learning problems, the first of its kind.

Update Logs

v1.1.1: Multiple examples added: unconstrained DL, feasibility problem, sphere manifold.

v1.1.0: L-BFGS Added.

v1.0.0: Initial release of NCVX. Main features: auto-differentiation, GPU acceleration, tensor input, scalable QP solver, and zero dependency on proprietary packages.

Acknowledgements

We would like to thank the GRANSO developers. This work was supported by UMII Seed Grant Program and NSF CMMI 2038403.

Citation

@article{liang2021ncvx,
    title={NCVX: A User-Friendly and Scalable Package for Nonconvex 
    Optimization in Machine Learning}, 
    author={Buyun Liang and Ju Sun},
    year={2021},
    eprint={2111.13984},
    archivePrefix={arXiv},
    primaryClass={cs.LG}
}

Contact

Codes written by Buyun Liang (https://buyunliang.org). Questions or bug reports please send email to Buyun Liang, [email protected].

Thanks to bug reporters:

NCVX (NonConVeX): A User-Friendly and Scalable Package for Nonconvex Optimization in Machine Learning.

Related tags

Overview

NCVX

Brief Introduction

Update Logs

Acknowledgements

Citation

Contact

You might also like...

A Python Automated Machine Learning tool that optimizes machine learning pipelines using genetic programming.

Python Extreme Learning Machine (ELM) is a machine learning technique used for classification/regression tasks.

Vowpal Wabbit is a machine learning system which pushes the frontier of machine learning with techniques

CD) in machine learning projectsImplementing continuous integration & delivery (CI/CD) in machine learning projects

ELI5 is a Python package which helps to debug machine learning classifiers and explain their predictions

Data science, Data manipulation and Machine learning package.

Python package for stacking (machine learning technique)

A Python Package to Tackle the Curse of Imbalanced Datasets in Machine Learning

Python package for machine learning for healthcare using a OMOP common data model

Releases(v1.1.1)

Owner

SUN Group @ UMN

BudouX is the successor to Budou, the machine learning powered line break organizer tool.

Flightfare-Prediction - It is a Flightfare Prediction Web Application Using Machine learning,Python and flask

Machine Learning approach for quantifying detector distortion fields

Combines Bayesian analyses from many datasets.

CorrProxies - Optimizing Machine Learning Inference Queries with Correlative Proxy Models

Fundamentals of Machine Learning

UpliftML: A Python Package for Scalable Uplift Modeling

Karate Club: An API Oriented Open-source Python Framework for Unsupervised Learning on Graphs (CIKM 2020)

Apache (Py)Spark type annotations (stub files).

🎛 Distributed machine learning made simple.

An open source framework that provides a simple, universal API for building distributed applications. Ray is packaged with RLlib, a scalable reinforcement learning library, and Tune, a scalable hyperparameter tuning library.

Machine-learning-dell - Repositório com as atividades desenvolvidas no curso de Machine Learning

Markov bot - A Writing bot based on Markov Chain for Data Structure Lab

Python package for causal inference using Bayesian structural time-series models.

Basic Docker Compose for Machine Learning Purposes

DeepSpeed is a deep learning optimization library that makes distributed training easy, efficient, and effective.

A repository of PyBullet utility functions for robotic motion planning, manipulation planning, and task and motion planning

ArviZ is a Python package for exploratory analysis of Bayesian models

Petastorm library enables single machine or distributed training and evaluation of deep learning models from datasets in Apache Parquet format. It supports ML frameworks such as Tensorflow, Pytorch, and PySpark and can be used from pure Python code.

Sequence learning toolkit for Python