A library for optimization on Riemannian manifolds

Overview

TensorFlow RiemOpt

PyPI arXiv Build Status Coverage Status Code style: black License

A library for manifold-constrained optimization in TensorFlow.

Installation

To install the latest development version from GitHub:

pip install git+https://github.com/master/tensorflow-riemopt.git

To install a package from PyPI:

pip install tensorflow-riemopt

Features

The core package implements concepts in differential geometry, such as manifolds and Riemannian metrics with associated exponential and logarithmic maps, geodesics, retractions, and transports. For manifolds, where closed-form expressions are not available, the library provides numerical approximations.

import tensorflow_riemopt as riemopt

S = riemopt.manifolds.Sphere()

x = S.projx(tf.constant([0.1, -0.1, 0.1]))
u = S.proju(x, tf.constant([1., 1., 1.]))
v = S.proju(x, tf.constant([-0.7, -1.4, 1.4]))

y = S.exp(x, v)

u_ = S.transp(x, y, u)
v_ = S.transp(x, y, v)

Manifolds

  • manifolds.Cholesky - manifold of lower triangular matrices with positive diagonal elements
  • manifolds.Euclidian - unconstrained manifold with the Euclidean metric
  • manifolds.Grassmannian - manifold of p-dimensional linear subspaces of the n-dimensional space
  • manifolds.Hyperboloid - manifold of n-dimensional hyperbolic space embedded in the n+1-dimensional Minkowski space
  • manifolds.Poincare - the Poincaré ball model of the hyperbolic space
  • manifolds.Product - Cartesian product of manifolds
  • manifolds.SPDAffineInvariant - manifold of symmetric positive definite (SPD) matrices endowed with the affine-invariant metric
  • manifolds.SPDLogCholesky - SPD manifold with the Log-Cholesky metric
  • manifolds.SPDLogEuclidean - SPD manifold with the Log-Euclidean metric
  • manifolds.SpecialOrthogonal - manifold of rotation matrices
  • manifolds.Sphere - manifold of unit-normalized points
  • manifolds.StiefelEuclidean - manifold of orthonormal p-frames in the n-dimensional space endowed with the Euclidean metric
  • manifolds.StiefelCanonical - Stiefel manifold with the canonical metric
  • manifolds.StiefelCayley - Stiefel manifold the retraction map via an iterative Cayley transform

Optimizers

Constrained optimization algorithms work as drop-in replacements for Keras optimizers for sparse and dense updates in both Eager and Graph modes.

  • optimizers.RiemannianSGD - Riemannian Gradient Descent
  • optimizers.RiemannianAdam - Riemannian Adam and AMSGrad
  • optimizers.ConstrainedRMSProp - Constrained RMSProp

Layers

  • layers.ManifoldEmbedding - constrained keras.layers.Embedding layer

Examples

  • SPDNet - Huang, Zhiwu, and Luc Van Gool. "A Riemannian network for SPD matrix learning." Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence. AAAI Press, 2017.
  • LieNet - Huang, Zhiwu, et al. "Deep learning on Lie groups for skeleton-based action recognition." Proceedings of the IEEE conference on computer vision and pattern recognition. 2017.
  • GrNet - Huang, Zhiwu, Jiqing Wu, and Luc Van Gool. "Building Deep Networks on Grassmann Manifolds." AAAI. AAAI Press, 2018.
  • Hyperbolic Neural Network - Ganea, Octavian, Gary Bécigneul, and Thomas Hofmann. "Hyperbolic neural networks." Advances in neural information processing systems. 2018.
  • Poincaré GloVe - Tifrea, Alexandru, Gary Becigneul, and Octavian-Eugen Ganea. "Poincaré Glove: Hyperbolic Word Embeddings." International Conference on Learning Representations. 2018.

References

If you find TensorFlow RiemOpt useful in your research, please cite:

@misc{smirnov2021tensorflow,
      title={TensorFlow RiemOpt: a library for optimization on Riemannian manifolds},
      author={Oleg Smirnov},
      year={2021},
      eprint={2105.13921},
      archivePrefix={arXiv},
      primaryClass={cs.MS}
}

Acknowledgment

TensorFlow RiemOpt was inspired by many similar projects:

  • Manopt, a matlab toolbox for optimization on manifolds
  • Pymanopt, a Python toolbox for optimization on manifolds
  • Geoopt: Riemannian Optimization in PyTorch
  • Geomstats, an open-source Python package for computations and statistics on nonlinear manifolds

License

The code is MIT-licensed.

You might also like...
Distributed Asynchronous Hyperparameter Optimization better than HyperOpt.
Distributed Asynchronous Hyperparameter Optimization better than HyperOpt.

UltraOpt : Distributed Asynchronous Hyperparameter Optimization better than HyperOpt. UltraOpt is a simple and efficient library to minimize expensive

Official code for paper "Optimization for Oriented Object Detection via Representation Invariance Loss".

Optimization for Oriented Object Detection via Representation Invariance Loss By Qi Ming, Zhiqiang Zhou, Lingjuan Miao, Xue Yang, and Yunpeng Dong. Th

Keras + Hyperopt: A very simple wrapper for convenient hyperparameter optimization

This project is now archived. It's been fun working on it, but it's time for me to move on. Thank you for all the support and feedback over the last c

Bayesian optimization in PyTorch

BoTorch is a library for Bayesian Optimization built on PyTorch. BoTorch is currently in beta and under active development! Why BoTorch ? BoTorch Prov

optimization routines for hyperparameter tuning
optimization routines for hyperparameter tuning

Optunity is a library containing various optimizers for hyperparameter tuning. Hyperparameter tuning is a recurrent problem in many machine learning t

Distributed Asynchronous Hyperparameter Optimization in Python

Hyperopt: Distributed Hyperparameter Optimization Hyperopt is a Python library for serial and parallel optimization over awkward search spaces, which

Hyper-parameter optimization for sklearn

hyperopt-sklearn Hyperopt-sklearn is Hyperopt-based model selection among machine learning algorithms in scikit-learn. See how to use hyperopt-sklearn

A Python implementation of global optimization with gaussian processes.
A Python implementation of global optimization with gaussian processes.

Bayesian Optimization Pure Python implementation of bayesian global optimization with gaussian processes. PyPI (pip): $ pip install bayesian-optimizat

Safe Bayesian Optimization
Safe Bayesian Optimization

SafeOpt - Safe Bayesian Optimization This code implements an adapted version of the safe, Bayesian optimization algorithm, SafeOpt [1], [2]. It also p

Comments
  • Projection on SPDs is not projecting onto SPDs

    Projection on SPDs is not projecting onto SPDs

    Hi, nice to see another package doing optimizationon manifolds! I have not yet had the time to check this versus what pymanopt is doing (I think they use tensor flow as a backend, too?) But I just noticed that

    https://github.com/master/tensorflow-manopt/blob/93402f6770d5b3c45f232340fddfa92a7126f19a/tensorflow_manopt/manifolds/symmetric_positive.py#L37-L41

    This might be wrong. For SPDs, the characteristic property is, that all eigenvalues are positive, so this projection is not projection onto the manifold (of SPDs) but onto the set of positive semidefinite matrices. There is no projection onto the SPDs since that set is open in the set of (symmetric) matrices.

    opened by kellertuer 2
  • GrNet produces NaN entries in input tensor

    GrNet produces NaN entries in input tensor

    Hi! First of all, really appreciate you guys taking the time to build a much required riemmannian geometry based package in tensorflow. It is proving to be quite useful for me. However, I recently ran the [GrNet code] (https://github.com/master/tensorflow-riemopt/tree/master/examples/grnet) with the AFEW dataset(the default dataset used in the code) on my machine and it seems at some point the input tensors get filled with NaN values. I tried tinkering with the learning rate and a few other usual things that could determine the cause of such NaN value in a dl model but it seems to be of no use. Any idea as to why this might be the case- is the code still been checked for bugs or am I missing something? Thanks in advance!

    opened by SouvikBan 2
Releases(v0.1.1)
Owner
Oleg Smirnov
Oleg Smirnov
A multi-mode modulator for multi-domain few-shot classification (ICCV)

A multi-mode modulator for multi-domain few-shot classification (ICCV)

Yanbin Liu 8 Apr 28, 2022
Long Expressive Memory (LEM)

Long Expressive Memory for Sequence Modeling This repository contains the implementation to reproduce the numerical experiments of the paper Long Expr

Konstantin Rusch 47 Dec 17, 2022
Implementation of paper "Decision-based Black-box Attack Against Vision Transformers via Patch-wise Adversarial Removal"

Patch-wise Adversarial Removal Implementation of paper "Decision-based Black-box Attack Against Vision Transformers via Patch-wise Adversarial Removal

4 Oct 12, 2022
Pseudo-mask Matters in Weakly-supervised Semantic Segmentation

Pseudo-mask Matters in Weakly-supervised Semantic Segmentation By Yi Li, Zhanghui Kuang, Liyang Liu, Yimin Chen, Wayne Zhang SenseTime, Tsinghua Unive

33 Oct 14, 2022
A Closer Look at Reference Learning for Fourier Phase Retrieval

A Closer Look at Reference Learning for Fourier Phase Retrieval This repository contains code for our NeurIPS 2021 Workshop on Deep Learning and Inver

Tobias Uelwer 1 Oct 28, 2021
Roger Labbe 13k Dec 29, 2022
Auto grind btdb2 exp for tower

Bloons TD Battles 2 EXP Grinder Auto grind btdb2 exp for towers Setup I suggest checking out every screenshot to see what they are supposed to be, so

Vincent 6 Jul 29, 2022
An Easy-to-use, Modular and Prolongable package of deep-learning based Named Entity Recognition Models.

DeepNER An Easy-to-use, Modular and Prolongable package of deep-learning based Named Entity Recognition Models. This repository contains complex Deep

Derrick 9 May 30, 2022
Code release for General Greedy De-bias Learning

General Greedy De-bias for Dataset Biases This is an extention of "Greedy Gradient Ensemble for Robust Visual Question Answering" (ICCV 2021, Oral). T

4 Mar 15, 2022
A tool for making map images from OpenTTD save games

OpenTTD Surveyor A tool for making map images from OpenTTD save games. This is not part of the main OpenTTD codebase, nor is it ever intended to be pa

Aidan Randle-Conde 9 Feb 15, 2022
Fast Neural Style for Image Style Transform by Pytorch

FastNeuralStyle by Pytorch Fast Neural Style for Image Style Transform by Pytorch This is famous Fast Neural Style of Paper Perceptual Losses for Real

Bengxy 81 Sep 03, 2022
UDP++ (ECCVW 2020 Oral), (Winner of COCO 2020 Keypoint Challenge).

UDP-Pose This is the pytorch implementation for UDP++, which won the Fisrt place in COCO Keypoint Challenge at ECCV 2020 Workshop. Top-Down Results on

20 Jul 29, 2022
Y. Zhang, Q. Yao, W. Dai, L. Chen. AutoSF: Searching Scoring Functions for Knowledge Graph Embedding. IEEE International Conference on Data Engineering (ICDE). 2020

AutoSF The code for our paper "AutoSF: Searching Scoring Functions for Knowledge Graph Embedding" and this paper has been accepted by ICDE2020. News:

AutoML Research 64 Dec 17, 2022
Pytorch implementation for DFN: Distributed Feedback Network for Single-Image Deraining.

DFN:Distributed Feedback Network for Single-Image Deraining Abstract Recently, deep convolutional neural networks have achieved great success for sing

6 Nov 05, 2022
SkipGNN: Predicting Molecular Interactions with Skip-Graph Networks (Scientific Reports)

SkipGNN: Predicting Molecular Interactions with Skip-Graph Networks Molecular interaction networks are powerful resources for the discovery. While dee

Kexin Huang 49 Oct 15, 2022
Repository for "Improving evidential deep learning via multi-task learning," published in AAAI2022

Improving evidential deep learning via multi task learning It is a repository of AAAI2022 paper, “Improving evidential deep learning via multi-task le

deargen 11 Nov 19, 2022
3D Pose Estimation for Vehicles

3D Pose Estimation for Vehicles Introduction This work generates 4 key-points and 2 key-edges from vertices and edges of vehicles as ground truth. The

Jingyi Wang 1 Nov 01, 2021
Pytorch-diffusion - A basic PyTorch implementation of 'Denoising Diffusion Probabilistic Models'

PyTorch implementation of 'Denoising Diffusion Probabilistic Models' This reposi

Arthur Juliani 76 Jan 07, 2023
The personal repository of the work: *DanceNet3D: Music Based Dance Generation with Parametric Motion Transformer*.

DanceNet3D The personal repository of the work: DanceNet3D: Music Based Dance Generation with Parametric Motion Transformer. Dataset and Results Pleas

南嘉Nanga 36 Dec 21, 2022
A tiny, friendly, strong baseline code for Person-reID (based on pytorch).

Pytorch ReID Strong, Small, Friendly A tiny, friendly, strong baseline code for Person-reID (based on pytorch). Strong. It is consistent with the new

Zhedong Zheng 3.5k Jan 08, 2023