A PyTorch Lightning Callback for pushing models to the Hugging Face Hub 🤗⚡️

Last update: Dec 14, 2022

Overview

hf-hub-lightning

A callback for pushing lightning models to the Hugging Face Hub.

Note: I made this package for myself, mostly...if folks seem to be interested in it, we'll move this into huggingface_hub or an official PyTorch Lightning repo directly and archive this one. Since there are a lot of options/considerations, I decided to just post it separately for now as a proof of concept.

Setup

pip install hf-hub-lightning

Usage

To periodically upload your model ckpt and TensorBoard logs while training, you can do the following...

import pytorch_lightning as pl
from hf_hub_lightning import HuggingFaceHubCallback

train_dataloader = ...
model = YourCoolModel()
trainer = pl.Trainer(callbacks=[HuggingFaceHubCallback('your_username/model_id')])
trainer.fit(model, train_dataloader)

To load your model back from the huggingface hub, just do...

from huggingface_hub import hf_hub_download

ckpt_path = hf_hub_download('your_username/model_id', 'lit_model.ckpt')
model = YourCoolModel.load_from_checkpoint(ckpt_path)

You might also like...

Code for HLA-Face: Joint High-Low Adaptation for Low Light Face Detection (CVPR21)

HLA-Face: Joint High-Low Adaptation for Low Light Face Detection The official PyTorch implementation for HLA-Face: Joint High-Low Adaptation for Low L

77 Dec 8, 2022

DVG-Face: Dual Variational Generation for Heterogeneous Face Recognition, TPAMI 2021

DVG-Face: Dual Variational Generation for HFR This repo is a PyTorch implementation of DVG-Face: Dual Variational Generation for Heterogeneous Face Re

52 Dec 30, 2022

[TIP 2021] SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction

SADRNet Paper link: SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction Requirements python

Multimedia Computing Group, Nanjing University

99 Dec 30, 2022

Realtime Face Anti Spoofing with Face Detector based on Deep Learning using Tensorflow/Keras and OpenCV

Realtime Face Anti-Spoofing Detection 🤖 Realtime Face Anti Spoofing Detection with Face Detector to detect real and fake faces Please star this repo

86 Aug 3, 2022

Swapping face using Face Mesh with TensorFlow Lite

17 Apr 26, 2022

Face Synthetics dataset is a collection of diverse synthetic face images with ground truth labels.

The Face Synthetics dataset Face Synthetics dataset is a collection of diverse synthetic face images with ground truth labels. It was introduced in ou

608 Jan 2, 2023

Face Library is an open source package for accurate and real-time face detection and recognition

Face Library Face Library is an open source package for accurate and real-time face detection and recognition. The package is built over OpenCV and us

52 Nov 9, 2022

VGGFace2-HQ - A high resolution face dataset for face editing purpose

The first open source high resolution dataset for face swapping!!! A high resolution version of VGGFace2 for academic face editing purpose

232 Dec 29, 2022

A large-scale face dataset for face parsing, recognition, generation and editing.

CelebAMask-HQ [Paper] [Demo] CelebAMask-HQ is a large-scale face image dataset that has 30,000 high-resolution face images selected from the CelebA da

1.7k Dec 26, 2022

Comments

Saving only the best models
Hi, just curious how it is interconnected with PL internal checkpointing callback, and saving only the best models according to some given monitoring criteria like max validation accuracy etc.?

# saves a file like: my/path/sample-mnist-epoch=02-val_loss=0.32.ckpt checkpoint_callback = ModelCheckpoint( monitor="val_loss", dirpath="my/path/", filename="sample-mnist-{epoch:02d}-{val_loss:.2f}", save_top_k=3, mode="min", ) trainer = Trainer(callbacks=[checkpoint_callback])

see: https://pytorch-lightning.readthedocs.io/en/latest/common/weights_loading.html

Thx :rabbit:
opened by Borda 1

Releases(v0.0.2)

v0.0.2(Oct 27, 2021)
fix logger updates so you can view updated TB logs as your model trains

fix args to the callback, adding missing ones I had hardcoded

Source code(tar.gz)
Source code(zip)
v0.0.1(Oct 27, 2021)

Initial Release!
Source code(tar.gz)
Source code(zip)

A PyTorch Lightning Callback for pushing models to the Hugging Face Hub 🤗⚡️

Related tags

Overview

hf-hub-lightning

Setup

Usage

You might also like...

Code for HLA-Face: Joint High-Low Adaptation for Low Light Face Detection (CVPR21)

DVG-Face: Dual Variational Generation for Heterogeneous Face Recognition, TPAMI 2021

[TIP 2021] SADRNet: Self-Aligned Dual Face Regression Networks for Robust 3D Dense Face Alignment and Reconstruction

Realtime Face Anti Spoofing with Face Detector based on Deep Learning using Tensorflow/Keras and OpenCV

Swapping face using Face Mesh with TensorFlow Lite

Face Synthetics dataset is a collection of diverse synthetic face images with ground truth labels.

Face Library is an open source package for accurate and real-time face detection and recognition

VGGFace2-HQ - A high resolution face dataset for face editing purpose

A large-scale face dataset for face parsing, recognition, generation and editing.

Comments

Saving only the best models

Releases(v0.0.2)

v0.0.2(Oct 27, 2021)

v0.0.1(Oct 27, 2021)

Owner

Nathan Raw

Second Order Optimization and Curvature Estimation with K-FAC in JAX.

PyTorch implementation of Algorithm 1 of "On the Anatomy of MCMC-Based Maximum Likelihood Learning of Energy-Based Models"

Repository relating to the CVPR21 paper TimeLens: Event-based Video Frame Interpolation

Deep Inside Convolutional Networks - This is a caffe implementation to visualize the learnt model

Improving Contrastive Learning by Visualizing Feature Transformation, ICCV 2021 Oral

Denoising images with Fourier Ring Correlation loss

Beginner-friendly repository for Hacktober Fest 2021. Start your contribution to open source through baby steps. 💜

Centroid-UNet is deep neural network model to detect centroids from satellite images.

AttentionGAN for Unpaired Image-to-Image Translation & Multi-Domain Image-to-Image Translation

PyTorch implementation of paper: HPNet: Deep Primitive Segmentation Using Hybrid Representations.

KeypointDeformer: Unsupervised 3D Keypoint Discovery for Shape Control

Contrastive unpaired image-to-image translation, faster and lighter training than cyclegan (ECCV 2020, in PyTorch)

Python and Julia in harmony.

On the Analysis of French Phonetic Idiosyncrasies for Accent Recognition

A PyTorch implementation of "From Two to One: A New Scene Text Recognizer with Visual Language Modeling Network" (ICCV2021)

Python implementation of O-OFDMNet, a deep learning-based optical OFDM system,

天勤量化开发包, 期货量化, 实时行情/历史数据/实盘交易

The repository contains source code and models to use PixelNet architecture used for various pixel-level tasks. More details can be accessed at .

Multi-query Video Retreival

Resources for the Ki testnet challenge