Repo for my Tensorflow/Keras CV experiments. Mostly revolving around the Danbooru20xx dataset

Last update: Dec 27, 2022

Related tags

Deep Learning SW-CV-ModelZoo

Overview

SW-CV-ModelZoo

Repo for my Tensorflow/Keras CV experiments. Mostly revolving around the Danbooru20xx dataset

Framework: TF/Keras 2.7

Training SQLite DB built using fire-egg's tools: https://github.com/fire-eggs/Danbooru2019

Currently training on Danbooru2021, 512px SFW subset (sans the rating:q images that had been included in the 2022-01-21 release of the dataset)

Reference:

Anonymous, The Danbooru Community, & Gwern Branwen; “Danbooru2021: A Large-Scale Crowdsourced and Tagged Anime Illustration Dataset”, 2022-01-21. Web. Accessed 2022-01-28 https://www.gwern.net/Danbooru2021

Journal

06/02/2022: great news crew! TRC allowed me to use a bunch of TPUs!

To make better use of this amount of compute I had to overhaul a number of components, so a bunch of things are likely to have fallen to bitrot in the process. I can only guarantee NFNet can work pretty much as before with the right arguments.
NFResNet changes should have left it retrocompatible with the previous version.
ResNet has been streamlined to be mostly in line with the Bag-of-Tricks paper (arXiv:1812.01187) with the exception of the stem. It is not compatible with the previous version of the code.

The training labels have been included in the 2021_0000_0899 folder for convenience.
The list of files used for training is going to be uploaded as a GitHub Release.

Now for some numbers:
compared to my previous best run, the one that resulted in NFNetL1V1-100-0.57141:

I'm using 1.86x the amount of images: 2.8M vs 1.5M
I'm training bigger models: 61M vs 45M params
... in less time: 232 vs 700 hours of processor time
don't get me started on actual wall clock time
with a few amenities thrown in: ECA for channel attention, SiLU activation

And it's all thanks to the folks at TRC, so shout out to them!

I currently have a few runs in progress across a couple of dimensions:

effect of model size with NFNet L0/L1/L2, with SiLU and ECA for all three of them
effect of activation function with NFNet L0, with SiLU/HSwish/ReLU, no ECA

Once the experiments are over, the plan is to select the network definitions that lay on the Pareto curve between throughput and F1 score and release the trained weights.

One last thing.
I'd like to call your attention to the tools/cleanlab_stuff.py script.
It reads two files: one with the binarized labels from the database, the other with the predicted probabilities.
It then uses the cleanlab package to estimate whether if an image in a set could be missing a given label. At the end it stores its conclusions in a json file.
This file could, potentially, be used in some tool to assist human intervention to add the missing tags.

You might also like...

Human head pose estimation using Keras over TensorFlow.

RealHePoNet: a robust single-stage ConvNet for head pose estimation in the wild.

71 Jan 5, 2023

Graph Neural Networks with Keras and Tensorflow 2.

Welcome to Spektral Spektral is a Python library for graph deep learning, based on the Keras API and TensorFlow 2. The main goal of this project is to

2.2k Jan 8, 2023

QKeras: a quantization deep learning library for Tensorflow Keras

QKeras github.com/google/qkeras QKeras 0.8 highlights: Automatic quantization using QKeras; Stochastic behavior (including stochastic rouding) is disa

437 Jan 3, 2023

Hyperparameter Optimization for TensorFlow, Keras and PyTorch

Hyperparameter Optimization for Keras Talos • Key Features • Examples • Install • Support • Docs • Issues • License • Download Talos radically changes

1.6k Dec 15, 2022

MMdnn is a set of tools to help users inter-operate among different deep learning frameworks. E.g. model conversion and visualization. Convert models between Caffe, Keras, MXNet, Tensorflow, CNTK, PyTorch Onnx and CoreML.

MMdnn MMdnn is a comprehensive and cross-framework tool to convert, visualize and diagnose deep learning (DL) models. The "MM" stands for model manage

5.7k Jan 9, 2023

Deep GPs built on top of TensorFlow/Keras and GPflow

GPflux Documentation | Tutorials | API reference | Slack What does GPflux do? GPflux is a toolbox dedicated to Deep Gaussian processes (DGP), the hier

107 Nov 2, 2022

tf2onnx - Convert TensorFlow, Keras and Tflite models to ONNX.

tf2onnx converts TensorFlow (tf-1.x or tf-2.x), tf.keras and tflite models to ONNX via command line or python api.

1.8k Jan 8, 2023

Build tensorflow keras model pipelines in a single line of code. Created by Ram Seshadri. Collaborators welcome. Permission granted upon request.

deep_autoviml Build keras pipelines and models in a single line of code! Table of Contents Motivation How it works Technology Install Usage API Image

102 Dec 17, 2022

Advanced Deep Learning with TensorFlow 2 and Keras (Updated for 2nd Edition)

1.5k Jan 3, 2023

Releases(models_db2021_5500_2022_10_21)

models_db2021_5500_2022_10_21(Oct 21, 2022)

ConvNext B, ViT B16
Trained on Danbooru2021 512px SFW subset, modulos 0000-0899
top 5500 tags (2021_0000_0899_5500/selected_tags.csv)
alpha to white
padding to make the image square is white
channel order is BGR, input is 0...255, scaled to -1...1 within the model

| run_name | definition_name | params_human | image_size | thres | F1 | |:---------------------------------|:------------------|:---------------|-------------:|--------:|-------:| | ConvNextBV1_09_25_2022_05h13m55s | B | 93.2M | 448 | 0.3673 | 0.6941 | | ViTB16_09_25_2022_04h53m38s | B16 | 90.5M | 448 | 0.3663 | 0.6918 |
Source code(tar.gz)
Source code(zip)
ConvNextBV1_09_25_2022_05h13m55s.7z(322.58 MB)
ViTB16_09_25_2022_04h53m38s.7z(312.96 MB)
convnexts_db2021_2022_03_22(Mar 22, 2022)

ConvNext, T/S/B
Trained on Danbooru2021 512px SFW subset, modulos 0000-0899
alpha to white
padding to make the image square is white
channel order is BGR, input is 0...255, scaled to -1...1 within the model

| run_name | definition_name | params_human | image_size | thres | F1 | |:---------------------------------|:------------------|:---------------|-------------:|--------:|-------:| | ConvNextBV1_03_10_2022_21h41m23s | B | 90.01M | 448 | 0.3372 | 0.6892 | | ConvNextSV1_03_11_2022_17h49m56s | S | 51.28M | 384 | 0.3301 | 0.6798 | | ConvNextTV1_03_05_2022_15h56m42s | T | 29.65M | 320 | 0.3259 | 0.6595 |
Source code(tar.gz)
Source code(zip)
ConvNextBV1_03_10_2022_21h41m23s.7z(311.29 MB)
ConvNextSV1_03_11_2022_17h49m56s.7z(177.36 MB)
ConvNextTV1_03_05_2022_15h56m42s.7z(102.96 MB)
nfnets_db2021_2022_03_04(Mar 4, 2022)

NFNet, L0/L1/L2 (based on timm Lx model definitions) Trained on Danbooru2021 512px SFW subset, modulos 0000-0899 alpha to white padding to make the image square is white channel order is BGR, input is 0...255, scaled to -1...1 within the model

| run_name | definition_name | params_human | image_size | thres | F1 | |:---------------------------------|:------------------|:---------------|-------------:|--------:|-------:| | NFNetL2V1_02_20_2022_10h27m08s | L2 | 60.96M | 448 | 0.3231 | 0.6785 | | NFNetL1V1_02_17_2022_20h18m38s | L1 | 45.65M | 384 | 0.3259 | 0.6691 | | NFNetL0V1_02_10_2022_17h50m14s | L0 | 27.32M | 320 | 0.3190 | 0.6509 |
Source code(tar.gz)
Source code(zip)
NFNetL0V1_02_10_2022_17h50m14s.7z(94.98 MB)
NFNetL1V1_02_17_2022_20h18m38s.7z(157.97 MB)
NFNetL2V1_02_20_2022_10h27m08s.7z(210.49 MB)
nfnet_tpu_training(Feb 6, 2022)

Attaching the lists of files currently in use for training and validation, along with the selected labels.

Training and validation files have been selected from the 512px SFW subset.
Source code(tar.gz)
Source code(zip)
2021_0000_0899.7z(1.32 MB)
NFNetL1V1-100-0.57141(Dec 31, 2021)
NFNet, L1 (based on timm Lx model definitions), 100 epochs, F1 @ 0.4 at the end of the 100th epoch was 0.57141

Trained on Danbooru2020 512px SFW subset, modulos 0000-0599

320px per side

alpha to white

padding to make the image square is white

channel order is BGR, scaled to 0-1

mixup alpha = 0.2 during epochs 76-100

analyze_metrics on Danbooru2020 original set, modulos 0984-0999: {'thres': 0.3485, 'F1': 0.6133, 'F2': 0.6133, 'MCC': 0.6094, 'A': 0.9923, 'R': 0.6133, 'P': 0.6133}

analyze_metrics on image IDs 4970000-5000000: {'thres': 0.3148, 'F1': 0.5942, 'F2': 0.5941, 'MCC': 0.5892, 'A': 0.9901, 'R': 0.5940, 'P': 0.5943}

Source code(tar.gz)
Source code(zip)
NFNetL1V1-100-0.57141.7z(158.09 MB)

Owner

GitHub Repository

[ICSE2020] MemLock: Memory Usage Guided Fuzzing

MemLock: Memory Usage Guided Fuzzing This repository provides the tool and the evaluation subjects for the paper "MemLock: Memory Usage Guided Fuzzing

54 Jan 07, 2023

Training Confidence-Calibrated Classifier for Detecting Out-of-Distribution Samples / ICLR 2018

Training Confidence-Calibrated Classifier for Detecting Out-of-Distribution Samples This project is for the paper "Training Confidence-Calibrated Clas

168 Nov 29, 2022

Minimal implementation of Denoised Smoothing: A Provable Defense for Pretrained Classifiers in TensorFlow.

Denoised-Smoothing-TF Minimal implementation of Denoised Smoothing: A Provable Defense for Pretrained Classifiers in TensorFlow. Denoised Smoothing is

19 Dec 11, 2022

Learning Versatile Neural Architectures by Propagating Network Codes

Learning Versatile Neural Architectures by Propagating Network Codes Mingyu Ding, Yuqi Huo, Haoyu Lu, Linjie Yang, Zhe Wang, Zhiwu Lu, Jingdong Wang,

36 Dec 06, 2022

More Photos are All You Need: Semi-Supervised Learning for Fine-Grained Sketch Based Image Retrieval

More Photos are All You Need: Semi-Supervised Learning for Fine-Grained Sketch Based Image Retrieval, CVPR 2021. Ayan Kumar Bhunia, Pinaki nath Chowdh

22 Aug 27, 2022

Code for Reciprocal Adversarial Learning for Brain Tumor Segmentation: A Solution to BraTS Challenge 2021 Segmentation Task

BRATS 2021 Solution For Segmentation Task This repo contains the supported pytorch code and configuration files to reproduce 3D medical image segmenta

6 Sep 15, 2022

An implementation of the AdaOPS (Adaptive Online Packing-based Search), which is an online POMDP Solver used to solve problems defined with the POMDPs.jl generative interface.

AdaOPS An implementation of the AdaOPS (Adaptive Online Packing-guided Search), which is an online POMDP Solver used to solve problems defined with th

9 Oct 05, 2022

Repo for my Tensorflow/Keras CV experiments. Mostly revolving around the Danbooru20xx dataset

Related tags

Overview

SW-CV-ModelZoo

Reference:

Journal

You might also like...

Human head pose estimation using Keras over TensorFlow.

Graph Neural Networks with Keras and Tensorflow 2.

QKeras: a quantization deep learning library for Tensorflow Keras

Hyperparameter Optimization for TensorFlow, Keras and PyTorch

MMdnn is a set of tools to help users inter-operate among different deep learning frameworks. E.g. model conversion and visualization. Convert models between Caffe, Keras, MXNet, Tensorflow, CNTK, PyTorch Onnx and CoreML.

Deep GPs built on top of TensorFlow/Keras and GPflow

tf2onnx - Convert TensorFlow, Keras and Tflite models to ONNX.

Build tensorflow keras model pipelines in a single line of code. Created by Ram Seshadri. Collaborators welcome. Permission granted upon request.

Advanced Deep Learning with TensorFlow 2 and Keras (Updated for 2nd Edition)

Releases(models_db2021_5500_2022_10_21)

models_db2021_5500_2022_10_21(Oct 21, 2022)

convnexts_db2021_2022_03_22(Mar 22, 2022)

nfnets_db2021_2022_03_04(Mar 4, 2022)

nfnet_tpu_training(Feb 6, 2022)

NFNetL1V1-100-0.57141(Dec 31, 2021)

Owner

[ICSE2020] MemLock: Memory Usage Guided Fuzzing

Training Confidence-Calibrated Classifier for Detecting Out-of-Distribution Samples / ICLR 2018

Minimal implementation of Denoised Smoothing: A Provable Defense for Pretrained Classifiers in TensorFlow.

Learning Versatile Neural Architectures by Propagating Network Codes

More Photos are All You Need: Semi-Supervised Learning for Fine-Grained Sketch Based Image Retrieval

Code for Reciprocal Adversarial Learning for Brain Tumor Segmentation: A Solution to BraTS Challenge 2021 Segmentation Task

An implementation of the AdaOPS (Adaptive Online Packing-based Search), which is an online POMDP Solver used to solve problems defined with the POMDPs.jl generative interface.

Simple image captioning model - CLIP prefix captioning.

The code repository for "PyCIL: A Python Toolbox for Class-Incremental Learning" in PyTorch.

A collection of IPython notebooks covering various topics.

[ICLR'19] Trellis Networks for Sequence Modeling

Python Rapid Artificial Intelligence Ab Initio Molecular Dynamics

Object Tracking and Detection Using OpenCV

Code for paper "Multi-level Disentanglement Graph Neural Network"

🚀 PyTorch Implementation of "Progressive Distillation for Fast Sampling of Diffusion Models(v-diffusion)"

Event queue (Equeue) dialect is an MLIR Dialect that models concurrent devices in terms of control and structure.

LSSY量化交易系统

Efficient neural networks for analog audio effect modeling

The Incredible PyTorch: a curated list of tutorials, papers, projects, communities and more relating to PyTorch.

Learning Features with Parameter-Free Layers (ICLR 2022)