A lightweight library to compare different PyTorch implementations of the same network architecture.

Related tags

Deep LearningTorchBug
Overview

TorchBug is a lightweight library designed to compare two PyTorch implementations of the same network architecture. It allows you to count, and compare, the different leaf modules (i.e., lowest level PyTorch modules, such as torch.nn.Conv2d) present both in the target model and the new model. These leaf modules are distinguished based on their attributes, so that an instance of Conv2d with a kernel_size of 3 and stride of 1 is counted separately from a Conv2d with kernel_size of 3 but stride 2.

Further, when the leaf modules match, the library also provides you the functionality to initialize both the models equivalently, by initializing the leaf modules with weights using seeds which are obtained from the hash of their attributes. TorchBug then lets you pass the same input through both the models, and compare their outputs, or the outputs of intermediate leaf modules, to help find where the new model implementaion deviates from the target model.

Setup | Usage | Docs | Examples

Setup

To install, simply clone the repository, cd into the TorchBug folder, and run the following command:

pip install .

Usage

To get started, check out demo.py.

Docs

Docstrings can be found for all the functions. Refer compare.py and model_summary.py for the main functions.

Examples

Summary of a model

Each row in the tables indicates a specific module type, along with a combination of its attributes, as shown in the columns.

  • The second row in the second table indicates, for example, that there are two instances of Conv2d with 6 in_channels and 6 out_channels in the Target Model. Each of these modules has 330 parameters.

Summary of a model

Comparison of leaf modules

TorchBug lets you compare the leaf modules present in both models, and shows you the missing/extraneous modules present in either.

Comparison of leaf modules

Comparison of leaf modules invoked in the forward pass

The comparison of leaf modules invoked in forward pass ensures that the registered leaf modules are indeed consumed in the forward function of the models.

Comparison of leaf modules

Comparison of outputs of all leaf modules

After instantiating the Target and New models equivalently, and passing the same data through both of them, the outputs of intermediate leaf modules (of the same types and attributes) are compared (by brute force).

  • The second row in the first table indicates, for example, that there are two instances of Conv2d with 6 in_channels and 6 out_channels in both the models, and their outputs match.

Module-wise comparison of models

Comparison of outputs of specific leaf modules only

TorchBug lets you mark specific leaf modules in the models, with names, and shows you whether the outputs of these marked modules match.

Comparison of outputs of marked modules

  • In the above example, a convolution and two linear layers in the New Model were marked with names "Second Convolution", "First Linear Layer", and "Second Linear Layer".
  • A convolution in the Target Model was marked with name "Second Convolution".
  • All the other leaf modules in the Target Model were marked using a convenience function, which set the names to a string describing the module.
Owner
Arjun Krishnakumar
Research Assistant (HiWi) | Master's in Computer Science @ University of Freiburg
Arjun Krishnakumar
JugLab 33 Dec 30, 2022
My course projects for the 2021 Spring Machine Learning course at the National Taiwan University (NTU)

ML2021Spring There are my projects for the 2021 Spring Machine Learning course at the National Taiwan University (NTU) Course Web : https://speech.ee.

Ding-Li Chen 15 Aug 29, 2022
A scikit-learn-compatible module for estimating prediction intervals.

MAPIE - Model Agnostic Prediction Interval Estimator MAPIE allows you to easily estimate prediction intervals (or prediction sets) using your favourit

588 Jan 04, 2023
An open source Jetson Nano baseboard and tools to design your own.

My Jetson Nano Baseboard This basic baseboard gives the user the foundation and the flexibility to design their own baseboard for the Jetson Nano. It

NVIDIA AI IOT 57 Dec 29, 2022
[ECCV2020] Content-Consistent Matching for Domain Adaptive Semantic Segmentation

[ECCV20] Content-Consistent Matching for Domain Adaptive Semantic Segmentation This is a PyTorch implementation of CCM. News: GTA-4K list is available

Guangrui Li 88 Aug 25, 2022
MEAL V2: Boosting Vanilla ResNet-50 to 80%+ Top-1 Accuracy on ImageNet without Tricks

MEAL-V2 This is the official pytorch implementation of our paper: "MEAL V2: Boosting Vanilla ResNet-50 to 80%+ Top-1 Accuracy on ImageNet without Tric

Zhiqiang Shen 653 Dec 19, 2022
Detail-Preserving Transformer for Light Field Image Super-Resolution

DPT Official Pytorch implementation of the paper "Detail-Preserving Transformer for Light Field Image Super-Resolution" accepted by AAAI 2022 . Update

50 Jan 01, 2023
Code for Two-stage Identifier: "Locate and Label: A Two-stage Identifier for Nested Named Entity Recognition"

Code for Two-stage Identifier: "Locate and Label: A Two-stage Identifier for Nested Named Entity Recognition", accepted at ACL 2021. For details of the model and experiments, please see our paper.

tricktreat 87 Dec 16, 2022
Load What You Need: Smaller Multilingual Transformers for Pytorch and TensorFlow 2.0.

Smaller Multilingual Transformers This repository shares smaller versions of multilingual transformers that keep the same representations offered by t

Geotrend 79 Dec 28, 2022
Code and data accompanying our SVRHM'21 paper.

Code and data accompanying our SVRHM'21 paper. Requires tensorflow 1.13, python 3.7, scikit-learn, and pytorch 1.6.0 to be installed. Python scripts i

5 Nov 17, 2021
Distinguishing Commercial from Editorial Content in News

Distinguishing Commercial from Editorial Content in News In this repository you can find the following: An anonymized version of the data used for my

Timo Kats 3 Sep 26, 2022
A check for whether the dependency jobs are all green.

alls-green A check for whether the dependency jobs are all green. Why? Do you have more than one job in your GitHub Actions CI/CD workflows setup? Do

Re:actors 33 Jan 03, 2023
A module for solving and visualizing Schrödinger equation.

qmsolve This is an attempt at making a solid, easy to use solver, capable of solving and visualize the Schrödinger equation for multiple particles, an

506 Dec 28, 2022
DilatedNet in Keras for image segmentation

Keras implementation of DilatedNet for semantic segmentation A native Keras implementation of semantic segmentation according to Multi-Scale Context A

303 Mar 15, 2022
Code for "My(o) Armband Leaks Passwords: An EMG and IMU Based Keylogging Side-Channel Attack" paper

Myo Keylogging This is the source code for our paper My(o) Armband Leaks Passwords: An EMG and IMU Based Keylogging Side-Channel Attack by Matthias Ga

Secure Mobile Networking Lab 7 Jan 03, 2023
SeqAttack: a framework for adversarial attacks on token classification models

A framework for adversarial attacks against token classification models

Walter 23 Nov 25, 2022
A toolkit for developing and comparing reinforcement learning algorithms.

Status: Maintenance (expect bug fixes and minor updates) OpenAI Gym OpenAI Gym is a toolkit for developing and comparing reinforcement learning algori

OpenAI 29.6k Jan 08, 2023
Unsupervised Video Interpolation using Cycle Consistency

Unsupervised Video Interpolation using Cycle Consistency Project | Paper | YouTube Unsupervised Video Interpolation using Cycle Consistency Fitsum A.

NVIDIA Corporation 100 Nov 30, 2022
A code generator from ONNX to PyTorch code

onnx-pytorch Generating pytorch code from ONNX. Currently support onnx==1.9.0 and torch==1.8.1. Installation From PyPI pip install onnx-pytorch From

Wenhao Hu 94 Jan 06, 2023
Cowsay - A rewrite of cowsay in python

Python Cowsay A rewrite of cowsay in python. Allows for parsing of existing .cow

James Ansley 3 Jun 27, 2022