Reverse engineer your pytorch vision models, in style

Related tags

Deep Learningrover
Overview

πŸ” Rover

Reverse engineer your CNNs, in style

Open In Colab

Rover will help you break down your CNN and visualize the features from within the model. No need to write weirdly abstract code to visualize your model's features anymore.

πŸ’» Usage

git clone https://github.com/Mayukhdeb/rover.git; cd rover

install requirements:

pip install -r requirements.txt
from rover import core
from rover.default_models import models_dict

core.run(models_dict = models_dict)

and then run the script with streamlit as:

$ streamlit run your_script.py

if everything goes right, you'll see something like:

You can now view your Streamlit app in your browser.

  Local URL: http://localhost:8501

πŸ§™ Custom models

rover supports pretty much any PyTorch model with an input of shape [N, 3, H, W] (even segmentation models/VAEs and all that fancy stuff) with imagenet normalization on input.

import torchvision.models as models 
model = models.resnet34(pretrained= True)  ## or any other model (need not be from torchvision.models)

models_dict = {
    'my model': model,  ## add in any number of models :)
}

core.run(
    models_dict = models_dict
)

πŸ–ΌοΈ Channel objective

Optimizes a single channel from one of the layer(s) selected.

  • layer index: specifies which layer you want to use out of the layers selected.
  • channel index: specifies the exact channel which needs to be visualized.

πŸ§™β€β™‚οΈ Writing your own objective

This is for the smarties who like to write their own objective function. The only constraint is that the function should be named custom_func.

Here's an example:

def custom_func(layer_outputs):
    '''
    layer_outputs is a list containing 
    the outputs (torch.tensor) of each layer you selected

    In this example we'll try to optimize the following:
    * the entire first layer -> layer_outputs[0].mean()
    * 20th channel of the 2nd layer -> layer_outputs[1][20].mean()
    '''
    loss = layer_outputs[0].mean() + layer_outputs[1][20].mean()
    return -loss

Running on google colab

Check out this notebook. I'll also include the instructions here just in case.

Clone the repo + install dependencies

!git clone https://github.com/Mayukhdeb/rover.git
!pip install torch-dreams --quiet
!pip install streamlit --quiet

Navigate into the repo

import os 
os.chdir('rover')

Write your file into a script from a cell. Here I wrote it into test.py

%%writefile  test.py

from rover import core
from rover.default_models import models_dict

core.run(models_dict = models_dict)

Run script on a thread

import threading

proc = threading.Thread(target= os.system, args=['streamlit run test.py'])
proc.start()

Download ngrok:

!wget https://bin.equinox.io/c/4VmDzA7iaHb/ngrok-stable-linux-amd64.zip
!unzip -o ngrok-stable-linux-amd64.zi

More ngrok stuff

get_ipython().system_raw('./ngrok http 8501 &')

Get your URL where rover is hosted

!curl -s http://localhost:4040/api/tunnels | python3 -c \
    "import sys, json; print(json.load(sys.stdin)['tunnels'][0]['public_url'])"

πŸ’» Args

  • width (int, optional): Width of image to be optimized
  • height (int, optional): Height of image to be optimized
  • iters (int, optional): Number of iterations, higher -> stronger visualization
  • lr (float, optional): Learning rate
  • rotate (deg) (int, optional): Max rotation in default transforms
  • scale max (float, optional): Max image size factor.
  • scale min (float, optional): Minimum image size factor.
  • translate (x) (float, optional): Maximum translation factor in x direction
  • translate (y) (float, optional): Maximum translation factor in y direction
  • weight decay (float, optional): Weight decay for default optimizer. Helps prevent high frequency noise.
  • gradient clip (float, optional): Maximum value of the norm of gradient.

Run locally

Clone the repo

git clone https://github.com/Mayukhdeb/rover.git

install requirements

pip install -r requirements.txt

showtime

streamlit run test.py
Owner
Mayukh Deb
Learning about life, one epoch at a time
Mayukh Deb
Official PyTorch implementation for Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers, a novel method to visualize any Transformer-based network. Including examples for DETR, VQA.

PyTorch Implementation of Generic Attention-model Explainability for Interpreting Bi-Modal and Encoder-Decoder Transformers 1 Using Colab Please notic

Hila Chefer 489 Jan 07, 2023
A universal memory dumper using Frida

Fridump Fridump (v0.1) is an open source memory dumping tool, primarily aimed to penetration testers and developers. Fridump is using the Frida framew

551 Jan 07, 2023
Improving Compound Activity Classification via Deep Transfer and Representation Learning

Improving Compound Activity Classification via Deep Transfer and Representation Learning This repository is the official implementation of Improving C

NingLab 2 Nov 24, 2021
[3DV 2021] A Dataset-Dispersion Perspective on Reconstruction Versus Recognition in Single-View 3D Reconstruction Networks

dispersion-score Official implementation of 3DV 2021 Paper A Dataset-dispersion Perspective on Reconstruction versus Recognition in Single-view 3D Rec

Yefan 7 May 28, 2022
COLMAP - Structure-from-Motion and Multi-View Stereo

COLMAP About COLMAP is a general-purpose Structure-from-Motion (SfM) and Multi-View Stereo (MVS) pipeline with a graphical and command-line interface.

4.7k Jan 07, 2023
Yolov5 + Deep Sort with PyTorch

λ”₯μ†ŒνŠΈ μˆ˜μ •μ€‘ Yolov5 + Deep Sort with PyTorch Introduction This repository contains a two-stage-tracker. The detections generated by YOLOv5, a family of obj

1 Nov 26, 2021
A large-scale database for graph representation learning

A large-scale database for graph representation learning

Scott Freitas 29 Nov 25, 2022
A script written in Python that returns a consensus string and profile matrix of a given DNA string(s) in FASTA format.

A script written in Python that returns a consensus string and profile matrix of a given DNA string(s) in FASTA format.

Zain 1 Feb 01, 2022
Source code for the paper "SEPP: Similarity Estimation of Predicted Probabilities for Defending and Detecting Adversarial Text" PACLIC 2021

Adversarial text generator Refer to "adversarial_text_generator"[https://github.com/quocnsh/SEPP_generator] project for generating adversarial texts A

0 Oct 05, 2021
Pyramid Grafting Network for One-Stage High Resolution Saliency Detection. CVPR 2022

PGNet Pyramid Grafting Network for One-Stage High Resolution Saliency Detection. CVPR 2022, CVPR 2022 (arXiv 2204.05041) Abstract Recent salient objec

CVTEAM 109 Dec 05, 2022
NVIDIA Deep Learning Examples for Tensor Cores

NVIDIA Deep Learning Examples for Tensor Cores Introduction This repository provides State-of-the-Art Deep Learning examples that are easy to train an

NVIDIA Corporation 10k Dec 31, 2022
Global-Local Attention for Emotion Recognition

Global-Local Attention for Emotion Recognition Requirements Python 3 Install tensorflow (or tensorflow-gpu) = 2.0.0 Install some other packages pip i

Minh Nhat Le 15 Apr 21, 2022
Automatic Differentiation Multipole Moment Molecular Forcefield

Automatic Differentiation Multipole Moment Molecular Forcefield Performance notes On a single gpu, using waterbox_31ang.pdb example from MPIDplugin wh

4 Jan 07, 2022
Predictive AI layer for existing databases.

MindsDB is an open-source AI layer for existing databases that allows you to effortlessly develop, train and deploy state-of-the-art machine learning

MindsDB Inc 12.2k Jan 03, 2023
Diabet Feature Engineering - Predict whether people have diabetes when their characteristics are specified

Diabet Feature Engineering - Predict whether people have diabetes when their characteristics are specified

Şebnem 6 Jan 18, 2022
Mesh TensorFlow: Model Parallelism Made Easier

Mesh TensorFlow - Model Parallelism Made Easier Introduction Mesh TensorFlow (mtf) is a language for distributed deep learning, capable of specifying

1.3k Dec 26, 2022
A simple, unofficial implementation of MAE using pytorch-lightning

Masked Autoencoders in PyTorch A simple, unofficial implementation of MAE (Masked Autoencoders are Scalable Vision Learners) using pytorch-lightning.

Connor Anderson 20 Dec 03, 2022
Back to the Feature: Learning Robust Camera Localization from Pixels to Pose (CVPR 2021)

Back to the Feature with PixLoc We introduce PixLoc, a neural network for end-to-end learning of camera localization from an image and a 3D model via

Computer Vision and Geometry Lab 610 Jan 05, 2023
Official PyTorch code of DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context Graph and Relation-based Optimization (ICCV 2021 Oral).

DeepPanoContext (DPC) [Project Page (with interactive results)][Paper] DeepPanoContext: Panoramic 3D Scene Understanding with Holistic Scene Context G

Cheng Zhang 66 Nov 16, 2022
Accommodating supervised learning algorithms for the historical prices of the world's favorite cryptocurrency and boosting it through LightGBM.

Accommodating supervised learning algorithms for the historical prices of the world's favorite cryptocurrency and boosting it through LightGBM.

1 Nov 27, 2021