PyTorch 1.0 inference in C++ on Windows10 platforms

Last update: Oct 15, 2022

Related tags

Overview

Serving PyTorch Models in C++ on Windows10 platforms

How to use

Prepare Data

examples/data/train/

	- 0
	- 1
	  .
	  .
	  .
	- n

examples/data/test/

	- 0
	- 1
	  .
	  .
	  .
	- n

Train Model

cd examples && python train.py

Transform Model

cd examples && python transform_model.py

Test Model

cd makefile/pytorch
mkdir build && cd build && cmake -A x64 ..

or

mkdir build && cd build && cmake -G "Visual Studio 15 2017 Win64" ..

set Command Arguments -> ..\..\..\examples\checkpoint ..\..\..\examples\images
set Environment -> path=%path%;../../../thirdparty/libtorch/lib;../../../thirdparty/opencv/build/x64/vc15/bin;

Test CUDA Softmax

cd makefile/cuda
mkdir build && cd build && cmake -A x64 ..

or

mkdir build && cd build && cmake -G "Visual Studio 15 2017 Win64" ..

Inference onnx model

cd makefile/tensorRT/classification
mkdir build && cd build && cmake -A x64 ..

or

mkdir build && cd build && cmake -G "Visual Studio 15 2017 Win64" ..
set Environment -> path=%path%;../../../../thirdparty/TensorRT/lib;

Inference caffe model for faster-rcnn

cd makefile/tensorRT/detection
mkdir build && cd build && cmake -A x64 ..

or

mkdir build && cd build && cmake -G "Visual Studio 15 2017 Win64" ..
set Environment -> path=%path%;../../../../thirdparty/TensorRT/lib;

download VGG16_faster_rcnn_final.caffemodel

Thirdparty

thirdparty/
	- libtorch  
	- opencv 
	- CUDA
	- TensorRT

download thirdparty from here.

Docker

docker pull zccyman/deepframe
nvidia-docker run -it --name=mydocker zccyman/deepframe /bin/bash
cd workspace && git clone https://github.com/zccyman/pytorch-inference.git

Environment

Windows10
VS2017
CMake3.13
CUDA10.0
CUDNN7.3
Pyton3.5
ONNX1.1.2
TensorRT5.0.1
Pytorch1.0
Libtorch
OpenCV4.0.1

Todo List

train and transform pytorch model
multi-batch inference pytorch model in C++
cpu and gpu softmax
transform pytorch model to ONNX model, and inference onnx model using tensorRT
inference caffe model for faster-rcnn using tensorRT
build classification network
compress pytorch model
object detection pytorch inference using C++ on Window platforms

Notes

"torch.jit.trace" doesn’t support nn.DataParallel so far.
TensorRT doesn’t supports opset 7 above so far, but pyTorch ONNX exporter seems to export opset 9.

PyTorch 1.0 inference in C++ on Windows10 platforms

Related tags

Overview

Serving PyTorch Models in C++ on Windows10 platforms

How to use

Prepare Data

Train Model

Transform Model

Test Model

Test CUDA Softmax

Inference onnx model

Inference caffe model for faster-rcnn

Thirdparty

Docker

Environment

Todo List

Notes

Acknowledgement

Owner

Henson

bespoke tooling for offensive security's Windows Usermode Exploit Dev course (OSED)

Pytorch implementation of paper: "NeurMiPs: Neural Mixture of Planar Experts for View Synthesis"

Implementation of a Transformer using ReLA (Rectified Linear Attention)

style mixing for animation face

这是一个deeplabv3-plus-pytorch的源码，可以用于训练自己的模型。

PyTorch implementation of a collections of scalable Video Transformer Benchmarks.

A basic implementation of Layer-wise Relevance Propagation (LRP) in PyTorch.

PyTorch version of the paper 'Enhanced Deep Residual Networks for Single Image Super-Resolution' (CVPRW 2017)

Co-GAIL: Learning Diverse Strategies for Human-Robot Collaboration

OpenPose: Real-time multi-person keypoint detection library for body, face, hands, and foot estimation

ICCV2021, Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet

[arXiv'22] Panoptic NeRF: 3D-to-2D Label Transfer for Panoptic Urban Scene Segmentation

the code for our CVPR 2021 paper Bilateral Grid Learning for Stereo Matching Network [BGNet]

Official PyTorch implementation of DD3D: Is Pseudo-Lidar needed for Monocular 3D Object detection? (ICCV 2021), Dennis Park, Rares Ambrus, Vitor Guizilini, Jie Li, and Adrien Gaidon.

Source code for "Progressive Transformers for End-to-End Sign Language Production" (ECCV 2020)

Social Fabric: Tubelet Compositions for Video Relation Detection

Visualizing Yolov5's layers using GradCam

Configure SRX interfaces with Scrapli

Relative Human dataset, CVPR 2022

An updated version of virtual model making

PyTorch 1.0 inference in C++ on Windows10 platforms

Related tags

Overview

Serving PyTorch Models in C++ on Windows10 platforms

How to use

Prepare Data

Train Model

Transform Model

Test Model

Test CUDA Softmax

Inference onnx model

Inference caffe model for faster-rcnn

Thirdparty

Docker

Environment

Todo List

Notes

Acknowledgement

Owner

Henson

bespoke tooling for offensive security's Windows Usermode Exploit Dev course (OSED)

Pytorch implementation of paper: "NeurMiPs: Neural Mixture of Planar Experts for View Synthesis"

Implementation of a Transformer using ReLA (Rectified Linear Attention)

style mixing for animation face

这是一个deeplabv3-plus-pytorch的源码，可以用于训练自己的模型。

PyTorch implementation of a collections of scalable Video Transformer Benchmarks.

A basic implementation of Layer-wise Relevance Propagation (LRP) in PyTorch.

PyTorch version of the paper 'Enhanced Deep Residual Networks for Single Image Super-Resolution' (CVPRW 2017)

Co-GAIL: Learning Diverse Strategies for Human-Robot Collaboration

OpenPose: Real-time multi-person keypoint detection library for body, face, hands, and foot estimation

ICCV2021, Tokens-to-Token ViT: Training Vision Transformers from Scratch on ImageNet

[arXiv'22] Panoptic NeRF: 3D-to-2D Label Transfer for Panoptic Urban Scene Segmentation

the code for our CVPR 2021 paper Bilateral Grid Learning for Stereo Matching Network [BGNet]

Official PyTorch implementation of DD3D: Is Pseudo-Lidar needed for Monocular 3D Object detection? (ICCV 2021), Dennis Park*, Rares Ambrus*, Vitor Guizilini, Jie Li, and Adrien Gaidon.

Source code for "Progressive Transformers for End-to-End Sign Language Production" (ECCV 2020)

Social Fabric: Tubelet Compositions for Video Relation Detection

Visualizing Yolov5's layers using GradCam

Configure SRX interfaces with Scrapli

Relative Human dataset, CVPR 2022

An updated version of virtual model making

Official PyTorch implementation of DD3D: Is Pseudo-Lidar needed for Monocular 3D Object detection? (ICCV 2021), Dennis Park, Rares Ambrus, Vitor Guizilini, Jie Li, and Adrien Gaidon.