HINet: Half Instance Normalization Network for Image Restoration

Related tags

Deep LearningHINet
Overview

PWC PWC PWC PWC PWC PWC PWC

HINet: Half Instance Normalization Network for Image Restoration

Liangyu Chen, Xin Lu, Jie Zhang, Xiaojie Chu, Chengpeng Chen

Paper: https://arxiv.org/abs/2105.06086

In this paper, we explore the role of Instance Normalization in low-level vision tasks. Specifically, we present a novel block: Half Instance Normalization Block (HIN Block), to boost the performance of image restoration networks. Based on HIN Block, we design a simple and powerful multi-stage network named HINet, which consists of two subnetworks. With the help of HIN Block, HINet surpasses the state-of-the-art (SOTA) on various image restoration tasks. For image denoising, we exceed it 0.11dB and 0.28 dB in PSNR on SIDD dataset, with only 7.5% and 30% of its multiplier-accumulator operations (MACs), 6.8 times and 2.9 times speedup respectively. For image deblurring, we get comparable performance with 22.5% of its MACs and 3.3 times speedup on REDS and GoPro datasets. For image deraining, we exceed it by 0.3 dB in PSNR on the average result of multiple datasets with 1.4 times speedup. With HINet, we won 1st place on the NTIRE 2021 Image Deblurring Challenge - Track2. JPEG Artifacts, with a PSNR of 29.70.

Network Architecture

arch

Installation

This implementation based on BasicSR which is a open source toolbox for image/video restoration tasks.

python 3.6.9
pytorch 1.5.1
cuda 10.1
git clone https://github.com/megvii-model/HINet
cd HINet
pip install -r requirements.txt
python setup.py develop --no_cuda_ext

Image Restoration Tasks

Image denoise, deblur, derain.

Image Denoise - SIDD dataset (Coming soon)
Image Deblur - GoPro dataset (Click to expand)

  • prepare data

    • mkdir ./datasets/GoPro

    • download the train set in ./datasets/GoPro/train and test set in ./datasets/GoPro/test (refer to MPRNet)

    • it should be like:

      ./datasets/
./datasets/GoPro/
./datasets/GoPro/train/
./datasets/GoPro/train/input/
./datasets/GoPro/train/target/
./datasets/GoPro/test/
./datasets/GoPro/test/input/
./datasets/GoPro/test/target/

    • python scripts/data_preparation/gopro.py

      • crop the train image pairs to 512x512 patches.

  • eval

    • download pretrained model to ./experiments/pretrained_models/HINet-GoPro.pth
    • python basicsr/test.py -opt options/test/REDS/HINet-GoPro.yml

  • train

    • python -m torch.distributed.launch --nproc_per_node=8 --master_port=4321 basicsr/train.py -opt options/train/GoPro/HINet.yml --launcher pytorch
Image Deblur - REDS dataset (Click to expand)

  • prepare data

    • mkdir ./datasets/REDS

    • download the train / val set from train_blur, train_sharp, val_blur, val_sharp to ./datasets/REDS/ and unzip them.

    • it should be like

      ./datasets/
./datasets/REDS/
./datasets/REDS/val/
./datasets/REDS/val/val_blur_jpeg/
./datasets/REDS/val/val_sharp/
./datasets/REDS/train/
./datasets/REDS/train/train_blur_jpeg/
./datasets/REDS/train/train_sharp/

    • python scripts/data_preparation/reds.py

      • flatten the folders and extract 300 validation images.

  • eval

    • download pretrained model to ./experiments/pretrained_models/HINet-REDS.pth
    • python basicsr/test.py -opt options/test/REDS/HINet-REDS.yml

  • train

    • python -m torch.distributed.launch --nproc_per_node=8 --master_port=4321 basicsr/train.py -opt options/train/REDS/HINet.yml --launcher pytorch
Image Derain - Rain13k dataset (Click to expand)

  • prepare data

    • mkdir ./datasets/Rain13k

    • download the train set and test set (refer to MPRNet)

    • it should be like

      ./datasets/
./datasets/Rain13k/
./datasets/Rain13k/train/
./datasets/Rain13k/train/input/
./datasets/Rain13k/train/target/
./datasets/Rain13k/test/
./datasets/Rain13k/test/Test100/
./datasets/Rain13k/test/Rain100H/
./datasets/Rain13k/test/Rain100L/
./datasets/Rain13k/test/Test2800/
./datasets/Rain13k/test/Test1200/

  • eval

    • download pretrained model to ./experiments/pretrained_models/HINet-Rain13k.pth

    • For Test100:

      • python basicsr/test.py -opt options/test/Rain13k/HINet-Test100.yml

    • For Rain100H

      • python basicsr/test.py -opt options/test/Rain13k/HINet-Rain100H.yml

    • For Rain100L

      • python basicsr/test.py -opt options/test/Rain13k/HINet-Rain100L.yml

    • For Test2800

      • python basicsr/test.py -opt options/test/Rain13k/HINet-Test2800.yml

    • For Test1200

      • python basicsr/test.py -opt options/test/Rain13k/HINet-Test1200.yml

  • train

    • python -m torch.distributed.launch --nproc_per_node=8 --master_port=4321 basicsr/train_rain.py -opt options/train/Rain13k/HINet.yml --launcher pytorch

Results

Some of the following results are higher than the original paper as we optimized some hyper-parameters.

NTIRE2021 Deblur Track2 ResultSIDD ResultGoPro Result
REDDS ResultRain13k Result

Citations

If HINet helps your research or work, please consider citing HINet.

@inproceedings{chen2021hinet,
  title={HINet: Half Instance Normalization Network for Image Restoration},
  author={Liangyu Chen and Xin Lu and Jie Zhang and Xiaojie Chu and Chengpeng Chen},
  booktitle={IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops},
  year={2021}
}

Contact

If you have any questions, please contact [email protected] or [email protected] .

Owner
GitHub Repository
Implementation of Online Label Smoothing in PyTorch

Online Label Smoothing Pytorch implementation of Online Label Smoothing (OLS) presented in Delving Deep into Label Smoothing. Introduction As the abst

83 Dec 14, 2022
This is the code for ACL2021 paper A Unified Generative Framework for Aspect-Based Sentiment Analysis

This is the code for ACL2021 paper A Unified Generative Framework for Aspect-Based Sentiment Analysis Install the package in the requirements.txt, the

108 Dec 23, 2022
Deep Learning with PyTorch made easy 🚀 !

Deep Learning with PyTorch made easy 🚀 ! Carefree? carefree-learn aims to provide CAREFREE usages for both users and developers. It also provides a c

381 Dec 22, 2022
Procedural 3D data generation pipeline for architecture

Synthetic Dataset Generator Authors: Stanislava Fedorova Alberto Tono Meher Shashwat Nigam Jiayao Zhang Amirhossein Ahmadnia Cecilia bolognesi Dominik

Computational Design Institute 49 Nov 25, 2022
An efficient framework for reinforcement learning.

rl: An efficient framework for reinforcement learning Requirements Introduction PPO Test Requirements name version Python =3.7 numpy =1.19 torch =1

16 Nov 30, 2022
Perform Linear Classification with Multi-way Data

MultiwayClassification This is an R package to perform linear classification for data with multi-way structure. The distance-weighted discrimination (

Eric F. Lock 2 Dec 15, 2020
This is the repository for the paper "Have I done enough planning or should I plan more?"

Metacognitive Learning Tool box https://re.is.mpg.de What Is This? This repository contains two modules used to analyse metacognitive learning in huma

0 Dec 01, 2021
Code and real data for the paper "Counterfactual Temporal Point Processes", available at arXiv.

counterfactual-tpp This is a repository containing code and real data for the paper Counterfactual Temporal Point Processes. Pre-requisites This code

Networks Learning 11 Dec 09, 2022
CTC segmentation python package

CTC segmentation CTC segmentation can be used to find utterances alignments within large audio files. This repository contains the ctc-segmentation py

Ludwig Kürzinger 217 Jan 04, 2023
BARF: Bundle-Adjusting Neural Radiance Fields 🤮 (ICCV 2021 oral)

BARF 🤮 : Bundle-Adjusting Neural Radiance Fields Chen-Hsuan Lin, Wei-Chiu Ma, Antonio Torralba, and Simon Lucey IEEE International Conference on Comp

Chen-Hsuan Lin 539 Dec 28, 2022
"MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction" (CVPRW 2022) & (Winner of NTIRE 2022 Challenge on Spectral Reconstruction from RGB)

MST++: Multi-stage Spectral-wise Transformer for Efficient Spectral Reconstruction (CVPRW 2022) Yuanhao Cai, Jing Lin, Zudi Lin, Haoqian Wang, Yulun Z

Yuanhao Cai 274 Jan 05, 2023
SIR model parameter estimation using a novel algorithm for differentiated uniformization.

TenSIR Parameter estimation on epidemic data under the SIR model using a novel algorithm for differentiated uniformization of Markov transition rate m

The Spang Lab 4 Nov 30, 2022
Download from Onlyfans.com.

OnlySave: Onlyfans downloader Getting Started: Download the setup executable from the latest release. Install and run. Only works on Windows currently

4 May 30, 2022
Image Fusion Transformer

Image-Fusion-Transformer Platform Python 3.7 Pytorch =1.0 Training Dataset MS-COCO 2014 (T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ram

Vibashan VS 68 Dec 23, 2022
Effect of Different Encodings and Distance Functions on Quantum Instance-based Classifiers

Effect of Different Encodings and Distance Functions on Quantum Instance-based Classifiers The repository contains the code to reproduce the experimen

Alessandro Berti 4 Aug 24, 2022
Meta Language-Specific Layers in Multilingual Language Models

Meta Language-Specific Layers in Multilingual Language Models This repo contains the source codes for our paper On Negative Interference in Multilingu

Zirui Wang 20 Feb 13, 2022
Official implementation of the Neurips 2021 paper Searching Parameterized AP Loss for Object Detection.

Parameterized AP Loss By Chenxin Tao, Zizhang Li, Xizhou Zhu, Gao Huang, Yong Liu, Jifeng Dai This is the official implementation of the Neurips 2021

46 Jul 06, 2022
A PyTorch Implementation of Gated Graph Sequence Neural Networks (GGNN)

A PyTorch Implementation of GGNN This is a PyTorch implementation of the Gated Graph Sequence Neural Networks (GGNN) as described in the paper Gated G

Ching-Yao Chuang 427 Dec 13, 2022
Detection of PCBA defect

Detection_of_PCBA_defect Detection_of_PCBA_defect Use yolov5 to train. $pip install -r requirements.txt Detect.py will detect file(jpg,mp4...) in cu

6 Nov 28, 2022
DenseCLIP: Language-Guided Dense Prediction with Context-Aware Prompting

DenseCLIP: Language-Guided Dense Prediction with Context-Aware Prompting Created by Yongming Rao*, Wenliang Zhao*, Guangyi Chen, Yansong Tang, Zheng Z

Yongming Rao 321 Dec 27, 2022