Pytorch implementation for DFN: Distributed Feedback Network for Single-Image Deraining.

Related tags

Deep LearningDFN
Overview

DFN:Distributed Feedback Network for Single-Image Deraining

Abstract

Recently, deep convolutional neural networks have achieved great success for single-image deraining. However, affected by the intrinsic overlapping between rain streaks and background texture patterns, a majority of these methods tend to almost remove texture details in rain-free regions and lead to over-smoothing effects in the recovered background. To generate reasonable rain streak layers and improve the reconstruction quality of the background, we propose a distributed feedback network (DFN) in recurrent structure. A novel feedback block is designed to implement the feedback mechanism. In each feedback block, the hidden state with high-level information (output) will flow into the next iteration to correct the low-level representations (input). By stacking multiple feedback blocks, the proposed network where the hidden states are distributed can extract powerful high-level representations for rain streak layers. Curriculum learning is employed to connect the loss of each iteration and ensure that hidden states contain the notion of output. In addition, a self-ensemble strategy for rain removal task, which can retain the approximate vertical character of rain streaks, is explored to maximize the potential performance of the deraining model. Extensive experimental results demonstrated the superiority of the proposed method in comparison with other deraining methods.

Image

Requirements

*Python 3.7,Pytorch >= 0.4.0
*Requirements: opencv-python
*Platforms: Ubuntu 18.04,cuda-10.2
*MATLAB for calculating PSNR and SSIM

Datasets

DFN is trained and tested on five benchamark datasets: Rain100L[1],Rain100H[1],RainLight[2],RainHeavy[2] and Rain12[3]. It should be noted that DFN is trained on strict 1,254 images for Rain100H.

*Note:

(i) The authors of [1] updated the Rain100L and Rain100H, we call the new datasets as RainLight and RainHeavy here.

(ii) The Rain12 contains only 12 pairs of testing images, we use the model trained on Rain100L to test on Rain12.

Getting Started

Test

All the pre-trained models were placed in ./logs/.

Run the test_DFN.py to obtain the deraining images. Then, you can calculate the evaluation metrics by run the MATLAB scripts in ./statistics/. For example, if you want to compute the average PSNR and SSIM on Rain100L, you can run the Rain100L.m.

Train

If you want to train the models, you can run the train_DFN.py and don't forget to change the args in this file. Or, you can run in the terminal by the following code:

python train_DFN.py --save_path path_to_save_trained_models --data_path path_of_the_training_dataset

Results

Average PSNR and SSIM values of DFN on five datasets are shown:

Datasets GMM DDN ResGuideNet JORDER-E SSIR PReNet BRN MSPFN DFN DFN+
Rain100L 28.66/0.865 32.16/0.936 33.16/0.963 - 32.37/0.926 37.48/0.979 38.16/0.982 37.5839/0.9784 39.22/0.985 39.85/0.987
Rain100H 15.05/0.425 21.92/0.764 25.25/0.841 - 22.47/0.716 29.62/0.901 30.73/0.916 30.8239/0.9055 31.40/0.926 31.81/0.930
RainLight - 31.66/0.922 - 39.13/0.985 32.20/0.929 37.93/0.983 38.86/0.985 39.7540/0.9862 39.53/0.987 40.12/0.988
RainHeavy - 22.03/0.713 - 29.21/0.891 22.17/0.719 29.36/0.903 30.27/0.917 30.7112/0.9129 31.07/0.927 31.47/0.931
Rain12 32.02/0.855 31.78/0.900 29.45/0.938 - 34.02/0.935 36.66/0.961 36.74/0.959 35.7780/0.9514 37.19/0.961 37.55/0.963

Image

References

[1]Yang W, Tan R, Feng J, Liu J, Guo Z, and Yan S. Deep joint rain detection and removal from a single image. In IEEE CVPR 2017.

[2]Yang W, Tan R, Feng J, Liu J, Yan S, and Guo Z. Joint rain detection and removal from a single image with contextualized deep networks. IEEE T-PAMI 2019.

[3]Li Y, Tan RT, Guo X, Lu J, and Brown M. Rain streak removal using layer priors. In IEEE CVPR 2016.

Citation

If you find our research or code useful for you, please cite our paper:

@article{DING2021,
  title = {Distributed Feedback Network for Single-Image Deraining},
  journal = {Information Sciences},
  year = {2021},
  issn = {0020-0255},
  doi = {https://doi.org/10.1016/j.ins.2021.02.080},
  url = {https://www.sciencedirect.com/science/article/pii/S0020025521002371},
  author = {Jiajun Ding and Huanlei Guo and Hang Zhou and Jun Yu and Xiongxiong He and Bo Jiang}
}
Owner
Zhejiang University of Technology(ZJUT). Research: Image Enhencement, Few-shot Learning, GAN.
Team Enigma at ArgMining 2021 Shared Task: Leveraging Pretrained Language Models for Key Point Matching

Team Enigma at ArgMining 2021 Shared Task: Leveraging Pretrained Language Models for Key Point Matching This is our attempt of the shared task on Quan

Manav Nitin Kapadnis 12 Jul 08, 2022
Consumer Fairness in Recommender Systems: Contextualizing Definitions and Mitigations

Consumer Fairness in Recommender Systems: Contextualizing Definitions and Mitigations This is the repository for the paper Consumer Fairness in Recomm

7 Nov 30, 2022
PiRapGenerator - Make anyone rap the digits of pi

PiRapGenerator Make anyone rap the digits of pi (sample files are of Ted Nivison

7 Oct 02, 2022
PyTorch implementation of "VRT: A Video Restoration Transformer"

VRT: A Video Restoration Transformer Jingyun Liang, Jiezhang Cao, Yuchen Fan, Kai Zhang, Rakesh Ranjan, Yawei Li, Radu Timofte, Luc Van Gool Computer

Jingyun Liang 837 Jan 09, 2023
TensorFlow Tutorials with YouTube Videos

TensorFlow Tutorials Original repository on GitHub Original author is Magnus Erik Hvass Pedersen Introduction These tutorials are intended for beginne

9.1k Jan 02, 2023
This repo is the official implementation for Multi-Scale Adaptive Graph Neural Network for Multivariate Time Series Forecasting

1 MAGNN This repo is the official implementation for Multi-Scale Adaptive Graph Neural Network for Multivariate Time Series Forecasting. 1.1 The frame

SZJ 12 Nov 08, 2022
Project page for End-to-end Recovery of Human Shape and Pose

End-to-end Recovery of Human Shape and Pose Angjoo Kanazawa, Michael J. Black, David W. Jacobs, Jitendra Malik CVPR 2018 Project Page Requirements Pyt

1.4k Dec 29, 2022
A Python package for time series augmentation

tsaug tsaug is a Python package for time series augmentation. It offers a set of augmentation methods for time series, as well as a simple API to conn

Arundo Analytics 278 Jan 01, 2023
Source code of article "Towards Toxic and Narcotic Medication Detection with Rotated Object Detector"

Towards Toxic and Narcotic Medication Detection with Rotated Object Detector Introduction This is the source code of article: Towards Toxic and Narcot

Woody. Wang 3 Oct 29, 2022
Software Platform for solving and manipulating multiparametric programs in Python

PPOPT Python Parametric OPtimization Toolbox (PPOPT) is a software platform for solving and manipulating multiparametric programs in Python. This pack

10 Sep 13, 2022
SegNet including indices pooling for Semantic Segmentation with tensorflow and keras

SegNet SegNet is a model of semantic segmentation based on Fully Comvolutional Network. This repository contains the implementation of learning and te

Yuta Kamikawa 172 Dec 23, 2022
Weight initialization schemes for PyTorch nn.Modules

nninit Weight initialization schemes for PyTorch nn.Modules. This is a port of the popular nninit for Torch7 by @kaixhin. ##Update This repo has been

Alykhan Tejani 69 Jan 26, 2021
[CVPR'21] MonoRUn: Monocular 3D Object Detection by Reconstruction and Uncertainty Propagation

MonoRUn MonoRUn: Monocular 3D Object Detection by Reconstruction and Uncertainty Propagation. CVPR 2021. [paper] Hansheng Chen, Yuyao Huang, Wei Tian*

同济大学智能汽车研究所综合感知研究组 ( Comprehensive Perception Research Group under Institute of Intelligent Vehicles, School of Automotive Studies, Tongji University) 96 Dec 10, 2022
The official code of Anisotropic Stroke Control for Multiple Artists Style Transfer

ASMA-GAN Anisotropic Stroke Control for Multiple Artists Style Transfer Proceedings of the 28th ACM International Conference on Multimedia The officia

Six_God 146 Nov 21, 2022
TiP-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling

TiP-Adapter: Training-free CLIP-Adapter for Better Vision-Language Modeling This is the official code release for the paper 'TiP-Adapter: Training-fre

peng gao 189 Jan 04, 2023
Binary Stochastic Neurons in PyTorch

Binary Stochastic Neurons in PyTorch http://r2rt.com/binary-stochastic-neurons-in-tensorflow.html https://github.com/pytorch/examples/tree/master/mnis

Onur Kaplan 54 Nov 21, 2022
Official MegEngine implementation of CREStereo(CVPR 2022 Oral).

[CVPR 2022] Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlation This repository contains MegEngine implementation of ou

MEGVII Research 309 Dec 30, 2022
Pure python implementations of popular ML algorithms.

Minimal ML algorithms This repo includes minimal implementations of popular ML algorithms using pure python and numpy. The purpose of these notebooks

Alexis Gidiotis 3 Jan 10, 2022
The Few-Shot Bot: Prompt-Based Learning for Dialogue Systems

Few-Shot Bot: Prompt-Based Learning for Dialogue Systems This repository includes the dataset, experiments results, and code for the paper: Few-Shot B

Andrea Madotto 103 Dec 28, 2022
Unsupervised Representation Learning via Neural Activation Coding

Neural Activation Coding This repository contains the code for the paper "Unsupervised Representation Learning via Neural Activation Coding" published

yookoon park 5 May 26, 2022