Pytorch implementation for DFN: Distributed Feedback Network for Single-Image Deraining.

Related tags

Deep LearningDFN
Overview

DFN:Distributed Feedback Network for Single-Image Deraining

Abstract

Recently, deep convolutional neural networks have achieved great success for single-image deraining. However, affected by the intrinsic overlapping between rain streaks and background texture patterns, a majority of these methods tend to almost remove texture details in rain-free regions and lead to over-smoothing effects in the recovered background. To generate reasonable rain streak layers and improve the reconstruction quality of the background, we propose a distributed feedback network (DFN) in recurrent structure. A novel feedback block is designed to implement the feedback mechanism. In each feedback block, the hidden state with high-level information (output) will flow into the next iteration to correct the low-level representations (input). By stacking multiple feedback blocks, the proposed network where the hidden states are distributed can extract powerful high-level representations for rain streak layers. Curriculum learning is employed to connect the loss of each iteration and ensure that hidden states contain the notion of output. In addition, a self-ensemble strategy for rain removal task, which can retain the approximate vertical character of rain streaks, is explored to maximize the potential performance of the deraining model. Extensive experimental results demonstrated the superiority of the proposed method in comparison with other deraining methods.

Image

Requirements

*Python 3.7,Pytorch >= 0.4.0
*Requirements: opencv-python
*Platforms: Ubuntu 18.04,cuda-10.2
*MATLAB for calculating PSNR and SSIM

Datasets

DFN is trained and tested on five benchamark datasets: Rain100L[1],Rain100H[1],RainLight[2],RainHeavy[2] and Rain12[3]. It should be noted that DFN is trained on strict 1,254 images for Rain100H.

*Note:

(i) The authors of [1] updated the Rain100L and Rain100H, we call the new datasets as RainLight and RainHeavy here.

(ii) The Rain12 contains only 12 pairs of testing images, we use the model trained on Rain100L to test on Rain12.

Getting Started

Test

All the pre-trained models were placed in ./logs/.

Run the test_DFN.py to obtain the deraining images. Then, you can calculate the evaluation metrics by run the MATLAB scripts in ./statistics/. For example, if you want to compute the average PSNR and SSIM on Rain100L, you can run the Rain100L.m.

Train

If you want to train the models, you can run the train_DFN.py and don't forget to change the args in this file. Or, you can run in the terminal by the following code:

python train_DFN.py --save_path path_to_save_trained_models --data_path path_of_the_training_dataset

Results

Average PSNR and SSIM values of DFN on five datasets are shown:

Datasets GMM DDN ResGuideNet JORDER-E SSIR PReNet BRN MSPFN DFN DFN+
Rain100L 28.66/0.865 32.16/0.936 33.16/0.963 - 32.37/0.926 37.48/0.979 38.16/0.982 37.5839/0.9784 39.22/0.985 39.85/0.987
Rain100H 15.05/0.425 21.92/0.764 25.25/0.841 - 22.47/0.716 29.62/0.901 30.73/0.916 30.8239/0.9055 31.40/0.926 31.81/0.930
RainLight - 31.66/0.922 - 39.13/0.985 32.20/0.929 37.93/0.983 38.86/0.985 39.7540/0.9862 39.53/0.987 40.12/0.988
RainHeavy - 22.03/0.713 - 29.21/0.891 22.17/0.719 29.36/0.903 30.27/0.917 30.7112/0.9129 31.07/0.927 31.47/0.931
Rain12 32.02/0.855 31.78/0.900 29.45/0.938 - 34.02/0.935 36.66/0.961 36.74/0.959 35.7780/0.9514 37.19/0.961 37.55/0.963

Image

References

[1]Yang W, Tan R, Feng J, Liu J, Guo Z, and Yan S. Deep joint rain detection and removal from a single image. In IEEE CVPR 2017.

[2]Yang W, Tan R, Feng J, Liu J, Yan S, and Guo Z. Joint rain detection and removal from a single image with contextualized deep networks. IEEE T-PAMI 2019.

[3]Li Y, Tan RT, Guo X, Lu J, and Brown M. Rain streak removal using layer priors. In IEEE CVPR 2016.

Citation

If you find our research or code useful for you, please cite our paper:

@article{DING2021,
  title = {Distributed Feedback Network for Single-Image Deraining},
  journal = {Information Sciences},
  year = {2021},
  issn = {0020-0255},
  doi = {https://doi.org/10.1016/j.ins.2021.02.080},
  url = {https://www.sciencedirect.com/science/article/pii/S0020025521002371},
  author = {Jiajun Ding and Huanlei Guo and Hang Zhou and Jun Yu and Xiongxiong He and Bo Jiang}
}
Owner
Zhejiang University of Technology(ZJUT). Research: Image Enhencement, Few-shot Learning, GAN.
Fog Simulation on Real LiDAR Point Clouds for 3D Object Detection in Adverse Weather

LiDAR fog simulation Created by Martin Hahner at the Computer Vision Lab of ETH Zurich. This is the official code release of the paper Fog Simulation

Martin Hahner 110 Dec 30, 2022
PyGCL: Graph Contrastive Learning Library for PyTorch

PyGCL: Graph Contrastive Learning for PyTorch PyGCL is an open-source library for graph contrastive learning (GCL), which features modularized GCL com

GCL: Graph Contrastive Learning Library for PyTorch 594 Jan 08, 2023
Multi-label Co-regularization for Semi-supervised Facial Action Unit Recognition (NeurIPS 2019)

MLCR This is the source code for paper Multi-label Co-regularization for Semi-supervised Facial Action Unit Recognition. Xuesong Niu, Hu Han, Shiguang

Edson-Niu 60 Nov 29, 2022
Python Implementation of the CoronaWarnApp (CWA) Event Registration

Python implementation of the Corona-Warn-App (CWA) Event Registration This is an implementation of the Protocol used to generate event and location QR

MaZderMind 17 Oct 05, 2022
Bottom-up attention model for image captioning and VQA, based on Faster R-CNN and Visual Genome

bottom-up-attention This code implements a bottom-up attention model, based on multi-gpu training of Faster R-CNN with ResNet-101, using object and at

Peter Anderson 1.3k Jan 09, 2023
Scale-aware Automatic Augmentation for Object Detection (CVPR 2021)

SA-AutoAug Scale-aware Automatic Augmentation for Object Detection Yukang Chen, Yanwei Li, Tao Kong, Lu Qi, Ruihang Chu, Lei Li, Jiaya Jia [Paper] [Bi

DV Lab 182 Dec 29, 2022
Bottom-up Human Pose Estimation

Introduction This is the official code of Rethinking the Heatmap Regression for Bottom-up Human Pose Estimation. This paper has been accepted to CVPR2

108 Dec 01, 2022
Encode and decode text application

Text Encoder and Decoder Encode and decode text in many ways using this application! Encode in: ASCII85 Base85 Base64 Base32 Base16 Url MD5 Hash SHA-1

Alice 1 Feb 12, 2022
Understanding and Improving Encoder Layer Fusion in Sequence-to-Sequence Learning (ICLR 2021)

Understanding and Improving Encoder Layer Fusion in Sequence-to-Sequence Learning (ICLR 2021) Citation Please cite as: @inproceedings{liu2020understan

Sunbow Liu 22 Nov 25, 2022
Code for ICML 2021 paper: How could Neural Networks understand Programs?

OSCAR This repository contains the source code of our ICML 2021 paper How could Neural Networks understand Programs?. Environment Run following comman

Dinglan Peng 115 Dec 17, 2022
TensorFlow Tutorial and Examples for Beginners (support TF v1 & v2)

TensorFlow Examples This tutorial was designed for easily diving into TensorFlow, through examples. For readability, it includes both notebooks and so

Aymeric Damien 42.5k Jan 08, 2023
BoxInst: High-Performance Instance Segmentation with Box Annotations

Introduction This repository is the code that needs to be submitted for OpenMMLab Algorithm Ecological Challenge, the paper is BoxInst: High-Performan

88 Dec 21, 2022
WiFi-based Multi-task Sensing

WiFi-based Multi-task Sensing Introduction WiFi-based sensing has aroused immense attention as numerous studies have made significant advances over re

zhangx289 6 Nov 24, 2022
Neural-fractal - Create Fractals Using Complex-Valued Neural Networks!

Neural Fractal Create Fractals Using Complex-Valued Neural Networks! Home Page Features Define Dynamical Systems Using Complex-Valued Neural Networks

Amirabbas Asadi 10 Dec 17, 2022
TuckER: Tensor Factorization for Knowledge Graph Completion

TuckER: Tensor Factorization for Knowledge Graph Completion This codebase contains PyTorch implementation of the paper: TuckER: Tensor Factorization f

Ivana Balazevic 296 Dec 06, 2022
Read number plates with https://platerecognizer.com/

HASS-plate-recognizer Read vehicle license plates with https://platerecognizer.com/ which offers free processing of 2500 images per month. You will ne

Robin 69 Dec 30, 2022
A PyTorch Implementation of Single Shot Scale-invariant Face Detector.

S³FD: Single Shot Scale-invariant Face Detector A PyTorch Implementation of Single Shot Scale-invariant Face Detector. Eval python wider_eval_pytorch.

carwin 235 Jan 07, 2023
My take on a practical implementation of Linformer for Pytorch.

Linformer Pytorch Implementation A practical implementation of the Linformer paper. This is attention with only linear complexity in n, allowing for v

Peter 349 Dec 25, 2022
Distributed Evolutionary Algorithms in Python

DEAP DEAP is a novel evolutionary computation framework for rapid prototyping and testing of ideas. It seeks to make algorithms explicit and data stru

Distributed Evolutionary Algorithms in Python 4.9k Jan 05, 2023
Tracking Progress in Question Answering over Knowledge Graphs

Tracking Progress in Question Answering over Knowledge Graphs Table of contents Question Answering Systems with Descriptions The QA Systems Table cont

Knowledge Graph Question Answering 47 Jan 02, 2023