PyTorch code for our paper "Gated Multiple Feedback Network for Image Super-Resolution" (BMVC2019)

Last update: Nov 03, 2022

Related tags

Overview

Gated Multiple Feedback Network for Image Super-Resolution

This repository contains the PyTorch implementation for the proposed GMFN [arXiv].

The framework of our proposed GMFN. The colored arrows among different time steps denote the multiple feedback connections. The high-level information carried by them helps low-level features become more representative.

Demo

Clone SRFBN as the backbone and satisfy its requirements.

Test

Copy ./networks/gmfn_arch.py into SRFBN_CVPR19/networks/
Download the pre-trained models from Google driver or Baidu Netdisk, unzip and place them into SRFBN_CVPR19/models.
Copy ./options/test/ to SRFBN_CVPR19/options/test/.
Run commands cd SRFBN_CVPR19 and one of followings for evaluation on Set5:

python test.py -opt options/test/test_GMFN_x2.json
python test.py -opt options/test/test_GMFN_x3.json
python test.py -opt options/test/test_GMFN_x4.json

Finally, PSNR/SSIM values for Set5 are shown on your screen, you can find the reconstruction images in ./results.

To test GMFN on other standard SR benchmarks or your own images, please refer to the instruction in SRFBN.

Train

Prepare the training set according to this (1-3).
Modify ./options/train/train_GMFN.json by following the instructions in ./options/train/README.md.
Run commands:

cd SRFBN_CVPR19
python train.py -opt options/train/train_GNFN.json

You can monitor the training process in ./experiments.
Finally, you can follow the test pipeline to evaluate the model trained by yourself.

Performance

Quantitative Results

Quantitative evaluation under scale factors x2, x3 and x4. The best performance is shown in bold and the second best performance is underlined.

More Qualitative Results (x4)

Acknowledgment

If you find our work useful in your research or publications, please consider citing:

@inproceedings{li2019gmfn,
    author = {Li, Qilei and Li, Zhen and Lu, Lu and Jeon, Gwanggil and Liu, Kai and Yang, Xiaomin},
    title = {Gated Multiple Feedback Network for Image Super-Resolution},
    booktitle = {The British Machine Vision Conference (BMVC)},
    year = {2019}
}

@inproceedings{li2019srfbn,
    author = {Li, Zhen and Yang, Jinglei and Liu, Zheng and Yang, Xiaomin and Jeon, Gwanggil and Wu, Wei},
    title = {Feedback Network for Image Super-Resolution},
    booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR)},
    year= {2019}
}

Pytorch implementation of our paper under review — Lottery Jackpots Exist in Pre-trained Models

Lottery Jackpots Exist in Pre-trained Models (Paper Link) Requirements Python = 3.7.4 Pytorch = 1.6.1 Torchvision = 0.4.1 Reproduce the Experiment

27 Jun 28, 2022

The repository offers the official implementation of our paper in PyTorch.

Cloth Interactive Transformer (CIT) Cloth Interactive Transformer for Virtual Try-On Bin Ren1, Hao Tang1, Fanyang Meng2, Runwei Ding3, Ling Shao4, Phi

49 Dec 1, 2022

PyTorch implementations for our SIGGRAPH 2021 paper: Editable Free-viewpoint Video Using a Layered Neural Representation.

st-nerf We provide PyTorch implementations for our paper: Editable Free-viewpoint Video Using a Layered Neural Representation SIGGRAPH 2021 Jiakai Zha

258 Jan 2, 2023

PyTorch implementation of our ICCV2021 paper: StructDepth: Leveraging the structural regularities for self-supervised indoor depth estimation

StructDepth PyTorch implementation of our ICCV2021 paper: StructDepth: Leveraging the structural regularities for self-supervised indoor depth estimat

112 Nov 28, 2022

Pytorch implementation for our ICCV 2021 paper "TRAR: Routing the Attention Spans in Transformers for Visual Question Answering".

TRAnsformer Routing Networks (TRAR) This is an official implementation for ICCV 2021 paper "TRAR: Routing the Attention Spans in Transformers for Visu

49 Nov 10, 2022

This is the official pytorch implementation for our ICCV 2021 paper "TRAR: Routing the Attention Spans in Transformers for Visual Question Answering" on VQA Task

🌈 ERASOR (RA-L'21 with ICRA Option) Official page of "ERASOR: Egocentric Ratio of Pseudo Occupancy-based Dynamic Object Removal for Static 3D Point C

225 Dec 29, 2022

PyTorch implementation of our ICCV 2021 paper, Interpretation of Emergent Communication in Heterogeneous Collaborative Embodied Agents.

4 May 8, 2022

PyTorch implementation for our NeurIPS 2021 Spotlight paper "Long Short-Term Transformer for Online Action Detection".

Long Short-Term Transformer for Online Action Detection Introduction This is a PyTorch implementation for our NeurIPS 2021 Spotlight paper "Long Short

77 Dec 16, 2022

Official PyTorch implemention of our paper "Learning to Rectify for Robust Learning with Noisy Labels".

WarPI The official PyTorch implemention of our paper "Learning to Rectify for Robust Learning with Noisy Labels". Run python main.py --corruption_type

3 Sep 3, 2022

Comments

Approximately how many epoches will reach the results in the paper (4x SR result)

Hi, liqilei After I have run about 700 epoches, the reult on val set is 32.41(highest result). I want to know if my training process seems to be problematic? How long did you reach 32.47 of SRFBN when you were training? How long does it take to reach 32.70? Thank you.

opened by Senwang98 7
train error size not match

CUDA_VISIBLE_DEVICES=0 python train.py -opt options/train/train_GMFN.json I use celeba dataset train

===> Training Epoch: [1/1000]... Learning Rate: 0.000200 Epoch: [1/1000]: 0%| | 0/251718 [00:00<?, ?it/s] Traceback (most recent call last): File "train.py", line 131, in main() File "train.py", line 69, in main iter_loss = solver.train_step() File "/exp_sr/SRFBN/solvers/SRSolver.py", line 104, in train_step loss_steps = [self.criterion_pix(sr, split_HR) for sr in outputs] File "/exp_sr/SRFBN/solvers/SRSolver.py", line 104, in loss_steps = [self.criterion_pix(sr, split_HR) for sr in outputs] File "/toolscnn/env_pyt0.4_py3.5_awsrn/lib/python3.5/site-packages/torch/nn/modules/module.py", line 477, in call result = self.forward(*input, **kwargs) File "/toolscnn/env_pyt0.4_py3.5_awsrn/lib/python3.5/site-packages/torch/nn/modules/loss.py", line 87, in forward return F.l1_loss(input, target, reduction=self.reduction) File "/toolscnn/env_pyt0.4_py3.5_awsrn/lib/python3.5/site-packages/torch/nn/functional.py", line 1702, in l1_loss input, target, reduction) File "/toolscnn/env_pyt0.4_py3.5_awsrn/lib/python3.5/site-packages/torch/nn/functional.py", line 1674, in _pointwise_loss return lambd_optimized(input, target, reduction) RuntimeError: input and target shapes do not match: input [16 x 3 x 192 x 192], target [16 x 3 x 48 x 48] at /pytorch/aten/src/THCUNN/generic/AbsCriterion.cu:12

opened by yja1 3
Not an Issue

Hey @Paper99,

Thanks for sharing your code! I wonder if it is possible to help with visualizing featuer-maps as you did in your paper figure 4.

Thanks

opened by Auth0rM0rgan 1
My training result with scale = 2
Hi, After I have trained the DIV2k, I get the final result(use best_ckp.pth to test):

set5:38.16/0.9610 set14:33.91/0.9203 urban100:32.81/0.9349 B100:32.30/0.9011 manga109:39.01/0.9776

It seems much lower than that in your paper.
opened by Senwang98 6

Releases(v1.0)

v1.0(Aug 26, 2020)

Source code(tar.gz)
Source code(zip)
GMFN_BMVC19_Models.zip(104.38 MB)

Owner

Qilei Li

GitHub Repository

LightLog is an open source deep learning based lightweight log analysis tool for log anomaly detection.

LightLog Introduction LightLog is an open source deep learning based lightweight log analysis tool for log anomaly detection. Function description [BG

25 Dec 17, 2022

A new codebase for Group Activity Recognition. It contains codes for ICCV 2021 paper: Spatio-Temporal Dynamic Inference Network for Group Activity Recognition and some other methods.

Spatio-Temporal Dynamic Inference Network for Group Activity Recognition The source codes for ICCV2021 Paper: Spatio-Temporal Dynamic Inference Networ

40 Dec 12, 2022

Implementation of Hierarchical Transformer Memory (HTM) for Pytorch

Hierarchical Transformer Memory (HTM) - Pytorch Implementation of Hierarchical Transformer Memory (HTM) for Pytorch. This Deepmind paper proposes a si

63 Dec 29, 2022

FluidNet re-written with ATen tensor lib

fluidnet_cxx: Accelerating Fluid Simulation with Convolutional Neural Networks. A PyTorch/ATen Implementation. This repository is based on the paper,

50 Jun 07, 2022

Pytorch implementation of our paper LIMUSE: LIGHTWEIGHT MULTI-MODAL SPEAKER EXTRACTION.

LiMuSE Overview Pytorch implementation of our paper LIMUSE: LIGHTWEIGHT MULTI-MODAL SPEAKER EXTRACTION. LiMuSE explores group communication on a multi

17 Oct 26, 2022

Paddle pit - Rethinking Spatial Dimensions of Vision Transformers

基于Paddle实现PiT ——Rethinking Spatial Dimensions of Vision Transformers,arxiv 官方原版代

4 Jan 15, 2022

[NeurIPS'20] Multiscale Deep Equilibrium Models

Multiscale Deep Equilibrium Models 💥 💥 💥 💥 This repo is deprecated and we will soon stop actively maintaining it, as a more up-to-date (and simple

221 Dec 26, 2022

Quickly and easily create / train a custom DeepDream model

Dream-Creator This project aims to simplify the process of creating a custom DeepDream model by using pretrained GoogleNet models and custom image dat

55 Dec 27, 2022

Code for "Training Neural Networks with Fixed Sparse Masks" (NeurIPS 2021).

Fisher Induced Sparse uncHanging (FISH) Mask This repo contains the code for Fisher Induced Sparse uncHanging (FISH) Mask training, from "Training Neu

37 Dec 30, 2022

FFTNet vocoder implementation

Unofficial Implementation of FFTNet vocode paper. implement the model. implement tests. overfit on a single batch (sanity check). linearize weights fo

81 Dec 08, 2022

This program automatically runs Python code copied in clipboard

CopyRun This program runs Python code which is copied in clipboard WARNING!! USE AT YOUR OWN RISK! NO GUARANTIES IF ANYTHING GETS BROKEN. DO NOT COPY

4 Sep 10, 2021

A PyTorch-based R-YOLOv4 implementation which combines YOLOv4 model and loss function from R3Det for arbitrary oriented object detection.

R-YOLOv4 This is a PyTorch-based R-YOLOv4 implementation which combines YOLOv4 model and loss function from R3Det for arbitrary oriented object detect

94 Dec 03, 2022

This is the repository of our article published on MDPI Entropy "Feature Selection for Recommender Systems with Quantum Computing".

Collaborative-driven Quantum Feature Selection This repository was developed by Riccardo Nembrini, PhD student at Politecnico di Milano. See the websi

10 Apr 21, 2022

Fuzzer for Linux Kernel Drivers

difuze: Fuzzer for Linux Kernel Drivers This repo contains all the sources (including setup scripts), you need to get difuze up and running. Tested on

344 Dec 27, 2022

CoRe: Contrastive Recurrent State-Space Models

CoRe: Contrastive Recurrent State-Space Models This code implements the CoRe model and reproduces experimental results found in Robust Robotic Control

21 Aug 11, 2022

YOLOv5 + ROS2 object detection package

YOLOv5-ROS YOLOv5 + ROS2 object detection package This program changes the input of detect.py (ultralytics/yolov5) to sensor_msgs/Image of ROS2. Requi

23 Dec 19, 2022

Nicely is a real-time Feedback and Intervention Program Depression is a prevalent issue across all age groups, socioeconomic classes, and cultural identities.

1 Jan 16, 2022

This repository contains numerical implementation for the paper Intertemporal Pricing under Reference Effects: Integrating Reference Effects and Consumer Heterogeneity.

6 Nov 18, 2022

HCQ: Hybrid Contrastive Quantization for Efficient Cross-View Video Retrieval

HCQ: Hybrid Contrastive Quantization for Efficient Cross-View Video Retrieval [toc] 1. Introduction This repository provides the code for our paper at

13 Dec 08, 2022

CurriculumNet: Weakly Supervised Learning from Large-Scale Web Images

CurriculumNet Introduction This repo contains related code and models from the ECCV 2018 CurriculumNet paper. CurriculumNet is a new training strategy

156 Jul 04, 2022