NCNN implementation of Real-ESRGAN. Real-ESRGAN aims at developing Practical Algorithms for General Image Restoration.

Overview

Real-ESRGAN ncnn Vulkan

CI License: MIT Open issue Closed issue

This project is the ncnn implementation of Real-ESRGAN. Real-ESRGAN ncnn Vulkan heavily borrows from realsr-ncnn-vulkan. Many thanks to nihui, ncnn and realsr-ncnn-vulkan 😁

Real-ESRGAN aims at developing Practical Algorithms for General Image Restoration. We also optimize it for anime images.

Contents


If Real-ESRGAN is helpful in your photos/projects, please help to this repo or recommend it to your friends. Thanks 😊
Other recommended projects:
▶️ Real-ESRGAN: A practical algorithm for general image restoration
▶️ GFPGAN: A practical algorithm for real-world face restoration
▶️ BasicSR: An open-source image and video restoration toolbox
▶️ facexlib: A collection that provides useful face-relation functions.
▶️ HandyView: A PyQt5-based image viewer that is handy for view and comparison.

📖 Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data

[Paper]   [Project Page]   [Demo]
Xintao Wang, Liangbin Xie, Chao Dong, Ying Shan
Tencent ARC Lab; Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences

TODO List

  • Support further cheap arbitrary resize (e.g., bicubic, bilinear) for the model outputs
  • Bug: Some PCs will output black images
  • Add the guidance for ncnn model conversion
  • Support face restoration - GFPGAN

💻 Usages

Example Command

realesrgan-ncnn-vulkan.exe -i input.jpg -o output.png -n realesrgan-x4plus-anime

Full Usages

Usage: realesrgan-ncnn-vulkan.exe -i infile -o outfile [options]...

  -h                   show this help
  -v                   verbose output
  -i input-path        input image path (jpg/png/webp) or directory
  -o output-path       output image path (jpg/png/webp) or directory
  -s scale             upscale ratio (4, default=4)
  -t tile-size         tile size (>=32/0=auto, default=0) can be 0,0,0 for multi-gpu
  -m model-path        folder path to pre-trained models(default=models)
  -n model-name        model name (default=realesrgan-x4plus, can be realesrgan-x4plus | realesrgan-x4plus-anime | realesrnet-x4plus)
  -g gpu-id            gpu device to use (default=0) can be 0,1,2 for multi-gpu
  -j load:proc:save    thread count for load/proc/save (default=1:2:2) can be 1:2,2,2:2 for multi-gpu
  -x                   enable tta mode
  -f format            output image format (jpg/png/webp, default=ext/png)
  • input-path and output-path accept either file path or directory path
  • scale = scale level, 4 = upscale 4x
  • tile-size = tile size, use smaller value to reduce GPU memory usage, default selects automatically
  • load:proc:save = thread count for the three stages (image decoding + model upscaling + image encoding), using larger values may increase GPU usage and consume more GPU memory. You can tune this configuration with "4:4:4" for many small-size images, and "2:2:2" for large-size images. The default setting usually works fine for most situations. If you find that your GPU is hungry, try increasing thread count to achieve faster processing.
  • format = the format of the image to be output, png is better supported, however webp generally yields smaller file sizes, both are losslessly encoded

If you encounter crash or error, try to upgrade your GPU driver

🌏 Other Open-Source Code Used

📜 BibTeX

@InProceedings{wang2021realesrgan,
    author    = {Xintao Wang and Liangbin Xie and Chao Dong and Ying Shan},
    title     = {Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data},
    booktitle = {International Conference on Computer Vision Workshops (ICCVW)},
    date      = {2021}
}

📧 Contact

If you have any question, please email [email protected] or [email protected].

Comments
  • problem running on aws

    problem running on aws

    I downloaded the ubuntu zip on a g3s.xlarge and the result is a black image. Is the zip missing files?

    
    ./realesrgan-ncnn-vulkan-v0.2.0-ubuntu/realesrgan-ncnn-vulkan -i input.jpg -o out/output.jpg -n realesrgan-x4plus -s 4 
    
    [0 Tesla M60]  queueC=0[16]  queueG=0[16]  queueT=1[2]
    [0 Tesla M60]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
    [0 Tesla M60]  fp16-p/s/a=1/1/0  int8-p/s/a=1/1/1
    [0 Tesla M60]  subgroup=32  basic=1  vote=1  ballot=1  shuffle=1
    [1 llvmpipe (LLVM 12.0.0, 256 bits)]  queueC=0[1]  queueG=0[1]  queueT=0[1]
    [1 llvmpipe (LLVM 12.0.0, 256 bits)]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
    [1 llvmpipe (LLVM 12.0.0, 256 bits)]  fp16-p/s/a=1/1/0  int8-p/s/a=1/1/0
    [1 llvmpipe (LLVM 12.0.0, 256 bits)]  subgroup=8  basic=1  vote=1  ballot=1  shuffle=0
    fopen /home/ubuntu/realesrgan-ncnn-vulkan-v0.2.0-ubuntu/models/realesrgan-x4plus.param failed
    fopen /home/ubuntu/realesrgan-ncnn-vulkan-v0.2.0-ubuntu/models/realesrgan-x4plus.bin failed
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    0.00%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    4.17%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    8.33%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    12.50%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    16.67%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    20.83%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    25.00%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    29.17%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    33.33%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    37.50%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    41.67%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    45.83%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    50.00%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    54.17%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    58.33%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    62.50%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    66.67%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    70.83%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    75.00%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    79.17%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    83.33%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    87.50%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    91.67%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    95.83%
    
    opened by kmulvey 2
  • How do I switch from integrated gpu to nvedia gpu

    How do I switch from integrated gpu to nvedia gpu

    [0 Intel(R) UHD Graphics 630] queueC=0[1] queueG=0[1] queueT=0[1] [0 Intel(R) UHD Graphics 630] bugsbn1=0 bugbilz=3 bugcopc=0 bugihfa=0 [0 Intel(R) UHD Graphics 630] fp16-p/s/a=1/1/1 int8-p/s/a=1/1/1 [0 Intel(R) UHD Graphics 630] subgroup=32 basic=1 vote=1 ballot=1 shuffle=1

    When upscaling images it only uses integrated gpu I want to use my gtx 1650 how do I switch this?

    opened by csAshish 0
  • in directory mode, option to skip if destination already exist

    in directory mode, option to skip if destination already exist

    if you stop the upscaler while processing a directory and you have to restart it, it will start from the beginning and overwrite existing files

    overwriting in file mode is fine but should be an option for direcotry mode ... so I propose a flag to disable overwrite and skip if exist

    opened by 6543 0
  • models: consider adding into README how to get them.

    models: consider adding into README how to get them.

    Hey,

    Please consider adding to README a section how to get the pre-trained model files. Currently you'd need to extract them from the following links

    • https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.5.0/realesrgan-ncnn-vulkan-20220424-ubuntu.zip
    • https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.5.0/realesr-animevideov3.pth
    • https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.5.0/realesr-general-x4v3.pth

    It took me a moment to figure out how to get a hold of them and I suspect many others would face similar challenge.

    opened by slashbeast 0
  • Segfault realesrnet-x4plus

    Segfault realesrnet-x4plus

    %  ./realesrgan-ncnn-vulkan -i FdLbqKcWIAAaCeY.jpeg  -o FdLbqKcWIAAaCeY.png -n realesrnet-x4plus
    zsh: segmentation fault  ./realesrgan-ncnn-vulkan -i FdLbqKcWIAAaCeY.jpeg -o FdLbqKcWIAAaCeY.png -n 
    
    opened by atomical 1
Owner
Xintao
Researcher at Tencent ARC Lab, (Applied Research Center)
Xintao
A novel pipeline framework for multi-hop complex KGQA task. About the paper title: Improving Multi-hop Embedded Knowledge Graph Question Answering by Introducing Relational Chain Reasoning

Rce-KGQA A novel pipeline framework for multi-hop complex KGQA task. This framework mainly contains two modules, answering_filtering_module and relati

金伟强 -上海大学人工智能小渣渣~ 16 Nov 18, 2022
This project aims to explore the deployment of Swin-Transformer based on TensorRT, including the test results of FP16 and INT8.

Swin Transformer This project aims to explore the deployment of SwinTransformer based on TensorRT, including the test results of FP16 and INT8. Introd

maggiez 87 Dec 21, 2022
Leveraging Instance-, Image- and Dataset-Level Information for Weakly Supervised Instance Segmentation

Leveraging Instance-, Image- and Dataset-Level Information for Weakly Supervised Instance Segmentation This paper has been accepted and early accessed

Yun Liu 39 Sep 20, 2022
This repository collects 100 papers related to negative sampling methods.

Negative-Sampling-Paper This repository collects 100 papers related to negative sampling methods, covering multiple research fields such as Recommenda

RUCAIBox 119 Dec 29, 2022
CUAD

Contract Understanding Atticus Dataset This repository contains code for the Contract Understanding Atticus Dataset (CUAD), a dataset for legal contra

The Atticus Project 273 Dec 17, 2022
LF-YOLO (Lighter and Faster YOLO) is used to detect defect of X-ray weld image.

This project is based on ultralytics/yolov3. LF-YOLO (Lighter and Faster YOLO) is used to detect defect of X-ray weld image. The related paper is avai

26 Dec 13, 2022
Repo for 2021 SDD assessment task 2, by Felix, Anna, and James.

SoftwareTask2 Repo for 2021 SDD assessment task 2, by Felix, Anna, and James. File/folder structure: helloworld.py - demonstrates various map backgrou

3 Dec 13, 2022
Official implementation of "Learning Proposals for Practical Energy-Based Regression", 2021.

ebms_proposals Official implementation (PyTorch) of the paper: Learning Proposals for Practical Energy-Based Regression, 2021 [arXiv] [project]. Fredr

Fredrik Gustafsson 10 Oct 22, 2022
This Deep Learning Model Predicts that from which disease you are suffering.

Deep-Learning-Project This Deep Learning Model Predicts that from which disease you are suffering. This Project Covers the Topics of Deep Learning Int

Jai Viral Doshi 0 Jan 20, 2022
Capsule endoscopy detection DACON challenge

capsule_endoscopy_detection (DACON Challenge) Overview Yolov5, Yolor, mmdetection기반의 모델을 사용 (총 11개 모델 앙상블) 모든 모델은 학습 시 Pretrained Weight을 yolov5, yolo

MAILAB 11 Nov 25, 2022
Animation of solving the traveling salesman problem to optimality using mixed-integer programming and iteratively eliminating sub tours

tsp-streamlit Animation of solving the traveling salesman problem to optimality using mixed-integer programming and iteratively eliminating sub tours.

4 Nov 05, 2022
DRLib:A concise deep reinforcement learning library, integrating HER and PER for almost off policy RL algos.

DRLib:A concise deep reinforcement learning library, integrating HER and PER for almost off policy RL algos A concise deep reinforcement learning libr

329 Jan 03, 2023
The code of paper 'Learning to Aggregate and Personalize 3D Face from In-the-Wild Photo Collection'

Learning to Aggregate and Personalize 3D Face from In-the-Wild Photo Collection Pytorch implemetation of paper 'Learning to Aggregate and Personalize

Tencent YouTu Research 136 Dec 29, 2022
[ICCV 2021] Official Pytorch implementation for Discriminative Region-based Multi-Label Zero-Shot Learning SOTA results on NUS-WIDE and OpenImages

Discriminative Region-based Multi-Label Zero-Shot Learning (ICCV 2021) [arXiv][Project page coming soon] Sanath Narayan*, Akshita Gupta*, Salman Kh

Akshita Gupta 54 Nov 21, 2022
The Face Mask recognition system uses AI technology to detect the person with or without a mask.

Face Mask Detection Face Mask Detection system built with OpenCV, Keras/TensorFlow using Deep Learning and Computer Vision concepts in order to detect

Rohan Kasabe 4 Apr 05, 2022
PyTorch Implementation of Spatially Consistent Representation Learning(SCRL)

Spatially Consistent Representation Learning (CVPR'21) Official PyTorch implementation of Spatially Consistent Representation Learning (SCRL). This re

Kakao Brain 102 Nov 03, 2022
Contains code for the paper "Vision Transformers are Robust Learners".

Vision Transformers are Robust Learners This repository contains the code for the paper Vision Transformers are Robust Learners by Sayak Paul* and Pin

Sayak Paul 103 Jan 05, 2023
PyTorch version of Stable Baselines, reliable implementations of reinforcement learning algorithms.

PyTorch version of Stable Baselines, reliable implementations of reinforcement learning algorithms.

DLR-RM 4.7k Jan 01, 2023
Implementation of Analyzing and Improving the Image Quality of StyleGAN (StyleGAN 2) in PyTorch

Implementation of Analyzing and Improving the Image Quality of StyleGAN (StyleGAN 2) in PyTorch

Kim Seonghyeon 2.2k Jan 01, 2023
A benchmark dataset for emulating atmospheric radiative transfer in weather and climate models with machine learning (NeurIPS 2021 Datasets and Benchmarks Track)

ClimART - A Benchmark Dataset for Emulating Atmospheric Radiative Transfer in Weather and Climate Models Official PyTorch Implementation Using deep le

21 Dec 31, 2022