NCNN implementation of Real-ESRGAN. Real-ESRGAN aims at developing Practical Algorithms for General Image Restoration.

Overview

Real-ESRGAN ncnn Vulkan

CI License: MIT Open issue Closed issue

This project is the ncnn implementation of Real-ESRGAN. Real-ESRGAN ncnn Vulkan heavily borrows from realsr-ncnn-vulkan. Many thanks to nihui, ncnn and realsr-ncnn-vulkan ๐Ÿ˜

Real-ESRGAN aims at developing Practical Algorithms for General Image Restoration. We also optimize it for anime images.

Contents


If Real-ESRGAN is helpful in your photos/projects, please help to โญ this repo or recommend it to your friends. Thanks ๐Ÿ˜Š
Other recommended projects:
โ–ถ๏ธ Real-ESRGAN: A practical algorithm for general image restoration
โ–ถ๏ธ GFPGAN: A practical algorithm for real-world face restoration
โ–ถ๏ธ BasicSR: An open-source image and video restoration toolbox
โ–ถ๏ธ facexlib: A collection that provides useful face-relation functions.
โ–ถ๏ธ HandyView: A PyQt5-based image viewer that is handy for view and comparison.

๐Ÿ“– Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data

[Paper] โ€ƒ [Project Page] โ€ƒ [Demo]
Xintao Wang, Liangbin Xie, Chao Dong, Ying Shan
Tencent ARC Lab; Shenzhen Institutes of Advanced Technology, Chinese Academy of Sciences

โณ TODO List

  • Support further cheap arbitrary resize (e.g., bicubic, bilinear) for the model outputs
  • Bug: Some PCs will output black images
  • Add the guidance for ncnn model conversion
  • Support face restoration - GFPGAN

๐Ÿ’ป Usages

Example Command

realesrgan-ncnn-vulkan.exe -i input.jpg -o output.png -n realesrgan-x4plus-anime

Full Usages

Usage: realesrgan-ncnn-vulkan.exe -i infile -o outfile [options]...

  -h                   show this help
  -v                   verbose output
  -i input-path        input image path (jpg/png/webp) or directory
  -o output-path       output image path (jpg/png/webp) or directory
  -s scale             upscale ratio (4, default=4)
  -t tile-size         tile size (>=32/0=auto, default=0) can be 0,0,0 for multi-gpu
  -m model-path        folder path to pre-trained models(default=models)
  -n model-name        model name (default=realesrgan-x4plus, can be realesrgan-x4plus | realesrgan-x4plus-anime | realesrnet-x4plus)
  -g gpu-id            gpu device to use (default=0) can be 0,1,2 for multi-gpu
  -j load:proc:save    thread count for load/proc/save (default=1:2:2) can be 1:2,2,2:2 for multi-gpu
  -x                   enable tta mode
  -f format            output image format (jpg/png/webp, default=ext/png)
  • input-path and output-path accept either file path or directory path
  • scale = scale level, 4 = upscale 4x
  • tile-size = tile size, use smaller value to reduce GPU memory usage, default selects automatically
  • load:proc:save = thread count for the three stages (image decoding + model upscaling + image encoding), using larger values may increase GPU usage and consume more GPU memory. You can tune this configuration with "4:4:4" for many small-size images, and "2:2:2" for large-size images. The default setting usually works fine for most situations. If you find that your GPU is hungry, try increasing thread count to achieve faster processing.
  • format = the format of the image to be output, png is better supported, however webp generally yields smaller file sizes, both are losslessly encoded

If you encounter crash or error, try to upgrade your GPU driver

๐ŸŒ Other Open-Source Code Used

๐Ÿ“œ BibTeX

@InProceedings{wang2021realesrgan,
    author    = {Xintao Wang and Liangbin Xie and Chao Dong and Ying Shan},
    title     = {Real-ESRGAN: Training Real-World Blind Super-Resolution with Pure Synthetic Data},
    booktitle = {International Conference on Computer Vision Workshops (ICCVW)},
    date      = {2021}
}

๐Ÿ“ง Contact

If you have any question, please email [email protected] or [email protected].

Comments
  • problem running on aws

    problem running on aws

    I downloaded the ubuntu zip on a g3s.xlarge and the result is a black image. Is the zip missing files?

    
    ./realesrgan-ncnn-vulkan-v0.2.0-ubuntu/realesrgan-ncnn-vulkan -i input.jpg -o out/output.jpg -n realesrgan-x4plus -s 4 
    
    [0 Tesla M60]  queueC=0[16]  queueG=0[16]  queueT=1[2]
    [0 Tesla M60]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
    [0 Tesla M60]  fp16-p/s/a=1/1/0  int8-p/s/a=1/1/1
    [0 Tesla M60]  subgroup=32  basic=1  vote=1  ballot=1  shuffle=1
    [1 llvmpipe (LLVM 12.0.0, 256 bits)]  queueC=0[1]  queueG=0[1]  queueT=0[1]
    [1 llvmpipe (LLVM 12.0.0, 256 bits)]  bugsbn1=0  bugbilz=0  bugcopc=0  bugihfa=0
    [1 llvmpipe (LLVM 12.0.0, 256 bits)]  fp16-p/s/a=1/1/0  int8-p/s/a=1/1/0
    [1 llvmpipe (LLVM 12.0.0, 256 bits)]  subgroup=8  basic=1  vote=1  ballot=1  shuffle=0
    fopen /home/ubuntu/realesrgan-ncnn-vulkan-v0.2.0-ubuntu/models/realesrgan-x4plus.param failed
    fopen /home/ubuntu/realesrgan-ncnn-vulkan-v0.2.0-ubuntu/models/realesrgan-x4plus.bin failed
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    0.00%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    4.17%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    8.33%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    12.50%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    16.67%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    20.83%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    25.00%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    29.17%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    33.33%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    37.50%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    41.67%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    45.83%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    50.00%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    54.17%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    58.33%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    62.50%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    66.67%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    70.83%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    75.00%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    79.17%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    83.33%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    87.50%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    91.67%
    find_blob_index_by_name data failed
    Try
    find_blob_index_by_name output failed
    Try
    95.83%
    
    opened by kmulvey 2
  • How do I switch from integrated gpu to nvedia gpu

    How do I switch from integrated gpu to nvedia gpu

    [0 Intel(R) UHD Graphics 630] queueC=0[1] queueG=0[1] queueT=0[1] [0 Intel(R) UHD Graphics 630] bugsbn1=0 bugbilz=3 bugcopc=0 bugihfa=0 [0 Intel(R) UHD Graphics 630] fp16-p/s/a=1/1/1 int8-p/s/a=1/1/1 [0 Intel(R) UHD Graphics 630] subgroup=32 basic=1 vote=1 ballot=1 shuffle=1

    When upscaling images it only uses integrated gpu I want to use my gtx 1650 how do I switch this?

    opened by csAshish 0
  • in directory mode, option to skip if destination already exist

    in directory mode, option to skip if destination already exist

    if you stop the upscaler while processing a directory and you have to restart it, it will start from the beginning and overwrite existing files

    overwriting in file mode is fine but should be an option for direcotry mode ... so I propose a flag to disable overwrite and skip if exist

    opened by 6543 0
  • models: consider adding into README how to get them.

    models: consider adding into README how to get them.

    Hey,

    Please consider adding to README a section how to get the pre-trained model files. Currently you'd need to extract them from the following links

    • https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.5.0/realesrgan-ncnn-vulkan-20220424-ubuntu.zip
    • https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.5.0/realesr-animevideov3.pth
    • https://github.com/xinntao/Real-ESRGAN/releases/download/v0.2.5.0/realesr-general-x4v3.pth

    It took me a moment to figure out how to get a hold of them and I suspect many others would face similar challenge.

    opened by slashbeast 0
  • Segfault realesrnet-x4plus

    Segfault realesrnet-x4plus

    %  ./realesrgan-ncnn-vulkan -i FdLbqKcWIAAaCeY.jpeg  -o FdLbqKcWIAAaCeY.png -n realesrnet-x4plus
    zsh: segmentation fault  ./realesrgan-ncnn-vulkan -i FdLbqKcWIAAaCeY.jpeg -o FdLbqKcWIAAaCeY.png -n 
    
    opened by atomical 1
Owner
Xintao
Researcher at Tencent ARC Lab, (Applied Research Center)
Xintao
U-Net implementation in PyTorch for FLAIR abnormality segmentation in brain MRI

U-Net for brain segmentation U-Net implementation in PyTorch for FLAIR abnormality segmentation in brain MRI based on a deep learning segmentation alg

562 Jan 02, 2023
PROJECT - Az Residential Real Estate Analysis

AZ RESIDENTIAL REAL ESTATE ANALYSIS -Decided on libraries to import. Includes pa

2 Jul 05, 2022
PyTorch code accompanying our paper on Maximum Entropy Generators for Energy-Based Models

Maximum Entropy Generators for Energy-Based Models All experiments have tensorboard visualizations for samples / density / train curves etc. To run th

Rithesh Kumar 135 Oct 27, 2022
PyTorch implementation of EigenGAN

PyTorch Implementation of EigenGAN Train python train.py [image_folder_path] --name [experiment name] Test python test.py [ckpt path] --traverse FFH

62 Nov 12, 2022
Robot Reinforcement Learning on the Constraint Manifold

Implementation of "Robot Reinforcement Learning on the Constraint Manifold"

31 Dec 05, 2022
A PyTorch library and evaluation platform for end-to-end compression research

CompressAI CompressAI (compress-ay) is a PyTorch library and evaluation platform for end-to-end compression research. CompressAI currently provides: c

InterDigital 680 Jan 06, 2023
Official code for CVPR2022 paper: Depth-Aware Generative Adversarial Network for Talking Head Video Generation

๐Ÿ“– Depth-Aware Generative Adversarial Network for Talking Head Video Generation (CVPR 2022) ๐Ÿ”ฅ If DaGAN is helpful in your photos/projects, please hel

Fa-Ting Hong 503 Jan 04, 2023
The aim of this project is to build an AI bot that can play the Wordle game, or more generally Squabble

Wordle RL The aim of this project is to build an AI bot that can play the Wordle game, or more generally Squabble I know there are more deterministic

Aditya Arora 3 Feb 22, 2022
PASTRIE: A Corpus of Prepositions Annotated with Supersense Tags in Reddit International English

PASTRIE Official release of the corpus described in the paper: Michael Kranzlein, Emma Manning, Siyao Peng, Shira Wein, Aryaman Arora, and Nathan Schn

NERT @ Georgetown 4 Dec 02, 2021
Fully Convolutional DenseNet (A.K.A 100 layer tiramisu) for semantic segmentation of images implemented in TensorFlow.

FC-DenseNet-Tensorflow This is a re-implementation of the 100 layer tiramisu, technically a fully convolutional DenseNet, in TensorFlow (Tiramisu). Th

Hasnain Raza 121 Oct 12, 2022
A Robust Non-IoU Alternative to Non-Maxima Suppression in Object Detection

Confluence: A Robust Non-IoU Alternative to Non-Maxima Suppression in Object Detection 1. ไป‹็ป ็”จไปฅๆ›ฟไปฃ NMS๏ผŒๅœจๆ‰€ๆœ‰ bbox ไธญๆŒ‘้€‰ๅ‡บๆœ€ไผ˜็š„้›†ๅˆใ€‚ NMS ไป…่€ƒ่™‘ไบ† bbox ็š„ๅพ—ๅˆ†๏ผŒ็„ถๅŽๆ นๆฎ IOU ๆฅ

44 Sep 15, 2022
A Strong Baseline for Image Semantic Segmentation

A Strong Baseline for Image Semantic Segmentation Introduction This project is an open source semantic segmentation toolbox based on PyTorch. It is ba

Clark He 49 Sep 20, 2022
The implementation our EMNLP 2021 paper "Enhanced Language Representation with Label Knowledge for Span Extraction".

LEAR The implementation our EMNLP 2021 paper "Enhanced Language Representation with Label Knowledge for Span Extraction". See below for an overview of

ๆจๆ”€ 93 Jan 07, 2023
Official implementation of the paper Chunked Autoregressive GAN for Conditional Waveform Synthesis

PyEmits, a python package for easy manipulation in time-series data. Time-series data is very common in real life. Engineering FSI industry (Financial

Descript 150 Dec 06, 2022
AEI: Actors-Environment Interaction with Adaptive Attention for Temporal Action Proposals Generation

AEI: Actors-Environment Interaction with Adaptive Attention for Temporal Action Proposals Generation A pytorch-version implementation codes of paper:

11 Dec 13, 2022
TensorFlow (Python) implementation of DeepTCN model for multivariate time series forecasting.

DeepTCN TensorFlow TensorFlow (Python) implementation of multivariate time series forecasting model introduced in Chen, Y., Kang, Y., Chen, Y., & Wang

Flavia Giammarino 21 Dec 19, 2022
In this project we predict the forest cover type using the cartographic variables in the training/test datasets.

Kaggle Competition: Forest Cover Type Prediction In this project we predict the forest cover type (the predominant kind of tree cover) using the carto

Marianne Joy Leano 1 Mar 15, 2022
Interactive Visualization to empower domain experts to align ML model behaviors with their knowledge.

An interactive visualization system designed to helps domain experts responsibly edit Generalized Additive Models (GAMs). For more information, check

InterpretML 83 Jan 04, 2023
A semantic segmentation toolbox based on PyTorch

Introduction vedaseg is an open source semantic segmentation toolbox based on PyTorch. Features Modular Design We decompose the semantic segmentation

407 Dec 15, 2022
Partial implementation of ODE-GAN technique from the paper Training Generative Adversarial Networks by Solving Ordinary Differential Equations

ODE GAN (Prototype) in PyTorch Partial implementation of ODE-GAN technique from the paper Training Generative Adversarial Networks by Solving Ordinary

Somshubra Majumdar 15 Feb 10, 2022