Automatically remove the mosaics in images and videos, or add mosaics to them.

Overview


DeepMosaics

English | 中文
You can use it to automatically remove the mosaics in images and videos, or add mosaics to them.
This project is based on "semantic segmentation" and "Image-to-Image Translation".
Try it at this website!

Examples

image

origin auto add mosaic auto clean mosaic
image image image
image image image
mosaic image DeepCreamPy ours
image image image
image image image
  • Style Transfer
origin to Van Gogh to winter
image image image

An interesting example:Ricardo Milos to cat

Run DeepMosaics

You can either run DeepMosaics via a pre-built binary package, or from source.

Try it on web

You can simply try to remove the mosaic on the face at this website.

Pre-built binary package

For Windows, we bulid a GUI version for easy testing.
Download this version, and a pre-trained model via [Google Drive] [百度云,提取码1x0a]

image
Attentions:

  • Requires Windows_x86_64, Windows10 is better.
  • Different pre-trained models are suitable for different effects.[Introduction to pre-trained models]
  • Run time depends on computers performance (GPU version has better performance but requires CUDA to be installed).
  • If output video cannot be played, you can try with potplayer.
  • GUI version updates slower than source.

Run From Source

Prerequisites

Dependencies

This code depends on opencv-python, torchvision available via pip install.

Clone this repo

git clone https://github.com/HypoX64/DeepMosaics.git
cd DeepMosaics

Get Pre-Trained Models

You can download pre_trained models and put them into './pretrained_models'.
[Google Drive] [百度云,提取码1x0a]
[Introduction to pre-trained models]

In order to add/remove mosaic, there must be a model file mosaic_position.pth at ./pretrained_models/mosaic/mosaic_position.pth

Install dependencies

(Optional) Create a virtual environment

virtualenv mosaic
source mosaic/bin/activate

Then install the dependencies

pip install -r requirements.txt

If you can not build scikit-image, running export CFLAGS='-Wno-implicit-function-declaration then try to rebuild.

Simple Example

  • Add Mosaic (output media will be saved in './result')
python deepmosaic.py --media_path ./imgs/ruoruo.jpg --model_path ./pretrained_models/mosaic/add_face.pth --gpu_id 0
  • Clean Mosaic (output media will save in './result')
python deepmosaic.py --media_path ./result/ruoruo_add.jpg --model_path ./pretrained_models/mosaic/clean_face_HD.pth --gpu_id 0

If you see the error Please check mosaic_position_model_path!, check if there is a model file named mosaic_position.pth at ./pretrained_models/mosaic/mosaic_position.pth

More Parameters

If you want to test other images or videos, please refer to this file.
[options_introduction.md]

Training With Your Own Dataset

If you want to train with your own dataset, please refer to training_with_your_own_dataset.md

Acknowledgements

This code borrows heavily from [pytorch-CycleGAN-and-pix2pix] [Pytorch-UNet] [pix2pixHD] [BiSeNet] [DFDNet] [GFRNet_pytorch_new].

Comments
  • 运行错误

    运行错误

    Clean temp... segment parameters: 12.4M netG parameters: 11.38M Clean Mosaic: C:\Users\OneRains\Desktop\mosaic.png --------------------ERROR-------------------- --------------Environment-------------- DeepMosaics: 0.4.0 Python: 3.6.9 |Anaconda, Inc.| (default, Jul 30 2019, 14:00:49) [MSC v.1915 64 bit (AMD64)] Pytorch: 0.4.1 OpenCV: 4.1.0 Platform: Windows-10-10.0.18362-SP0 --------------BUG-------------- Error Type: <class 'cv2.error'> OpenCV(4.1.0) c:\projects\opencv-python\opencv\modules\imgproc\src\filterengine.hpp:363: error: (-215:Assertion failed) anchor.inside(Rect(0, 0, ksize.width, ksize.height)) in function 'cv::normalizeAnchor'

    <FrameSummary file deepmosaic.py, line 64 in > <FrameSummary file deepmosaic.py, line 40 in main> <FrameSummary file cores\core.py, line 169 in cleanmosaic_img> <FrameSummary file models\runmodel.py, line 75 in get_mosaic_position> <FrameSummary file util\image_processing.py, line 201 in mask_threshold> Please press any key to exit.

    bug todo 
    opened by OneRain233 7
  • Update dependency version and docs

    Update dependency version and docs

    1. Update readme.md to include:
      • How to install dependencies
      • Troubleshoot issue Please check mosaic_position_model_path!
      • Troubleshoot issue when building scikit-image
    2. Bump version of numpy to 1.19.3 to solve conflict with opencv_python (openvc_python require numpy >=1.19.3)
    opened by conanak99 5
  • i got error and i don't know why

    i got error and i don't know why

    --------------------ERROR-------------------- Error Type: <class 'UnboundLocalError'> local variable 'img_fake' referenced before assignment <FrameSummary file deepmosaic.py, line 66 in > <FrameSummary file deepmosaic.py, line 45 in main> <FrameSummary file D:\DeepMosaics-master\cores\core.py, line 142 in cleanmosaic_video_byframe> Please press any key to exit.

    i run the program on windows 10 with newest python and got this error.

    i don't know how to fix it.

    opened by skyzombie89 5
  • Find mosaic location error: too many values to unpack (expected 2)

    Find mosaic location error: too many values to unpack (expected 2)

    Hi, I'm getting following error when running on python 3.6. Do you know what could cause the issue? Thanks

    frame=26508 fps=194 q=0.0 Lsize=N/A time=00:14:43.60 bitrate=N/A dup=4 drop=0 speed=6.45x
    video:1418716kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
    Find mosaic location:
    --------------------ERROR--------------------
    Error Type: <class 'ValueError'>
    too many values to unpack (expected 2)
    <FrameSummary file deepmosaic.py, line 64 in <module>>
    <FrameSummary file deepmosaic.py, line 43 in main>
    <FrameSummary file C:\temp\DeepMosaics-master\cores\core.py, line 233 in cleanmosaic_video_fusion>
    <FrameSummary file C:\temp\DeepMosaics-master\cores\core.py, line 144 in get_mosaic_positions>
    <FrameSummary file C:\temp\DeepMosaics-master\models\runmodel.py, line 77 in get_mosaic_position>
    <FrameSummary file C:\temp\DeepMosaics-master\util\image_processing.py, line 149 in find_mostlikely_ROI>
    Please press any key to exit.
    
    opened by j4m3s0z 4
  • Can't Detect Mosaic

    Can't Detect Mosaic

    I just tested out on Google Colab with image sample on readme (white cloth girl), and result is still the same (blurred face)

    Tested on: Google Colab
    Runtime: GPU
    Cuda: True
    

    More information & screenshot here

    opened by BroNils 4
  • JSONDecodeError when running from Source

    JSONDecodeError when running from Source

    After attempting to change the ffmpeg parameters in the pre build version, unsuccessful, i am now trying to run DeepMasaics from source.

    So far i stuck to the installation guide on the front page, i cloned the repository and installed all the dependencies. When im trying to run it from the command line (i'm using Power Shell) i get this error: --------------BUG-------------- Error Type: <class 'json.decoder.JSONDecodeError'> Expecting value: line 1 column 1 (char 0) <FrameSummary file V:\DeepMosaics\deepmosaic.py, line 77 in <module>> <FrameSummary file V:\DeepMosaics\deepmosaic.py, line 51 in main> <FrameSummary file V:\DeepMosaics\cores\clean.py, line 169 in cleanmosaic_video_fusion> <FrameSummary file V:\DeepMosaics\cores\init.py, line 8 in video_init> <FrameSummary file V:\DeepMosaics\util\ffmpeg.py, line 63 in get_video_infos> <FrameSummary file C:\Users\First User\AppData\Local\Programs\Python\Python39\lib\json\__init__.py, line 346 in loads> <FrameSummary file C:\Users\First User\AppData\Local\Programs\Python\Python39\lib\json\decoder.py, line 337 in decode> <FrameSummary file C:\Users\First User\AppData\Local\Programs\Python\Python39\lib\json\decoder.py, line 355 in raw_decode> Please press any key to exit.

    This is what i have installed: (im Running Windows 10) --------------Environment-------------- DeepMosaics: 0.5.1 Python: 3.9.4 (tags/v3.9.4:1f2e308, Apr 6 2021, 13:40:21) [MSC v.1928 64 bit (AMD64)] Pytorch: 1.10.0+cu102 OpenCV: 4.5.5

    This is the command im running from within the DeepMosaics Folder: python deepmosaic.py --mode clean --media_path V:\test.mp4 --model_path V:\DeepMosaics\pretrained_models\clean_youknow_video.pth --fps 0 --gpu_id 0 -ss 00:00:00 -t 00:00:00 --result_dir V:\DeepResult\ --mask_threshold 64 --tempimage_type png

    To me it looks like it cant get useful data from the video, but its a bog standard h264 encoded .mp4 (SD resolution, constant frame rate)

    opened by Lukas-52 2
  • [Type of Generator error!] Test error occur when testing model trained with pix2pixHD

    [Type of Generator error!] Test error occur when testing model trained with pix2pixHD

    Type of Generator error! Please press any key to exit.

    If set --netG to be HD, the error is as follows: segment parameters: 12.4M --------------------ERROR-------------------- --------------Environment-------------- DeepMosaics: 0.5.0 Python: 3.7.5 (default, Nov 7 2019, 10:50:52) [GCC 8.3.0] Pytorch: 1.6.0 OpenCV: 4.5.1 Platform: Linux-5.4.0-65-generic-x86_64-with-Ubuntu-18.04-bionic --------------BUG-------------- Error Type: <class 'RuntimeError'> Error(s) in loading state_dict for BiSeNet: Missing key(s) in state_dict: "saptial_path.convblock1.conv1.weight", "saptial_path.convblock1.bn.weight", "saptial_path.convblock1.bn.bias", "saptial_path.convblock1.bn.running_mean", "saptial_path.convblock1.bn.running_var", "saptial_path.convblock2.conv1.weight", "saptial_path.convblock2.bn.weight", "saptial_path.convblock2.bn.bias", "saptial_path.convblock2.bn.running_mean", "saptial_path.convblock2.bn.running_var", "saptial_path.convblock3.conv1.weight", "saptial_path.convblock3.bn.weight", "saptial_path.convblock3.bn.bias", "saptial_path.convblock3.bn.running_mean", "saptial_path.convblock3.bn.running_var", "context_path.features.conv1.weight", "context_path.features.bn1.weight", "context_path.features.bn1.bias", "context_path.features.bn1.running_mean", "context_path.features.bn1.running_var", "context_path.features.layer1.0.conv1.weight", "context_path.features.layer1.0.bn1.weight", "context_path.features.layer1.0.bn1.bias", "context_path.features.layer1.0.bn1.running_mean", "context_path.features.layer1.0.bn1.running_var", "context_path.features.layer1.0.conv2.weight", "context_path.features.layer1.0.bn2.weight", "context_path.features.layer1.0.bn2.bias", "context_path.features.layer1.0.bn2.running_mean", "context_path.features.layer1.0.bn2.running_var", "context_path.features.layer1.1.conv1.weight", "context_path.features.layer1.1.bn1.weight", "context_path.features.layer1.1.bn1.bias", "context_path.features.layer1.1.bn1.running_mean", "context_path.features.layer1.1.bn1.running_var", "context_path.features.layer1.1.conv2.weight", "context_path.features.layer1.1.bn2.weight", "context_path.features.layer1.1.bn2.bias", "context_path.features.layer1.1.bn2.running_mean", "context_path.features.layer1.1.bn2.running_var", "context_path.features.layer2.0.conv1.weight", "context_path.features.layer2.0.bn1.weight", "context_path.features.layer2.0.bn1.bias", "context_path.features.layer2.0.bn1.running_mean", "context_path.features.layer2.0.bn1.running_var", "context_path.features.layer2.0.conv2.weight", "context_path.features.layer2.0.bn2.weight", "context_path.features.layer2.0.bn2.bias", "context_path.features.layer2.0.bn2.running_mean", "context_path.features.layer2.0.bn2.running_var", "context_path.features.layer2.0.downsample.0.weight", "context_path.features.layer2.0.downsample.1.weight", "context_path.features.layer2.0.downsample.1.bias", "context_path.features.layer2.0.downsample.1.running_mean", "context_path.features.layer2.0.downsample.1.running_var", "context_path.features.layer2.1.conv1.weight", "context_path.features.layer2.1.bn1.weight", "context_path.features.layer2.1.bn1.bias", "context_path.features.layer2.1.bn1.running_mean", "context_path.features.layer2.1.bn1.running_var", "context_path.features.layer2.1.conv2.weight", "context_path.features.layer2.1.bn2.weight", "context_path.features.layer2.1.bn2.bias", "context_path.features.layer2.1.bn2.running_mean", "context_path.features.layer2.1.bn2.running_var", "context_path.features.layer3.0.conv1.weight", "context_path.features.layer3.0.bn1.weight", "context_path.features.layer3.0.bn1.bias", "context_path.features.layer3.0.bn1.running_mean", "context_path.features.layer3.0.bn1.running_var", "context_path.features.layer3.0.conv2.weight", "context_path.features.layer3.0.bn2.weight", "context_path.features.layer3.0.bn2.bias", "context_path.features.layer3.0.bn2.running_mean", "context_path.features.layer3.0.bn2.running_var", "context_path.features.layer3.0.downsample.0.weight", "context_path.features.layer3.0.downsample.1.weight", "context_path.features.layer3.0.downsample.1.bias", "context_path.features.layer3.0.downsample.1.running_mean", "context_path.features.layer3.0.downsample.1.running_var", "context_path.features.layer3.1.conv1.weight", "context_path.features.layer3.1.bn1.weight", "context_path.features.layer3.1.bn1.bias", "context_path.features.layer3.1.bn1.running_mean", "context_path.features.layer3.1.bn1.running_var", "context_path.features.layer3.1.conv2.weight", "context_path.features.layer3.1.bn2.weight", "context_path.features.layer3.1.bn2.bias", "context_path.features.layer3.1.bn2.running_mean", "context_path.features.layer3.1.bn2.running_var", "context_path.features.layer4.0.conv1.weight", "context_path.features.layer4.0.bn1.weight", "context_path.features.layer4.0.bn1.bias", "context_path.features.layer4.0.bn1.running_mean", "context_path.features.layer4.0.bn1.running_var", "context_path.features.layer4.0.conv2.weight", "context_path.features.layer4.0.bn2.weight", "context_path.features.layer4.0.bn2.bias", "context_path.features.layer4.0.bn2.running_mean", "context_path.features.layer4.0.bn2.running_var", "context_path.features.layer4.0.downsample.0.weight", "context_path.features.layer4.0.downsample.1.weight", "context_path.features.layer4.0.downsample.1.bias", "context_path.features.layer4.0.downsample.1.running_mean", "context_path.features.layer4.0.downsample.1.running_var", "context_path.features.layer4.1.conv1.weight", "context_path.features.layer4.1.bn1.weight", "context_path.features.layer4.1.bn1.bias", "context_path.features.layer4.1.bn1.running_mean", "context_path.features.layer4.1.bn1.running_var", "context_path.features.layer4.1.conv2.weight", "context_path.features.layer4.1.bn2.weight", "context_path.features.layer4.1.bn2.bias", "context_path.features.layer4.1.bn2.running_mean", "context_path.features.layer4.1.bn2.running_var", "context_path.features.fc.weight", "context_path.features.fc.bias", "context_path.conv1.weight", "context_path.bn1.weight", "context_path.bn1.bias", "context_path.bn1.running_mean", "context_path.bn1.running_var", "context_path.layer1.0.conv1.weight", "context_path.layer1.0.bn1.weight", "context_path.layer1.0.bn1.bias", "context_path.layer1.0.bn1.running_mean", "context_path.layer1.0.bn1.running_var", "context_path.layer1.0.conv2.weight", "context_path.layer1.0.bn2.weight", "context_path.layer1.0.bn2.bias", "context_path.layer1.0.bn2.running_mean", "context_path.layer1.0.bn2.running_var", "context_path.layer1.1.conv1.weight", "context_path.layer1.1.bn1.weight", "context_path.layer1.1.bn1.bias", "context_path.layer1.1.bn1.running_mean", "context_path.layer1.1.bn1.running_var", "context_path.layer1.1.conv2.weight", "context_path.layer1.1.bn2.weight", "context_path.layer1.1.bn2.bias", "context_path.layer1.1.bn2.running_mean", "context_path.layer1.1.bn2.running_var", "context_path.layer2.0.conv1.weight", "context_path.layer2.0.bn1.weight", "context_path.layer2.0.bn1.bias", "context_path.layer2.0.bn1.running_mean", "context_path.layer2.0.bn1.running_var", "context_path.layer2.0.conv2.weight", "context_path.layer2.0.bn2.weight", "context_path.layer2.0.bn2.bias", "context_path.layer2.0.bn2.running_mean", "context_path.layer2.0.bn2.running_var", "context_path.layer2.0.downsample.0.weight", "context_path.layer2.0.downsample.1.weight", "context_path.layer2.0.downsample.1.bias", "context_path.layer2.0.downsample.1.running_mean", "context_path.layer2.0.downsample.1.running_var", "context_path.layer2.1.conv1.weight", "context_path.layer2.1.bn1.weight", "context_path.layer2.1.bn1.bias", "context_path.layer2.1.bn1.running_mean", "context_path.layer2.1.bn1.running_var", "context_path.layer2.1.conv2.weight", "context_path.layer2.1.bn2.weight", "context_path.layer2.1.bn2.bias", "context_path.layer2.1.bn2.running_mean", "context_path.layer2.1.bn2.running_var", "context_path.layer3.0.conv1.weight", "context_path.layer3.0.bn1.weight", "context_path.layer3.0.bn1.bias", "context_path.layer3.0.bn1.running_mean", "context_path.layer3.0.bn1.running_var", "context_path.layer3.0.conv2.weight", "context_path.layer3.0.bn2.weight", "context_path.layer3.0.bn2.bias", "context_path.layer3.0.bn2.running_mean", "context_path.layer3.0.bn2.running_var", "context_path.layer3.0.downsample.0.weight", "context_path.layer3.0.downsample.1.weight", "context_path.layer3.0.downsample.1.bias", "context_path.layer3.0.downsample.1.running_mean", "context_path.layer3.0.downsample.1.running_var", "context_path.layer3.1.conv1.weight", "context_path.layer3.1.bn1.weight", "context_path.layer3.1.bn1.bias", "context_path.layer3.1.bn1.running_mean", "context_path.layer3.1.bn1.running_var", "context_path.layer3.1.conv2.weight", "context_path.layer3.1.bn2.weight", "context_path.layer3.1.bn2.bias", "context_path.layer3.1.bn2.running_mean", "context_path.layer3.1.bn2.running_var", "context_path.layer4.0.conv1.weight", "context_path.layer4.0.bn1.weight", "context_path.layer4.0.bn1.bias", "context_path.layer4.0.bn1.running_mean", "context_path.layer4.0.bn1.running_var", "context_path.layer4.0.conv2.weight", "context_path.layer4.0.bn2.weight", "context_path.layer4.0.bn2.bias", "context_path.layer4.0.bn2.running_mean", "context_path.layer4.0.bn2.running_var", "context_path.layer4.0.downsample.0.weight", "context_path.layer4.0.downsample.1.weight", "context_path.layer4.0.downsample.1.bias", "context_path.layer4.0.downsample.1.running_mean", "context_path.layer4.0.downsample.1.running_var", "context_path.layer4.1.conv1.weight", "context_path.layer4.1.bn1.weight", "context_path.layer4.1.bn1.bias", "context_path.layer4.1.bn1.running_mean", "context_path.layer4.1.bn1.running_var", "context_path.layer4.1.conv2.weight", "context_path.layer4.1.bn2.weight", "context_path.layer4.1.bn2.bias", "context_path.layer4.1.bn2.running_mean", "context_path.layer4.1.bn2.running_var", "attention_refinement_module1.conv.weight", "attention_refinement_module1.conv.bias", "attention_refinement_module1.bn.weight", "attention_refinement_module1.bn.bias", "attention_refinement_module1.bn.running_mean", "attention_refinement_module1.bn.running_var", "attention_refinement_module2.conv.weight", "attention_refinement_module2.conv.bias", "attention_refinement_module2.bn.weight", "attention_refinement_module2.bn.bias", "attention_refinement_module2.bn.running_mean", "attention_refinement_module2.bn.running_var", "supervision1.weight", "supervision1.bias", "supervision2.weight", "supervision2.bias", "feature_fusion_module.convblock.conv1.weight", "feature_fusion_module.convblock.bn.weight", "feature_fusion_module.convblock.bn.bias", "feature_fusion_module.convblock.bn.running_mean", "feature_fusion_module.convblock.bn.running_var", "feature_fusion_module.conv1.weight", "feature_fusion_module.conv1.bias", "feature_fusion_module.conv2.weight", "feature_fusion_module.conv2.bias", "conv.weight", "conv.bias". Unexpected key(s) in state_dict: "model.1.weight", "model.1.bias", "model.4.weight", "model.4.bias", "model.7.weight", "model.7.bias", "model.10.weight", "model.10.bias", "model.13.weight", "model.13.bias", "model.16.conv_block.1.weight", "model.16.conv_block.1.bias", "model.16.conv_block.5.weight", "model.16.conv_block.5.bias", "model.17.conv_block.1.weight", "model.17.conv_block.1.bias", "model.17.conv_block.5.weight", "model.17.conv_block.5.bias", "model.18.conv_block.1.weight", "model.18.conv_block.1.bias", "model.18.conv_block.5.weight", "model.18.conv_block.5.bias", "model.19.conv_block.1.weight", "model.19.conv_block.1.bias", "model.19.conv_block.5.weight", "model.19.conv_block.5.bias", "model.20.conv_block.1.weight", "model.20.conv_block.1.bias", "model.20.conv_block.5.weight", "model.20.conv_block.5.bias", "model.21.conv_block.1.weight", "model.21.conv_block.1.bias", "model.21.conv_block.5.weight", "model.21.conv_block.5.bias", "model.22.conv_block.1.weight", "model.22.conv_block.1.bias", "model.22.conv_block.5.weight", "model.22.conv_block.5.bias", "model.23.conv_block.1.weight", "model.23.conv_block.1.bias", "model.23.conv_block.5.weight", "model.23.conv_block.5.bias", "model.24.conv_block.1.weight", "model.24.conv_block.1.bias", "model.24.conv_block.5.weight", "model.24.conv_block.5.bias", "model.25.weight", "model.25.bias", "model.28.weight", "model.28.bias", "model.31.weight", "model.31.bias", "model.34.weight", "model.34.bias", "model.38.weight", "model.38.bias". <FrameSummary file /mnt/diskc/even/DeepMosaic/deepmosaic.py, line 81 in > <FrameSummary file /mnt/diskc/even/DeepMosaic/deepmosaic.py, line 40 in run> <FrameSummary file /mnt/diskc/even/DeepMosaic/models/loadmodel.py, line 86 in bisenet> <FrameSummary file /home/jaya/.local/lib/python3.7/site-packages/torch/nn/modules/module.py, line 1045 in load_state_dict> Please press any key to exit.

    opened by CaptainEven 2
  • Recommended Hardware or Adjust Parameter for low end GPU

    Recommended Hardware or Adjust Parameter for low end GPU

    I'm running into a CUDA out of memory error. Currently, I'm using i3-6100 with 16GB of memory and a Nvidia GTX 750 that only has 1GB of memory. Is there a way to adjust the parameter for low end GPU? If not, what's the minimum memory required for the GPU?

    opened by Engrish 2
  • class 'RuntimeError'

    class 'RuntimeError'

    I run this in python 3.6.8 and 3.7.10. Got the following error. Do you know why? Error Type: <class 'RuntimeError'> Given groups=1, weight of size [64, 3, 3, 3], expected input[1, 4, 500, 360] to have 3 channels, but got 4 channels instead <FrameSummary file deepmosaic.py, line 66 in > <FrameSummary file deepmosaic.py, line 40 in main> <FrameSummary file cores\core.py, line 112 in cleanmosaic_img> <FrameSummary file models\runmodel.py, line 73 in get_mosaic_position> <FrameSummary file models\runmodel.py, line 13 in run_segment> <FrameSummary file site-packages\torch\nn\modules\module.py, line 477 in call> <FrameSummary file models\BiSeNet_model.py, line 236 in forward> <FrameSummary file site-packages\torch\nn\modules\module.py, line 477 in call> <FrameSummary file models\BiSeNet_model.py, line 122 in forward> <FrameSummary file site-packages\torch\nn\modules\module.py, line 477 in call> <FrameSummary file models\BiSeNet_model.py, line 111 in forward> <FrameSummary file site-packages\torch\nn\modules\module.py, line 477 in call> <FrameSummary file site-packages\torch\nn\modules\conv.py, line 301 in forward> Please press any key to exit.

    opened by nichijou147 2
  • exiting due to unkown error

    exiting due to unkown error

    deep mosiac exits early when rebuilding the video when using google colab. The video being processed has been cut using the windows photo app which might be an issue. The input command is !python3 deepmosaic.py --media_path="/content/DeepMosaics/imgs/MIDD-817(1a).mp4" --model_path="/content/DeepMosaics/pretrained_models/clean_youknow_video.pth" --mode="clean" --no_preview --gpu_id=0 --fps 0 -ss 00:00:00 -t 00:00:00 --mask_threshold 64 and the output is attached below.

    makedir: ./result segment parameters: 12.4M netG parameters: 26.65M Step:1/4 -- Convert video to images Clean temp... ffmpeg version 3.4.8-0ubuntu0.2 Copyright (c) 2000-2020 the FFmpeg developers built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04) configuration: --prefix=/usr --extra-version=0ubuntu0.2 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared libavutil 55. 78.100 / 55. 78.100 libavcodec 57.107.100 / 57.107.100 libavformat 57. 83.100 / 57. 83.100 libavdevice 57. 10.100 / 57. 10.100 libavfilter 6.107.100 / 6.107.100 libavresample 3. 7. 0 / 3. 7. 0 libswscale 4. 8.100 / 4. 8.100 libswresample 2. 9.100 / 2. 9.100 libpostproc 54. 7.100 / 54. 7.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/content/DeepMosaics/imgs/MIDD-817(1a).mp4': Metadata: major_brand : mp42 minor_version : 0 compatible_brands: mp41isom creation_time : 2020-02-12T18:31:23.000000Z Duration: 00:29:00.67, start: 0.000000, bitrate: 3902 kb/s Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv), 1920x1080 [SAR 1:1 DAR 16:9], 3707 kb/s, 30 fps, 30 tbr, 30k tbn, 60 tbc (default) Metadata: creation_time : 2021-07-27T15:07:05.000000Z handler_name : VideoHandler encoder : AVC Coding Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 192 kb/s (default) Metadata: creation_time : 2021-07-27T15:07:05.000000Z handler_name : SoundHandler Stream mapping: Stream #0:1 -> #0:0 (aac (native) -> mp3 (libmp3lame)) Press [q] to stop, [?] for help -async is forwarded to lavfi similarly to -af aresample=async=1:min_hard_comp=0.100000:first_pts=0. Output #0, mp3, to './tmp/DeepMosaics_temp/voice_tmp.mp3': Metadata: major_brand : mp42 minor_version : 0 compatible_brands: mp41isom TSSE : Lavf57.83.100 Stream #0:0(und): Audio: mp3 (libmp3lame), 48000 Hz, stereo, fltp, 320 kb/s (default) Metadata: creation_time : 2021-07-27T15:07:05.000000Z handler_name : SoundHandler encoder : Lavc57.107.100 libmp3lame [aac @ 0x560c0eb5f900] decode_band_types: Input buffer exhausted before END element found Error while decoding stream #0:1: Invalid data found when processing input size= 67997kB time=00:29:00.67 bitrate= 320.0kbits/s speed=41.7x
    video:0kB audio:67996kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.001577% ffmpeg version 3.4.8-0ubuntu0.2 Copyright (c) 2000-2020 the FFmpeg developers built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04) configuration: --prefix=/usr --extra-version=0ubuntu0.2 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared libavutil 55. 78.100 / 55. 78.100 libavcodec 57.107.100 / 57.107.100 libavformat 57. 83.100 / 57. 83.100 libavdevice 57. 10.100 / 57. 10.100 libavfilter 6.107.100 / 6.107.100 libavresample 3. 7. 0 / 3. 7. 0 libswscale 4. 8.100 / 4. 8.100 libswresample 2. 9.100 / 2. 9.100 libpostproc 54. 7.100 / 54. 7.100 Input #0, mov,mp4,m4a,3gp,3g2,mj2, from '/content/DeepMosaics/imgs/MIDD-817(1a).mp4': Metadata: major_brand : mp42 minor_version : 0 compatible_brands: mp41isom creation_time : 2020-02-12T18:31:23.000000Z Duration: 00:29:00.67, start: 0.000000, bitrate: 3902 kb/s Stream #0:0(und): Video: h264 (Main) (avc1 / 0x31637661), yuv420p(tv), 1920x1080 [SAR 1:1 DAR 16:9], 3707 kb/s, 30 fps, 30 tbr, 30k tbn, 60 tbc (default) Metadata: creation_time : 2021-07-27T15:07:05.000000Z handler_name : VideoHandler encoder : AVC Coding Stream #0:1(und): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 192 kb/s (default) Metadata: creation_time : 2021-07-27T15:07:05.000000Z handler_name : SoundHandler Stream mapping: Stream #0:0 -> #0:0 (h264 (native) -> mjpeg (native)) Press [q] to stop, [?] for help [swscaler @ 0x55a76142c000] deprecated pixel format used, make sure you did set range correctly Output #0, image2, to './tmp/DeepMosaics_temp/video2image/output_%06d.jpg': Metadata: major_brand : mp42 minor_version : 0 compatible_brands: mp41isom encoder : Lavf57.83.100 Stream #0:0(und): Video: mjpeg, yuvj420p(pc), 1920x1080 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 30 fps, 30 tbn, 30 tbc (default) Metadata: creation_time : 2021-07-27T15:07:05.000000Z handler_name : VideoHandler encoder : Lavc57.107.100 mjpeg Side data: cpb: bitrate max/min/avg: 0/0/200000 buffer size: 0 vbv_delay: -1 frame=52220 fps= 52 q=0.0 Lsize=N/A time=00:29:00.66 bitrate=N/A speed=1.73x
    video:5994909kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown Step:2/4 -- Find mosaic location 52220/52220 [###################################] 100.00% 00:35:41/00:35:41 Optimize mosaic locations... Step:3/4 -- Clean Mosaic: 52220/52220 [###################################] 100.00% 01:22:43/01:22:43 Step:4/4 -- Convert images to video Traceback (most recent call last): File "/usr/lib/python3.7/multiprocessing/queues.py", line 242, in _feed send_bytes(obj) File "/usr/lib/python3.7/multiprocessing/connection.py", line 200, in send_bytes self._send_bytes(m[offset:offset + size]) File "/usr/lib/python3.7/multiprocessing/connection.py", line 397, in _send_bytes self._send(header) File "/usr/lib/python3.7/multiprocessing/connection.py", line 368, in _send n = write(self._handle, buf) BrokenPipeError: [Errno 32] Broken pipe Traceback (most recent call last): File "/usr/lib/python3.7/multiprocessing/queues.py", line 242, in _feed send_bytes(obj) File "/usr/lib/python3.7/multiprocessing/connection.py", line 200, in send_bytes self._send_bytes(m[offset:offset + size]) File "/usr/lib/python3.7/multiprocessing/connection.py", line 397, in _send_bytes self._send(header) File "/usr/lib/python3.7/multiprocessing/connection.py", line 368, in _send n = write(self._handle, buf) BrokenPipeError: [Errno 32] Broken pipe Traceback (most recent call last): File "/usr/lib/python3.7/multiprocessing/queues.py", line 242, in _feed send_bytes(obj) File "/usr/lib/python3.7/multiprocessing/connection.py", line 200, in send_bytes self._send_bytes(m[offset:offset + size]) File "/usr/lib/python3.7/multiprocessing/connection.py", line 397, in _send_bytes self._send(header) File "/usr/lib/python3.7/multiprocessing/connection.py", line 368, in _send n = write(self._handle, buf) BrokenPipeError: [Errno 32] Broken pipe Traceback (most recent call last): File "/usr/lib/python3.7/multiprocessing/queues.py", line 242, in _feed send_bytes(obj) File "/usr/lib/python3.7/multiprocessing/connection.py", line 200, in send_bytes self._send_bytes(m[offset:offset + size]) File "/usr/lib/python3.7/multiprocessing/connection.py", line 397, in _send_bytes self._send(header) File "/usr/lib/python3.7/multiprocessing/connection.py", line 368, in _send n = write(self._handle, buf) BrokenPipeError: [Errno 32] Broken pipe Exception in thread Thread-52222: Traceback (most recent call last): File "/usr/lib/python3.7/threading.py", line 926, in _bootstrap_inner self.run() File "/usr/lib/python3.7/threading.py", line 870, in run self._target(*self._args, **self._kwargs) File "/content/DeepMosaics/cores/clean.py", line 183, in write_result save_ori,imagepath,img_origin,img_fake,x,y,size = write_pool.get() File "/usr/lib/python3.7/multiprocessing/queues.py", line 94, in get res = self._recv_bytes() File "/usr/lib/python3.7/multiprocessing/connection.py", line 212, in recv_bytes self._check_closed() File "/usr/lib/python3.7/multiprocessing/connection.py", line 136, in _check_closed raise OSError("handle is closed") OSError: handle is closed

    ffmpeg version 3.4.8-0ubuntu0.2 Copyright (c) 2000-2020 the FFmpeg developers built with gcc 7 (Ubuntu 7.5.0-3ubuntu1~18.04) configuration: --prefix=/usr --extra-version=0ubuntu0.2 --toolchain=hardened --libdir=/usr/lib/x86_64-linux-gnu --incdir=/usr/include/x86_64-linux-gnu --enable-gpl --disable-stripping --enable-avresample --enable-avisynth --enable-gnutls --enable-ladspa --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librubberband --enable-librsvg --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwavpack --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzmq --enable-libzvbi --enable-omx --enable-openal --enable-opengl --enable-sdl2 --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libopencv --enable-libx264 --enable-shared libavutil 55. 78.100 / 55. 78.100 libavcodec 57.107.100 / 57.107.100 libavformat 57. 83.100 / 57. 83.100 libavdevice 57. 10.100 / 57. 10.100 libavfilter 6.107.100 / 6.107.100 libavresample 3. 7. 0 / 3. 7. 0 libswscale 4. 8.100 / 4. 8.100 libswresample 2. 9.100 / 2. 9.100 libpostproc 54. 7.100 / 54. 7.100 [mjpeg @ 0x55bac588ca00] EOI missing, emulating Input #0, image2, from './tmp/DeepMosaics_temp/replace_mosaic/output_%06d.jpg': Duration: 00:34:48.64, start: 0.000000, bitrate: N/A Stream #0:0: Video: mjpeg, yuvj420p(pc, bt470bg/unknown/unknown), 1920x1080 [SAR 1:1 DAR 16:9], 25 fps, 25 tbr, 25 tbn, 25 tbc Stream mapping: Stream #0:0 -> #0:0 (mjpeg (native) -> h264 (libx264)) Press [q] to stop, [?] for help [libx264 @ 0x55bac588de00] using SAR=1/1 [libx264 @ 0x55bac588de00] using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX FMA3 BMI2 AVX2 [libx264 @ 0x55bac588de00] profile High, level 4.0 [libx264 @ 0x55bac588de00] 264 - core 152 r2854 e9a5903 - H.264/MPEG-4 AVC codec - Copyleft 2003-2017 - http://www.videolan.org/x264.html - options: cabac=1 ref=3 deblock=1:0:0 analyse=0x3:0x113 me=hex subme=7 psy=1 psy_rd=1.00:0.00 mixed_ref=1 me_range=16 chroma_me=1 trellis=1 8x8dct=1 cqm=0 deadzone=21,11 fast_pskip=1 chroma_qp_offset=-2 threads=3 lookahead_threads=1 sliced_threads=0 nr=0 decimate=1 interlaced=0 bluray_compat=0 constrained_intra=0 bframes=3 b_pyramid=2 b_adapt=1 b_bias=0 direct=1 weightb=1 open_gop=0 weightp=2 keyint=250 keyint_min=25 scenecut=40 intra_refresh=0 rc_lookahead=40 rc=crf mbtree=1 crf=23.0 qcomp=0.60 qpmin=0 qpmax=69 qpstep=4 ip_ratio=1.40 aq=1:1.00 Output #0, mp4, to './tmp/DeepMosaics_temp/video_tmp.mp4': Metadata: encoder : Lavf57.83.100 Stream #0:0: Video: h264 (libx264) (avc1 / 0x31637661), yuvj420p(pc), 1920x1080 [SAR 1:1 DAR 16:9], q=-1--1, 30 fps, 15360 tbn, 30 tbc Metadata: encoder : Lavc57.107.100 libx264 Side data: cpb: bitrate max/min/avg: 0/0/0 buffer size: 0 vbv_delay: -1 frame=52216 fps=6.8 q=-1.0 Lsize= 953084kB time=00:29:00.43 bitrate=4486.0kbits/s speed=0.228x
    video:952465kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: 0.064936% [libx264 @ 0x55bac588de00] frame I:272 Avg QP:19.24 size: 58788 [libx264 @ 0x55bac588de00] frame P:13709 Avg QP:22.37 size: 33376 [libx264 @ 0x55bac588de00] frame B:38235 Avg QP:24.08 size: 13124 [libx264 @ 0x55bac588de00] consecutive B-frames: 1.8% 0.5% 3.4% 94.3% [libx264 @ 0x55bac588de00] mb I I16..4: 31.2% 64.6% 4.2% [libx264 @ 0x55bac588de00] mb P I16..4: 19.8% 31.7% 0.7% P16..4: 27.7% 7.5% 1.8% 0.0% 0.0% skip:10.9% [libx264 @ 0x55bac588de00] mb B I16..4: 4.3% 6.8% 0.1% B16..8: 40.9% 4.0% 0.3% direct: 2.0% skip:41.6% L0:55.9% L1:41.1% BI: 3.0% [libx264 @ 0x55bac588de00] 8x8 transform intra:60.8% inter:90.6% [libx264 @ 0x55bac588de00] coded y,uvDC,uvAC intra: 30.0% 61.4% 4.8% inter: 8.4% 19.9% 0.2% [libx264 @ 0x55bac588de00] i16 v,h,dc,p: 29% 26% 15% 30% [libx264 @ 0x55bac588de00] i8 v,h,dc,ddl,ddr,vr,hd,vl,hu: 32% 24% 31% 2% 2% 2% 2% 2% 2% [libx264 @ 0x55bac588de00] i4 v,h,dc,ddl,ddr,vr,hd,vl,hu: 36% 20% 13% 3% 8% 8% 6% 4% 2% [libx264 @ 0x55bac588de00] i8c dc,h,v,p: 42% 25% 27% 5% [libx264 @ 0x55bac588de00] Weighted P-Frames: Y:0.8% UV:0.7% [libx264 @ 0x55bac588de00] ref P L0: 68.0% 8.8% 17.5% 5.7% 0.0% [libx264 @ 0x55bac588de00] ref B L0: 90.9% 7.4% 1.6% [libx264 @ 0x55bac588de00] ref B L1: 96.0% 4.0% [libx264 @ 0x55bac588de00] kb/s:4482.87 sh: 1: Syntax error: "(" unexpected Clean temp... Finished!

    opened by calvin360 1
  • test video error!

    test video error!

    when I tese my own videos,the error occur in the last step:

    ./tmp/DeepMosaics_temp/voice_tmp.mp3: No such file or directory

    and this error only occured in several of videos! I want to konw the reasons! thank you!

    opened by yanganlan0310 1
  • dependence error on caffe2_detectron_ops_gpu.dll

    dependence error on caffe2_detectron_ops_gpu.dll

    On x64 win10 and 11, I got the follow error even if I only try the CPU version. I have installed VC_redist.x64.exe while win 11 is a clean system.

    [WinError 126] The specified module could not be found. Error loading "C:\Users\vm\Downloads\DeepMosaics_0.5.1_gpu\core\torch\lib\caffe2_detectron_ops_gpu.dll" or one of its dependencies.

    opened by funnymoon-ddsun 1
  • Could not run with video trained models

    Could not run with video trained models

    Please refer the log as following. I've tried both in GUI and shell, given the same error. Other models without 'video' in filename could run properly.

    D:\DeepMosaics_0.5.1_gpu\core\torchvision\__init__.py:26: UserWarning: You are importing torchvision within its own root folder (D:\DeepMosaics_0.5.1_gpu\core). This is not expected to work and may give errors. Please exit the torchvision project source and relaunch your python interpreter.
    segment parameters: 12.4M
    netG parameters: 26.65M
    --------------------ERROR--------------------
    --------------Environment--------------
    DeepMosaics: 0.5.1
    Python: 3.7.3 (default, Apr 24 2019, 15:29:51) [MSC v.1915 64 bit (AMD64)]
    Pytorch: 1.7.1
    OpenCV: 4.1.2
    Platform: Windows-10-10.0.22621-SP0
    --------------BUG--------------
    Error Type: <class 'RuntimeError'>
    Error(s) in loading state_dict for BVDNet:
            Missing key(s) in state_dict: "encoder3d.model.0.weight_orig", "encoder3d.model.0.weight", "encoder3d.model.0.weight_u", "encoder3d.model.0.bias", "encoder3d.model.0.weight_orig", "encoder3d.model.0.weight_u", "encoder3d.model.0.weight_v", "encoder3d.model.2.weight_orig", "encoder3d.model.2.weight", "encoder3d.model.2.weight_u", "encoder3d.model.2.bias", "encoder3d.model.2.weight_orig", "encoder3d.model.2.weight_u", "encoder3d.model.2.weight_v", "encoder3d.model.4.weight_orig", "encoder3d.model.4.weight", "encoder3d.model.4.weight_u", "encoder3d.model.4.bias", "encoder3d.model.4.weight_orig", "encoder3d.model.4.weight_u", "encoder3d.model.4.weight_v", "encoder3d.model.6.weight_orig", "encoder3d.model.6.weight", "encoder3d.model.6.weight_u", "encoder3d.model.6.bias", "encoder3d.model.6.weight_orig", "encoder3d.model.6.weight_u", "encoder3d.model.6.weight_v", "encoder2d.model.1.weight_orig", "encoder2d.model.1.weight", "encoder2d.model.1.weight_u", "encoder2d.model.1.bias", "encoder2d.model.1.weight_orig", "encoder2d.model.1.weight_u", "encoder2d.model.1.weight_v", "encoder2d.model.4.weight_orig", "encoder2d.model.4.weight", "encoder2d.model.4.weight_u", "encoder2d.model.4.bias", "encoder2d.model.4.weight_orig", "encoder2d.model.4.weight_u", "encoder2d.model.4.weight_v", "encoder2d.model.7.weight_orig", "encoder2d.model.7.weight", "encoder2d.model.7.weight_u", "encoder2d.model.7.bias", "encoder2d.model.7.weight_orig", "encoder2d.model.7.weight_u", "encoder2d.model.7.weight_v", "encoder2d.model.10.weight_orig", "encoder2d.model.10.weight", "encoder2d.model.10.weight_u", "encoder2d.model.10.bias", "encoder2d.model.10.weight_orig", "encoder2d.model.10.weight_u", "encoder2d.model.10.weight_v", "blocks.0.conv_block.1.weight_orig", "blocks.0.conv_block.1.weight", "blocks.0.conv_block.1.weight_u", "blocks.0.conv_block.1.bias", "blocks.0.conv_block.1.weight_orig", "blocks.0.conv_block.1.weight_u", "blocks.0.conv_block.1.weight_v", "blocks.0.conv_block.4.weight_orig", "blocks.0.conv_block.4.weight", "blocks.0.conv_block.4.weight_u", "blocks.0.conv_block.4.bias", "blocks.0.conv_block.4.weight_orig", "blocks.0.conv_block.4.weight_u", "blocks.0.conv_block.4.weight_v", "blocks.1.conv_block.1.weight_orig", "blocks.1.conv_block.1.weight", "blocks.1.conv_block.1.weight_u", "blocks.1.conv_block.1.bias", "blocks.1.conv_block.1.weight_orig", "blocks.1.conv_block.1.weight_u", "blocks.1.conv_block.1.weight_v", "blocks.1.conv_block.4.weight_orig", "blocks.1.conv_block.4.weight", "blocks.1.conv_block.4.weight_u", "blocks.1.conv_block.4.bias", "blocks.1.conv_block.4.weight_orig", "blocks.1.conv_block.4.weight_u", "blocks.1.conv_block.4.weight_v", "blocks.2.conv_block.1.weight_orig", "blocks.2.conv_block.1.weight", "blocks.2.conv_block.1.weight_u", "blocks.2.conv_block.1.bias", "blocks.2.conv_block.1.weight_orig", "blocks.2.conv_block.1.weight_u", "blocks.2.conv_block.1.weight_v", "blocks.2.conv_block.4.weight_orig", "blocks.2.conv_block.4.weight", "blocks.2.conv_block.4.weight_u", "blocks.2.conv_block.4.bias", "blocks.2.conv_block.4.weight_orig", "blocks.2.conv_block.4.weight_u", "blocks.2.conv_block.4.weight_v", "blocks.3.conv_block.1.weight_orig", "blocks.3.conv_block.1.weight", "blocks.3.conv_block.1.weight_u", "blocks.3.conv_block.1.bias", "blocks.3.conv_block.1.weight_orig", "blocks.3.conv_block.1.weight_u", "blocks.3.conv_block.1.weight_v", "blocks.3.conv_block.4.weight_orig", "blocks.3.conv_block.4.weight", "blocks.3.conv_block.4.weight_u", "blocks.3.conv_block.4.bias", "blocks.3.conv_block.4.weight_orig", "blocks.3.conv_block.4.weight_u", "blocks.3.conv_block.4.weight_v", "decoder.0.convup.2.weight_orig", "decoder.0.convup.2.weight", "decoder.0.convup.2.weight_u", "decoder.0.convup.2.bias", "decoder.0.convup.2.weight_orig", "decoder.0.convup.2.weight_u", "decoder.0.convup.2.weight_v", "decoder.1.convup.2.weight_orig", "decoder.1.convup.2.weight", "decoder.1.convup.2.weight_u", "decoder.1.convup.2.bias", "decoder.1.convup.2.weight_orig", "decoder.1.convup.2.weight_u", "decoder.1.convup.2.weight_v", "decoder.2.convup.2.weight_orig", "decoder.2.convup.2.weight", "decoder.2.convup.2.weight_u", "decoder.2.convup.2.bias", "decoder.2.convup.2.weight_orig", "decoder.2.convup.2.weight_u", "decoder.2.convup.2.weight_v", "decoder.4.weight", "decoder.4.bias".
            Unexpected key(s) in state_dict: "encoder_2d.model.1.weight", "encoder_2d.model.1.bias", "encoder_2d.model.5.weight", "encoder_2d.model.5.bias", "encoder_2d.model.9.weight", "encoder_2d.model.9.bias", "encoder_2d.model.13.weight", "encoder_2d.model.13.bias", "encoder_2d.model.17.weight", "encoder_2d.model.17.bias", "encoder_3d.inconv.conv.0.weight", "encoder_3d.inconv.conv.0.bias", "encoder_3d.down1.conv.0.weight", "encoder_3d.down1.conv.0.bias", "encoder_3d.down2.conv.0.weight", "encoder_3d.down2.conv.0.bias", "encoder_3d.down3.conv.0.weight", "encoder_3d.down3.conv.0.bias", "encoder_3d.down4.conv.0.weight", "encoder_3d.down4.conv.0.bias", "decoder_2d.model.0.conv_block.1.weight", "decoder_2d.model.0.conv_block.1.bias", "decoder_2d.model.0.conv_block.5.weight", "decoder_2d.model.0.conv_block.5.bias", "decoder_2d.model.1.conv_block.1.weight", "decoder_2d.model.1.conv_block.1.bias", "decoder_2d.model.1.conv_block.5.weight", "decoder_2d.model.1.conv_block.5.bias", "decoder_2d.model.2.conv_block.1.weight", "decoder_2d.model.2.conv_block.1.bias", "decoder_2d.model.2.conv_block.5.weight", "decoder_2d.model.2.conv_block.5.bias", "decoder_2d.model.3.conv_block.1.weight", "decoder_2d.model.3.conv_block.1.bias", "decoder_2d.model.3.conv_block.5.weight", "decoder_2d.model.3.conv_block.5.bias", "decoder_2d.model.4.conv_block.1.weight", "decoder_2d.model.4.conv_block.1.bias", "decoder_2d.model.4.conv_block.5.weight", "decoder_2d.model.4.conv_block.5.bias", "decoder_2d.model.5.conv_block.1.weight", "decoder_2d.model.5.conv_block.1.bias", "decoder_2d.model.5.conv_block.5.weight", "decoder_2d.model.5.conv_block.5.bias", "decoder_2d.model.6.conv_block.1.weight", "decoder_2d.model.6.conv_block.1.bias", "decoder_2d.model.6.conv_block.5.weight", "decoder_2d.model.6.conv_block.5.bias", "decoder_2d.model.7.conv_block.1.weight", "decoder_2d.model.7.conv_block.1.bias", "decoder_2d.model.7.conv_block.5.weight", "decoder_2d.model.7.conv_block.5.bias", "decoder_2d.model.8.conv_block.1.weight", "decoder_2d.model.8.conv_block.1.bias", "decoder_2d.model.8.conv_block.5.weight", "decoder_2d.model.8.conv_block.5.bias", "decoder_2d.model.9.weight", "decoder_2d.model.9.bias", "decoder_2d.model.12.weight", "decoder_2d.model.12.bias", "decoder_2d.model.15.weight", "decoder_2d.model.15.bias", "decoder_2d.model.18.weight", "decoder_2d.model.18.bias", "decoder_2d.model.22.weight", "decoder_2d.model.22.bias", "merge1.conv.1.weight", "merge1.conv.1.bias".
    <FrameSummary file deepmosaic.py, line 77 in <module>>
    <FrameSummary file deepmosaic.py, line 41 in main>
    <FrameSummary file models\loadmodel.py, line 56 in video>
    <FrameSummary file torch\nn\modules\module.py, line 1052 in load_state_dict>
    Please press any key to exit.
    
    opened by vxfranky 0
  • GPU source code never finds mosaic

    GPU source code never finds mosaic

    I have using the CPU GUI version .exe file that works, but so slow. I have wanted to get the GPU command prompt version to work as the GUI version.exe kept getting errors that DeepMosaics said where too complicated on another post. After getting the command prompt version to work, the video output is the same as the input. There is no error while running the source code.

    While using source code GPU version when step two starts the python program goes to not responding. After 15 minutes it will work again, but never finds any mosaic. The program runs without any errors. The only thing I can think off is my CPU is too weak and is causing the not responding on start of step two.

    opened by heartingNinja 0
  • Source Code GPU not seeing mosaic

    Source Code GPU not seeing mosaic

    I have using the CPU GUI version .exe file that works, but so slow. I have wanted to get the GPU command prompt version to work as the GUI version.exe kept getting errors that DeepMosaics said where too complicated on another post. After getting the command prompt version to work, the video output is the same as the input. There is no error while running the source code.

    While using source code GPU version when step two starts the python program goes to not responding. After 15 minutes it will work again but never finds the mosaic. The only thing I can think off is my CPU is too weak and is causing the not responding on start of step two.

    opened by heartingNinja 0
  • Regarding the empty train_A and train_B folders generated when creating the Clean mosaic dataset. Solution.

    Regarding the empty train_A and train_B folders generated when creating the Clean mosaic dataset. Solution.

    make_pix2pix_dataset.pyを実行すると発生しました。このpyファイルを調べてみると、108行目から152行目まで余分なインデントが1つあることがわかりました。これをインデント1つ分削除して修正します。

    また、私の場合、最初はインデントを消しても画像が生成されなかったので、品質オプションを20にしたところ、train_Aフォルダにモザイクをかけた画像が、train_Bフォルダに元画像が格納されるようになりました。

    おそらく、私の動画の画質が悪すぎて、フィルタリングされていたのでしょう。やっと解決しましたー。 ちなみにimage_processing.pyを見ると、(quality)>20が正常で、argオプションのデフォルト設定は40と書いてあります。

    opened by mossan0101 0
Releases(v0.5.1)
Owner
Hypo
An indie developer / Focus on DL&DSP.
Hypo
Detect textlines in document images

Textline Detection Detect textlines in document images Introduction This tool performs border, region and textline detection from document image data

QURATOR-SPK 70 Jun 30, 2022
A python scripts that uses 3 different feature extraction methods such as SIFT, SURF and ORB to find a book in a video clip and project trailer of a movie based on that book, on to it.

A python scripts that uses 3 different feature extraction methods such as SIFT, SURF and ORB to find a book in a video clip and project trailer of a movie based on that book, on to it.

tooraj taraz 3 Feb 10, 2022
Total Text Dataset. It consists of 1555 images with more than 3 different text orientations: Horizontal, Multi-Oriented, and Curved, one of a kind.

Total-Text-Dataset (Official site) Updated on April 29, 2020 (Detection leaderboard is updated - highlighted E2E methods. Thank you shine-lcy.) Update

Chee Seng Chan 671 Dec 27, 2022
deployment of a hybrid model for automatic weapon detection/ anomaly detection for surveillance applications

Automatic Weapon Detection Deployment of a hybrid model for automatic weapon detection/ anomaly detection for surveillance applications. Loved the pro

Janhavi 4 Mar 04, 2022
Image augmentation library in Python for machine learning.

Augmentor is an image augmentation library in Python for machine learning. It aims to be a standalone library that is platform and framework independe

Marcus D. Bloice 4.8k Jan 04, 2023
question‘s area recognition using image processing and regular expression

======================================== Paper-Question-recognition ======================================== question‘s area recognition using image p

Yuta Mizuki 7 Dec 27, 2021
PyNeuro is designed to connect NeuroSky's MindWave EEG device to Python and provide Callback functionality to provide data to your application in real time.

PyNeuro PyNeuro is designed to connect NeuroSky's MindWave EEG device to Python and provide Callback functionality to provide data to your application

Zach Wang 45 Dec 30, 2022
Generates a message from the infamous Jerma Impostor image

Generate your very own jerma sus imposter message. Modes: Default Mode: Only supports the characters " ", !, a, b, c, d, e, h, i, m, n, o, p, q, r, s,

Giorno420 1 Oct 27, 2022
Let's explore how we can extract text from forms

Form Segmentation Let's explore how we can extract text from any forms / scanned pages. Objectives The goal is to find an algorithm that can extract t

Philip Doxakis 42 Jun 05, 2022
CRAFT-Pyotorch:Character Region Awareness for Text Detection Reimplementation for Pytorch

CRAFT-Reimplementation Note:If you have any problems, please comment. Or you can join us weChat group. The QR code will update in issues #49 . Reimple

453 Dec 28, 2022
kaldi-asr/kaldi is the official location of the Kaldi project.

Kaldi Speech Recognition Toolkit To build the toolkit: see ./INSTALL. These instructions are valid for UNIX systems including various flavors of Linux

Kaldi 12.3k Jan 05, 2023
OCR engine for all the languages

Description kraken is a turn-key OCR system optimized for historical and non-Latin script material. kraken's main features are: Fully trainable layout

431 Jan 04, 2023
Convolutional Recurrent Neural Networks(CRNN) for Scene Text Recognition

CRNN_Tensorflow This is a TensorFlow implementation of a Deep Neural Network for scene text recognition. It is mainly based on the paper "An End-to-En

MaybeShewill-CV 1000 Dec 27, 2022
graph learning code for ogb

The final code for OGB Installation Requirements: ogb=1.3.1 torch=1.7.0 torch-geometric=1.7.0 torch-scatter=2.0.6 torch-sparse=0.6.9 Baseline models T

PierreHao 20 Nov 10, 2022
Code for generating synthetic text images as described in "Synthetic Data for Text Localisation in Natural Images", Ankush Gupta, Andrea Vedaldi, Andrew Zisserman, CVPR 2016.

SynthText Code for generating synthetic text images as described in "Synthetic Data for Text Localisation in Natural Images", Ankush Gupta, Andrea Ved

Ankush Gupta 1.8k Dec 28, 2022
Official implementation of "An Image is Worth 16x16 Words, What is a Video Worth?" (2021 paper)

An Image is Worth 16x16 Words, What is a Video Worth? paper Official PyTorch Implementation Gilad Sharir, Asaf Noy, Lihi Zelnik-Manor DAMO Academy, Al

213 Nov 12, 2022
Text recognition (optical character recognition) with deep learning methods.

What Is Wrong With Scene Text Recognition Model Comparisons? Dataset and Model Analysis | paper | training and evaluation data | failure cases and cle

Clova AI Research 3.2k Jan 04, 2023
~1000 book pages + OpenCV + python = page regions identified as paragraphs, lines, images, captions, etc.

cosc428-structor I had an open-ended Computer Vision assignment to complete, and an out-of-copyright book that I wanted to turn into an ebook. Convent

Chad Oliver 45 Dec 06, 2022
OCR-D-compliant page segmentation

ocrd_segment This repository aims to provide a number of OCR-D-compliant processors for layout analysis and evaluation. Installation In your virtual e

OCR-D 59 Sep 10, 2022
Histogram specification using openCV in python .

histogram specification using openCV in python . Have to input miu and sigma to draw gausssian distribution which will be used to map the input image . Example input can be miu = 128 sigma = 30

Tamzid hasan 6 Nov 17, 2021