Official implementation of SynthTIGER (Synthetic Text Image GEneratoR) ICDAR 2021

Overview

🐯 SynthTIGER: Synthetic Text Image GEneratoR

Official implementation of SynthTIGER | Paper | Datasets

Moonbin Yim1, Yoonsik Kim1, Han-cheol Cho1, Sungrae Park2

1 Clova AI Research, NAVER Corp.

2 Upstage AI Research





Contents

Updates

Datasets

SynthTIGER is available for download at google drive.

synthtiger_v1.0.zip (36G) (md5: 5b5365f4fe15de24e403a9256079be70)

  • Original paper version.

synthtiger_v1.1.zip (38G) (md5: b2757a7e2b5040b14ed64c473533b592)

  • Used MJ/ST lexicon instead of MJ/ST label.
  • Fixed a bug that applies transformation twice on curved text.
  • Fixed a bug that incorrectly converts grayscale to RGB.
Version IIIT5k SVT IC03 IC13 IC15 SVTP CUTE80 Total
1.0 93.2 87.3 90.5 92.9 72.1 77.7 80.6 85.9
1.1 93.4 87.6 91.4 93.2 73.9 77.8 80.6 86.6

Structure

The structure of the dataset is as follows. The dataset contains 10M images.

gt.txt
images/
    0/
        0.jpg
        1.jpg
        ...
        9998.jpg
        9999.jpg
    1/
    ...
    998/
    999/

The format of gt.txt is as follows. Image path and label are separated by tab. (<image_path>\t<label>)

images/0/0.jpg	10
images/0/1.jpg	date:
...
images/999/9999998.jpg	STUFFIER
images/999/9999999.jpg	Re:

Usage

# for macOS
$ export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES

# install python packages
$ pip install -r requirements.txt

$ python gen.py --template TEMPLATE
                --config CONFIG
                --output OUTPUT
                [--count COUNT]
                [--worker WORKER]

Requirements

  • python >= 3.6
  • libraqm

Parameters

Name Type Default Description
template string Template module path
config string Config file path
output string Folder path to save data
count integer 100 Number of data
worker integer 1 Number of workers

Examples

Default text images

# horizontal
python gen.py --template templates/default.py --config templates/default_horizontal.yaml --output results --worker 4

# vertical
python gen.py --template templates/default.py --config templates/default_vertical.yaml --output results --worker 4

Multiline text images

python gen.py --template templates/multiline.py --config templates/multiline.yaml --output results --worker 4

Advanced Usage

Non-Latin language data generation

  1. Prepare corpus and fonts

    corpus - txt file, line by line (example)

    font - ttf/otf file (example)

  2. Extract renderable charsets

    python tools/extract_font_charset.py --input fonts --worker 4

    This script extracts renderable charsets for all font files. (example)

    Text files are generated in the input path with the same names as the fonts.

  3. Edit corpus path and font path in config file

  4. Run gen.py

Colormap customization

  1. Prepare images

    image - jpg/jpeg/png/bmp file

  2. Create colormaps

    python tools/create_colormap.py --input images --output colormap.txt --worker 4

    This script creates colormaps for all image files. (example)

  3. Edit colormap path in config file

  4. Run gen.py

Citation

@article{yim2021synthtiger,
  title={SynthTIGER: Synthetic Text Image GEneratoR Towards Better Text Recognition Models},
  author={Yim, Moonbin and Kim, Yoonsik and Cho, Han-Cheol and Park, Sungrae},
  journal={arXiv preprint arXiv:2107.09313},
  year={2021}
}

License

SynthTIGER
Copyright (c) 2021-present NAVER Corp.

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

The following directories and their subdirectories are licensed the same as their origins. Please refer to NOTICE

docs/
docsrc/
resources/font/
Comments
  • How to get character bbox annotation?

    How to get character bbox annotation?

    this project is very helpful for generating synth text for scene text recognition, and it seem to generate text image by combine several character images, but the outputs doesn't contain information of each character, is it possible to get character annotation, for example, each character and its location?

    opened by GuokunWang 5
  • Question about background image

    Question about background image

    Hi, @moonbings Thank you for sharing nice work.

    I wonder where's the code that load background image such as below?

    image

    And I also wonder when the Synthdog will be released

    opened by yellowjs0304 3
  • training configuration of STR model

    training configuration of STR model

    Hi, thank you for open-sourcing your work. I have a question about the training configuration of STR model used in your paper. How did you set the sensitive character mode and data_filtering_off option in BEST model?

    opened by tzm-tora 2
  • gen.py throws munmap_chunk(): invalid pointer and stops generating images

    gen.py throws munmap_chunk(): invalid pointer and stops generating images

    On running gen.py with the command posted in the readme, the process throws

    munmap_chunk(): invalid pointer

    and is stuck. It stops generating images. Each worker generates a maximum of 1 image before this error is thrown.

    opened by arundprabhu 2
  • libraqm dependency error occurs

    libraqm dependency error occurs

    Thanks for the nice package! When i followed your descriptions and used it, i met some error. KeyError: 'setting text direction, language or font features is not supported without libraqm' I have installed python package 'synthtiger' and dependency following the shell script How can i solve this issue? Thanks,

    opened by mandal4 1
  • How to generate Non-Broken bengali synthetic data for text recognition?

    How to generate Non-Broken bengali synthetic data for text recognition?

    thank you for your awesome sharing. i tried to generate text recognition data using your library for bangla printed documents,i made all the necessary changes to make this synthesis engine working for bengali text recognition data generation, however for bangla it is breaking text during text to image conversion just like this trdg issue of mine : https://github.com/Belval/TextRecognitionDataGenerator/issues/253 (where the authors of that repo is not responsive at all)

    here is a sample that i got while using this synthesis engine.

    label : নির্লজ্জ

    image : https://i.ibb.co/0Y0W5GG/image.jpg ( নির্লজ্জ word got broken )

    do you know how to solve this issue? thanks a lot in advance for your great work.

    opened by mobassir94 1
  • Other latin language

    Other latin language

    Hi, great work ! What exactly should one modify to change language. I found this: https://github.com/clovaai/synthtiger/tree/master/resources/charset but not sure if I need to generate something from that.

    thanks in advance :)

    opened by Globolik 1
  • Fix seed bug and update test code

    Fix seed bug and update test code

    Description

    • Fixed a bug for seed with none value. (https://github.com/clovaai/synthtiger/pull/45#issuecomment-1311229534)
    • Added retry parameter in generator function to make testing easier.
      • If retry is False, when an error occurs during data generation, it does not retry and return data of None value.
    • Updated test code.

    Changes in this PR

    • Updated main.py, gen.py.
    • Updated test code.

    How has this been tested?

    Checked directly whether images are generated correctly.

    opened by moonbings 0
  • Update seed feature

    Update seed feature

    Description

    • Random seed is also applied to main process.
    • Added get_global_random_states, set_global_random_states, set_global_random_seed functions. This functions can be used in templates.

    Changes in this PR

    • Updated main.py, gen.py.
    • Added functions (get_global_random_states, set_global_random_states, set_global_random_seed).

    How has this been tested?

    Checked directly whether images are generated correctly.

    opened by moonbings 0
  • Add seed option

    Add seed option

    Description

    This engine cannot reproduce same data. So I added random seed option to solve this issue.

    I also changed multi processing structure. When the new engine puts tasks in the task queue, each worker gets and runs it. And each task is assigned an index. Note that the incoming indexes are random when storing data because the task completion times are different.

    Changes in this PR

    • Added random seed option. (-s or --seed)
    • Refactored main.py, gen.py
    • Updated README.md
    • Updated .pylintrc

    How has this been tested?

    Checked directly whether images are generated correctly.

    opened by moonbings 0
  • Update ubuntu depends

    Update ubuntu depends

    Description

    Updated shell script to get latest package list before installing dependencies in ubuntu.

    Changes in this PR

    • Updated install_ubuntu_depends.sh.

    How has this been tested?

    Check CI test.

    opened by moonbings 0
  • Issue with synthtiger_v1.1.zip

    Issue with synthtiger_v1.1.zip

    Hi All,

    I have downloaded all parts of synthtiger_v1.1.zip. When I combined into a single zip file, the resulting zip file is not valid.

    Anyone faces similar problem ?

    Appreciate your advices.

    opened by rinabuoy 1
  • Improving documentation

    Improving documentation

    Hi team,

    Is there any chance you could improve the documentation for this library to describe what the functions and classes actually do? I'm having trouble figuring out how to utilize this library for different kinds of unstructured & semi-structured text images.

    opened by haydenedelson 0
  • Images are very distorted

    Images are very distorted

    Hello @moonbings During synthetic dataset generation, some image are very distorted and i don't have any idea how to fix it. I played with some parameters but it didnt work for me. Any solution?

    Samples image are not clear 23 33 83

    opened by khawar-islam 1
  • RuntimeError: Text is not visible

    RuntimeError: Text is not visible

    Hello @moonbings While generating Korean dataset, I have added all given requirements like fonts and everything. I am using below command to generate dataset but getting an error between data generation process.

    python -m synthtiger -o results -w 4 -v examples/synthtiger/template.py SynthTiger examples/synthtiger/config_vertical.yaml

    Generated 56 data
    Generated 57 data
    Generated 58 data
    Generated 59 data
    Generated 60 data
    Traceback (most recent call last):
      File "/media/cvpr/CM_22/synthtiger/synthtiger/gen.py", line 71, in _generate
        data = template.generate()
      File "/media/cvpr/CM_22/synthtiger/examples/synthtiger/template.py", line 117, in generate
        image = _blend_images(fg_image, bg_image, self.visibility_check)
      File "/media/cvpr/CM_22/synthtiger/examples/synthtiger/template.py", line 246, in _blend_images
        raise RuntimeError("Text is not visible")
    RuntimeError: Text is not visible
    
    Generated 61 data
    Generated 62 data
    Generated 63 data
    Generated 64 data
    Generated 65 data
    Generated 66 data
    
    opened by khawar-islam 1
Releases(1.2.1)
  • 1.2.1(Nov 11, 2022)

    • Fix seed bug and update test code (https://github.com/clovaai/synthtiger/pull/46)
      • Fixed a bug for seed with none value. (https://github.com/clovaai/synthtiger/pull/45#issuecomment-1311229534)
      • Added retry parameter in generator function to make testing easier.
      • Updated test code.
    Source code(tar.gz)
    Source code(zip)
  • 1.2.0(Nov 10, 2022)

    • Updated seed feature (https://github.com/clovaai/synthtiger/pull/44)
      • Random seed is also applied to main process.
      • Added get_global_random_states, set_global_random_states, set_global_random_seed functions.
    Source code(tar.gz)
    Source code(zip)
  • 1.1.0(Nov 9, 2022)

    • Added character bbox and mask outputs (https://github.com/clovaai/synthtiger/pull/33, https://github.com/clovaai/synthtiger/pull/39)
    • Added seed option (https://github.com/clovaai/synthtiger/pull/42)
    • Changed multi processing structure (https://github.com/clovaai/synthtiger/pull/42)
      • The incoming indexes are random when storing data because the task completion times are different.
    • Updated ubuntu depends (https://github.com/clovaai/synthtiger/pull/41)
    • Updated some style and add some information in README.md (https://github.com/clovaai/synthtiger/pull/40)
    Source code(tar.gz)
    Source code(zip)
  • 1.0.2(May 8, 2022)

  • 1.0.1(Feb 17, 2022)

  • 1.0.0(Feb 15, 2022)

Owner
Clova AI Research
Open source repository of Clova AI Research, NAVER & LINE
Clova AI Research
Navigating StyleGAN2 w latent space using CLIP

Navigating StyleGAN2 w latent space using CLIP an attempt to build sth with the official SG2-ADA Pytorch impl kinda inspired by Generating Images from

Mike K. 55 Dec 06, 2022
Object Detection and Multi-Object Tracking

Object Detection and Multi-Object Tracking

Bobby Chen 1.6k Jan 04, 2023
Apache Spark - A unified analytics engine for large-scale data processing

Apache Spark Spark is a unified analytics engine for large-scale data processing. It provides high-level APIs in Scala, Java, Python, and R, and an op

The Apache Software Foundation 34.7k Jan 04, 2023
UniFormer - official implementation of UniFormer

UniFormer This repo is the official implementation of "Uniformer: Unified Transformer for Efficient Spatiotemporal Representation Learning". It curren

SenseTime X-Lab 573 Jan 04, 2023
TCTrack: Temporal Contexts for Aerial Tracking (CVPR2022)

TCTrack: Temporal Contexts for Aerial Tracking (CVPR2022) Ziang Cao and Ziyuan Huang and Liang Pan and Shiwei Zhang and Ziwei Liu and Changhong Fu In

Intelligent Vision for Robotics in Complex Environment 100 Dec 19, 2022
Implementation detail for paper "Multi-level colonoscopy malignant tissue detection with adversarial CAC-UNet"

Multi-level-colonoscopy-malignant-tissue-detection-with-adversarial-CAC-UNet Implementation detail for our paper "Multi-level colonoscopy malignant ti

CVSM Group - email: <a href=[email protected]"> 84 Nov 22, 2022
PyTorch implementation of PNASNet-5 on ImageNet

PNASNet.pytorch PyTorch implementation of PNASNet-5. Specifically, PyTorch code from this repository is adapted to completely match both my implemetat

Chenxi Liu 314 Nov 25, 2022
Large scale embeddings on a single machine.

Marius Marius is a system under active development for training embeddings for large-scale graphs on a single machine. Training on large scale graphs

Marius 107 Jan 03, 2023
PyTorch3D is FAIR's library of reusable components for deep learning with 3D data

Introduction PyTorch3D provides efficient, reusable components for 3D Computer Vision research with PyTorch. Key features include: Data structure for

Facebook Research 6.8k Jan 01, 2023
Video Frame Interpolation with Transformer (CVPR2022)

VFIformer Official PyTorch implementation of our CVPR2022 paper Video Frame Interpolation with Transformer Dependencies python = 3.8 pytorch = 1.8.0

DV Lab 63 Dec 16, 2022
Predicting Price of house by considering ,house age, Distance from public transport

House-Price-Prediction Predicting Price of house by considering ,house age, Distance from public transport, No of convenient stores around house etc..

Musab Jaleel 1 Jan 08, 2022
torchlm is aims to build a high level pipeline for face landmarks detection, it supports training, evaluating, exporting, inference(Python/C++) and 100+ data augmentations

💎A high level pipeline for face landmarks detection, supports training, evaluating, exporting, inference and 100+ data augmentations, compatible with torchvision and albumentations, can easily instal

DefTruth 142 Dec 25, 2022
Code for "OctField: Hierarchical Implicit Functions for 3D Modeling (NeurIPS 2021)"

OctField(Jittor): Hierarchical Implicit Functions for 3D Modeling Introduction This repository is code release for OctField: Hierarchical Implicit Fun

55 Dec 08, 2022
Benchmark library for high-dimensional HPO of black-box models based on Weighted Lasso regression

LassoBench LassoBench is a library for high-dimensional hyperparameter optimization benchmarks based on Weighted Lasso regression. Note: LassoBench is

Kenan Šehić 5 Mar 15, 2022
Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Data

Seasonal Contrast: Unsupervised Pre-Training from Uncurated Remote Sensing Data This is the official PyTorch implementation of the SeCo paper: @articl

ElementAI 101 Dec 12, 2022
Pop-Out Motion: 3D-Aware Image Deformation via Learning the Shape Laplacian (CVPR 2022)

Pop-Out Motion Pop-Out Motion: 3D-Aware Image Deformation via Learning the Shape Laplacian (CVPR 2022) Jihyun Lee*, Minhyuk Sung*, Hyunjin Kim, Tae-Ky

Jihyun Lee 88 Nov 22, 2022
Code for the paper: Fighting Fake News: Image Splice Detection via Learned Self-Consistency

Fighting Fake News: Image Splice Detection via Learned Self-Consistency [paper] [website] Minyoung Huh *12, Andrew Liu *1, Andrew Owens1, Alexei A. Ef

minyoung huh (jacob) 174 Dec 09, 2022
Realtime Face Anti Spoofing with Face Detector based on Deep Learning using Tensorflow/Keras and OpenCV

Realtime Face Anti-Spoofing Detection 🤖 Realtime Face Anti Spoofing Detection with Face Detector to detect real and fake faces Please star this repo

Prem Kumar 86 Aug 03, 2022
CSPML (crystal structure prediction with machine learning-based element substitution)

CSPML (crystal structure prediction with machine learning-based element substitution) CSPML is a unique methodology for the crystal structure predicti

8 Dec 20, 2022
LSTM-VAE Implementation and Relevant Evaluations

LSTM-VAE Implementation and Relevant Evaluations Before using any file in this repository, please create two directories under the root directory name

Lan Zhang 5 Oct 08, 2022