Official implementation of SynthTIGER (Synthetic Text Image GEneratoR) ICDAR 2021

Overview

đŸ¯ SynthTIGER: Synthetic Text Image GEneratoR

Official implementation of SynthTIGER | Paper | Datasets

Moonbin Yim1, Yoonsik Kim1, Han-cheol Cho1, Sungrae Park2

1 Clova AI Research, NAVER Corp.

2 Upstage AI Research





Contents

Updates

Datasets

SynthTIGER is available for download at google drive.

synthtiger_v1.0.zip (36G) (md5: 5b5365f4fe15de24e403a9256079be70)

  • Original paper version.

synthtiger_v1.1.zip (38G) (md5: b2757a7e2b5040b14ed64c473533b592)

  • Used MJ/ST lexicon instead of MJ/ST label.
  • Fixed a bug that applies transformation twice on curved text.
  • Fixed a bug that incorrectly converts grayscale to RGB.
Version IIIT5k SVT IC03 IC13 IC15 SVTP CUTE80 Total
1.0 93.2 87.3 90.5 92.9 72.1 77.7 80.6 85.9
1.1 93.4 87.6 91.4 93.2 73.9 77.8 80.6 86.6

Structure

The structure of the dataset is as follows. The dataset contains 10M images.

gt.txt
images/
    0/
        0.jpg
        1.jpg
        ...
        9998.jpg
        9999.jpg
    1/
    ...
    998/
    999/

The format of gt.txt is as follows. Image path and label are separated by tab. (<image_path>\t<label>)

images/0/0.jpg	10
images/0/1.jpg	date:
...
images/999/9999998.jpg	STUFFIER
images/999/9999999.jpg	Re:

Usage

# for macOS
$ export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES

# install python packages
$ pip install -r requirements.txt

$ python gen.py --template TEMPLATE
                --config CONFIG
                --output OUTPUT
                [--count COUNT]
                [--worker WORKER]

Requirements

  • python >= 3.6
  • libraqm

Parameters

Name Type Default Description
template string Template module path
config string Config file path
output string Folder path to save data
count integer 100 Number of data
worker integer 1 Number of workers

Examples

Default text images

# horizontal
python gen.py --template templates/default.py --config templates/default_horizontal.yaml --output results --worker 4

# vertical
python gen.py --template templates/default.py --config templates/default_vertical.yaml --output results --worker 4

Multiline text images

python gen.py --template templates/multiline.py --config templates/multiline.yaml --output results --worker 4

Advanced Usage

Non-Latin language data generation

  1. Prepare corpus and fonts

    corpus - txt file, line by line (example)

    font - ttf/otf file (example)

  2. Extract renderable charsets

    python tools/extract_font_charset.py --input fonts --worker 4

    This script extracts renderable charsets for all font files. (example)

    Text files are generated in the input path with the same names as the fonts.

  3. Edit corpus path and font path in config file

  4. Run gen.py

Colormap customization

  1. Prepare images

    image - jpg/jpeg/png/bmp file

  2. Create colormaps

    python tools/create_colormap.py --input images --output colormap.txt --worker 4

    This script creates colormaps for all image files. (example)

  3. Edit colormap path in config file

  4. Run gen.py

Citation

@article{yim2021synthtiger,
  title={SynthTIGER: Synthetic Text Image GEneratoR Towards Better Text Recognition Models},
  author={Yim, Moonbin and Kim, Yoonsik and Cho, Han-Cheol and Park, Sungrae},
  journal={arXiv preprint arXiv:2107.09313},
  year={2021}
}

License

SynthTIGER
Copyright (c) 2021-present NAVER Corp.

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

The following directories and their subdirectories are licensed the same as their origins. Please refer to NOTICE

docs/
docsrc/
resources/font/
Comments
  • How to get character bbox annotation?

    How to get character bbox annotation?

    this project is very helpful for generating synth text for scene text recognition, and it seem to generate text image by combine several character images, but the outputs doesn't contain information of each character, is it possible to get character annotation, for example, each character and its location?

    opened by GuokunWang 5
  • Question about background image

    Question about background image

    Hi, @moonbings Thank you for sharing nice work.

    I wonder where's the code that load background image such as below?

    image

    And I also wonder when the Synthdog will be released

    opened by yellowjs0304 3
  • training configuration of STR model

    training configuration of STR model

    Hi, thank you for open-sourcing your work. I have a question about the training configuration of STR model used in your paper. How did you set the sensitive character mode and data_filtering_off option in BEST model?

    opened by tzm-tora 2
  • gen.py throws munmap_chunk(): invalid pointer and stops generating images

    gen.py throws munmap_chunk(): invalid pointer and stops generating images

    On running gen.py with the command posted in the readme, the process throws

    munmap_chunk(): invalid pointer

    and is stuck. It stops generating images. Each worker generates a maximum of 1 image before this error is thrown.

    opened by arundprabhu 2
  • libraqm dependency error occurs

    libraqm dependency error occurs

    Thanks for the nice package! When i followed your descriptions and used it, i met some error. KeyError: 'setting text direction, language or font features is not supported without libraqm' I have installed python package 'synthtiger' and dependency following the shell script How can i solve this issue? Thanks,

    opened by mandal4 1
  • How to generate Non-Broken bengali synthetic data for text recognition?

    How to generate Non-Broken bengali synthetic data for text recognition?

    thank you for your awesome sharing. i tried to generate text recognition data using your library for bangla printed documents,i made all the necessary changes to make this synthesis engine working for bengali text recognition data generation, however for bangla it is breaking text during text to image conversion just like this trdg issue of mine : https://github.com/Belval/TextRecognitionDataGenerator/issues/253 (where the authors of that repo is not responsive at all)

    here is a sample that i got while using this synthesis engine.

    label : āĻ¨āĻŋāĻ°ā§āĻ˛āĻœā§āĻœ

    image : https://i.ibb.co/0Y0W5GG/image.jpg ( āĻ¨āĻŋāĻ°ā§āĻ˛āĻœā§āĻœ word got broken )

    do you know how to solve this issue? thanks a lot in advance for your great work.

    opened by mobassir94 1
  • Other latin language

    Other latin language

    Hi, great work ! What exactly should one modify to change language. I found this: https://github.com/clovaai/synthtiger/tree/master/resources/charset but not sure if I need to generate something from that.

    thanks in advance :)

    opened by Globolik 1
  • Fix seed bug and update test code

    Fix seed bug and update test code

    Description

    • Fixed a bug for seed with none value. (https://github.com/clovaai/synthtiger/pull/45#issuecomment-1311229534)
    • Added retry parameter in generator function to make testing easier.
      • If retry is False, when an error occurs during data generation, it does not retry and return data of None value.
    • Updated test code.

    Changes in this PR

    • Updated main.py, gen.py.
    • Updated test code.

    How has this been tested?

    Checked directly whether images are generated correctly.

    opened by moonbings 0
  • Update seed feature

    Update seed feature

    Description

    • Random seed is also applied to main process.
    • Added get_global_random_states, set_global_random_states, set_global_random_seed functions. This functions can be used in templates.

    Changes in this PR

    • Updated main.py, gen.py.
    • Added functions (get_global_random_states, set_global_random_states, set_global_random_seed).

    How has this been tested?

    Checked directly whether images are generated correctly.

    opened by moonbings 0
  • Add seed option

    Add seed option

    Description

    This engine cannot reproduce same data. So I added random seed option to solve this issue.

    I also changed multi processing structure. When the new engine puts tasks in the task queue, each worker gets and runs it. And each task is assigned an index. Note that the incoming indexes are random when storing data because the task completion times are different.

    Changes in this PR

    • Added random seed option. (-s or --seed)
    • Refactored main.py, gen.py
    • Updated README.md
    • Updated .pylintrc

    How has this been tested?

    Checked directly whether images are generated correctly.

    opened by moonbings 0
  • Update ubuntu depends

    Update ubuntu depends

    Description

    Updated shell script to get latest package list before installing dependencies in ubuntu.

    Changes in this PR

    • Updated install_ubuntu_depends.sh.

    How has this been tested?

    Check CI test.

    opened by moonbings 0
  • Issue with synthtiger_v1.1.zip

    Issue with synthtiger_v1.1.zip

    Hi All,

    I have downloaded all parts of synthtiger_v1.1.zip. When I combined into a single zip file, the resulting zip file is not valid.

    Anyone faces similar problem ?

    Appreciate your advices.

    opened by rinabuoy 1
  • Improving documentation

    Improving documentation

    Hi team,

    Is there any chance you could improve the documentation for this library to describe what the functions and classes actually do? I'm having trouble figuring out how to utilize this library for different kinds of unstructured & semi-structured text images.

    opened by haydenedelson 0
  • Images are very distorted

    Images are very distorted

    Hello @moonbings During synthetic dataset generation, some image are very distorted and i don't have any idea how to fix it. I played with some parameters but it didnt work for me. Any solution?

    Samples image are not clear 23 33 83

    opened by khawar-islam 1
  • RuntimeError: Text is not visible

    RuntimeError: Text is not visible

    Hello @moonbings While generating Korean dataset, I have added all given requirements like fonts and everything. I am using below command to generate dataset but getting an error between data generation process.

    python -m synthtiger -o results -w 4 -v examples/synthtiger/template.py SynthTiger examples/synthtiger/config_vertical.yaml

    Generated 56 data
    Generated 57 data
    Generated 58 data
    Generated 59 data
    Generated 60 data
    Traceback (most recent call last):
      File "/media/cvpr/CM_22/synthtiger/synthtiger/gen.py", line 71, in _generate
        data = template.generate()
      File "/media/cvpr/CM_22/synthtiger/examples/synthtiger/template.py", line 117, in generate
        image = _blend_images(fg_image, bg_image, self.visibility_check)
      File "/media/cvpr/CM_22/synthtiger/examples/synthtiger/template.py", line 246, in _blend_images
        raise RuntimeError("Text is not visible")
    RuntimeError: Text is not visible
    
    Generated 61 data
    Generated 62 data
    Generated 63 data
    Generated 64 data
    Generated 65 data
    Generated 66 data
    
    opened by khawar-islam 1
Releases(1.2.1)
  • 1.2.1(Nov 11, 2022)

    • Fix seed bug and update test code (https://github.com/clovaai/synthtiger/pull/46)
      • Fixed a bug for seed with none value. (https://github.com/clovaai/synthtiger/pull/45#issuecomment-1311229534)
      • Added retry parameter in generator function to make testing easier.
      • Updated test code.
    Source code(tar.gz)
    Source code(zip)
  • 1.2.0(Nov 10, 2022)

    • Updated seed feature (https://github.com/clovaai/synthtiger/pull/44)
      • Random seed is also applied to main process.
      • Added get_global_random_states, set_global_random_states, set_global_random_seed functions.
    Source code(tar.gz)
    Source code(zip)
  • 1.1.0(Nov 9, 2022)

    • Added character bbox and mask outputs (https://github.com/clovaai/synthtiger/pull/33, https://github.com/clovaai/synthtiger/pull/39)
    • Added seed option (https://github.com/clovaai/synthtiger/pull/42)
    • Changed multi processing structure (https://github.com/clovaai/synthtiger/pull/42)
      • The incoming indexes are random when storing data because the task completion times are different.
    • Updated ubuntu depends (https://github.com/clovaai/synthtiger/pull/41)
    • Updated some style and add some information in README.md (https://github.com/clovaai/synthtiger/pull/40)
    Source code(tar.gz)
    Source code(zip)
  • 1.0.2(May 8, 2022)

  • 1.0.1(Feb 17, 2022)

  • 1.0.0(Feb 15, 2022)

Owner
Clova AI Research
Open source repository of Clova AI Research, NAVER & LINE
Clova AI Research
Reference models and tools for Cloud TPUs.

Cloud TPUs This repository is a collection of reference models and tools used with Cloud TPUs. The fastest way to get started training a model on a Cl

5k Jan 05, 2023
UIUCTF 2021 Public Challenge Repository

UIUCTF-2021-Public UIUCTF 2021 Public Challenge Repository Notes: every challenge folder contains a challenge.yml file in the format for ctfcli, CTFd'

SIGPwny 15 Nov 03, 2022
Auto-Lama combines object detection and image inpainting to automate object removals

Auto-Lama Auto-Lama combines object detection and image inpainting to automate object removals. It is build on top of DE:TR from Facebook Research and

44 Dec 09, 2022
The code from the paper Character Transformations for Non-Autoregressive GEC Tagging

Character Transformations for Non-Autoregressive GEC Tagging Milan Straka, Jakub NÃĄplava, Jana StrakovÃĄ Charles University Faculty of Mathematics and

ÚFAL 5 Dec 10, 2022
Python package to generate image embeddings with CLIP without PyTorch/TensorFlow

imgbeddings A Python package to generate embedding vectors from images, using OpenAI's robust CLIP model via Hugging Face transformers. These image em

Max Woolf 81 Jan 04, 2023
《Lerning n Intrinsic Grment Spce for Interctive Authoring of Grment Animtion》

Learning an Intrinsic Garment Space for Interactive Authoring of Garment Animation Overview This is the demo code for training a motion invariant enco

YuanBo 213 Dec 14, 2022
PyTorch code for training MM-DistillNet for multimodal knowledge distillation

There is More than Meets the Eye: Self-Supervised Multi-Object Detection and Tracking with Sound by Distilling Multimodal Knowledge MM-DistillNet is a

51 Dec 20, 2022
[ICCV-2021] An Empirical Study of the Collapsing Problem in Semi-Supervised 2D Human Pose Estimation

An Empirical Study of the Collapsing Problem in Semi-Supervised 2D Human Pose Estimation (ICCV 2021) Introduction This is an official pytorch implemen

rongchangxie 42 Jan 04, 2023
Establishing Strong Baselines for TripClick Health Retrieval; ECIR 2022

TripClick Baselines with Improved Training Data Welcome 🙌 to the hub-repo of our paper: Establishing Strong Baselines for TripClick Health Retrieval

Sebastian Hofstätter 3 Nov 03, 2022
CowHerd is a partially-observed reinforcement learning environment

CowHerd is a partially-observed reinforcement learning environment, where the player walks around an area and is rewarded for milking cows. The cows try to escape and the player can place fences to h

Danijar Hafner 6 Mar 06, 2022
TorchXRayVision: A library of chest X-ray datasets and models.

torchxrayvision A library for chest X-ray datasets and models. Including pre-trained models. ( đŸŽŦ promo video about the project) Motivation: While the

Machine Learning and Medicine Lab 575 Jan 08, 2023
Computational modelling of ray propagation through optical elements using the principles of geometric optics (Ray Tracer)

Computational modelling of ray propagation through optical elements using the principles of geometric optics (Ray Tracer) Introduction By applying the

Son Gyo Jung 1 Jul 09, 2022
MAUS: A Dataset for Mental Workload Assessment Using Wearable Sensor - Baseline system

MAUS: A Dataset for Mental Workload Assessment Using Wearable Sensor - Baseline system Getting started To start working on this assignment, you should

2 Aug 06, 2022
Differentiable Factor Graph Optimization for Learning Smoothers @ IROS 2021

Differentiable Factor Graph Optimization for Learning Smoothers Overview Status Setup Datasets Training Evaluation Acknowledgements Overview Code rele

Brent Yi 60 Nov 14, 2022
https://sites.google.com/cornell.edu/recsys2021tutorial

Counterfactual Learning and Evaluation for Recommender Systems (RecSys'21 Tutorial) Materials for "Counterfactual Learning and Evaluation for Recommen

yuta-saito 45 Nov 10, 2022
N-RPG - Novel role playing game da turfu

N-RPG Ce README sera la page de garde du projet. Contenu Il contiendra la prÊsen

4 Mar 15, 2022
Demystifying How Self-Supervised Features Improve Training from Noisy Labels

Demystifying How Self-Supervised Features Improve Training from Noisy Labels This code is a PyTorch implementation of the paper "[Demystifying How Sel

<a href=[email protected]"> 4 Oct 14, 2022
Training a Resilient Q-Network against Observational Interference, Causal Inference Q-Networks

Obs-Causal-Q-Network AAAI 2022 - Training a Resilient Q-Network against Observational Interference Preprint | Slides | Colab Demo | Environment Setup

23 Nov 21, 2022
Anti-UAV base on PaddleDetection

Paddle-Anti-UAV Anti-UAV base on PaddleDetection Background UAVs are very popular and we can see them in many public spaces, such as parks and playgro

Qingzhong Wang 2 Apr 20, 2022
Measures input lag without dedicated hardware, performing motion detection on recorded or live video

What is InputLagTimer? This tool can measure input lag by analyzing a video where both the game controller and the game screen can be seen on a webcam

Bruno Gonzalez 4 Aug 18, 2022