Official implementation of SynthTIGER (Synthetic Text Image GEneratoR) ICDAR 2021

Overview

🐯 SynthTIGER: Synthetic Text Image GEneratoR

Official implementation of SynthTIGER | Paper | Datasets

Moonbin Yim1, Yoonsik Kim1, Han-cheol Cho1, Sungrae Park2

1 Clova AI Research, NAVER Corp.

2 Upstage AI Research





Contents

Updates

Datasets

SynthTIGER is available for download at google drive.

synthtiger_v1.0.zip (36G) (md5: 5b5365f4fe15de24e403a9256079be70)

  • Original paper version.

synthtiger_v1.1.zip (38G) (md5: b2757a7e2b5040b14ed64c473533b592)

  • Used MJ/ST lexicon instead of MJ/ST label.
  • Fixed a bug that applies transformation twice on curved text.
  • Fixed a bug that incorrectly converts grayscale to RGB.
Version IIIT5k SVT IC03 IC13 IC15 SVTP CUTE80 Total
1.0 93.2 87.3 90.5 92.9 72.1 77.7 80.6 85.9
1.1 93.4 87.6 91.4 93.2 73.9 77.8 80.6 86.6

Structure

The structure of the dataset is as follows. The dataset contains 10M images.

gt.txt
images/
    0/
        0.jpg
        1.jpg
        ...
        9998.jpg
        9999.jpg
    1/
    ...
    998/
    999/

The format of gt.txt is as follows. Image path and label are separated by tab. (<image_path>\t<label>)

images/0/0.jpg	10
images/0/1.jpg	date:
...
images/999/9999998.jpg	STUFFIER
images/999/9999999.jpg	Re:

Usage

# for macOS
$ export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES

# install python packages
$ pip install -r requirements.txt

$ python gen.py --template TEMPLATE
                --config CONFIG
                --output OUTPUT
                [--count COUNT]
                [--worker WORKER]

Requirements

  • python >= 3.6
  • libraqm

Parameters

Name Type Default Description
template string Template module path
config string Config file path
output string Folder path to save data
count integer 100 Number of data
worker integer 1 Number of workers

Examples

Default text images

# horizontal
python gen.py --template templates/default.py --config templates/default_horizontal.yaml --output results --worker 4

# vertical
python gen.py --template templates/default.py --config templates/default_vertical.yaml --output results --worker 4

Multiline text images

python gen.py --template templates/multiline.py --config templates/multiline.yaml --output results --worker 4

Advanced Usage

Non-Latin language data generation

  1. Prepare corpus and fonts

    corpus - txt file, line by line (example)

    font - ttf/otf file (example)

  2. Extract renderable charsets

    python tools/extract_font_charset.py --input fonts --worker 4

    This script extracts renderable charsets for all font files. (example)

    Text files are generated in the input path with the same names as the fonts.

  3. Edit corpus path and font path in config file

  4. Run gen.py

Colormap customization

  1. Prepare images

    image - jpg/jpeg/png/bmp file

  2. Create colormaps

    python tools/create_colormap.py --input images --output colormap.txt --worker 4

    This script creates colormaps for all image files. (example)

  3. Edit colormap path in config file

  4. Run gen.py

Citation

@article{yim2021synthtiger,
  title={SynthTIGER: Synthetic Text Image GEneratoR Towards Better Text Recognition Models},
  author={Yim, Moonbin and Kim, Yoonsik and Cho, Han-Cheol and Park, Sungrae},
  journal={arXiv preprint arXiv:2107.09313},
  year={2021}
}

License

SynthTIGER
Copyright (c) 2021-present NAVER Corp.

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in
all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
THE SOFTWARE.

The following directories and their subdirectories are licensed the same as their origins. Please refer to NOTICE

docs/
docsrc/
resources/font/
Comments
  • How to get character bbox annotation?

    How to get character bbox annotation?

    this project is very helpful for generating synth text for scene text recognition, and it seem to generate text image by combine several character images, but the outputs doesn't contain information of each character, is it possible to get character annotation, for example, each character and its location?

    opened by GuokunWang 5
  • Question about background image

    Question about background image

    Hi, @moonbings Thank you for sharing nice work.

    I wonder where's the code that load background image such as below?

    image

    And I also wonder when the Synthdog will be released

    opened by yellowjs0304 3
  • training configuration of STR model

    training configuration of STR model

    Hi, thank you for open-sourcing your work. I have a question about the training configuration of STR model used in your paper. How did you set the sensitive character mode and data_filtering_off option in BEST model?

    opened by tzm-tora 2
  • gen.py throws munmap_chunk(): invalid pointer and stops generating images

    gen.py throws munmap_chunk(): invalid pointer and stops generating images

    On running gen.py with the command posted in the readme, the process throws

    munmap_chunk(): invalid pointer

    and is stuck. It stops generating images. Each worker generates a maximum of 1 image before this error is thrown.

    opened by arundprabhu 2
  • libraqm dependency error occurs

    libraqm dependency error occurs

    Thanks for the nice package! When i followed your descriptions and used it, i met some error. KeyError: 'setting text direction, language or font features is not supported without libraqm' I have installed python package 'synthtiger' and dependency following the shell script How can i solve this issue? Thanks,

    opened by mandal4 1
  • How to generate Non-Broken bengali synthetic data for text recognition?

    How to generate Non-Broken bengali synthetic data for text recognition?

    thank you for your awesome sharing. i tried to generate text recognition data using your library for bangla printed documents,i made all the necessary changes to make this synthesis engine working for bengali text recognition data generation, however for bangla it is breaking text during text to image conversion just like this trdg issue of mine : https://github.com/Belval/TextRecognitionDataGenerator/issues/253 (where the authors of that repo is not responsive at all)

    here is a sample that i got while using this synthesis engine.

    label : নির্লজ্জ

    image : https://i.ibb.co/0Y0W5GG/image.jpg ( নির্লজ্জ word got broken )

    do you know how to solve this issue? thanks a lot in advance for your great work.

    opened by mobassir94 1
  • Other latin language

    Other latin language

    Hi, great work ! What exactly should one modify to change language. I found this: https://github.com/clovaai/synthtiger/tree/master/resources/charset but not sure if I need to generate something from that.

    thanks in advance :)

    opened by Globolik 1
  • Fix seed bug and update test code

    Fix seed bug and update test code

    Description

    • Fixed a bug for seed with none value. (https://github.com/clovaai/synthtiger/pull/45#issuecomment-1311229534)
    • Added retry parameter in generator function to make testing easier.
      • If retry is False, when an error occurs during data generation, it does not retry and return data of None value.
    • Updated test code.

    Changes in this PR

    • Updated main.py, gen.py.
    • Updated test code.

    How has this been tested?

    Checked directly whether images are generated correctly.

    opened by moonbings 0
  • Update seed feature

    Update seed feature

    Description

    • Random seed is also applied to main process.
    • Added get_global_random_states, set_global_random_states, set_global_random_seed functions. This functions can be used in templates.

    Changes in this PR

    • Updated main.py, gen.py.
    • Added functions (get_global_random_states, set_global_random_states, set_global_random_seed).

    How has this been tested?

    Checked directly whether images are generated correctly.

    opened by moonbings 0
  • Add seed option

    Add seed option

    Description

    This engine cannot reproduce same data. So I added random seed option to solve this issue.

    I also changed multi processing structure. When the new engine puts tasks in the task queue, each worker gets and runs it. And each task is assigned an index. Note that the incoming indexes are random when storing data because the task completion times are different.

    Changes in this PR

    • Added random seed option. (-s or --seed)
    • Refactored main.py, gen.py
    • Updated README.md
    • Updated .pylintrc

    How has this been tested?

    Checked directly whether images are generated correctly.

    opened by moonbings 0
  • Update ubuntu depends

    Update ubuntu depends

    Description

    Updated shell script to get latest package list before installing dependencies in ubuntu.

    Changes in this PR

    • Updated install_ubuntu_depends.sh.

    How has this been tested?

    Check CI test.

    opened by moonbings 0
  • Issue with synthtiger_v1.1.zip

    Issue with synthtiger_v1.1.zip

    Hi All,

    I have downloaded all parts of synthtiger_v1.1.zip. When I combined into a single zip file, the resulting zip file is not valid.

    Anyone faces similar problem ?

    Appreciate your advices.

    opened by rinabuoy 1
  • Improving documentation

    Improving documentation

    Hi team,

    Is there any chance you could improve the documentation for this library to describe what the functions and classes actually do? I'm having trouble figuring out how to utilize this library for different kinds of unstructured & semi-structured text images.

    opened by haydenedelson 0
  • Images are very distorted

    Images are very distorted

    Hello @moonbings During synthetic dataset generation, some image are very distorted and i don't have any idea how to fix it. I played with some parameters but it didnt work for me. Any solution?

    Samples image are not clear 23 33 83

    opened by khawar-islam 1
  • RuntimeError: Text is not visible

    RuntimeError: Text is not visible

    Hello @moonbings While generating Korean dataset, I have added all given requirements like fonts and everything. I am using below command to generate dataset but getting an error between data generation process.

    python -m synthtiger -o results -w 4 -v examples/synthtiger/template.py SynthTiger examples/synthtiger/config_vertical.yaml

    Generated 56 data
    Generated 57 data
    Generated 58 data
    Generated 59 data
    Generated 60 data
    Traceback (most recent call last):
      File "/media/cvpr/CM_22/synthtiger/synthtiger/gen.py", line 71, in _generate
        data = template.generate()
      File "/media/cvpr/CM_22/synthtiger/examples/synthtiger/template.py", line 117, in generate
        image = _blend_images(fg_image, bg_image, self.visibility_check)
      File "/media/cvpr/CM_22/synthtiger/examples/synthtiger/template.py", line 246, in _blend_images
        raise RuntimeError("Text is not visible")
    RuntimeError: Text is not visible
    
    Generated 61 data
    Generated 62 data
    Generated 63 data
    Generated 64 data
    Generated 65 data
    Generated 66 data
    
    opened by khawar-islam 1
Releases(1.2.1)
  • 1.2.1(Nov 11, 2022)

    • Fix seed bug and update test code (https://github.com/clovaai/synthtiger/pull/46)
      • Fixed a bug for seed with none value. (https://github.com/clovaai/synthtiger/pull/45#issuecomment-1311229534)
      • Added retry parameter in generator function to make testing easier.
      • Updated test code.
    Source code(tar.gz)
    Source code(zip)
  • 1.2.0(Nov 10, 2022)

    • Updated seed feature (https://github.com/clovaai/synthtiger/pull/44)
      • Random seed is also applied to main process.
      • Added get_global_random_states, set_global_random_states, set_global_random_seed functions.
    Source code(tar.gz)
    Source code(zip)
  • 1.1.0(Nov 9, 2022)

    • Added character bbox and mask outputs (https://github.com/clovaai/synthtiger/pull/33, https://github.com/clovaai/synthtiger/pull/39)
    • Added seed option (https://github.com/clovaai/synthtiger/pull/42)
    • Changed multi processing structure (https://github.com/clovaai/synthtiger/pull/42)
      • The incoming indexes are random when storing data because the task completion times are different.
    • Updated ubuntu depends (https://github.com/clovaai/synthtiger/pull/41)
    • Updated some style and add some information in README.md (https://github.com/clovaai/synthtiger/pull/40)
    Source code(tar.gz)
    Source code(zip)
  • 1.0.2(May 8, 2022)

  • 1.0.1(Feb 17, 2022)

  • 1.0.0(Feb 15, 2022)

Owner
Clova AI Research
Open source repository of Clova AI Research, NAVER & LINE
Clova AI Research
A PyTorch Implementation of Neural IMage Assessment

NIMA: Neural IMage Assessment This is a PyTorch implementation of the paper NIMA: Neural IMage Assessment (accepted at IEEE Transactions on Image Proc

yunxiaos 418 Dec 29, 2022
fcn by tensorflow

Update An example on how to integrate this code into your own semantic segmentation pipeline can be found in my KittiSeg project repository. tensorflo

9 May 22, 2022
Unofficial Implementation of RobustSTL: A Robust Seasonal-Trend Decomposition Algorithm for Long Time Series (AAAI 2019)

RobustSTL: A Robust Seasonal-Trend Decomposition Algorithm for Long Time Series (AAAI 2019) This repository contains python (3.5.2) implementation of

Doyup Lee 222 Dec 21, 2022
Adversarial examples to the new ConvNeXt architecture

Adversarial examples to the new ConvNeXt architecture To get adversarial examples to the ConvNeXt architecture, run the Colab: https://github.com/stan

Stanislav Fort 19 Sep 18, 2022
JAXDL: JAX (Flax) Deep Learning Library

JAXDL: JAX (Flax) Deep Learning Library Simple and clean JAX/Flax deep learning algorithm implementations: Soft-Actor-Critic (arXiv:1812.05905) Transf

Patrick Hart 4 Nov 27, 2022
A C implementation for creating 2D voronoi diagrams

Branch OSX/Linux Windows master dev jc_voronoi A fast C/C++ header only implementation for creating 2D Voronoi diagrams from a point set Uses Fortune'

Mathias Westerdahl 481 Dec 29, 2022
Convolutional Neural Network for Text Classification in Tensorflow

This code belongs to the "Implementing a CNN for Text Classification in Tensorflow" blog post. It is slightly simplified implementation of Kim's Convo

Denny Britz 5.5k Jan 02, 2023
Hierarchical Cross-modal Talking Face Generation with Dynamic Pixel-wise Loss (ATVGnet)

Hierarchical Cross-modal Talking Face Generation with Dynamic Pixel-wise Loss (ATVGnet) By Lele Chen , Ross K Maddox, Zhiyao Duan, Chenliang Xu. Unive

Lele Chen 218 Dec 27, 2022
Official pytorch implementation of the AAAI 2021 paper Semantic Grouping Network for Video Captioning

Semantic Grouping Network for Video Captioning Hobin Ryu, Sunghun Kang, Haeyong Kang, and Chang D. Yoo. AAAI 2021. [arxiv] Environment Ubuntu 16.04 CU

Hobin Ryu 43 Nov 25, 2022
Deep Learning ❤️ OneFlow

Deep Learning with OneFlow made easy 🚀 ! Carefree? carefree-learn aims to provide CAREFREE usages for both users and developers. User Side Computer V

21 Oct 27, 2022
LinkNet - This repository contains our Torch7 implementation of the network developed by us at e-Lab.

LinkNet This repository contains our Torch7 implementation of the network developed by us at e-Lab. You can go to our blogpost or read the article Lin

e-Lab 158 Nov 11, 2022
IAST: Instance Adaptive Self-training for Unsupervised Domain Adaptation (ECCV 2020)

This repo is the official implementation of our paper "Instance Adaptive Self-training for Unsupervised Domain Adaptation". The purpose of this repo is to better communicate with you and respond to y

CVSM Group - email: <a href=[email protected]"> 84 Dec 12, 2022
This is the repository of our article published on MDPI Entropy "Feature Selection for Recommender Systems with Quantum Computing".

Collaborative-driven Quantum Feature Selection This repository was developed by Riccardo Nembrini, PhD student at Politecnico di Milano. See the websi

Quantum Computing Lab @ Politecnico di Milano 10 Apr 21, 2022
Build a medical knowledge graph based on Unified Language Medical System (UMLS)

UMLS-Graph Build a medical knowledge graph based on Unified Language Medical System (UMLS) Requisite Install MySQL Server 5.6 and import UMLS data int

Donghua Chen 6 Dec 25, 2022
DGL-TreeSearch and the Gurobi-MWIS interface

Independent Set Benchmarking Suite This repository contains the code for our maximum independent set benchmarking suite as well as our implementations

Maximilian Böther 19 Nov 22, 2022
Github Traffic Insights as Prometheus metrics.

github-traffic Github Traffic collects your repository's traffic data and exposes it as Prometheus metrics. Grafana dashboard that displays the metric

Grafana Labs 34 Oct 27, 2022
Revisiting Video Saliency: A Large-scale Benchmark and a New Model (CVPR18, PAMI19)

DHF1K =========================================================================== Wenguan Wang, J. Shen, M.-M Cheng and A. Borji, Revisiting Video Sal

Wenguan Wang 126 Dec 03, 2022
Veri Setinizi Yolov5 Formatına Dönüştürün

Veri Setinizi Yolov5 Formatına Dönüştürün! Bu Repo da Neler Var? Xml Formatındaki Veri Setini .Txt Formatına Çevirme Xml Formatındaki Dosyaları Silme

Kadir Nar 4 Aug 22, 2022
SIR model parameter estimation using a novel algorithm for differentiated uniformization.

TenSIR Parameter estimation on epidemic data under the SIR model using a novel algorithm for differentiated uniformization of Markov transition rate m

The Spang Lab 4 Nov 30, 2022
Create and implement a deep learning library from scratch.

In this project, we create and implement a deep learning library from scratch. Table of Contents Deep Leaning Library Table of Contents About The Proj

Rishabh Bali 22 Aug 23, 2022